Adjoint ODE behaviour

Hi!

Thanks for trying. I found a few issues, which are fixed now. For now I cannot run your model, so if you could try on your side:

I’d assume going forward with the above version is an improvement. This version does run without these Inf error messages for the example from @Teddy_Groves1 . However, the sampler only produces divergent transitions on that example still. Nevertheless, I must point out that the older rc2 version is not part of the ODE testing framework… which is the released version is. Thus, I do trust the released code more at the moment, given that the new code is far better tested.

I hope the above code also addresses your problems.

Best,
Sebastian

EDIT: Things are a bit urgent as I would love to include the changes in the about to be released 2.28.1. So testing now and iterating would be great. From my view all unit tests pass and the Inf error vanished, which means we have an improvement as it looks.

Hi Sebastian,

I tried again, and nothing changes in terms of errors. Given the urgency of the testing, I have made available a toy example on our project repo here.

Thanks.

1 Like

Great!

One quick q: Does RK45 sample ok (but slowly)?

Correct. This is what motivated us to consider the adjoint solver - scalability to larger problems.

Could you post the model, data and initial?

… and… does the famous rc2 version of the adjoint solver produce results which you trust?

EDIT: At the moment I am really not sure if we mess something up on the Stan side or if the adjoint solver is not being used correctly or even outputs non-sense. Again, our unit tests are quite massive and they do pass.

Sure, I’ve added an .html file in the same folder with the model description, data and initial. Just open it with your browser and let me know of any questions.

does the famous rc2 version of the adjoint solver produce results which you trust?

So far I have only tested “cmdstan-2.27.0” and “cmdstan-ode-adjoint-v2”. How do I access the 2.27.0-rc2 version? I see that 2.28.0-rc2 was made available recently, if this is what you are refering to.

I was referring to cmdstan-ode-adjoint-v2.

Thanks for the model description… I would like to have a stan model file, a data file and an init file. That’s a lot easier for me to handle rather than running a script with many dependencies. Thanks!

cmdstan-ode-adjoint-v2 did not work for me, either.

The .stan model file is already available. I have added a data.RData and init.RData file, containing the respective objects in the desired format.

Cool. Next time export it directly in stan_rdump flavour such that I can directly call cmdstan on it… I will do that now.

I will write a model variant using RK45 and compare that to the adjoint outputs… let’s see.

1 Like

Thanks a lot for this work @wds15 - I have tried the files you linked and this also removes the Inf errors for me. On Monday I should be able to update my repo with a less problematic model configuration, or at least one that seems to work ok with the bdf solver.

HI!

In case you test anything, please use the version from the most u ptr date branch here:

A few more issues were resolved.

Sebastian

2 Likes

Hi Sebastian,

I copied the contents of the files cvodes_integrator_adjoint.hpp & ode_adjoint.hpp from the branch to the respective files in my cmdstan-2.27.0 installation. Tested on the same model , but the same errors persist.

Happy to discuss further once the testing on the toy example is completed from your end.

Thanks.

We should definitely now start with a model which runs ok - but slow - with one of the existing forward solvers (bdf or rk45)… and then switch to adjoint.

Given that we struggle that much it would also make sense to go back to the SBC simulation I wrote a while ago. That simulation showed that the Adjoint ODE implementation at the time was OK.

For now, I do think that the changes made in the branch are an improvement and I’d hope that @stevebronder finds the time to review them so that it is included in 2.28.1 for now. It seems we need to investigate further apparently.

1 Like

Yes will look them over today!

1 Like

Thanks.

We should definitely now start with a model which runs ok - but slow - with one of the existing forward solvers (bdf or rk45)… and then switch to adjoint.

Cool! Posterior outputs for the single-type SIR model with a time-varying effective contact rate (trapezoidal rule) are available. It’s slow, but it is indeed working nicely. Aiming to extend to a multi-type SIR, we take a step back and wish to examine the performance of the single-type SIR with the adjoint solver first. This implementation fails, with the errors reported earlier in this thread.

For now, I do think that the changes made in the branch are an improvement and I’d hope that @stevebronder finds the time to review them so that it is included in 2.28.1 for now. It seems we need to investigate further apparently.

… and this is great news! At this stage, I am trying to cross out possible reasons from my list as to why the single-type SIR model implementation with the adjoint solver fails. I.e. is it the solver itself that struggles for this model, etc. Or it might be the case that the route proposed by @Funko_Unko should be followed, see also the CoDatMo thread.

Is this the stan file you are running?

I was trying to replicate this locally but getting compiler errors because some of the vectors do not have dimensions. You can also dump the data and init into the folder such as

cmdstanr::write_stan_json(cov_data, file="./Test_Adjoint/adjoint_data.json")
cmdstanr::write_stan_json(ini_1(), file="./Test_Adjoint/init.json")
1 Like

Yes, this is the .stan file with the model.

You can also dump the data and init into the folder

I have just placed the adjoint_data.json & init.json files in the folder, thank you.

I was trying to replicate this locally but getting compiler errors because some of the vectors do not have dimensions.

This is strange, I did not get any compiler errors in my machine with cmdstanr 2.27.0, RTools 4.0 and:

R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=Greek_Greece.1253  LC_CTYPE=Greek_Greece.1253    LC_MONETARY=Greek_Greece.1253
[4] LC_NUMERIC=C                  LC_TIME=Greek_Greece.1253    
system code page: 1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] Bernadette_0.3.0     mgcv_1.8-35          nlme_3.1-152         gridExtra_2.3       
 [5] bayesplot_1.8.1      cmdstanr_0.4.0.9000  rstan_2.21.2         StanHeaders_2.21.0-7
 [9] vroom_1.5.2          rvest_1.0.0          forcats_0.5.1        stringr_1.4.0       
[13] dplyr_1.0.7          purrr_0.3.4          readr_1.4.0          tidyr_1.1.3         
[17] tibble_3.1.2         tidyverse_1.3.1      ggplot2_3.3.5       

loaded via a namespace (and not attached):
 [1] httr_1.4.2         bit64_4.0.5        jsonlite_1.7.2     splines_4.1.0      modelr_0.1.8      
 [6] RcppParallel_5.1.4 assertthat_0.2.1   stats4_4.1.0       cellranger_1.1.0   pillar_1.6.3      
[11] backports_1.2.1    lattice_0.20-44    glue_1.4.2         colorspace_2.0-2   Matrix_1.3-3      
[16] plyr_1.8.6         pkgconfig_2.0.3    broom_0.7.8        haven_2.4.1        scales_1.1.1      
[21] processx_3.5.2     tzdb_0.1.1         generics_0.1.0     ellipsis_0.3.2     withr_2.4.2       
[26] cli_3.0.0          magrittr_2.0.1     crayon_1.4.1       readxl_1.3.1       ps_1.6.0          
[31] fs_1.5.0           fansi_0.5.0        MASS_7.3-54        xml2_1.3.2         pkgbuild_1.2.0    
[36] tools_4.1.0        loo_2.4.1          prettyunits_1.1.1  hms_1.1.0          lifecycle_1.0.1   
[41] matrixStats_0.59.0 scoringRules_1.0.1 V8_3.4.2           munsell_0.5.0      reprex_2.0.0      
[46] callr_3.7.0        compiler_4.1.0     rlang_0.4.11       grid_4.1.0         ggridges_0.5.3    
[51] rstudioapi_0.13    gtable_0.3.0       codetools_0.2-18   inline_0.3.19      DBI_1.1.1         
[56] curl_4.3.2         R6_2.5.1           lubridate_1.7.10   knitr_1.33         bit_4.0.4         
[61] utf8_1.2.1         stringi_1.6.2      parallel_4.1.0     Rcpp_1.0.7         vctrs_0.3.8       
[66] dbplyr_2.1.1       tidyselect_1.1.1   xfun_0.24 

Can you double check you are running the same file as the one in your repo I’ve linked below

In that stan program you have data with no size which is illegal syntax in Stan. i.e. there should be a [whatever_size_abs_tol_backward_is] on the line vector abs_tol_backward; like

vector[whatever_size_abs_tol_backward_is] abs_tol_backward;
1 Like

Hi @stevebronder !

We have two models which are problematic reported from users. The one I posted in the PR is the one from @Teddy_Groves1 . I updated the comment where I post stan file, data and initial to also now include the missing functions.stan. Better take that one as this is the only example which I ran myself and confirmed the issues. The one from @bernadette-eu was not run by myself yet as problems surfaced with the other model already and I was able to fix them with the changes in the code which are in the PR.

I wanted to get back to the model from @bernadette-eu at some point.

Sebastian

1 Like

Hi @stevebronder ,

Good catch, it’s weird that this was not picked up in my machine while compiling the .stan file. I list below the steps followed, for reproducibility on the testing:

  1. Copied the files cvodes_integrator_adjoint.hpp and ode_adjoint.hpp to my cmdrstan 2.27.0 installation.
  2. Addressed the issue with the missing vector size for the data objects abs_tol_forward and abs_tol_backward. It should now read:
vector[n_difeq] abs_tol_forward;
vector[n_difeq] abs_tol_backward; 

Hopefully there is now an agreement with the guidelines presented in the Stan Functions reference doc and the Stan Users Guide doc.

  1. This updated .stan file is available in my repo. The init and data .json files lie there as well.

  2. Compiled the file Test_Adjoint_v2.stan on my machine. I get the following messages:



  3. Test run: receiving the error messages " Exception: ode_adjoint_tol_ctl: ode parameters and data[1] is nan".

Thanks for the great work.

EDIT: (Steve) Fixed link to stan file