Hi @Teddy_Groves1 & @bernadette-eu !
Can you please download the files
and replace the respective on in Stan-math (sitting under cmdstan/stan/lib/stan_math/…)? I hope this will make things work for you.
Should this work, then I need to build back a lot of changes I made in a speculative way… at which point I would ask you once more to test if you don’t mind.
Best,
Sebastian
Hi Sebastian,
Thank you for your efforts. I have attempted a longer MCMC run but I’m afraid the errors persist. I attach a screenshot below. A couple of points:
The error code changes to “-3” after using the debugged .hpp files. Previously the error would show “-1”, please refer to my screenshots above.
Given that my daily time series consists of 395 points, I miss why I receive an error message for “data[397] is inf, but must be finite” (see middle of screenshot).
Best,
Lampros
Hi!
Thanks for trying. I found a few issues, which are fixed now. For now I cannot run your model, so if you could try on your side:
I’d assume going forward with the above version is an improvement. This version does run without these Inf error messages for the example from @Teddy_Groves1 . However, the sampler only produces divergent transitions on that example still. Nevertheless, I must point out that the older rc2 version is not part of the ODE testing framework… which is the released version is. Thus, I do trust the released code more at the moment, given that the new code is far better tested.
I hope the above code also addresses your problems.
Best,
Sebastian
EDIT: Things are a bit urgent as I would love to include the changes in the about to be released 2.28.1. So testing now and iterating would be great. From my view all unit tests pass and the Inf error vanished, which means we have an improvement as it looks.
Hi Sebastian,
I tried again, and nothing changes in terms of errors. Given the urgency of the testing, I have made available a toy example on our project repo here.
Thanks.
Great!
One quick q: Does RK45 sample ok (but slowly)?
Correct. This is what motivated us to consider the adjoint solver - scalability to larger problems.
Could you post the model, data and initial?
… and… does the famous rc2 version of the adjoint solver produce results which you trust?
EDIT: At the moment I am really not sure if we mess something up on the Stan side or if the adjoint solver is not being used correctly or even outputs non-sense. Again, our unit tests are quite massive and they do pass.
Sure, I’ve added an .html file in the same folder with the model description, data and initial. Just open it with your browser and let me know of any questions.
does the famous rc2 version of the adjoint solver produce results which you trust?
So far I have only tested “cmdstan-2.27.0” and “cmdstan-ode-adjoint-v2”. How do I access the 2.27.0-rc2 version? I see that 2.28.0-rc2 was made available recently, if this is what you are refering to.
I was referring to cmdstan-ode-adjoint-v2.
Thanks for the model description… I would like to have a stan model file, a data file and an init file. That’s a lot easier for me to handle rather than running a script with many dependencies. Thanks!
cmdstan-ode-adjoint-v2 did not work for me, either.
The .stan model file is already available. I have added a data.RData and init.RData file, containing the respective objects in the desired format.
Cool. Next time export it directly in stan_rdump flavour such that I can directly call cmdstan on it… I will do that now.
I will write a model variant using RK45 and compare that to the adjoint outputs… let’s see.
Thanks a lot for this work @wds15 - I have tried the files you linked and this also removes the Inf errors for me. On Monday I should be able to update my repo with a less problematic model configuration, or at least one that seems to work ok with the bdf solver.
HI!
In case you test anything, please use the version from the most u ptr date branch here:
A few more issues were resolved.
Sebastian
Hi Sebastian,
I copied the contents of the files cvodes_integrator_adjoint.hpp & ode_adjoint.hpp from the branch to the respective files in my cmdstan-2.27.0 installation. Tested on the same model , but the same errors persist.
Happy to discuss further once the testing on the toy example is completed from your end.
Thanks.
We should definitely now start with a model which runs ok - but slow - with one of the existing forward solvers (bdf or rk45)… and then switch to adjoint.
Given that we struggle that much it would also make sense to go back to the SBC simulation I wrote a while ago. That simulation showed that the Adjoint ODE implementation at the time was OK.
For now, I do think that the changes made in the branch are an improvement and I’d hope that @stevebronder finds the time to review them so that it is included in 2.28.1 for now. It seems we need to investigate further apparently.
Yes will look them over today!
Thanks.
We should definitely now start with a model which runs ok - but slow - with one of the existing forward solvers (bdf or rk45)… and then switch to adjoint.
Cool! Posterior outputs for the single-type SIR model with a time-varying effective contact rate (trapezoidal rule) are available. It’s slow, but it is indeed working nicely. Aiming to extend to a multi-type SIR, we take a step back and wish to examine the performance of the single-type SIR with the adjoint solver first. This implementation fails, with the errors reported earlier in this thread.
For now, I do think that the changes made in the branch are an improvement and I’d hope that @stevebronder finds the time to review them so that it is included in 2.28.1 for now. It seems we need to investigate further apparently.
… and this is great news! At this stage, I am trying to cross out possible reasons from my list as to why the single-type SIR model implementation with the adjoint solver fails. I.e. is it the solver itself that struggles for this model, etc. Or it might be the case that the route proposed by @Funko_Unko should be followed, see also the CoDatMo thread.
Is this the stan file you are running?
I was trying to replicate this locally but getting compiler errors because some of the vectors do not have dimensions. You can also dump the data and init into the folder such as
cmdstanr::write_stan_json(cov_data, file="./Test_Adjoint/adjoint_data.json")
cmdstanr::write_stan_json(ini_1(), file="./Test_Adjoint/init.json")
Yes, this is the .stan file with the model.
You can also dump the data and init into the folder
I have just placed the adjoint_data.json & init.json files in the folder, thank you.
I was trying to replicate this locally but getting compiler errors because some of the vectors do not have dimensions.
This is strange, I did not get any compiler errors in my machine with cmdstanr 2.27.0, RTools 4.0 and:
R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)
Matrix products: default
locale:
[1] LC_COLLATE=Greek_Greece.1253 LC_CTYPE=Greek_Greece.1253 LC_MONETARY=Greek_Greece.1253
[4] LC_NUMERIC=C LC_TIME=Greek_Greece.1253
system code page: 1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] Bernadette_0.3.0 mgcv_1.8-35 nlme_3.1-152 gridExtra_2.3
[5] bayesplot_1.8.1 cmdstanr_0.4.0.9000 rstan_2.21.2 StanHeaders_2.21.0-7
[9] vroom_1.5.2 rvest_1.0.0 forcats_0.5.1 stringr_1.4.0
[13] dplyr_1.0.7 purrr_0.3.4 readr_1.4.0 tidyr_1.1.3
[17] tibble_3.1.2 tidyverse_1.3.1 ggplot2_3.3.5
loaded via a namespace (and not attached):
[1] httr_1.4.2 bit64_4.0.5 jsonlite_1.7.2 splines_4.1.0 modelr_0.1.8
[6] RcppParallel_5.1.4 assertthat_0.2.1 stats4_4.1.0 cellranger_1.1.0 pillar_1.6.3
[11] backports_1.2.1 lattice_0.20-44 glue_1.4.2 colorspace_2.0-2 Matrix_1.3-3
[16] plyr_1.8.6 pkgconfig_2.0.3 broom_0.7.8 haven_2.4.1 scales_1.1.1
[21] processx_3.5.2 tzdb_0.1.1 generics_0.1.0 ellipsis_0.3.2 withr_2.4.2
[26] cli_3.0.0 magrittr_2.0.1 crayon_1.4.1 readxl_1.3.1 ps_1.6.0
[31] fs_1.5.0 fansi_0.5.0 MASS_7.3-54 xml2_1.3.2 pkgbuild_1.2.0
[36] tools_4.1.0 loo_2.4.1 prettyunits_1.1.1 hms_1.1.0 lifecycle_1.0.1
[41] matrixStats_0.59.0 scoringRules_1.0.1 V8_3.4.2 munsell_0.5.0 reprex_2.0.0
[46] callr_3.7.0 compiler_4.1.0 rlang_0.4.11 grid_4.1.0 ggridges_0.5.3
[51] rstudioapi_0.13 gtable_0.3.0 codetools_0.2-18 inline_0.3.19 DBI_1.1.1
[56] curl_4.3.2 R6_2.5.1 lubridate_1.7.10 knitr_1.33 bit_4.0.4
[61] utf8_1.2.1 stringi_1.6.2 parallel_4.1.0 Rcpp_1.0.7 vctrs_0.3.8
[66] dbplyr_2.1.1 tidyselect_1.1.1 xfun_0.24
Can you double check you are running the same file as the one in your repo I’ve linked below
In that stan program you have data with no size which is illegal syntax in Stan. i.e. there should be a [whatever_size_abs_tol_backward_is] on the line vector abs_tol_backward; like
vector[whatever_size_abs_tol_backward_is] abs_tol_backward;