Hi,
Recently I have performed the same stan code with same input data on a Windows computer with rstan (Version 2.21.2) and cmdstanr (version 0.3.0) separately. For both packages I have set the same random seed and chains within Stan (chains=4, seed = 42 when calling the function and other optional arguments are set to be the default value). However, I found that output results are silightly different and may I ask that will this have any potential bad consequences?
Thx!
Hi,
there is no guarantee that rstan and cmdstanr will produce the exact same samples. Exact same here meaning equal on the bit-level.
Reasons:
version of Stan used
rstan uses 2.21, cmdstanr uses 2.26. There were a lot of changes in the Math library between these two versions. The versions of dependencies like Sundials, Boost and TBB also differ.
output precision
rstan outputs directly to R, keeping all the precision. Cmdstanr writes the results to a CSV file and by default keeps 6 significant figures which is more than enough for most cases. For higher precisin in cmdstanr use the sig_figs argument.
differences in compilation and compile flags
rstan compiles the model to an object file with its own flags, cmdstanr compile to executables with its own flags. These can cause slight differences in the numerical results which causes sampler to have different paths.
However, if the results differ completely, meaning that the posterior distribution are not “similar”, then that is a bug. While numerical differences can cause the sampler to sample with different paths, it still sampling from the same posterior distribution meaning that the resulting chains should mix well and all that.
Thanks so much!
The results are slightly different but not exactly same:)