Continuing the evaluations done in Compilation time evolution in cmdstan I ran some performance tests on our benchmark models and a few other models on Cmdstan 2.18 to 2.23.
For fast models, which finish in under 2 seconds, I ran tests with exaggerated numbers of iterations to reduce the noise and eliminate the effect of IO which was not the point here.
Some results are awesome, for some it might be worth taking a closer look. Versions labeled -sc2 are compiled with Stanc2.
SIR model: https://github.com/stan-dev/stat_comp_benchmarks/blob/master/benchmarks/sir/sir.stan
base.stan model from @bbbales2 https://discourse.mc-stan.org/uploads/short-url/jD15kpGMMMtMySAeKwZEsU7L884.stan
logistic regression (no GLM) https://github.com/bstatcomp/gpu-stan-paper-materials/blob/e8a1dfb30b1e33acca45d3968567460875ae36e5/replicationScripts/endToEnd/bernoulli_glm/Stan/lr_basic.stan
logistic regression with GLM (same input as above)
irt_2pl model https://github.com/stan-dev/stat_comp_benchmarks/tree/master/benchmarks/irt_2pl
arma https://github.com/stan-dev/stat_comp_benchmarks/tree/master/benchmarks/arma
arK https://github.com/stan-dev/stat_comp_benchmarks/tree/master/benchmarks/arK
gp_pois_regr https://github.com/stan-dev/stat_comp_benchmarks/tree/master/benchmarks/gp_pois_regr
