@Bob_Carpenter @betanalpha @andrewgelman @avehtari @paul.buerkner (kindly tag others for whom this may be relevant) I came across this paper that compares JAGS, NIMBLE, and Stan using a detailed and consistent framework across 4 different classes of models (Linear, Logistic, Mixture and Accelerated…

@andrjohns It would be nice to change these specs to the more efficient Mixture model setup and re-test the model results and subsequent comparisons.

Hi, yes, I saw that. I don’t think iterations per second is the right thing to use. It might not make much of a difference in this case–I have no idea–but I know that users coming from Bugs/Jags often get into the habit of running 100,000 iterations, and when they come to Stan we have to train the…

Here is a link to one of the authors page with greater in depth details on how they compare across different software / tools / algorithms: MCMC_comparisons They acknowledge that their code for Stan might not be the most efficient as follows: To put that another way, the other package developers …

[image] sree_d: Since HMC relies on the gradient of the log posterior density, considering non-informative priors such as the uniform distribution, for which the gradient is computationally cheap to compute, provides a speed-up for the sampling time required by Stan. This goes counter to ever…

[image] andrjohns: Unfortunately, it looks like their models are using the less efficient constructions for the various likelihoods (e.g., not using the _glm distributions, and using log_sum_exp instead of log_mix for the mixture distributions) The log_mix documentation still states that it o…

I suppose it marginally decreases the cost of each gradient evaluation, but frequently at the expense of more evaluations per iteration and more iterations per effective sample. I imagine that in virtually all important classes of model, the cost of evaluating the gradient of the prior pales in com…

Can anyone ascertain how they chose the number of warmup iterations in Stan, and how they chose the number of adaptation and burn-in iterations in JAGS?

Thanks for sharing this. I notice a couple things about their metrics that are interesting. Their measures are (as @sree_d noted): Average effective sample size for the regression coefficients (\mathcal E_\beta), compilation time in seconds (tc) and iterations per second (N_it/t_s) Their raw me…

I have to respectfully disagree with @sree_d : if anything this paper demonstrates many common mistakes that are made when trying to compare Markov chain Monte Carlo methods. Every probabilisitic computation method approximates expectation values with respect to a given target distribution; in the B…

@betanalpha Thank you so much for your detailed note. Your note and some of the observations others have posted above is precisely why I posted the link to this paper. It is beyond my pay grade to critically examine such articles, as I consider myself a novice in this space. So I was hoping the more…

Excellent paper on Comparing JAGS, NIMBLE, and Stan

Publicity

Richard_Erickson May 3, 2024, 3:46pm 24

Thank you. I forgot to add “this link doesn’t work”.

I’ve edited my original post.

JAGS vs Stan performance - how to compare 'workflow efficiency'?

Topic		Replies	Views
"Selling" Stan General	14	5607	April 7, 2018
New preprint: Investigating the efficiency of marginalising over discrete parameters in Bayesian computations Publicity rstan	9	1493	September 21, 2022
Blog: A gentle Stan to INLA comparison Publicity	18	6795	June 5, 2018
How to compare different algorithms for Bayesian inference in terms of speed (and subject to effectiveness) General	16	4231	February 17, 2018
JAGS vs Stan performance - how to compare 'workflow efficiency'? General techniques	7	247	August 6, 2026

Excellent paper on Comparing JAGS, NIMBLE, and Stan

Related topics