I’m writing to notify the Stan developers and broader community about a new NUTS variant that may be of interest. This is a follow up to @Bob_Carpenter 's recent blog post with a reprex and some more details. The basic idea is to precondition (i.e., decorrelate and descale) the posterior using spars…

That looks really cool! Making use of more structural information about the model (like conditional independence) in the sampler sounds like a great way to improve performance long term. I usually work with pymc instead of stan, where we have a lot of that information around in some way, but I don’…

@aseyboldt Thanks for the feedback. That’s a good idea to try it on nutpie. I haven’t used that actually but Bob keeps raving about it so I should have thought to do it. Could you please file an issue about the parallel failure here with details about your platform? The efficiency for nutpie is no…

[image] monnahc: I do find it cleaner to just write out the model in standard notation and have the correlations handled by the preconditioner. @monnahc and I have discussed this, but I’ll share here. I agree that we’d ideally just have a way to write down any proper posterior density and f…

@monnahc Good point about the logp, I’ll run that tomorrow. @Bob_Carpenter The post I linked above had an example of how to use the ZeroSumNormal strictly as a reparametrization. If we just write the effect as a normal distribution effect ~ normal(0, sigma), we measure the effect size relative to …

@aseyboldt If you send me an updated script with logp tracked I can run on my end and update the benchmark. Note that SNUTS can deal with correlations/scales caused by other things too. For instance the ‘kilpisjarvi’ model in posteriordb has three parameters (and is not hierarchical) but has a max …

Sorry, I was traveling for a few days. Here is the updated plot that includes the lp ess (script attached) [snuts_vs_stan_vs_nutpie_glmm] I’d also be interested to see what this looks like for more models. I would expect nutpie to do decently well in low dim problems, but as the dimensionality i…

Thanks for doing it, it’s much clearer now. It motivates me to keep working on minimizing the overhead of SNUTS. I also thought about adding compile time, which doesn’t exist for RTMB. In low dimensions compiling often takes longer than sampling so in that sense SNUTS would do very well too. Do you…

Not sure if this of any help, but any models in particular you are looking for? I think if this is particularly applicable to hierarchical models, it might be easy to just generate a bunch of stan code for hierarchical (multilevel models) using brms to test, to see what happens. This might be fast. …

SNUTS can’t simply run all models in posteriordb, because it needs extra information that’s not directly visible from a stan model, so it needs some recoding. But apart from that concrete question, I think it would be incredibly valuable to add more models to posteriordb. It has a lot of small mode…

Probably easy to add, lots of models, I think no one has uploaded any models. I’m not funded to do this, but I’m working on C++, this is more beneficial for my skills and career. Contributed to posteriordb is just annoying, and would require me re-running data simulations so that the models could ru…

Sparse NUTS: preconditioning with sparse matrix operations

General

Topic		Replies	Views
Comparing Stan's adaptation phase to that of nuts-rs? Algorithms	20	1920	August 11, 2023
Inefficient sampling and divergent transitions in network meta-analysis (ported from WinBUGS) Modeling fitting-issues , meta-analysis	12	1284	July 24, 2020
Case study on spatial models for areal data - Poisson CAR/IAR Modeling	116	11588	July 12, 2018
How to speed up sampling in rstan? Modeling rstan , performance , hierarchical-model	6	6247	July 24, 2020
Efficient LogL Computation and / or changing NUTS controls Modeling techniques , specification	10	620	October 7, 2022

Sparse NUTS: preconditioning with sparse matrix operations

Related topics