Hi all -
As part of the paper I am writing for my idealstan package (GitHub - saudiwin/idealstan: idealstan offers item-response theory (IRT) ideal-point estimation for binary, ordinal, counts and continuous responses with time-varying and missing-data inference. Latent space model also included. Full and approximate Bayesian sampling with 'Stan' (www.mc-stan.org).), I did a simulation study of idealstan compared to traditional models and between HMC and Pathfinder/Laplace. I thought you all might find this interesting as it’s a Monte Carlo study of fairly complex models with these new approximations in Stan. The tl; dr is that the approximations perform reasonably well, but only approach HMC on simpler models like splines and struggle with more complex models like GPs and AR(1). They also perform noticeably worse when incorporating a 2-stage hurdle adjustment for missing data that induces a lot of posterior correlation.
One interesting thing that emerged as well is that Laplace had much better coverage than Pathfinder. Still lower than HMC, but noticeably higher across specifications. (see plot in the SI): idalstan_compare/kubinec_SI_anon_simversion.pdf at master · saudiwin/idalstan_compare · GitHub
Link to the paper here: OSF