Hello Stan users,
I’m using Windows 10, rstan 2.18.2 to run a model with 680 parameters and I would like to calculate the multivariate potential scale reduction factor (MPSRF) for the stanfit object. I checked the rstan user’s guide and it appears rstan does not have a built in function to calculate MPSPR. Is there anyway to obtain a reliable MPSRF estimate?
Thank you very much.
You would have to call As.mcmc.list() on your stanfit object and pass it to coda.
Thank you for your instant response.
I tried this function to obtain MPSRF with coda and the result baffled me. I ran the same model in runjags and it took only 30000 iterations (half for burn-in/warm-up) to obtain both PSRF and MPSRF < 1.1. rstan took about the same amount of iterations to have PSRF < 1.1, but when the converted stanfit object is passed to coda, MPSRF is pretty large at 1.55. Again I experimented with chain length in rstan, and when it reached 80,000 iterations (half for warm-up) MPSRF went below 1.1. What might be possible reasons to explain this difference? What might I have overlooked?
My guess is that JAGS didn’t mix throughout the typical set, and likely Stan didn’t either. If it takes Stan more than the default number of warmup iterations, there is almost certainly something wrong with your model, or it is not consistent with the data, or it is not feasible to draw from the posterior distribution efficiently. Did you get some divergent transition warnings?
2 Likes
Yes, rstan gave warnings about divergent transitions, even when I tried to run the model with up to 100,000 iterations.
I should have added some contextual information. I’m running a simulation, and both runjags and rstan produced accurate results with only 30,000 iterations when posterior distribution means were compared to the generated parameters. runjags did not give any warnings with 30,000 iterations and the run terminated normally with both PSRF and MPSRF < 1.1. rstan gave warnings about divergent transitions despite being slightly more accurate than runjags in my simulation.
With Stan, if there are any divergent transitions, you can’t trust any of the results. You also can’t really trust the results from other MCMC algorithms because Stan is telling you there is a part of the typical set that is difficult to get into and out of with the right posterior probability.
To avoid divergent transitions, the first thing to try is increasing the adapt_delta parameter. However, that might not suffiice and the real fix involves changing your model. There is more information about those warnings at
https://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
But you should not be departing from the default number of warmup iterations. If there are going to be problems, they are usually apparent with the defaults.
3 Likes
Thank you Dr. Goodrich for your suggestion. In my model, one parameter type is difficult to estimate. Marginal maximum likelihood tends to bring about parameter drift and estimate with large standard error. I changed the prior for this parameter from uninformative to informative and rstan is doing fine now. The divergent transition warnings disappear (in fact there are no longer any warnings), and MPSRF goes down to 1.003 with only 5000 samples after the default warm-up.
However, some PSRF values are slightly less than 1, like 0.99999. I notice that some PSRF values were below 1 even before I made the above revision to the model. Is it a worrying sign about the sampling process?
The estimated PSRF can be less than 1, in which case you should consider it to be 1.
1 Like
Thank you Dr. Goodrich for your quick response.