In cmdstanr, I’m running ADVI,
fit_vi <- mod$variational(...)
and often, the algorithm does not converge and I get the following message:
Informational Message: The maximum number of iterations is reached! The algorithm may not have converged.
For my experiments, I run ADVI with multiple seeds and then I look at the results from different attempts at a fit. Is there some summary in the fit_vi that I can extract which warns me that the median ELBO did not converge?
Didn’t @Jonah give you an answer during the Stan meeting? If so, would you mind repeating it here?
(I think it was that you get a non-zero return code if it terminates before convergence.)
I tried, but it didn’t work. Whether convergence works or fails, the return is 0.
Yeah I think we discovered live in the meeting that return codes weren’t implemented properly (or at all?) for advi. If optimize fails you get a non zero return code but not for advi. This would have to be fixed outside of CmdStanR.
Without proper return codes, you could try to parse the console output (which you can obtain even after fitting from
fit$output()).
Another thing to check is whether there’s anything useful in the “latent dymamics”
files. You would set save_latent_dynamics=TRUE and that would write some extra info to a csv file whose path is given by fit$latent_dynamics_files(). I’m more familiar with what’s in those files for MCMC, but I think advi also writes something there.