Hi, I am trying to use variational inference to fit a hierarchical bayes model through brms. The main reason for doing this is to increase the speed (especially during early testing of different models). In the brms document, it says that I can change the “algorithm” parameter from “sampling” to “meanfield” to use variational inference. However, this is all the information I can find online about how to use variational inference through brms. I tried a simple example and received a warning message suggesting me to decrease tol_rel_obj. But where can I specify this parameter? More generally, what parameters do I have when using variational inference and how to specify them? Is there a tutorial on how to use variational inference and diagnose the result?
brms::brm() can accept further parameters that are passed to either rstan::sampling() or rstan::vb(). You can find more about tol_rel_obj and other parameters in the rstan::vb() function help.
1 Like
Thank you for the reply! I am using rstan as backend so I assume that when I use variational inference, I am passing the further parameters to rstan::vb(). Here is an example code showing where I put the additional parameter (tol_rel_obj):
test ← brm(
bf(response ~ 0 + Intercept + X + (1 | group),
data = data2fit,
family = zero_inflated_negbinomial(),
algorithm = “meanfield”,
tol_rel_obj = 0.00000001,
iter = 2000, chains = 4, cores = 4
)
Is this correct? I tried this on a toy example. When I decrease tol_rel_obj, the Pareto k diagnostic value is sometimes decreasing and sometimes increasing. Does this mean that the tol_rel_obj is working or not? I thought the Pareto k diagnostic value will keep decreasing when I decrease tol_rel_obj.
@YidaZhang0628 sorry but I cannot help you. I don’t have experience with VI. But I can point you towards the following literature that can help you:
Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2016). Variational Inference: A Review for Statisticians. Journal of the American Statistical Association , 112 (518), 859–877. https://doi.org/10.1080/01621459.2017.1285773
Kucukelbir, A., Ranganath, R., Gelman, A., & Blei, D. M. (2015). Automatic Variational Inference in Stan. ArXiv:1506.03431 [Stat] . http://arxiv.org/abs/1506.03431
Yao, Y., Vehtari, A., Simpson, D., & Gelman, A. (2018). Yes, but Did It Work?: Evaluating Variational Inference. 35th International Conference on Machine Learning, ICML 2018 . http://arxiv.org/abs/1802.02538
Also don’t forget to check the section 18 of Variational Inference in the Stan Reference Manual.
2 Likes
Thank you! I will take a look at these materials!