Hi, I have been using EP-stan in its simulation mode. I start reading Gelman’s EP paper which mentions the variational approach as one of the alternative methods. I know ADVI is part of stan, but not parallelized. I am wondering whether there is any parallel implementation of the variational approach in the works along the line of EP-stan.
No. We need to get ADVI first more reliable and we are working on that.
As I understand advi is not a parallel computing implementation right now. Will it be once you complete it’s improvement?
We will make further decisions after we have something we can recommend to be used (ie it needs to have reliable diagnostic (solved) and it needs to work reliably in a regime we can describe sufficiently). EP-life style parallel ADVI is not on the top of todo list. ADVI will benefit from MPI and GPU parallelizations which are in progress.
Nothing’s ever complete! But no, we’re not planning on parallelizing it until we can understand the serial version. I think PyMC3 and Edward both have stochastic variational inference implemented.