ADVI with Diffusion model

Modeling
1

Hi,

I’m trying to apply ADVI(vb) for Diffusion model.
But I hardly succeed to get estimates.
If you have any advices or solutions, please tell me.

R code and stan code is taken from this thread.
rcode_diffusion.R (655 Bytes)
model_diff.stan (379 Bytes)

Sometimes I get the error as below;
Error in evalq(sys.calls(), ) :
stan::variational::advi::adapt_eta: Cannot compute ELBO using the initial variational distribution. Your model may be either severely ill-conditioned or misspecified.

and the other I get the error as below;
Error in evalq(sys.calls(), ) :
stan::variational::advi::calc_ELBO: The number of dropped evaluations has reached its maximum amount (100). Your model may be either severely ill-conditioned or misspecified.

Thank you.

2

I think the current status of ADVI is that it either works or it doesn’t. We’re looking into ways to robustify it and/or flag cases where it returns an answer but that answer is probably not very good, but we don’t have any answers yet.

So the best advice I have is that if ADVI doesn’t work, don’t use ADVI. See if you can do it in the NUTS version of Stan.

(Generally, Variational Inference is a tool that either works or doesn’t with minimal theoretical guidance as to which is going to occur. So I don’t want to flag ADVI as unusually bad. Variational inference is just hard to do generically.)

3

Thanks Daniel,

Of course I know I’d better not to use ADVI as long as it doesn’t work stably.
But I want to know when can I use ADVI and when(under what conditions) cannot.
I know wiener distribution is complicated, so I suppose it’s difficult to use ADVI, but actually, sometimes I’m succeeding to estimate.
If anyone know the condition that cannot apply Variational Bayes, I’d like to know. Or, is it still not clear?

4

I don’t think anyone knows the answer to that yet. Sorry.