As usual I seem to be working with SLOW models, and this really makes debugging hard. ;-) I’ve been having trouble with some chains sticking in place at the initial conditions and never moving at all… where other chains seem to converge, etc. Part of it is probably poorly parameterized components to the model, but other issues are that it’s just a big model for a spatially variable dataset with lots of observations… lots of parameters etc.
Currently I’m doing an optimize step, then running the vb algorithm to get a sample, and then picking one of those samples as the initial location for a sampling statement. This at least seems to stop me from getting fully stuck.
It occurs to me though that the adaptation period for sampling could potentially be speeded up by using the variational approximation to inform the mass matrix. Is this something that has been tried by the stan team?
Right now, my model takes about 30 seconds to optimize (and I can use that point estimate to debug obvious problems, so that’s good), about 2 or 3 minutes to get 1000 samples from the vb algorithm (this also helps with finding obvious problems quickly), and then maybe a couple hours to give me 200 iterations of full Bayes, I have it do 150 warmups, and then pass the last 50 samples to me on 4 chains. This is annoyingly slow, and it’d be super cool to be able to get an initial mass matrix from VB and then run say 50 more warmups, and 50 real samples on 4 chains and have the whole thing take… 20 mins
Has anyone looked into the idea of using the VB approximation to short-cut some of the time it takes to estimate the warmup stuff?