Model requires small stepsize and large treedepth, what steps to do?

Modeling
1

I have a complex model that I’ve mentioned here before. After lots of debugging and adjusting model assumptions and soforth, I’m at the point where it seems to run ok, but the stepsize is about 2e-5 and I give it 15 treedepth and it uses the full 32767 steps each sample, when it tries to adapt stepsize above about 5e-5 it gets divergent transitions. Took it about 30 samples to get into the typical set, and has been sampling successfully since then, I think. (i’m monitoring the csv files). It’s slow as a dog though, those 30 samples took something like 12 hours.

So, a lot of this I think is down to some kind of inherent difficulty in this model, maybe certain parameters are extremely concentrated in the posterior, and there are lots of correlations. For example I have one parameter that’s a national level parameter for each year, and I’m using something like 70,000 data points across 5 years, so this national average is my guess something like 0.25 ± .003 the posterior uncertainty is pretty small. And, the national parameter multiplies every single local estimate in each of the 2000+ public use microdata areas (PUMA), so there are correlations between the national level, and 2000+ dimensions of local level models… etc

Each PUMA has a function that is parameterized by about 6 parameters, so there are something like 12,000 parameters in my model and they are all correlated through spatial and temporal pooling relationships.

Yes, I have already tried the non-centered approach to parameterization, so for example those national averages are implemented using a natav_raw parameter that’s normal(0,1) and then a transformation natav = natav_raw*.05+.22 or something like that to get a normal(.22,.05) so that the mass matrix isn’t too out of whack.

Is this kind of thing with very high dimensional highly correlated parameters just inherently very difficult to sample and I should just be satisfied with letting it run 32k timesteps at each sample, or are there obvious things to try when stepsize crashes down and long integration times are required?

2

What you are describing are good first steps. I would use the few samples you can squeeze out at this stage to translate/rescale everything so its about N(0,1) and do some visualization of the parameters you expect correlations for. Restart the model from there and the warmup period should be much less painful.

Yesbut: You can almost always reparameterize this stuff. Which reparameterization to use is another story…

1 Like