I have a complex model that I’ve mentioned here before. After lots of debugging and adjusting model assumptions and soforth, I’m at the point where it seems to run ok, but the stepsize is about 2e-5 and I give it 15 treedepth and it uses the full 32767 steps each sample, when it tries to adapt stepsize above about 5e-5 it gets divergent transitions. Took it about 30 samples to get into the typical set, and has been sampling successfully since then, I think. (i’m monitoring the csv files). It’s slow as a dog though, those 30 samples took something like 12 hours.
So, a lot of this I think is down to some kind of inherent difficulty in this model, maybe certain parameters are extremely concentrated in the posterior, and there are lots of correlations. For example I have one parameter that’s a national level parameter for each year, and I’m using something like 70,000 data points across 5 years, so this national average is my guess something like 0.25 ± .003 the posterior uncertainty is pretty small. And, the national parameter multiplies every single local estimate in each of the 2000+ public use microdata areas (PUMA), so there are correlations between the national level, and 2000+ dimensions of local level models… etc
Each PUMA has a function that is parameterized by about 6 parameters, so there are something like 12,000 parameters in my model and they are all correlated through spatial and temporal pooling relationships.
Yes, I have already tried the non-centered approach to parameterization, so for example those national averages are implemented using a natav_raw parameter that’s normal(0,1) and then a transformation natav = natav_raw*.05+.22 or something like that to get a normal(.22,.05) so that the mass matrix isn’t too out of whack.
Is this kind of thing with very high dimensional highly correlated parameters just inherently very difficult to sample and I should just be satisfied with letting it run 32k timesteps at each sample, or are there obvious things to try when stepsize crashes down and long integration times are required?