I need some help to understand how the following things can occur:
the model optimizes just fine, with the parameters looking normal.
if I run the sampler without warmup, the model gets stuck in a single state, never moving.
if I run with warmup, the sampler slows to a crawl.
The model is an experiment where I model a latent matrix F with two independent factor models
F = BW+B_0W_0
where the first factor model uses sparse weights and heavy-tailed factors, while the second factor model is PCA-like with Gaussian factors/weights. The model has some additional confounding features (e.g. ordinal likelihood), but it fits alright if I constrain myself to one of the two factor decompositions.
Check out the manual section on warmup. In optimization the algorithm follows one path up to optimal parameter values. In sampling it has to explore the whole space so apparently there’s a well defined optimum but some part of the space is too complicated w.r.t. to how HMC behaves. When “the sampler slows to a crawl”, it usually means the stepsize is dropping as the sampler tries to avoid divergences and get an acceptable acceptance rate. … so next step is figuring out if the stepsize is the issue and figure out why the model is causing a problem.
I got it to complete a short run of 20 samples with warmup of 10, and it is definitely taking a lot longer than the internal time estimates claim. But issues with the step length does not sound implausible, although I feel like the type of model I described should be fairly forgiving and tend towards too much mass rather than too little.