But at the expense of increased computation. Users pushing adapt_delta = 0.999, or interfaces defaulting to adapt_delta = 0.99 just tries to hide degeneracy problems behind increased computation that at best makes it hard for users to identify the underlying problems and at worst burns a load of carbon only to turn more divergences.
Having a proper posterior density doesn’t imply that we should be able to fit the model accurately, let alone efficiently. The infamous inv_gamma(epsilon, epsilon) priors from the early BUGS era are proper but cause no end of problems in practice. Do you expect Stan to fit models with those proper priors?
Divergences aren’t just indicating that Stan can’t accurate quantify a given posterior density function. The pathologies that mess up Hamiltonian Monte Carlo and induce divergences are almost always (I have yet to see an exception) due to degeneracies in the posterior density function which indicate that the chosen parameters are not independently well-informed by the data or the prior model. The model is not wrong in any philosophical sense but rather in the sense that it conflicts with the tacit assumption that the parameters were sufficiently well informed to be able to learn anything useful about them.
In particular, you can burn more compute to push Hamiltonian Monte Carlo to explore the degenerate posterior more accurately but will it be worthwhile if the final inferences aren’t all that useful because of the limited information?
Reparameterizations, more careful priors, and the like are all ways of adding information or concentration available information in more effective ways. We know to consider these approaches, however, only when we follow up on the divergences. Conveniently I just published a case study on this topic, Identity Crisis.
For example in hierarchical models do you have unbalanced data? Have you centered and non-centered each individual parameter separately based on the sparsity of the data (as a proxy for the width of the likelihood function)? Do you have enough individuals, or enough individuals with enough data, to learn the population scale well enough to break the population location/population scale degeneracy?