Divergences happen when the simulated Hamiltonian dynamics diverges from the true path. In practice, I see these come up in models in two ways: in hierarchical models as in the Michael’s case study, when parameters take on extreme values (\approx e^{\pm 15}).
You can identify these by looking at pairs plots in shinystan. If you see a funnel shape, it is likely the former, if they tend to accumulate near extreme values, then the latter.
Another plot that people find helpful is the parallel coordinate plot. There is a thread on it on the forum: Concentration of divergences
If you have parameters that are related to each other, such as in hierarchical models, this is a good place to start. Another place is looking at the traceplots of potentially problematic parameters. Look for places where the parameter gets ‘stuck’ for several iterations.
If you are getting the latter, you may be able to avoid it by having tighter priors. If you can’t change the priors, it can be helpful to keep the parameters in log space as much as possible. If you have positive parameters that you later need to take the log of in the model, you can sample on the unconstrained space to avoid doing and then undoing the transform. At this point, I do everything but simplex transformations manually for numeric stability.
If you go this route, make sure to include the Jacobian correction.