Hi all, is it common to have good parameter recovery (e.g., correlations and biases) but bad sampling diagnostics? I had good parameter recovery but with warnings including low tail ESS and divergent transitions. After I got rid of the warnings, I now have relatively worse parameter recovery. Any ideas why this happens?
You can have good parameter recovery with low tail ESS and divergences if it’s only the tails of the posterior that the sampler struggles with (in general). This means that the bulk of the posterior is properly explored, but the tails have not (hence the ‘Bulk’ and ‘Tail’ ESS distinction).
As for the second part of your question, that’s going to depend on what you did to ‘get rid’ of the warnings, and what you mean by worse recovery
If I have good paramter recovery while having divergent transitions, can I trust the parameter recovery results? After I got rid of the divergent transitions, my paramter recovery was much worse than it when I had divergent transitions.
No, if you have divergent transitions then this means that there are areas of the posterior that the sampler has not explored (or not adequately explored). This means that model estimates can be biased, as they are not fully representing the posterior distribution
thanks for your reply. After I got rid of the divergent transitions, my parameter recovery was much worse than it when I had divergent transitions. Do you know why this happens?
How are you getting rid of the divergent transitions and how are you assessing parameter recovery?
I changed the prior distributions I used for the estimated parameters. the paramteter recovery indices are correlations between true and estimated parameters, bias measuring the difference between true and estimated parameters, and RMSE. the correlations look similar, but bias and RMSE are much worse
I would guess that the priors are changing how the posterior is explored, but it’s hard to give a good answer without knowing more about your workflow. Are you able to post the code you use to generate data (with the ‘true’ parameters) and the two models (old and new priors)?
1 Like