Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable

Hi,I recently encountered the error below related to Tail ESS and I adjusted my total number of iterations from 4000 to 5000 and used default warm-up, still couldn’t get rid of the warning. Any suggestions on what might cause this warning?

Thanks!

hash mismatch so recompiling; make sure Stan code ends with a blank line
Warning messages:
1: In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  '-E' not found
2: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#tail-ess 

The sampler is struggling to properly explore and sample from the tails of the posterior distribution of some (or all) parameters. There could be all sorts of reasons for that, but as it says you shouldn’t necessarily trust your estimates of parameter uncertainty. If you are planning to do inference based on intervals of the posterior distribution, this is a warning you may need to get rid of.

Perhaps you could tell a bit more about the model you are fitting, and what data you are using? Which parameters are you getting low tail ESS for?

2 Likes

Thanks for your reply! I just took a look at the tail ESS and only one out of 5000 parameters has relatively low tail ESS (281), other parameters have at least 1090 tail ESS, the largest is 7000. I ran 5000 iterations with 3 chains. Does the one 281 have a large impact on my estimates? Thanks!

You can check that by computing Monte Carlo standard error estimates for your estimates. rstan and posterior packages have functions to compute MCSE for mean, sd, median, and any quantiles. You can then compare MCSEs to your domain expertise on what is sufficient accuracy. 281 might be enough for your purposes, but the warning is given because it’s low enough value that there might be convergence or mixing problems and you need to proceed with more care.

Thanks!
Comparing to other parameter estimated mean se (0.002-0.06), the se for the parameter that has 281 tail ESS is relatively large (~0.05).

By se you mean MCSE? And Tail ESS specifically warns that either 5% or 95% percent quantile has low ESS, and thus you may expect that tail probabilities, tail quantiles, and thus end points of intervals have high MCSE.

Oh, I just read the se_mean column in the summary output, I guess I need to use the function in the posterior package to calculate MCSE.

1 Like