Hi everyone!
I have a simple question, what does “Min. n_eff” means in the pareto_k_table please?
For example:
Thank you!
-S
It’s the minimum effective sample size achieved with Pareto smoothed importance sampling (PSIS). The minimum is over parameters, each of which has its own effective sample size (written “ESS” or “n_eff” in different places), because in general, every expectation leads to a different ESS.
See the doc: Diagnostics for Pareto smoothed importance sampling (PSIS) — pareto-k-diagnostic • loo
and the paper on which it’s based: [1507.02646v6] Pareto Smoothed Importance Sampling
2 Likes
What @Bob_Carpenter said is correct except I think he meant to say that the minimum is over observations not parameters. We do PSIS on the draws of the log likelihood and get an effective sample size from PSIS. This is done separately for each observation, so there is one effective sample size value per observation. On the documentation page Bob linked to there’s also a function for getting all the effective sample sizes. In the table we report the worst (lowest) one for each category in the table.
2 Likes
Bob and Jonah had good answers
It seems you’re using the old version of loo. The new version has dropped the range (0.5, 0.7] and uses label ESS instead of n_eff.
See also LOO package glossary — loo-glossary • loo
The link to the journal version of the paper is Pareto Smoothed Importance Sampling
EDIT: fixed a typo (was typing on my phone)
1 Like
That’s a generous interpretation of my intent! I just didn’t think it through well enough.