ArviZ loo_i

Interfaces
,
8

Amen ! Thank you so much for taking the time to go through it all with me.

for independent observations, both elpd and elppd are equivalent.

I think they’re equivalent (up to multiplying by n) for identically distributed observations, whether independent or not ?

Point 2 includes both the Monte Carlo and the PSIS/Importance sampling error.

p(y_i | y_{-i}) = \int p(y_i | \theta) p(\theta | y_{-i}) d\theta = \int p(y_i | \theta) \underbrace{\frac{p(\theta | y_{-i})}{p(\theta|y)}}_{r_i(\theta)} p(\theta|y) d\theta

We can imagine 3 possible Monte Carlo estimators based on these integrals:

a. Monte Carlo estimator with samples from p(\theta | y_{-i}) is \frac{1}{S} \sum_{s=1}^S p(y_i|\theta^s)

b. Monte Carlo (raw) Importance Sampling estimator with samples from p(\theta | y) is \frac{\sum_{s=1}^S r_i(\theta^s) p(y_i|\theta^s)}{\sum_{s=1}^S r_i(\theta^s)}

c. Monte Carlo Pareto Smoothed Importance Sampling (PSIS) estimator with samples from p(\theta | y) is \frac{\sum_{s=1}^S w_i(\theta^s) p(y_i|\theta^s)}{\sum_{s=1}^S w_i(\theta^s)}

Is your point that the Importance Sampling estimators (b and c) have higher Monte Carlo error than estimator a ? Though I think that estimator c (PSIS) should have less error than estimator b (raw IS).

What is IWMM ?

Thanks again !!

2 Likes