Can I compare two distance decay models with ELPD?

Modeling
1

Hi,

I’m fitted two hierarchical distance decay models, one with a linear decay and another with an exponential decay.

Linear decay

similarity ~ normal(mu, sigma)
mu[i] =a[basin_id[i]] + b[basin_id[i]] * distance[i]

Exponential decay

similarity ~ normal(mu, sigma)
mu[i] =a[basin_id[i]]) * exp(b[basin_id[i]] * distance[i]

Posterior predictive checks seem to be ok for both models (please correct me If I’m wrong).

Linear model

linear_ppd
linear_ppd1269×711 51.3 KB

Exponential model

exp_ppd
exp_ppd1275×719 51 KB

I would like to formally compare both models and determine which one fits the data best. I read:

http://mc-stan.org/loo/articles/loo2-with-rstan.html

and was thinking if I can/should compare both models with ELPD as shown in the above vignette.

Thanks

Filipe

2

Yes, you can use ELPD. But whatever function you have underneath does not look so good for the linear model.

3

Thanks. The linear model is just a standard linear regression. I take it the skewness plot indicates the linear model does not fit the data well and that I should select the exponential model (i.e. no need for ELPD comparisons)?

4

Do the ELPD comparison.

5

I did and it seems the linear model is “better”. In this case, which model should I choose?
lin_mod → linear model
exp_mod-> exponential model

library(loo)
loglik_linear<-extract_log_lik(lin_mod@stanfit, merge_chains = FALSE)
r_eff_lin ← relative_eff(exp(loglik_linear))
loo_lin ← loo(loglik_linear, r_eff = r_eff_lin)
loglik_exp<-extract_log_lik(exp_mod@stanfit, merge_chains = FALSE)
r_eff_exp ← relative_eff(exp(loglik_exp))
loo_exp ← loo(loglik_exp, r_eff = r_eff_exp)
model_comp ← loo_compare(loo_lin,loo_exp)
print(model_comp)

elpd_diff se_diff
model1 0.0 0.0
model2 -24.7 7.3

6

linear

7

Thanks for the help