fsdias
January 14, 2020, 6:36pm
1
Hi,
I’m fitted two hierarchical distance decay models, one with a linear decay and another with an exponential decay.
Linear decay
similarity ~ normal(mu, sigma)
mu[i] =a[basin_id[i]] + b[basin_id[i]] * distance[i]
Exponential decay
similarity ~ normal(mu, sigma)
mu[i] =a[basin_id[i]]) * exp(b[basin_id[i]] * distance[i]
Posterior predictive checks seem to be ok for both models (please correct me If I’m wrong).
Linear model
Exponential model
I would like to formally compare both models and determine which one fits the data best. I read:
http://mc-stan.org/loo/articles/loo2-with-rstan.html
and was thinking if I can/should compare both models with ELPD as shown in the above vignette.
Thanks
Filipe
1 Like
Yes, you can use ELPD. But whatever function you have underneath does not look so good for the linear model.
1 Like
fsdias
January 15, 2020, 9:22am
3
Thanks. The linear model is just a standard linear regression. I take it the skewness plot indicates the linear model does not fit the data well and that I should select the exponential model (i.e. no need for ELPD comparisons)?
fsdias
January 15, 2020, 5:09pm
5
I did and it seems the linear model is “better”. In this case, which model should I choose?
lin_mod → linear model
exp_mod-> exponential model
library(loo)
loglik_linear<-extract_log_lik(lin_mod@stanfit, merge_chains = FALSE)
r_eff_lin ← relative_eff(exp(loglik_linear))
loo_lin ← loo(loglik_linear, r_eff = r_eff_lin)
loglik_exp<-extract_log_lik(exp_mod@stanfit, merge_chains = FALSE)
r_eff_exp ← relative_eff(exp(loglik_exp))
loo_exp ← loo(loglik_exp, r_eff = r_eff_exp)
model_comp ← loo_compare(loo_lin,loo_exp)
print(model_comp)
elpd_diff se_diff
model1 0.0 0.0
model2 -24.7 7.3