Hi all - first time poster, long time admirer.
I’m trying to compare hierarchical models using the compare_models function in rstanarm. I’ve fit the models using stan_glmer and applied leave-one-out using the loo function of the loo package. All models use the same dataset, same # observations, same response variable, same family of distributions, same transformation applied to the response variable.
When I call compare_models to compare all models, I receive the following error message:
“Error: Not all models have the same y variable.”
By leaving out the most complex model, I was able to run compare_models successfully (no error message). So that’s the model that’s causing the problems.
I checked this page (https://github.com/stan-dev/rstanarm/blob/master/tests/testthat/test_loo.R), but I have not committed one of the example “sins” listed under this error message.
Other (maybe) helpful info:
I did end up having use the “k_threshold=0.7” argument to two of my models. The problematic model (the most complex one) is one of the models that uses the k_threshold argument. The only other difference between the problematic model and the others is that I use a different “random effect” term in it. Here are two of my model specifications - one that does NOT throw an error message (model 1) and the other that DOES throw an error message (model 2):
model 1
stan_glmer(growth ~ diam1 + gap +
dens.c + dens.h +
mycBAc5 + mycBAh5 +
(1 | plot) + (1 | species),
prior = student_t(df=7, location=0, autoscale=TRUE),
prior_intercept = student_t(df=7, location=0, autoscale=TRUE),
data = dat)
model 2 (PROBLEMATIC)
stan_glmer(growth ~ diam1 + gap +
dens.c + dens.h +
BAc5 + BAh5 +
(BAc5+BAh5 | species) + (1 | plot),
prior = student_t(df=7, location=0, autoscale=TRUE),
prior_intercept = student_t(df=7, location=0, autoscale=TRUE),
data = dat)
I have quite complex models that take a while to run, so not sure it will be helpful to include my data for reproducibility (they will take hours to run). My hope is that my error is apparent from my description, but if not, I can try to link to my .RData file … somehow.
Greatly appreciate any guidance. Apologize if I’m not including enough info - I can work with responders to provide needed info.
Operating System: Windows 10
Interface Version: rstanarm 2.15.2
Compiler/Toolkit: