I’m using loo to compare categorical models fit using brms with cmdstan on remote linux instances.
Today I ran into a problem where the loo objects of models fit to the same dataset using different versions of brms and cmdstanr cannot be compared without the warning:
Not all models have the same y variable. (‘yhash’ attributes do not match)
And indeed, when I call attributes(loo_object)$yhash
, the strings do differ, like so:
attributes(Mod04_Person3pl_StemX3pl_loo)$yhash
[1] “cbdddd13d96776ed82ff6339b88005b4fc406aa8”
attributes(Mod04_Person3pl_StemX3pl_Corr_loo)$yhash
[1] “29c95c57a1eda060bc20b2281b44333b6edeeddc”
But the datasets are the same. The two models above differ only in whether there’s a correlation parameter between varying slopes and intercepts.
The first model was fit using brms 2.17, cmdstan 2.30.1, and cmdstanr 0.2.0.
The second model was fit using brms 2.18.7, cmdstan 2.30.1, and cmdstanr 0.5.3.
Is it safe to ignore the warning? For what it’s worth, the reported elpd_loo statistic do look plausible.