I am using a virtual machine (e2-standard-16) on the Google Compute Engine to run some brms models. The models are 2PL item response theory models for a test with 30 items administered to 1050 participants. General syntax for the models looks like this (+ my fixed and random effects):
brm(bf(Resp ~ exp(logalpha) * eta, eta ~ 1 + (1 |i| Item) + (1 | ID), logalpha ~ 1 + (1 |i| Item), nl = TRUE), data = df, family = bernoulli("logit"), prior = TwoPL_priors, seed = SEED, iter = 3000, warmup = 1000, sample_prior = "yes")
, with the following priors:
TwoPL_priors <- prior("normal(0, 1)", class = "b", nlpar = "eta") + prior("normal(0, 1)", class = "b", nlpar = "logalpha") + prior("constant(1)" , class = "sd", group = "ID", nlpar = "eta") + prior("normal(0, 2)", class = "sd", group = "Item", nlpar = "eta") + prior("normal(0, 1)", class = "sd", group = "Item", nlpar = "logalpha")
I’ve run many different models now all without any significant issues (e.g., no divergent transitions, good chain mixing). I obviously want to be able to compare various models, so I have been using the
add_criterion(..., criterion = "loo") function. In addition to loo comparisons, I was also hoping to use
model_weights(..., weights = "pseudobma"). Due to some missing data, I have 5 imputed datasets via the
mice package that are being fit with the
brm_multiple() function. When I attempt to add the loo criterion (or even the Bayesian R2 for that matter) for these models, my R session is terminated and I am returned to the console with the line “KILLED”. Trying them on my own PC, I get the classic Windows error of “unable to allocate a vector of size 9.4GB”.
On the cloud, I am running a machine with Ubuntu 20.10 and R 4.0.2. Aside from R, the only thing I have installed on the machine is screen (and then the various dependencies for stan and future). Within R, the only packages I have installed are
future (plus their dependencies). I assume the issue is the machine running out of memory (16 virtual cores, 64GB memory), though I thought that running Ubuntu in R (and not RStudio) would avoid memory issues.
Is there something that I’m missing other than running out of memory that would cause R to be terminated like this? I’m not too familiar with the
futurepackage or running
plan(multicore), so I’m not sure whether maybe there’s a memory leak in there. Similarly, I’m used to doing everything in Windows, so all problems in Linux are new problems to me.
If this is just an issue with running out of memory, is it worth it to create a more memory intensive machine, or are there maybe other ways of doing the model comparisons? I’ve used
model_weights()to compare simpler versions of the models, but I’m open to doing something else – I just want a consistent method used for all of the models.