# Ubuntu kills brms loo process

**URL:** <https://discourse.mc-stan.org/t/ubuntu-kills-brms-loo-process/20590>\
**Category:** brms\
**Created:** [February 3, 2021, 12:30am UTC](https://discourse.mc-stan.org/t/ubuntu-kills-brms-loo-process/20590 "2021-02-03T00:30:11Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![wgoette](https://avatars.discourse-cdn.com/v4/letter/w/35a633/32.png) [@wgoette](https://discourse.mc-stan.org/u/wgoette)\
**Post date:** [February 3, 2021, 12:30am UTC](https://discourse.mc-stan.org/t/ubuntu-kills-brms-loo-process/20590/1 "2021-02-03T00:30:11Z")

</div>

**Background**  
I am using a virtual machine (e2-standard-16) on the Google Compute Engine to run some brms models. The models are 2PL item response theory models for a test with 30 items administered to 1050 participants. General syntax for the models looks like this (+ my fixed and random effects):

```
brm(bf(Resp ~ exp(logalpha) * eta,
                      eta ~ 1 + (1 |i| Item) + (1 | ID),
                      logalpha ~ 1 + (1 |i| Item),
                      nl = TRUE),
                   data = df, family = bernoulli("logit"),
                   prior = TwoPL_priors, seed = SEED, iter = 3000, warmup = 1000,
                   sample_prior = "yes")

```

, with the following priors:

```
TwoPL_priors <- 
  prior("normal(0, 1)", class = "b", nlpar = "eta") +
  prior("normal(0, 1)", class = "b", nlpar = "logalpha") +
  prior("constant(1)" , class = "sd", group = "ID", nlpar = "eta") +
  prior("normal(0, 2)", class = "sd", group = "Item", nlpar = "eta") +
  prior("normal(0, 1)", class = "sd", group = "Item", nlpar = "logalpha")

```

I’ve run many different models now all without any significant issues (e.g., no divergent transitions, good chain mixing). I obviously want to be able to compare various models, so I have been using the `add_criterion(..., criterion = "loo")` function. In addition to loo comparisons, I was also hoping to use `model_weights(..., weights = "pseudobma")`. Due to some missing data, I have 5 imputed datasets via the `mice` package that are being fit with the `brm_multiple()` function. When I attempt to add the loo criterion (or even the Bayesian R2 for that matter) for these models, my R session is terminated and I am returned to the console with the line “KILLED”. Trying them on my own PC, I get the classic Windows error of “unable to allocate a vector of size 9.4GB”.

**Technical Specs**  
On the cloud, I am running a machine with Ubuntu 20.10 and R 4.0.2. Aside from R, the only thing I have installed on the machine is screen (and then the various dependencies for stan and future). Within R, the only packages I have installed are `rstan`, `brms`, and `future` (plus their dependencies). I assume the issue is the machine running out of memory (16 virtual cores, 64GB memory), though I thought that running Ubuntu in R (and not RStudio) would avoid memory issues.

**Questions**

1. Is there something that I’m missing other than running out of memory that would cause R to be terminated like this? I’m not too familiar with the `future` package or running `plan(multicore)`, so I’m not sure whether maybe there’s a memory leak in there. Similarly, I’m used to doing everything in Windows, so all problems in Linux are new problems to me.

2. If this is just an issue with running out of memory, is it worth it to create a more memory intensive machine, or are there maybe other ways of doing the model comparisons? I’ve used `loo_compare()` and `model_weights()` to compare simpler versions of the models, but I’m open to doing something else – I just want a consistent method used for all of the models.

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [February 3, 2021, 1:36am UTC](https://discourse.mc-stan.org/t/ubuntu-kills-brms-loo-process/20590/2 "2021-02-03T01:36:17Z")

</div>

As you mentioned, this is most likely caused by running out of memory. A couple of things to investigate here.

After running your `brm` model on your local Windows machine, how much RAM is used up by the R/RStudio process and how much RAM does your computer have in total?

Some things to try are to run `add_criterion(..., criterion = "loo",pointwise=TRUE)`, as it uses less memory. You can also try making sure that the `loo` calculations are only using a single core, as using multi-core processing can require more memory for `loo`. You can do this by specifying `options(mc.cores=1)` prior to the `add_criterion` call. I’m not sure how the `future` package interacts with `add_criterion`, so if you’ve also setup a `multicore` session then also run `plan(sequential)` prior to the call

---

<div class="post-metadata">

**Author:** ![wgoette](https://avatars.discourse-cdn.com/v4/letter/w/35a633/32.png) [@wgoette](https://discourse.mc-stan.org/u/wgoette)\
**Post date:** [February 3, 2021, 2:54am UTC](https://discourse.mc-stan.org/t/ubuntu-kills-brms-loo-process/20590/3 "2021-02-03T02:54:46Z")

</div>

I’m giving `add_criterion(..., criterion = "loo", pointwise = TRUE)` a shot right now. Looks like, per Task Manager, RStudio is taking up about 2500 MB at the moment. On my PC, I’ve got 32 GB of RAM (31.7 useable).

In the past, I’ve had to extend the memory available to R with `memory.limit(size = ...)` because Windows sets a hard limit by default, and I find that usually RStudio still kills the process well before my actual RAM is used up (even if I set the memory limit to something like 25 GB). My hope in setting up a Linux virtual machine running R and not RStudio was to get around some of these memory usage limits, but maybe that’s not the case.

---

<div class="post-metadata">

**Author:** ![wgoette](https://avatars.discourse-cdn.com/v4/letter/w/35a633/32.png) [@wgoette](https://discourse.mc-stan.org/u/wgoette)\
**Post date:** [February 5, 2021, 3:02am UTC](https://discourse.mc-stan.org/t/ubuntu-kills-brms-loo-process/20590/4 "2021-02-05T03:02:02Z")

</div>

The addition of `pointwise = TRUE` to the `add_criterion()` function does let me get a loo estimate for the large models. It takes a good length of time, but it works.

Any recommendations/workarounds for getting `model_weights()` to work with `weights = "pseudobma"`? I can get weights with `weights = "loo"`, but my preference is with pseudo-Bayesian Model Averaging. I’ll give it another shot on a high memory virtual machine without calling `future` to see if that fixes the Ubuntu console kill, but just from a practical sense, wondering whether there’s any memory saving tricks like `pointwise = TRUE` but for `model_weights()`.

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [February 5, 2021, 3:15am UTC](https://discourse.mc-stan.org/t/ubuntu-kills-brms-loo-process/20590/5 "2021-02-05T03:15:27Z")

</div>

I’m not familiar with pseudo-bma weights, so I can’t speak to that unfortunately.

However if the `pointwise` loo is working, then it is likely a memory issue. I’d try using the regular `loo` & `pseudobma` calls, but ensuring no parallelism (i.e.,`options(mc.cores=1)`, `plan(sequential)`) to see how it compares

---

<div class="post-metadata">

**Author:** ![t-pollington](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/t-pollington/32/14497_2.png) [@t-pollington](https://discourse.mc-stan.org/u/t-pollington)\
**Post date:** [July 27, 2022, 6:33am UTC](https://discourse.mc-stan.org/t/ubuntu-kills-brms-loo-process/20590/6 "2022-07-27T06:33:31Z")

</div>

I actually found that on a cluster its easier to max out the cores to get the job done faster. And since the clusters usually have a fixed amount of memory per slot, this also maxes the RAM available (for me 24\*15G=815G). Thanks for the speedy command though @andrjohns .

```nohighlight
options(mc.cores = 24)
brms::add_criterion(brmsfitobject, criterion = "loo", pointwise=TRUE)

```