# Filesize error in loo

**URL:** <https://discourse.mc-stan.org/t/filesize-error-in-loo/23324>\
**Category:** Modeling\
**Tags:** loo\
**Created:** [July 5, 2021, 6:57am UTC](https://discourse.mc-stan.org/t/filesize-error-in-loo/23324 "2021-07-05T06:57:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![nitzan\_shahar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/nitzan_shahar/32/12123_2.png) [@nitzan\_shahar](https://discourse.mc-stan.org/u/nitzan_shahar)\
**Post date:** [July 5, 2021, 6:57am UTC](https://discourse.mc-stan.org/t/filesize-error-in-loo/23324/1 "2021-07-05T06:57:06Z")

</div>

Hi All,

I fitted a behavioral data set with 150 subjects and 700 trials each using an hierarchical RL model with stan ([code](https://github.com/shahar-lab/0002_Maayan-Pereg_free_vs_forced_learning/blob/main/modeling/stan_models/stan_alpha_beta_bias.stan?fbclid=IwAR2gJ6WnZSUcd4rFEi4bus-PwylMpGLVWWJIbyMsyjQMTKSobm6aCk88ukg)). This gets me from the transformed parameters block a log\_like matrix (subjects x trials), which I then try to use with loo.

```stan
rl_fit<- stan(file = my_model, 
              data=data_for_stan, 
              iter=2000,                         
              chains=4,
              cores =4, 
              save_warmup=F)

loo_1<-loo(rl_fit,   
           pars = "log_lik",    
           save_psis = FALSE,   
           cores = 4,   
           moment_match = FALSE,  
           k_threshold = 0.7)

```

I then get this:

**Error: cannot allocate vector of size 3.8 Gb**  
**In addition: Warning message:**  
**Some Pareto k diagnostic values are too high. See help(‘pareto-k-diagnostic’) for details.**

Any idea whats going on and how I can fix it?

Many thanks,  
Nitzan

---

<div class="post-metadata">

**Author:** ![erognli](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/erognli/32/5181_2.png) [@erognli](https://discourse.mc-stan.org/u/erognli)\
**Post date:** [July 10, 2021, 5:50am UTC](https://discourse.mc-stan.org/t/filesize-error-in-loo/23324/2 "2021-07-10T05:50:57Z")

</div>

I’m by far not the most solid help you might find on this forum, but as your question has gone unanswered for some time: The error seems just to say that the size of the output from loo exceeds what R can allocate to memory. That is usually not a problem, so it might signal something is not right.

I had a brief look at your code, and I couldn’t quite figure out how the log\_lik matrix coming out of your code is appropriate to use for the loo function. Are you confident that it’s the right quantity? Usually, it’s calculated in the generated quantities block, as the log-probability of the observations conditional on the parameters.

Also, what are you trying to use loo for? Estimating the predictive precision of your model for a new subject, or something else?

---

<div class="post-metadata">

**Author:** ![nitzan\_shahar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/nitzan_shahar/32/12123_2.png) [@nitzan\_shahar](https://discourse.mc-stan.org/u/nitzan_shahar)\
**Post date:** [July 10, 2021, 6:41am UTC](https://discourse.mc-stan.org/t/filesize-error-in-loo/23324/3 "2021-07-10T06:41:42Z")

</div>

Thank you so much for this. I am trying to use loo for model comparison (at this point only in simulations).

So on the more technical side, the loo documentation notes that the first argument can be a “A log-likelihood array, matrix, or function”, so I figured this should be ok… (but obviously I’m doing something wrong).

1. I am estimating log\_lik for N subjects with T trials and I was wondering whether it might be appropriate/helpfull to aggregate over trials (and then have an N vector with the sum of log\_lik). Do you have any idea whether this should make a difference for psis-loo estimation?

2. Yes - I can def calculate log\_lik in the generated quantities block. But will it make a difference in the actual estimates (other then speeding up the code - which is also very useful)?

Much appreciated.  
Nitzan.

---

<div class="post-metadata">

**Author:** ![erognli](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/erognli/32/5181_2.png) [@erognli](https://discourse.mc-stan.org/u/erognli)\
**Post date:** [July 10, 2021, 7:20am UTC](https://discourse.mc-stan.org/t/filesize-error-in-loo/23324/4 "2021-07-10T07:20:16Z")

</div>

I can’t really follow what your model is doing, but as long as you are getting reasonable fits to simulated data and understand it yourself, I guess all is well. And apart from the computational gain, I don’t think there is a specific reason to have it in the generated quantities. Also, a lot of (most?) models aren’t set up with the log-likelihood of the data as a parameter.

Anyway, as it stands now, I think you are doing “leave-one-trial-out”, which estimates the predictive utility of your model for a new trial, given all the other observations for a subject. That may or may not be what you want. Summing the log-likelihoods over trials would give you leave-one-subject out, but there are often issues with loo for hierarchical models at the subject level.

If you haven’t looked at it already, there is a lot of useful information in the [Cross-validation FAQ](https://avehtari.github.io/modelselection/CV-FAQ.html) .

---

<div class="post-metadata">

**Author:** ![nitzan\_shahar](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/nitzan_shahar/32/12123_2.png) [@nitzan\_shahar](https://discourse.mc-stan.org/u/nitzan_shahar)\
**Post date:** [July 11, 2021, 9:52am UTC](https://discourse.mc-stan.org/t/filesize-error-in-loo/23324/5 "2021-07-11T09:52:00Z")

</div>

Thank you for this. I will try to dig deeper to understand whether it make sense to do subject-wise loo which will probably be easier to implement…  
Best,  
Nitzan
