Projpred error on model with lots of observations

Interfaces
,
1

I am trying to run either varsel or cvarsel from projpred but I am getting Error: vector memory exhausted (limit reached?)

This is a Gaussian family model with 39 predictors (no group-level effects) fitted to 303897 observations. Many of the predictors are multicollinear. In case relevant, the code is below:

mdl_tapped_pulse_tap_vel_full_rstan <-
    stan_glm(
      tap_vel ~
        cue * 
        (tap_l_1 + 
           perf_num_sc + repetition_sc + 
           N_sc + K_sc + mean_IOI_sc + evenness_sc + 
           all_ent_sc + step_ent_sc + SQ_sc + CQ_sc + balance_sc + 
           duple_triple +
           reg_exp_sc + proj_cent_sc + height_sc + 
           Markov3_sc + APM_sc + edge_sc),
      data = WF_rhythm_data_tapped,
      family = gaussian(),
      prior = student_t(3, 0, 1), 
      prior_intercept = student_t(3, 0, 1),
      QR = TRUE
    )

tapped_pulse_tap_vs <-
  varsel(
    mdl_tapped_pulse_tap_vel_full_rstan,
    ndraws = 2,
    ndraws_pred = 2,
    verbose = TRUE)

I have experimented with the ndraws and ndraws_pred arguments – even trying silly low numbers (as shown above), but I still get the error message. Thank you in advance for any advice or suggestions to get around this problem.

1 Like
2

Hi,

Maybe @AlejandroCatalina or @avehtari can help you faster.

Good day

3

This error doesn’t come from projpred itself, it comes from R because you are surpassing the maximum memory that it can allocate. One possible reason is that we store all intermediate projections, which may take some space. At the moment we haven’t thought about this but a first idea that comes to mind is to not store the projections and only project when the user wants a specific submodes. Nonetheless, before pushing a fix we’ll have to think about it.

Thank you for reporting this!

3 Likes
4

Thank you - good to know this is fixable and there’s no urgency at my end.

1 Like