Projpred: Fixing Group Effects in Search Terms and Tips for Speed?

[l-gorman]

Hi Frank,

Thanks so much for your response! I’ve always been so impressed seeing how active this community is!!! I really appreciate you taking the time to write such a helpful response!

@fweber144

Are you using the most recent CRAN version of projpred (2.6.0)? Because it comes with a reduced peak memory usage in K-fold CV

No I wasn’t actually, I have now updated the package, let’s see how that effects things. Sounds like this would be a big help, and exactly what I need.

@fweber144

Do I get your question concerning the multilevel terms correctly, in that you want (1 | village) and (1 | country) to be included in all submodels? In other words, should these terms be forced to be selected first? If yes, then you can achieve this through argument search_terms, as you guessed.

Yes that’s exactly right. I was looking at this post. The impression I got was that I would need to do something like this:

my_search_terms <- c(
    "1",
    "(1 | iso_country_code) + (1 | iso_country_code:village)",
    "(1 | iso_country_code) + (1 | iso_country_code:village) + x1",
    "(1 | iso_country_code) + (1 | iso_country_code:village) + x2",
    "(1 | iso_country_code) + (1 | iso_country_code:village) + ...",
    "(1 | iso_country_code) + (1 | iso_country_code:village) + x10"
)

cv_varsel_res <- cv_varsel(ref_model,
                          method = 'forward', 
                          cv_method = 'kfold', 
                          K = 5,
                          verbose = TRUE, 
                          seed = seed,
                          search_terms = my_search_terms
)

Is that correct, or have I misunderstood? This seems to line up with the issue you linked to?

@fweber144

There are many ways to speed up projpred, but you need to be careful not to become too approximate. In general, with such speed-ups, you can quickly get some results, but they are only rough and should only be considered as preliminary results. Some speed-up possibilities are…

Thanks so much for the suggestions, I will have play around with the different options! I will let you know how I get on.

@fweber144

Finally, just a minor remark: In your code, you are specifying projpred:: for the get_refmodel() call, but not for the cv_varsel() call. Does that mean you call library(projpred) beforehand?

Ah yes, I see the inconsistency! Yes I do call library(projpred) beforehand, I don’t know why I have presented it like this!

Thanks again,

Léo