Brm_multiple object submitted to projpred

Modeling
1

I submitted a brm_multiple object to cv_varsel.

A warning appeared during the fitting:

Posterior_linpred (transform = TRUE) is deprecated
Please use pp_expect() instead without the ‘transform’ argument
Quick_TRANSfer stage steps exceeded maximum (= 16000000)

The cvs object was eventually created (long run time - ? 1 hour +)

The cvs object is 3.3 Gb in memory and required 16 Gb to write to disc (Rds object).

Is there an argument in projpred to use pp_expect?

Will this change the cv_varsel run time and the object size?

Code details below.

Nathan

Running in Windows
R 3.6.2
brms 2.12.0
projpred 1.1.6

binary dv; predictors 7 numeric, 11 binary factors
n = 680
Handled missing values by 10 fold multiple imputation (mice)
data = tmp.df.imp

Model estimated with

bprior <- c(prior(student_t(3, 0, 2.5), class = ‘b’),
prior(student_t(3, 0, 2.5), class = ‘Intercept’))

tmp.imp.t.stan_glm <- brm_multiple(formula =
OpioidStillTaking ~
Age +
Sex +
BMI +
c.Occupation +
c.Relationship +
c.ASAPhysicalStatus +
c.SurgeryType +
PreopAnxietyTScore +
PreopDepressionTScore +
PreopPhysicalFunctionTScore +
SitePainLastWeek +
PreopCompositeNonOpioid +
PreopCompositeOpioid +
OME +
AnesthesiaDuration +
c.NonOpioidAnalgesicsCount +
General +
Neuraxial,
family = ‘bernoulli’,
prior = bprior,
data = tmp.df.imp,
control = list(adapt_delta = 0.99, max_treedepth = 20),
cores = 8,
chains = 8,
seed = 12345678,
iter = 8000)

tmp.imp.t.stan_glm.cvs <- cv_varsel(
tmp.imp.t.stan_glm,
method = ‘forward’,
data = tmp.df.imp,
cv_method = ‘LOO’,
verbose = FALSE
)

1 Like
2

I don’t know what is brm_multiple and I suspect that the current projpred doesn’t know either. @AlejandroCatalina and @paul.buerkner could comment on this. There is new version coming which might work with brm_mutiple.

This warning is temporary due to the change in posterior_linpred and will go away when all the relevant packages have been updated. You can ignore this warning for a few months, and if it still pops up after updating the latest package versions it’s good to remind the maintaners.

This warning implies that projection didn’t work which my be due to projpred not knowing what to do with brm_multiple.

No these are not connected.

1 Like
3

As of right now we are not working with brm_multiple outputs. They might work if the output is a brmsfit object, but the behaviour is not clear. I mean it’s not clear because, as far as I know (and @paul.buerkner will know better), one uses brm_multiple to fit the same model to several datasets, in which case it’s not clear to me how to handle the response. If all that is handled internally and the resulting object is a single brmsfit object that makes sense, then projpred might work as expected, but we haven’t tried.

1 Like
4

It should work because the returned object is just brmsfit and only a single data set is retained (for now). However, when running brm_multiple you typically have a lot of posterior draws so projpred may run in trouble when it projects the model for each of the thousands of posterior draws.

1 Like
5

Number of projections can be controlled. See arguments in Run search and performance evaluation with cross-validation — cv_varsel • projpred

Also if cv_varsel is slow, you can try method=‘L1’

Eventually we should also allow use of sub-sampling LOO which is implemented in loo package, but not yet enabled in projpred as it requires some careful design choices. projpred was one of the initial motivations for developing sub-sampling LOO.

1 Like