Reducing large model file size

I’ve fit a time series model like so:

fit <- brm(event_chances ~ arma(time=age, gr=group, cov=TRUE) + poly(age,2), 
           train_data, 
           family=exponential(),
           prior=priors,
           chains=4, cores=4, iter=2000,
           control=list(adapt_delta=0.95))

Where train_data is around 18,000 rows. This yields a saved model of around 1GB which presents some issues for me as I run some parallelized prediction tasks that export the model N times. I was wondering if there is there a way to reduce the model size either before or after the model is fit?

Look for the package shredder on github

1 Like

Oh man, I’d forgotten about that awesome package. Thanks for the reminder!

Thanks for the suggestion, I wasn’t aware of shredder. My guess is that the large file size is located in fit$fit@sim$samples where there are a large number of residuals (err[n]):

> object.size(fit$fit@sim$samples)
1177236424 bytes

Not quite sure how I should be using shredder to reduce the size of this object. Any thoughts?

I’d appreciate any pointers on this too – not to distract from the OP’s question – but specifically on how to delete things like all the individual random effects from a brms model object while keeping the ability to predict new levels from the population SD and correlation estimates. [EDIT: assuming I stupidly used save_ranef = TRUE initially and not wanting to take another few days to refit].

1 Like

I have a similar question about the residuals in my model. Can they be safely deleted and still allow prediction to function and if so can shredder or some other method be used to delete them out of the model object?

Maybe tagging @yonicd could help elicit their help in discerning how to use shredder with brms.

Alternatively, I’d been planning the addition of keep & toss arguments to aria::compose() that would permit the desired behaviour and could bump that up in priority if you’re ok with a workflow whereby you use brms to specify the model then have it export to a .stan file. Not sure if the resulting rvars representation of the posterior samples would play nicely with any code you have that expects an brms fit though.

My solution to this was to refit the model to omit saving the residuals. This reduced the file size from 1GB to a much more managable 28MB:

fit <- brm(event_chances ~ arma(time=age, gr=group, cov=TRUE) + poly(age,2), 
           train_data, 
           family=exponential(),
           prior=priors,
           chains=4, cores=4, iter=2000,
           control=list(adapt_delta=0.95),
           save_pars=save_pars(group=FALSE))

Thanks for all the suggestions!

1 Like