I am trying to compute multiple models in parallel on my computer with 16 logical cores. To do so, I use brms::brm() inside furrr::future_map(), and using four workers to run furrr::future_map() by setting plan(tweak(multisession, workers = 4)). In addition, I use brms::brm() with its backend set to…

@rok_cesnovar Thank you for your suggestion and apologies for my late reply. I finally managed to write more efficient code, separating the compilation part and sampling part, as shown below. Since I fit one model to multiple simulation data, I put the compilation procedure outside of future_map, …

It’s possible to do this with {rstan} (you might try it with the up to date experimental branch) and {future} but as I understand it because {cmdstanr} is just a light-weight wrapper that calls cmdstan behind the scenes, parallelizing models via {future} and a cmdstanr backend is complicated if not…

@ajnafa Thank you for your reply. parallelizing models via {future} and a cmdstanr backend is complicated if not impossible and is not currently supported by {brms} Would you please let me know how to fit ‘raw’ cmdstanr models parallelly using {future}, or would you tell me any reference that m…

I am investigating the cmdstanr issue, but I am almost certain that it is stemming from compiling the model. I have been able to run sampling with future, but have never used it for the compiling part as well.

@rok_cesnovar Thank you for your comment. I am almost certain that it is stemming from compiling the model. I have been able to run sampling with future , but have never used it for the compiling part as well Do you think it would work if I separate the compiling part and sampling part and pass…

I actually se no reason to parallelize compilation. Just compile the model once and then use the same executable for all $sample() calls.

Is it impossible to compute multiple models in parallel using brms with `backend = "cmdstanr"`?

Interfaces brms

rok_cesnovar July 25, 2022, 12:47pm 8

Great news!

Topic		Replies	Views
Possible to quasi-automatically implement within-chain parallelization in cmdrstan with brms? Modeling cmdstanr , brms	3	583	August 13, 2022
Brms model will not sample with threading enabled CmdStan fitting-issues , brms	1	490	June 27, 2023
Parallel with cmdstanr? Interfaces cmdstanr	8	1957	April 25, 2020
Within-chain parallelization misbehaves with brms in a model with measurement errors brms	21	2094	November 1, 2020
Facing error while compiling brms code Developers brms	22	1441	February 11, 2021

Is it impossible to compute multiple models in parallel using brms with `backend = "cmdstanr"`?

Related topics