Error with brms (that calls rstan) when using mclapply() on an AWS instance

Interfaces brms
1

I am using mclapply() to do parallel computing of a brms model on an AWS instance. When the number of parallel computations is greater than the number of cores, I get this error for some computations - Error in sink(type = “output”) : invalid connection

res.null <- mclapply(1:100, function(x) sim(N=500, cores=1), mc.cores = 70)

The sim() in the above code snippet has a call to brms() as -

fit.brms <- brm(y ~ trt*day + day.0 + (1|id),
                  data = subset(df.l), family=cumulative("logit"), chains=1, cores=cores)
  • Operating System: Ubuntu 16.04.6
  • brms Version: 2.12.0
2

Hi, sorry that your question fell through. Did you manage to resolve this? You usually get more info out of stan when running with cores=1, also it appears weird that you essentially embed two paralellizations - I would do all the paralellization in the top level-call and then run brm with cores=1 otherwise you might get some weird competition for resources (at least on a normal computer, not sure about AWS)

3

A post was split to a new topic: Paralellizing brms across both chains and fits