Brms: within-chain parallelization fails with negative binomial likelihood

I’m running brms version 2.14.4. With no within-chain parallelization, the model seems fine. However, with within-chain parallelization:

brm(y ~ grp+offset(log(X))+(1|cell), data=dat, family = negbinomial(), chains = 4, iter=1000, backend = “cmdstanr”, threads = threading(6))

it throws out the following error:

Start sampling

Running MCMC with 4 chains, at most 24 in parallel, with 6 thread(s) per chain…

Warning: Chain 1 finished unexpectedly!
Warning: Chain 2 finished unexpectedly!
Warning: Chain 3 finished unexpectedly!
Warning: Chain 4 finished unexpectedly!

Warning: Use read_cmdstan_csv() to read the results of the failed chains.
Error in rstan::read_stan_csv(out$output_files()) :
csvfiles does not contain any CSV file name
Calls: brm … eval2 -> eval -> eval -> .fun -> .fit_model ->
In addition: Warning messages:
1: All chains finished unexpectedly!
2: No chains finished successfully. Unable to retrieve the fit.

Execution halted

Let me know if you need more information. Thanks!

Can you try installing the latest version with remotes::install_github("paul-buerkner/brms") and see if that fixes it?

1 Like

The error message is different now after brms was updated:

Compiling Stan program…
Start sampling
Running MCMC with 4 chains, at most 24 in parallel, with 6 thread(s) per chain…

Chain 1 Exception: Exception: neg_binomial_2_log_glm_lpmf: Vector of dependent variables has dimension = 1088, expecting dimension = 1087; a function was called with arguments of different scalar, array, vector, or matrix types, and they were not consistently sized; all arguments must be scalars or multidimensional values of the same shape. (in ‘/tmp/Rtmp37G3Ph/model-2ab8608dabbb.stan’, line 40, column 4 to column 87) (in ‘/tmp/Rtmp37G3Ph/model-2ab8608dabbb.stan’, line 40, column 4 to column 87)

Warning: Chain 1 finished unexpectedly!
Warning: Chain 2 finished unexpectedly!
Warning: Chain 3 finished unexpectedly!
Warning: Chain 4 finished unexpectedly!

Warning: Use read_cmdstan_csv() to read the results of the failed chains.

Error in rstan::read_stan_csv(out$output_files()) :
csvfiles does not contain any CSV file name
Calls: brm … eval2 -> eval -> eval -> .fun -> .fit_model ->

In addition: Warning messages:

1: All chains finished unexpectedly!
2: No chains finished successfully. Unable to retrieve the fit.

Execution halted

In that case, this is an unresolved issue/bug with the brms code generation in threading. Please post your example to https://github.com/paul-buerkner/brms/issues and mention that you tried the most recent version.

1 Like