`Recompiling the model with 'rstan'` fails with crash when calculating Bayes factor using `bayes_factor()`

Interfaces brms
, , ,
1

Using brms::bayes_factor() (or bridgesampling::bayes_factor()), I’m calculating Bayes Factors from the models that I created using brms::brm(..., backend = "cmdstanr").

As reported as a GitHub issue here, Bayes factors cannot be calculated using CmdStan and its interfaces. However, after brms Ver. 2.16.3, or brms::bayes_factor() (or bridgesampling::bayes_factor()) automatically recompiles the models of brms::brm(..., backend = "cmdstanr") using rstan.

Although this functionality eliminates the need for manual recompilation of models (as posted here), R crashes when bayes_factor(mod1, mode2) is trying to recompile the second model (i.e. mod2 here).

Does anybody replicate this problem, or tell me how to avoid such a problem?

MWE

mod1_cmd <- brm(
  Sepal.Length ~ Petal.Length + Petal.Width,
  iris,
  family = lognormal,
  warmup = 2000,
  iter = 30000,
  chains = 8,
  control = list(adapt_delta = 0.95),
  save_pars = save_pars(all = TRUE),
  backend = "cmdstanr",
  cores = parallelly::availableCores() - 1
)

mod2_cmd <- brm(
  Sepal.Length ~ Petal.Length,
  iris,
  family = lognormal,
  warmup = 2000,
  iter = 30000,
  chains = 8,
  control = list(adapt_delta = 0.95),
  save_pars = save_pars(all = TRUE),
  backend = "cmdstanr",
  cores = parallelly::availableCores() - 1
)

bf_cmd_brms <- brms::bayes_factor(mod1_cmd, mod2_cmd)
#bf_cmd_bridge <- bridgesampling::bayes_factor(mod1_cmd, mod2_cmd)

bf

Session Info

  • Operating System: Windows 10
  • R 4.1.0
  • CmdStan Version: 2.28.2
    • cmdstanr Version: 0.4.0
  • rstan: 2.21.2
  • StanHeaders: 2.21.0-7
  • brms: 2.16.3
  • Rcpp: 1.0.7
  • Compiler/Toolkit: Rtools40
R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
 [1] rstan_2.21.2         StanHeaders_2.21.0-7 cmdstanr_0.4.0
 [4] brms_2.16.3          Rcpp_1.0.7           hypr_0.2.2
 [7] MASS_7.3-54          forcats_0.5.1        stringr_1.4.0
[10] dplyr_1.0.7          purrr_0.3.4          readr_2.1.0
[13] tidyr_1.1.4          tibble_3.1.6         ggplot2_3.3.5
[16] tidyverse_1.3.1
2

Your code works for me without any crashes. However, it does recompile the models using rstan for the bridge sampling.

> bf_cmd_brms <- brms::bayes_factor(mod1_cmd, mod2_cmd)
Recompiling the model with 'rstan'
Iteration: 1
Iteration: 2
Iteration: 3
Iteration: 4
Recompiling the model with 'rstan'
Iteration: 1
Iteration: 2
Iteration: 3
Iteration: 4
> bf_cmd_brms
Estimated Bayes factor in favor of mod1_cmd over mod2_cmd: 0.27864

My info

> packageVersion('brms')
[1] ‘2.16.1’
> packageVersion('cmdstanr')
[1] ‘0.4.0’

> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Arch Linux

Matrix products: default
BLAS:   /usr/lib/libblas.so.3.10.0
LAPACK: /usr/lib/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C               LC_TIME=en_GB.UTF-8        LC_COLLATE=en_GB.UTF-8     LC_MONETARY=en_GB.UTF-8   
 [6] LC_MESSAGES=en_GB.UTF-8    LC_PAPER=en_GB.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] brms_2.16.1 Rcpp_1.0.7
3

Thank you for your reply! Ummm, then the problem might be specific to me, or OS specific (your OS is Linux and mine is Windows).

Could you tell us which version of the following packages you use?

  • rstan
  • StanHeaders
  • Rcpp
4

Upgrading rstan version to 2.26.6 (Stan version 2.26.1; StanHeaders version 2.26.6) by running the following code (as described here) solved the problem.

install.packages(
  "rstan",
  repos = c(
    "https://mc-stan.org/r-packages/",
    getOption("repos")
  )
)
5

I realise this is solved, but just for the record, since I was asked:

> packageVersion('StanHeaders')
[1] ‘2.21.0.7’
> packageVersion('rstan')
[1] ‘2.21.2’
> packageVersion('Rcpp')
[1] ‘1.0.7’