Hi all,

I have been stuck with this problem for a few days now and I would really appreciate anyone’s input:

I would like to compare two brm models with Bayes Factor but something goes wrong during bridesampling.

Dataset: The outcome variable is a likert scale (1-8 → RatingNum) so I opted for using cumulative models. The predictor variables include an ordinal variable with 5 levels (–> LevelN) and a continuous predictor (for the sake of simplicity I describe the reduced model below that does not include the continuous predictor). I also include random slope for the ordinal variable and random intercepts for stimuli and participants.

testdataset.csv (115.9 KB)

Problem: The dataset has 8100 observations (I’m attaching a subset of the data with 15 participants only) so I opted to use within chain parallelization with the cmdstanr backend. The model fit works fine but when I try to use bridgesampling I get this error after recompilation:

Error : Exception: stan::math::simplex_free: Simplex variable is not a valid simplex. sum(Simplex variable) = 0.9999996, but should be 1 (in ‘anon_model’, line 106, column 2 to column 25).

I tried to play around with the prior setting etc. with no luck. Then, I tried NOT using the cmdstanr backend and then the bridgesampling works. So I assume something goes wrong during recompilation?

It would be really good if I could use the within chain parallelization because fitting the model takes many hours and to use Bayes Factors in a acceptable manner, I want to run several simulations and testing

(following the recommendations of: Schad, D. J., Nicenboim, B., Bürkner, P.-C., Betancourt, M., & Vasishth, S. (2022). Workflow techniques for the robust use of bayes factors. Psychological Methods. Advance online publication. APA PsycNet)

Thank you for your input in advance!

Cheers,

Aniko

```
library(cmdstanr)
library(brms)
library(bridgesampling)
formulanull = RatingNum ~ 1 + mo(LevelN) +(1+mo(LevelN)|Img) + (1+mo(LevelN) |subject)
priors_mixed.m0 <- c(
set_prior('normal(-1.1503494, 1)', class = 'Intercept', coef = '1'),
set_prior('normal(-0.6744898, 1)', class = 'Intercept', coef = '2'),
set_prior('normal(-0.3186394, 1)', class = 'Intercept', coef = '3'),
set_prior('normal(0, 1)', class = 'Intercept', coef = '4'),
set_prior('normal(0.3186394, 1)', class = 'Intercept', coef = '5'),
set_prior('normal(0.6744898, 1)', class = 'Intercept', coef = '6'),
set_prior('normal(1.1503494, 1)', class = 'Intercept', coef = '7'),
set_prior('normal(0, 1)', class = 'b'),
set_prior('lkj(2)', class = 'cor', group = 'Img'),
set_prior('lkj(2)', class = 'cor', group = 'subject'),
set_prior('dirichlet(2, 2, 2, 2)', class = 'simo', coef = 'moLevelN1'),
set_prior('exponential(.1)', class = 'sd', group = 'Img'),
set_prior('exponential(.1)', class = 'sd', group = 'subject'))
modelnull <- brm(formula = formulanull,
family = cumulative,
prior = priors_mixed.m0,
init = 0.2,
chain = 4,
cores = 4,
iter = 2000, #for actual BF calculations I use 10000 with 2000 warmup
warmup = 1000,
control=list(adapt_delta=0.99,max_treedepth=15),
save_pars = save_pars(all = TRUE),
data = subset)
bridgesampling::bridge_sampler(modelnull)
model1null <- brm(formula = formulanull,
family = cumulative,
prior = priors_mixed.m0,
init = 0.2,
chain = 4,
cores = 4, #set-up using 8 CPU cores
backend = "cmdstanr",
threads = threading(2),
iter = 2000,
warmup = 1000,
control=list(adapt_delta=0.99,max_treedepth=15),
save_pars = save_pars(all = TRUE),
data = subset)
bridgesampling::bridge_sampler(model1null)
```

- Operating System: macOS Ventura v13.2.1
- R version 4.2.1
- brms Version: brms_2.18.0

*cmdstanr_0.5.3

*bridgesampling_1.1-2