Setting priors to produce bayesian comparison - syntax

Modeling
, , ,
1

I’m trying to replicate the analysis from this code (OSF) and compare two models that have either 2 or 6 predictor variables, but I don’t really understand the syntax for setting the priors.
My models are:

prior1 <- c(prior(normal(0, 1), class="b"), 
             #prior(normal(0, 1), class="b", coef="varA"),
            #prior(normal(0, 1), class="b", coef="varB"),
            prior(normal(0, 1), class="b", coef="WORDTYPE:varA"),
            prior(normal(0, 1), class="b", coef="WORDTYPE:varB"),
            prior(normal(0.5, 1), class="b", coef="WORDTYPE"))


prior2 <- c(prior(normal(0, 1), class="b"), 
             prior(normal(0, 1), class="b", coef="varA"),
            prior(normal(0, 1), class="b", coef="varB"),
             prior(normal(0, 1), class="b", coef="varC"),
            prior(normal(0, 1), class="b", coef="varD"),
             prior(normal(0, 1), class="b", coef="varE"),
            prior(normal(0, 1), class="b", coef="varF"),
            prior(normal(0.5, 1), class="b", coef="WORDTYPE"))

null_model <- brm(WordResp ~ 1 + WORDTYPE*varA+ WORDTYPE*varB + (1|PsNumber)+(1|WordNumber),
             data=MemoryData, family=bernoulli(link="probit"), prior=prior1,
             control=list(adapt_delta=.99), init=0, cores=4, iter=1000,
             sample_prior=TRUE)
Step2_model <- brm(WordResp ~ 1 + WORDTYPE*varA+ WORDTYPE*varB +WORDTYPE*varC+ WORDTYPE*varD +WORDTYPE*varE+ WORDTYPE*varF+ (1|PsNumber) + (1|WordNumber),
             data=MemoryData, family=bernoulli(link="probit"), prior=prior2,
             control=list(adapt_delta=.99), init=0, cores=4, iter=1000,
             sample_prior=TRUE)

all the variables are standardised to have normal distribution and mean at 0. I want to set the same prior for all of them, but I’m confused as to whether I need to enumerate them in the syntax, with or without interaction, or if maybe setting priors for all coefficients (class=b) in one line of code is enough?
The reason I’m struggling is that whenever I try to run a bayes factor comparison, I get a null result as if the model is not read correctly, even though it seems like a simple procedure. That suggests that something is wrong with the model even though it runs and produces an output properly, so I suspect the problem is with the priors. More specifically when I run the code above for bayes factor my R session typically crashes, and when I try:

exp(Step2_model$ml - null_model$ml)

I get “numeric(0)” as a result.
any advice would be appreciated!

2

See ?set_prior:

To put the same prior on all population-level effects at once, we may write as a shortcut set_prior("<prior>", class = "b"). This also leads to faster sampling, because priors can be vectorized in this case.

So if you set prior(normal(0, 1), class="b") all the b parameters will get that prior, but you can still set other priors for specific coefficients if you want. You can use prior_summary(your_model) to see what priors were actually used.

Where does that syntax come from? I don’t think brmsfit objects have a ml slot, so Step2_model$ml will return NULL. brms has built in functionality for Bayes factors, so you can run:

bayes_factor(Step2_model, null_model)

Do read the guidance at ?bayes_factor.

3

Thanks for the response! When I run

bayes_factor(Step2_model, null_model)

on my laptop, R basically crashes. I managed to run it once on a more powerful computer and it gave me an error, something to do with not being able to read the model or the priors, unfortunately I don’t remember exactly so I’ll have to try it again to extract the error. But that’s what prompted me to look into the priors, because I’m not sure that I even set them at all initially. But perhaps the problem is with the installation of a package so I will also look into that again.

4

Hi again, I have an updated question. I ran the models again, but bayesfactor again results in errors.

Error : Exception: variable does not exist; processing stage=parameter initialization; variable name=z_1; base type=double (in ‘string’, line 36, column 2 to column 29)
Error: Bridgesampling failed. Perhaps you did not set ‘save_pars = save_pars(all = TRUE)’ when fitting your model? If you are running bridge sampling on another machine than the one used to fit the model, you may need to set recompile = TRUE.

I will try adding save_pars(all=TRUE) as a parameter, but any idea what the z_1 variable refers to?
Thanks