I’m trying to model experimental data in which participants make two dichotomous forced choice decisions. One of the hypotheses is that each decision (and how the other predictors affect it) depends on the other in a certain way. I’m new to brm and Baysian modelling, so apologies if this is either trivial or confused. My first question is whether the following kind of model specification is legitimate:
decision1_fm <- bf( decision1 ~ otherPredictors*decision2 + (1|p|participant)) + bernoulli() decision2_fm <- bf(decision2 ~ otherPredictors*decision1 + (1|p|participant)) + bernoulli() joint.md <- brm( decision1_fm + decision2_fm + set_rescor(FALSE), data = data, chains = 4, cores = 4)
The model seems to work and the predictions of the model make sense, but I’m just not sure whether this is a legitimate way of modeling given the mutual co-dependence of the two response variables.
If this is in principle a viable model, then I have two questions about how to interpret and evaluate the model:
- When I assess whether each decision affects the other based on the post_samples, it seems that there is strong evidence (with a weakly informative skeptical prior) that the relevant coefficients (the main effect of one response on the other, as well as some of the interaction terms with the otherPredictors) are different from zero, so the decisions do seem to mutually depend on each. This is compatible with the fact that when I do separate traditional logistic mixed effects regression on each decision: the main effect of the other response and some of the interaction terms with the predictors come out significant.
However, when try to check whether the decisions affect each other simply by a model comparison between joint.md and a simpler model that lacks the decision1/decision2 predictor terms in the respective formulas (using loo_compare based on WAIC criterion), the comparison suggests that the more complex model is not better at capturing the data.
So I’m not sure what to make of this–should I conclude that the decisions affect each other (based on post_samples), or that they do not (based on model comparison)?
- I would also like to compute predictions for the likelihood of pairs of decisions given values for the predictors, but I’m not sure how. I can compute predictions with predict() for decision2 given decision1 and vice versa, but I don’t know how to compute predictions for pairs of decisions based just based on the otherPredictors. Can I somehow compute such predictions directly, or do I have to compute such predictions about pairs of decisions based on the conditional predictions for each of the responses? If the latter, how do I do this?
Any leads would be appreciated (or examples where someone has done something similar)
- Operating System: macOs 2.15.4
- brms Version: 2.11.1