# Modelling two mutually informative decisions

**URL:** <https://discourse.mc-stan.org/t/modelling-two-mutually-informative-decisions/15113>\
**Category:** brms\
**Tags:** mixed-model, model-comparison, loo\
**Created:** [May 13, 2020, 10:59pm UTC](https://discourse.mc-stan.org/t/modelling-two-mutually-informative-decisions/15113 "2020-05-13T22:59:12Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![chael](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/chael/32/6159_2.png) [@chael](https://discourse.mc-stan.org/u/chael)\
**Post date:** [May 13, 2020, 10:59pm UTC](https://discourse.mc-stan.org/t/modelling-two-mutually-informative-decisions/15113/1 "2020-05-13T22:59:12Z")

</div>

I’m trying to model experimental data in which participants make two dichotomous forced choice decisions. One of the hypotheses is that each decision (and how the other predictors affect it) depends on the other in a certain way. I’m new to brm and Baysian modelling, so apologies if this is either trivial or confused. My first question is whether the following kind of model specification is legitimate:

```
decision1_fm <- bf( decision1 ~
					  otherPredictors*decision2 +
                      (1|p|participant)) +
  bernoulli()

decision2_fm <- bf(decision2 ~
                    otherPredictors*decision1 + 
                    (1|p|participant)) +
  bernoulli()

joint.md <- brm(
  decision1_fm + 
  decision2_fm + 
  set_rescor(FALSE),
  data = data,
  chains = 4,
  cores = 4)

```

The model seems to work and the predictions of the model make sense, but I’m just not sure whether this is a legitimate way of modeling given the mutual co-dependence of the two response variables.

If this is in principle a viable model, then I have two questions about how to interpret and evaluate the model:

- When I assess whether each decision affects the other based on the post\_samples, it seems that there is strong evidence (with a weakly informative skeptical prior) that the relevant coefficients (the main effect of one response on the other, as well as some of the interaction terms with the otherPredictors) are different from zero, so the decisions do seem to mutually depend on each. This is compatible with the fact that when I do separate traditional logistic mixed effects regression on each decision: the main effect of the other response and some of the interaction terms with the predictors come out significant.

However, when try to check whether the decisions affect each other simply by a model comparison between joint.md and a simpler model that lacks the decision1/decision2 predictor terms in the respective formulas (using loo\_compare based on WAIC criterion), the comparison suggests that the more complex model is not better at capturing the data.

So I’m not sure what to make of this–should I conclude that the decisions affect each other (based on post\_samples), or that they do not (based on model comparison)?

- I would also like to compute predictions for the likelihood of pairs of decisions given values for the predictors, but I’m not sure how. I can compute predictions with predict() for decision2 given decision1 and vice versa, but I don’t know how to compute predictions for pairs of decisions based just based on the otherPredictors. Can I somehow compute such predictions directly, or do I have to compute such predictions about pairs of decisions based on the conditional predictions for each of the responses? If the latter, how do I do this?

Any leads would be appreciated (or examples where someone has done something similar)

- Operating System: macOs 2.15.4
- brms Version: 2.11.1

---

<div class="post-metadata">

**Author:** ![Ara\_Winter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/ara_winter/32/4284_2.png) [@Ara\_Winter](https://discourse.mc-stan.org/u/Ara_Winter)\
**Post date:** [May 14, 2020, 1:51am UTC](https://discourse.mc-stan.org/t/modelling-two-mutually-informative-decisions/15113/2 "2020-05-14T01:51:49Z")

</div>

Hi and welcome! I am wondering why you have set\_rescor(FALSE) for the model?

Usually (coming from ecology) when I have tow things that likely depend on each other I set my model to set\_rescor(TRUE). Since this is outside of my discipline I could be way off about how your model is working.

---

<div class="post-metadata">

**Author:** ![chael](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/chael/32/6159_2.png) [@chael](https://discourse.mc-stan.org/u/chael)\
**Post date:** [May 14, 2020, 2:37am UTC](https://discourse.mc-stan.org/t/modelling-two-mutually-informative-decisions/15113/3 "2020-05-14T02:37:41Z")

</div>

I may be wrong but I think it has to be set to false because it’s a logistic model (and it returns an error if I set it to true)

---

<div class="post-metadata">

**Author:** ![martinmodrak](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/martinmodrak/32/133_2.png) [@martinmodrak](https://discourse.mc-stan.org/u/martinmodrak)\
**Post date:** [May 14, 2020, 8:29pm UTC](https://discourse.mc-stan.org/t/modelling-two-mutually-informative-decisions/15113/4 "2020-05-14T20:29:41Z")

</div>

> [@chael](#):
>
> The model seems to work and the predictions of the model make sense, but I’m just not sure whether this is a legitimate way of modeling given the mutual co-dependence of the two response variables.

I think using one decision to predict the other is _slightly_ weird. It makes the model non-generative (i.e. that you can’t simulate data from it), which is then source of your additional problems with prediction. Nevertheless I don’t think there is “one best way” to model this. I’ve seen people handle this by moving to long format (i.e. so you know have columns `decision` and `decision_id` (a factor taking values “1” and “2”) and each row in the original dataset is know two rows). Then you would use formula `decision ~ (otherPredictors | decision_id) + (1 | participant)` where the correlation terms for the `otherPredictors | decision_id` can be interpreted as a dependence between the two decisions.

You can play with that making only some effects/intercepts depend on `decision_id` or even add a varying intercept `decision_id | row_id_in_the_original_datset`.

Alternatively, you might have some more theory to guide a more precise form of the depenedency.

> [@chael](#):
>
> However, when try to check whether the decisions affect each other simply by a model comparison between joint.md and a simpler model that lacks the decision1/decision2 predictor terms in the respective formulas (using loo\_compare based on WAIC criterion), the comparison suggests that the more complex model is not better at capturing the data.

I am guessing here, but I think it is possible that the other terms in your model are flexible enough to capture the variability (despite the other decision being a good predictor). Bernoulli response provides very limited information. You probably should run posterior predictive checks to see if any of the models has problems capturing some aspect of the data.

Hope that helps!

---

<div class="post-metadata">

**Author:** ![chael](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/chael/32/6159_2.png) [@chael](https://discourse.mc-stan.org/u/chael)\
**Post date:** [May 14, 2020, 9:12pm UTC](https://discourse.mc-stan.org/t/modelling-two-mutually-informative-decisions/15113/5 "2020-05-14T21:12:12Z")

</div>

Thanks, Martin, this is incredibly helpful! I understand now better what was weird about how I tried to model this, and why I couldn’t get the predictions I wanted. I am playing with your alternative way of modelling this, and I think it’ll take a me a while to understand how to interpret the correlation terms, but in principle I can see where this is going.

Just to clarify, apart from the joint model, I am also looking at (i) a model in which both decisions are independently of each other, and (ii) models in which one is made independently, and the other one is based on the first (decision1 before decision2 or vice versa). Just looking at these options, the empirical data looks more like one of the latter ordered hypotheses is correct. I guess what I tried to do with the above model is to avoid deciding in the model itself that one necessarily precedes the other—just because it’s not clear to me why perception should necessarily proceed one way or the other. It seems to me that this must be a recurring issue not just in speech perception (two in principle orthogonal decisions are made based no overlapping cues), but I haven’t found any discussion about how to model this best. This general issue is not a question about brms of course, I just thought there might be a way to do this with it. Anyway, I will explore your alternative of coding the model more.

Thanks also for the comment about the model comparison, that makes a lot of sense to me!