Correction for running multiple hypothesis when the goal is decision making?

Interfaces brms
1

Suppose we have an IV about body site with two levels a and b. We have three DVs dv1, dv2, and dv3. For example how well the participant attends to vibrations, how well the participant differentiate the viberations, and how well the participant breaths with the vibrations. We have three hypotheses: a > b for dv1, dv2, and dv3.

I compute three posterior probabilities p1, p2 and p3 testing the hypotheses a > b for dv1, dv2, and dv3. Let’s say p1 =0.8, p2 = 0.9, and p3 = 0.1. Again, we’re looking for evidence that body site a is better than site b, so we can recommend using site a. If we wanted to give a joint probability, it would be 0.80.90.1 = 0.072 for a > b, and 0.2 * 0.1 * 0.9 = 0.018 for b > a. The ratio of these probabilities is 0.072/0.018 = 4 so it seems that we could still recommend a.

Is this a justifiable method? Is it an approach that has been used?

2

I am not sure. There are multiple aspects of this. First, your approach assumes independence of the DVs which may not necessarily be true. Second, you are just contrasting the extremes, that is all 3 tests going in direction “a” vs. all 3 tests going in direction “b”. There are quite a lot of other alternative outcomes in between which may be meaningfull as well.

3

Hi Paul,
I just read your response here. Could you help me understand your second point a bit more please?

As for the first point, when DVs are not independent, what should I do? [sorry for my ignorance, still second week through taking a bayesian class].