Post-hoc tests for mixed design

Modeling
1

Dear Stan/Bayesian experts,

I have a question related to the post-hoc comparison. If i have a between subject design variable with three dim and two within subject variable. If i want to check a dim in the within subject variable (let’s say A variable equals = 1) is significantly different in between subject design variable. Can i perform three regression for different between design variable dim and then compare the model prediction when variable A = 1? Or because i fit three models, each model has its own standard deviation, so it’s not allowed to do this analysis? Thanks in advance!

Best,

2

I’m not exactly sure what you mean by checking a dimension. Yes, you can build different models of any kind and compare their predictions using something like leave-one-out cross-validation. That implicitly adjusts for uncertainty in both estimation and in the sampling distribution.

A better idea might be to combine all the covariates you have in a single model and just do posterior predictive inference. That will automatically discount the less relevant covariates.

1 Like