Hi! I’m trying to determine what interaction terms (if any) I should include in a phylogenetically informed brms model with a binary response variable. I have several predictors, including ordinal, binary, and continuous variables, as well as the phylogenetic variance-covariance matrix (coded as a random effect). As you can imagine, there are a lot of possible interaction terms. My tree is comprised of nearly 3400 taxa, so each model is pretty computationally intensive even without any interactions.
Is there a good way (akin to stepwise AIC in more “standard” regressions) to assess which interaction terms improve the model in an automated way? I’ve seen a couple of people use LOO or WAIC techniques using the update() function to remove interaction terms one by one, but I’m worried that this method will be too intensive time-wise, given that each model takes a while to run and that I’ll have dozens and dozens of possible interaction terms to sift through.
I’d be cautious about stepwise approaches in general, as they introduce selection bias — estimates from the final model are conditional on the selection process, which standard inference methods don’t account for.
You might find projpred (projection predictive feature selection) useful? While I haven’t used it extensively myself, I believe it allows you to specify which variables should be included in the selection process (via function force_search_terms). This could be particularly helpful given your focus on testing the contributions of interaction terms.
There are also other approaches like Bayesian stacking / model weights but I believe those would require you to first separately fit all model variants of interest, which might not be feasible in your case.
projpred is awesome, but it is possible that it doesn’t work with phylogenetic model, but you can try. We wrote the above-mentioned paper specifically for the cases when projpred is not applicable, but stepwise LOO would be easy to implement.