How to interpret brm_multiple warning when using pp_check and loo

Interfaces brms
1

I’m using brm_multiple to fit a hierarchical logistic regression model on multiply imputed datasets. As I’m relatively new to brms and am using it on multiply imputed datasets for the first time, I have a somewhat general question regarding handling of the multiply imputed datasets.

When using pp_check and loo on the brmsfit_multiple object, the following warning is printed:

Warning message:
Using only the first imputed data set. Please interpret the results with caution until a more principled 
approach has been implemented.

In this case, I have no missing data for the outcome variable (which is thus 100% identical across all multiply imputed datasets); only one of the predictors variables were imputed and thus differ across the imputed datasets.

Does the above warning simply mean that pp_check only uses the observed outcome data from the first imputed dataset, but uses the full model (combined across all imputed datasets) to generate the data for the comparison? If that understanding is correct, then I should be able to safely ignore the warning here, as the outcome data is the same in all imputed datasets.

Further, does the above interpretation apply to the same warning from loo?

Thanks in advance.

Please also provide the following information in addition to your question:

  • Operating System: Windows 7
  • brms Version: 2.9.0
2

pp_check uses the predictors to generate y_rep that is the predicted observations. That is, regardless of where your missings happen to be (in the response or predictors), it will apply to pp_check and to loo as they use both the response and the predictors only from the first data set, unless you specify other data via newdata.

1 Like
3

Understood. Thanks for the clarification!