Predictive Performance Metrics for Imputed Outcomes

General
1

Hi all,

I’m working on developing a predictive model for a continuous outcome that is a sum of two measurements. One measurement is taken for all points in the dataset and one is only taken for a small percentage (~10%). I’m planning on implementing an imputation model in Stan and using some form of cross validation/leave one out validation to assess predictive performance. Does anyone know of any methods to account for the large percentage of imputed outcomes and the potential impacts when calculating performance metrics?

2

It’s valid to compute the predictive performance using just the observations. For the imputed ones you can’t really assess anything else than sharpness, which may be useful in addition of cross-validation for the observed ones.

3

That makes sense, thanks.