I’m trying to familiarize myself with the projection predictive approach and projpred package, and thinking about implementing projpred methods for particular custom model class. However, the documentation on how to do this is somewhat sparse, scattered, and perhaps outdated (init_refmodel example with custom model · Issue #125 · stan-dev/projpred · GitHub). Any suggestions on how to decipher the minimal structure for this, or should I just try to mimic the implementations for brsm and/or rstanarm fits? Or are there cases where it could be more reasonable to just implement the whole method from scratch for specific model types according to the source literature? The models I am thinking are essentially constrained linear regression models.
Tagging @fweber144@avehtari as they seem to be actively working on Github on projpred and also present here.
The documentation for the use of the latent projection with custom families is pretty comprehensive and has a nice worked example for the negative binomial.
Projection is based on minimizing the KL-divergence from the reference model predictive distribution to the constrained model predictive distribution for each reference posterior draw separately. For many data model distributions, this is equivalent or can be approximated with optimization of the constrained model parameters given mean of the reference model prediction for each reference model posterior draw. In case of joint missing data imputation, each reference model posterior draw includes draw from the missing data distribution, too. The optimization approach to minimize KL does not work well for these latent data parameters. We could approximate by keeping the latent data parameters as fixed, and optimize only other parameters, but then this would be the same as using multiple imputation approach, which is a big task to add as discussed in a github issue.