I find that data reduction (unsupervised learning) is a valuable part of my modeling strategy when there are too many predictors for the effective sample size available. I also like penalized regression but tools like principal components, sparse principal components, and variable clustering also have roles.

Does anyone know of a Bayesian modeling approach that provides a one-step approach to restricting a model to emphasize orthogonal collapsed covariate dimensions as principal components regression does? I would imagine that scaling issues are tough to deal with in this context.