Guidance on prior predictive checks in ordinal model

[natpac]

@avehtari

@natpac

I am concerned about the justification of the choice of the prior, normal(0, 1) is meaningful but normal(0, 2) or normal(0, 3) could be valid too.

If you think they are all valid, then you can report the results with all of them as prior sensitivity analysis.

Coefficients not sensitive to prior specification remain small when using a wide prior and I do not have a reason to believe that the coefficient of the parameters that are sensitive to prior spec could have larger impacts than the other.

@avehtari

Which model did you use as the reference model? The predictive performance estimates going above the reference model result indicates the reference model might be overitting,

My reference model is the model with all 11 predictors (some are categorical with several levels) and is likely overfitting above 7-8 parameters.

@avehtari

@natpac

Would keeping 6 predictors and adding back the monotonic effects make sense?

Probably that 1 predictor is enough, unless you get better results by improving the reference model.

1 predictor is the most parsimonious model but I guess interpreting carefully the model with 6 predictors would be fine given the small improvement in elpd (and that the projpred elpd plot is based on models without monotonic effects) and the results make sense. Is that correct?

A last question would be on computing a R2. I read that bayes_R2() is probably biased for ordinal models and I saw in another post R^2 calculation for brm model with cumulative family type - #4 by andymilne that a Bayesian McKelvey-Zavoina R2 could do a better approximation?

---

On projpred and monotonic effect:

@avehtari

You can create an issue in the repo, so we don’t forget this.

Will do, thanks!