Not a Stan-specific Q, but I’m curious if there are any standard practices for computing posterior distributions for effect sizes of any sort for a hierarchical model with “within-subjects” effects. The approach I’m using now is to compute two kinds of effect size:
A “between-Ss” effect size where, in every sample from the posterior, I divide the value of the parameter representing the effect by the value of the parameter representing the SD of deviations in how this effect manifests across participants.
A “within-Ss” effect size where, for each subject, I get the variance of the posterior for that subject’s effect, compute the mean of these variances across subjects, then use the square-root of this value as the denominator when computing an effect size from each sample of the coefficient.
I’m pretty confident in the appropriateness of method #1 (though feel free to correct me), but #2 feels a bit ad hoc and has the likely problematic behaviour of reducing the effect size estimates when there is less data and therefore more widely dispersed estimates for each subject’s effect. Any thoughts?