I am running a model in brms with a couple of smooth terms. I thought it would be interesting to use the priorsense package. It is finding several prior-data conflicts for the various smooth terms used in brms. Changing the sds priors seems to not help; indeed, I cannot find any combination of priors that doesn’t result in several prior-data conflicts. However, if I replace the smooths with cubic terms, the prior-data conflicts disappear (but the fits to the data are less convincing). I am curious to know if prior-data conflicts reported by priorsense are not actually problematic for smooth terms due to there being some form of combined/joint information being shared between the various terms.
1 Like
Tagging some of the authors of priorsense … @topipa, @paul.buerkner, @avehtari
I have no experience with using priorsense with smooth terms yet. Do you have a reproducible example that demonstrates the behavior?
Have you looked at the GP example in [2107.14054] Detecting and diagnosing prior and likelihood sensitivity with power-scaling (updated 19 Dec 2022)? That GP implementation is practically the same as the spline implementation, so the advice there holds for splines, too.
That’s very informative and probably corresponds to what I am seeing in my model – thank you for the pointer. I will investigate further by focussing on the sensitivity of the predictions.
1 Like
Thank you so much for your helpful reply @avehtari, Is there a code example that goes with Figure 11 by any chance?
Great that you asked, as the code was missing from the repo by accident.
@n-kall just pushed to code to powerscaling-sensitivity/case-studies/motorcycle at master · n-kall/powerscaling-sensitivity · GitHub
2 Likes
Amazing code examples, @n-kall, a ton to learn from this on so many levels. I’m trying to figure out how to best use draws from brms::posterior_epred with priorsense and am a little stuck, but I’ll post a new question since that’s a bit off topic here.
1 Like
I ended up opening an issue with some example code. It’s just an interface issue, really, so hopefully this helps others prior to the interface update @n-kall is working on.
1 Like
A quick question about this model: why aren’t beta_f and beta_g included in the lprior calculation?
If I remember correctly, this is because the model is using a non-centered parametrisation where beta_f and beta_g have a standard normal prior and then are multiplied by the diagSPD terms (lines 66 and 67). Because of this it doesn’t make sense to perturb the normal(0,1) prior on these parameters but to focus on the other parameters that determine the scales.
1 Like
Ok, I see it now. Thanks!
I’ve been away, but wanted to add even a bit late that also in hierarchical models we need anyway to select the level of the focus and in this case we’re not interested in beta_f and beta_g as they do not have direct interpretation and they can be considered as nuisance parameters.
1 Like