Presenting influence of different priors

Modeling
1

Are there good ways for showing the influence of two different priors on results? Or a good way showing to a friend-frequentist that my priors do not influence the results.

I have two models. One with a weak prior.

library(tidyverse)
library(brms)
library(janitor)
n = 20
a = tibble(weight = rnorm(n, 80, 7), sex = rep("man"))
b = tibble(weight = rnorm(n, 60, 5), sex = rep("woman"))
df = rbind(a, b)

m_weak_prior = brm(weight ~ sex, data = df, sample_prior = "yes", prior = c(set_prior("normal(0, 20)", class = "b", coef = "sexwoman")))

post1 = m_weak_prior %>% posterior_samples() %>% clean_names()

post1 %>% ggplot()+
  geom_density(aes(b_sexwoman), alpha = 0.3, fill = "darkblue")+
  geom_density(aes(prior_b_sexwoman), alpha = 0.3, fill = "lightblue")+
  ggtitle("posterior and prior")

image
image700Ɨ432 3.82 KB

And one with a stronger prior.

m_strong_prior = brm(weight ~ sex, data = df, sample_prior = "yes", prior = c(set_prior("normal(-40, 5)", class = "b", coef = "sexwoman")))

post2 = m_strong_prior %>% posterior_samples() %>% clean_names()

post2 %>% ggplot()+
  geom_density(aes(b_sexwoman), alpha = 0.3, fill = "darkblue")+
  geom_density(aes(prior_b_sexwoman), alpha = 0.3, fill = "lightblue")+
  ggtitle("posterior and prior")

image
image700Ɨ432 4.78 KB

My possibly bad solution for showing the effect of the stronger prior on b coefficient for sex

I calculate the difference between the two modelā€™s b coefficients and then pass it to posterior_summary() function

post1 = post1 %>% select(b_sexwoman) %>% rename(b_sexwoman1 = b_sexwoman) %>% mutate(id = row_number())
post2 = post2 %>% select(b_sexwoman) %>% rename(b_sexwoman2 = b_sexwoman) %>% mutate(id = row_number())
merge(post1, post2, by = "id") %>% mutate(influence = b_sexwoman2 - b_sexwoman1) %>% posterior_summary()

               Estimate   Est.Error       Q2.5      Q97.5
id          2000.500000 1154.844867 100.975000 3900.02500
b_sexwoman1  -17.978276    1.528503 -20.929719  -14.97290
b_sexwoman2  -20.062139    1.562302 -23.276515  -17.10929
influence     -2.083863    2.170942  -6.401877    2.04397

This shows that the results of the two models differ. Stronger prior makes women 2 kilograms lighter but the relevant CIā€™s (for ā€œinfluenceā€) does not have a good interpretation?

2 Likes
2

I have a few nonexhaustive suggestions:

a) Is it possible to plot all four distributions (weak_prior, weak_posterior, strong_prior, strong_posterior) on the same graph?
b) I would separate things out into: (i) mean difference between posterior draws and (ii) width of the posterior credible intervals.
c) A third thing you can compute is the reduction in standard deviation between prior and posterior:

1 - \frac{\sigma_{\textrm{posterior}}}{\sigma_{\textrm{prior}}},

for both pairs of prior/posterior. See Towards A Principled Bayesian Workflow.

3 Likes
3

I agree with @maxbiostat that potentially just plotting the distributions on the same graph can be very convincing (provided that they do indeed overlap!).

Another consideration which could be added to a plot in which they are overlaid is to actually compute the degree of overlap between the different posterior distributions, using the R Package ā€˜overlappingā€™:

1 Like
4

Some great suggestions here already. If calculating the percent overlap between the prior and posterior (an indicator of how much the data inform the posterior beyond what the prior is doing) is something thatā€™s of interest, R package MCMCvis does this (using the previously mentioned overlap package) with the MCMCtrace function. Vignette here.