Diagnostics on predictions

Modeling
, ,
1

Suppose I fit a model in brms that includes a random intercept (group-specific intercept). What I care about is the group-specific intercepts (\alpha_0 + \alpha_j), which I can get through posterior_epred(). Is there an easy way to calculate diagnostics for these predictions, such as Rhats and ESS?

In general, are there a functions to calculate Rhats and ESS for any quantity, if you have the draws (separated into chains)?

Getting rhats for posterior predictions, using tidybayes
2

The posterior package has functions for that.

3

Perfect, thanks.

1 Like
4

Following up on this:

What is the easiest way to calculate the group-specific intercepts (\alpha_0 + \alpha_j) and get the posteriors separated by chains? The two easy ways that I know of (posterior_epred() and coef()) don’t separate the posterior into chains. This is needed for the rhat() and ess() functions from posterior. I could always use rstan::extract() on the rstan object, but that’s not very convenient.

5

You may have to either manually compute them with as_draws_df() or you might look into some of the functions in the tidybayes package.

6

Thanks for pointing me in the right direction. The following code worked, using the rvar data type:

fit_rvars <- as_draws_rvars(fit)
a0 <- fit_rvars$b_Intercept
aj <- fit_rvars$r_school_id[,1]
a0j <- a0 + aj

rhats <- posterior::rhat(a0j)[,1]
bess  <- posterior::ess_bulk(a0j)[,1]
tess  <- posterior::ess_tail(a0j)[,1]
1 Like