Calculate bayes_R2 for each group in a multilevel model?

Interfaces brms
1

Hi,

I’m using the bayes_R2() command. I have a non-nested multi-level model, looking at firms, within 15 countries across 25 years: (...| country:year).

Can I calculate a bayes_R2() for each year? In a similar way to how the pp_check() function has a group option, so that I can look at the posterior predictive distribution of a test statistic (mean or median say) for every year in my model, e.g:

pp_check (fit, type = “stat_grouped”, stat = “median”, group =“year”, nsamples = 250)

In doing something equivalent to this for the bayes_R2(), the aim is to provide my academic reader with a simple yet accurate measure of how well my model ‘explains’ the data (my y variable) for each year, in a table form.

Best wishes,

P.S. The loo_R2 calculation is seriously dragging down my upgraded Macbook pro processor – unlike anything else I have done to this model fit in brms() etc. Calculates something after about 10-15 minutes. Model is run on a large dataset (around 100,000 points). And gives error with doing the loo_R2 “Warning message: Some Pareto k diagnostic values are too high. See help(‘pareto-k-diagnostic’) for details.” So maybe due to that?

2

You can supply year specific data via argument newdata.

3

Thanks for this. Trying to understand the answer: how will this create year specific predictions though – from which the variance for the R^2_{Bayesian} formulae is calculated? Like in a pp_check by group (year)?

4

You subset the data per year and then run bayes_R2 separately for each year.

5

Thanks!

6

100,000 is a lot and thus 10-15 mins is not that much. The speed depends also how many cores you are using. Pareto k warning is not an error and doesn’t affect the speed. See the links in the other thread you started. We’ll soon have sub-sampling LOO implemented which an provide faster estimate in case of very large n (Magnusson, Andersen, Jonasson, Vehtari (2019)).

1 Like