I am trying to understand the bayes_R2 function to estimate the proportion of deviance explained for new data and I have been through the information from Gelman et al…
My concern was raised when analyzing my own data. I will illustrate my issue, using the model suggested in the vignette from Paul Bürkner on custom response distributions with the study case using the cbpp data (from the lme4 package).
The model that he suggests in a first step is the following:
fit1 <- brm(incidence | trials(size) ~ period + (1|herd), data = cbpp, family = binomial())
To play around with the data, I wanted to perform the same model using bernouilli as the probability distribution instead of the Binomial distribution. I therefore performed a similar model but on a transformed version of the initial dataset in a way that each row is one individual observed:
fit2 <- brm(incidence ~ period + (1|herd), data = cbpp_individual, family = bernouilli())
the 2 models give the same estimates and errors on the parameters, which is obviously what I would have expected. However, the bayes_R2 give completely different results for fit1=0.64 and fit2=0.10.
I cannot figure out what may explains this gap. I would be delighted to be enlightened on this point.