Calculate loo_R2 with Cox family in brms package in R

Interfaces brms
1

Hello Stan community,

I’ve been using the package brms in R to estimate a survival model, I took the following exmaple from another thread:

library(simsurv)
library(survival)
set.seed(1234)

# simulated data from the rstanarm::stan_surv example
covs <- data.frame(id  = 1:200, trt = stats::rbinom(200, 1L, 0.5))

d1 <- simsurv(lambdas = 0.1,
              gammas  = 1.5,
              betas   = c(trt = -0.5),
              x       = covs,
              maxt    = 5)

d1 <- merge(d1, covs)

fit_brm <- brm(eventtime | cens(1 - status) ~ 1 + trt,
               data = d1, family = brmsfamily("cox"))

This example works perfectly, but I want also to estimate the out of sample R^2 with the function loo_R2. The problem is that this function cannot handle the cox family yet. Is there any way to estimate the out of sample R^2 using the Cox family in brms? Any idea or recommendation will be appreciated

Thanks.

2

Not yet. This still needs to be figured out.

1 Like
3

Paul, is this even recommended practice, i.e., use R^2 for anything but Gaussian likelihoods?

4

It is a generalised R^2 so yes it can be used outside of gaussian (but not for categorical families for example).

5

Interesting, thanks. I need to read the Gelman et al. paper.

6

To be honest, even though I’m one of the authors of that paper, I still don’t love the idea of using R^2 for non-Gaussian models. Then again, I don’t really like using it for Gaussian models either ;)

Since we published that paper a lot of people seem to be using bayes_R2 and loo_R2 (the latter definitely has fewer problems), but I hope they keep in mind what we say in the discussion section, which is that R^2

has well-known problems as a measure of model fit, but it can be a handy quick summary for linear regressions and generalized linear models

Basically we wanted to provide a way for people who insist on using R^2 to have a Bayesian option, but we’re not recommending people put too much faith in (Bayesian or any version of) R^2. I pretty much never look at it myself. By “handy quick summary” we mean more or less something that may be useful to glance at when fitting models but not really something to put too much emphasis on in publications.

2 Likes
7

Thanks Paul for your answer.

8

Thanks Jonah for your opinion. I agree people insist on using any Bayesian R^2 especially reviewers. I’ll pay attention to the discussion in your article.

1 Like