Hello, I am using brms.predict on a brms model to get a set of predicted y values using the model weights. I then want to see how much variation in the original y labels can be explained by the predicted y labels. However, I can’t use the Bayes_R2 function because the output of brms.predict is a vector of estimates, not a brmsfit object. Therefore I am calculating the coefficient of determination manually in the following way:
SSresidual <- sum((originalY - predictedY)^2,na.rm=TRUE) SStotal <- sum((originalY - mean(originalY, na.rm=TRUE))^2,na.rm=TRUE) my_R2 <- 1 - (SSresidual/SStotal)
My next step is to use a bootstrapping approach to see how accurate my prediction is compared to noise. I compute a series of R^2 calculated in the same way as above but with randomly permuted Y labels as the
The problem is, when I randomly permute the original Y labels to form a ‘prediction’ and manually calculate R^2 as shown above, I often get negative R^2 values. I am wondering whether the above approach is sensible, and if so why my approach yields negative R^2 and what to do about it. Thanks, I appreciate any advice.