# Manually computing R^2 and receiving negative output

**URL:** <https://discourse.mc-stan.org/t/manually-computing-r-2-and-receiving-negative-output/17404>\
**Category:** brms\
**Created:** [August 12, 2020, 5:54pm UTC](https://discourse.mc-stan.org/t/manually-computing-r-2-and-receiving-negative-output/17404 "2020-08-12T17:54:04Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![htrier](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/htrier/32/9397_2.png) [@htrier](https://discourse.mc-stan.org/u/htrier)\
**Post date:** [August 12, 2020, 5:54pm UTC](https://discourse.mc-stan.org/t/manually-computing-r-2-and-receiving-negative-output/17404/1 "2020-08-12T17:54:04Z")

</div>

Hello, I am using brms.predict on a brms model to get a set of predicted y values using the model weights. I then want to see how much variation in the original y labels can be explained by the predicted y labels. However, I can’t use the Bayes\_R2 function because the output of brms.predict is a vector of estimates, not a brmsfit object. Therefore I am calculating the coefficient of determination manually in the following way:

```
SSresidual <- sum((originalY - predictedY)^2,na.rm=TRUE)
SStotal <- sum((originalY - mean(originalY, na.rm=TRUE))^2,na.rm=TRUE)
my_R2 <- 1 - (SSresidual/SStotal)

```

My next step is to use a bootstrapping approach to see how accurate my prediction is compared to noise. I compute a series of R^2 calculated in the same way as above but with randomly permuted Y labels as the `predictedY`.

The problem is, when I randomly permute the original Y labels to form a ‘prediction’ and manually calculate R^2 as shown above, I often get negative R^2 values. I am wondering whether the above approach is sensible, and if so why my approach yields negative R^2 and what to do about it. Thanks, I appreciate any advice.

---

<div class="post-metadata">

**Author:** ![matti](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/matti/32/13631_2.png) [@matti](https://discourse.mc-stan.org/u/matti)\
**Post date:** [August 13, 2020, 8:12am UTC](https://discourse.mc-stan.org/t/manually-computing-r-2-and-receiving-negative-output/17404/2 "2020-08-13T08:12:02Z")

</div>

> [@htrier](#):
>
> However, I can’t use the Bayes\_R2 function because the output of brms.predict is a vector of estimates, not a brmsfit object.

Why don’t you use the function on your model fit object? `bayes_R2(your_model)`

---

<div class="post-metadata">

**Author:** ![htrier](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/htrier/32/9397_2.png) [@htrier](https://discourse.mc-stan.org/u/htrier)\
**Post date:** [August 13, 2020, 10:00am UTC](https://discourse.mc-stan.org/t/manually-computing-r-2-and-receiving-negative-output/17404/3 "2020-08-13T10:00:44Z")

</div>

My understanding of the bayes\_R2 function is that it outputs the proportion of variation of `originalY` that is explained by the model predictors in the brmsfit object – however, what I want to compute is the proportion of variation in `originalY` that is explained by `predictedY`, which is not part of a brmsfit object. So I’ve computed R^2 manually, but I’m not sure why the output is sometimes negative.

---

<div class="post-metadata">

**Author:** ![matti](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/matti/32/13631_2.png) [@matti](https://discourse.mc-stan.org/u/matti)\
**Post date:** [August 13, 2020, 1:16pm UTC](https://discourse.mc-stan.org/t/manually-computing-r-2-and-receiving-negative-output/17404/4 "2020-08-13T13:16:51Z")

</div>

I see, sorry for my misunderstanding.

---

<div class="post-metadata">

**Author:** ![Zhanglong](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/zhanglong/32/7539_2.png) [@Zhanglong](https://discourse.mc-stan.org/u/Zhanglong)\
**Post date:** [August 14, 2020, 6:10am UTC](https://discourse.mc-stan.org/t/manually-computing-r-2-and-receiving-negative-output/17404/5 "2020-08-14T06:10:59Z")

</div>

I see. You are using the formula R^2=1-\frac{SS\_{res}}{SS\_{total}} which is the classical way to calculate R^2 for regression problem. However, `brms` uses Bayesian R^2=\frac{ var(y^{prd})}{var(y^{prd})+var(res)}

> var(predictedY) / ( var(predictedY)+ var(originalY - predictedY))

Reference: [R-squared for Bayesian Regression Models](https://doi.org/10.1080/00031305.2018.1549100)

---

<div class="post-metadata">

**Author:** ![htrier](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/htrier/32/9397_2.png) [@htrier](https://discourse.mc-stan.org/u/htrier)\
**Post date:** [August 14, 2020, 1:12pm UTC](https://discourse.mc-stan.org/t/manually-computing-r-2-and-receiving-negative-output/17404/6 "2020-08-14T13:12:01Z")

</div>

Thank you for your quick response and reference - I have re-run the script using the Bayesian R^2 equation and I no longer get negative R^2 values. Much appreciated!