# Why is posterior\_linpred so slow?

**URL:** <https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550>\
**Category:** brms\
**Tags:** techniques\
**Created:** [December 2, 2021, 5:44pm UTC](https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550 "2021-12-02T17:44:44Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Maxime](https://avatars.discourse-cdn.com/v4/letter/m/a9adbd/32.png) [@Maxime](https://discourse.mc-stan.org/u/Maxime)\
**Post date:** [December 2, 2021, 5:44pm UTC](https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550/1 "2021-12-02T17:44:44Z")

</div>

Hi,

I fitted this model with brms on a large dataset (70 000 observations) :

```
model_formula <- brmsformula(hunting_success | trials(4) ~
                                        Zspeed +
                                        Zspace_covered_rate +
                                        Zprox_mid_PreyGuarding +
                                        Zhook_start_time +
                                        Zgame_duration +
                                        (1 | map_name) +
                                        (1 | player_id) +
                                        (1 | obs))

```

```
base_model <- brm(formula = model_formula,
                  family = binomial(link = "logit"),
                  warmup = 3000, 
                  iter = 11000,
                  thin = 32,
                  chains = 4, 
                  inits = "0", 
                  threads = threading(10),
                  backend = "cmdstanr",
                  seed = 123,
                  prior = priors,
                  control = list(adapt_delta = 0.95),
                  save_pars = save_pars(all = TRUE),
                  sample_prior = TRUE,
                  data = data)

```

The size of the output is ~1.5 Go. I want to extract the predicted values on the response scale to compute the linear trends for each fixed effects. However, I don’t understand why extracting the draws takes so much time.

For instance I ran this to have the predicted values of the model, but it’s been running for +20 minutes and it is only 30 draws:

```
draws <- posterior_linpred(base_model,
                            re_formula = NA,
                            transform = FALSE,
                            ndraws = 30,
                            seed = 123)

```

Am I doing something wrong? Is it because of the overdispersion term? I didn’t have this problem before with an earlier version of brms so I don’t understand what might be going on. Is there a way to extract the predicted values in a faster way so I can easily manipulate them myself?

This is my computer setup :  
R version 4.0.4 (2021-02-15)  
Platform: x86\_64-w64-mingw32/x64 (64-bit)  
Running under: Windows 10 x64 (build 19042)  
brms\_2.16.1, Rcpp\_1.0.7

Thank you very much for your help!

Maxime

---

<div class="post-metadata">

**Author:** ![martinmodrak](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/martinmodrak/32/133_2.png) [@martinmodrak](https://discourse.mc-stan.org/u/martinmodrak)\
**Post date:** [December 16, 2021, 9:47am UTC](https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550/2 "2021-12-16T09:47:53Z")

</div>

This might be related to some recent changes in brms that now delegates more stuff to the `posterior` package. Tagging @paul.buerkner who previously suggested that maybe the relevant code in posterior is slower than we thought.

You can always use the `$fit` member to access the underlying Stan fit and do whatever summaries you need directly.

Could you please confirm that reverting to an older version of brms improves the performance?

---

<div class="post-metadata">

**Author:** ![paul.buerkner](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/paul.buerkner/32/3303_2.png) [@paul.buerkner](https://discourse.mc-stan.org/u/paul.buerkner)\
**Post date:** [December 17, 2021, 10:15pm UTC](https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550/4 "2021-12-17T22:15:47Z")

</div>

(This post became obsolete after the post before was deleted)

---

<div class="post-metadata">

**Author:** ![patrick-eng](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/patrick-eng/32/13327_2.png) [@patrick-eng](https://discourse.mc-stan.org/u/patrick-eng)\
**Post date:** [December 17, 2021, 10:45pm UTC](https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550/5 "2021-12-17T22:45:40Z")

</div>

Sorry, Paul!

---

<div class="post-metadata">

**Author:** ![patrick-eng](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/patrick-eng/32/13327_2.png) [@patrick-eng](https://discourse.mc-stan.org/u/patrick-eng)\
**Post date:** [December 17, 2021, 10:56pm UTC](https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550/6 "2021-12-17T22:56:29Z")

</div>

(Posted something before about a possible bug before realising I had a typo in my code!)

I have been having the same issue after updating brms yesterday.

After reading this thread, I ran **posterior\_epred** on the same model using three different package versions:

- 2.16.3 : 13 minutes 37 seconds
- 2.15.0 : 11 minutes 57 seconds
- 2.14.0 : 11 minutes 49 seconds

Model has an n of about 9000, and I’m predicting over a pretty hefty post-stratification frame.

Code:

```
pp <- posterior_epred(test.mod,
                       newdata=ps,
                       allow_new_levels=TRUE,
                       cores=6,
                       ndraws = 20,
                       seed=12345)

```

Scale that up and the differences start getting quick fast quite quickly, I guess?

---

<div class="post-metadata">

**Author:** ![Maxime](https://avatars.discourse-cdn.com/v4/letter/m/a9adbd/32.png) [@Maxime](https://discourse.mc-stan.org/u/Maxime)\
**Post date:** [December 20, 2021, 3:25pm UTC](https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550/7 "2021-12-20T15:25:41Z")

</div>

Thank you @patrick-eng for providing a reproducible example I did not have the time in the last days! And thank you @martinmodrak for the suggestion of using $fit, I did not know that we could do that.

Maxime

---

<div class="post-metadata">

**Author:** ![paul.buerkner](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/paul.buerkner/32/3303_2.png) [@paul.buerkner](https://discourse.mc-stan.org/u/paul.buerkner)\
**Post date:** [December 20, 2021, 4:00pm UTC](https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550/8 "2021-12-20T16:00:32Z")

</div>

Yeah, I assume this is because the extraction via posterior is not the quickest yet apparently. Do things get quicker if you install the latest dev version of posterior from github (`stan-dev/posterior`)?

---

<div class="post-metadata">

**Author:** ![patrick-eng](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/patrick-eng/32/13327_2.png) [@patrick-eng](https://discourse.mc-stan.org/u/patrick-eng)\
**Post date:** [January 5, 2022, 5:34pm UTC](https://discourse.mc-stan.org/t/why-is-posterior-linpred-so-slow/25550/9 "2022-01-05T17:34:15Z")

</div>

That does appear to help Paul, yes! :-)