# Rstanarm survival - random effects

**URL:** <https://discourse.mc-stan.org/t/rstanarm-survival-random-effects/22662>\
**Category:** rstanarm\
**Created:** [May 23, 2021, 9:27pm UTC](https://discourse.mc-stan.org/t/rstanarm-survival-random-effects/22662 "2021-05-23T21:27:55Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sbhuey](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sbhuey/32/7759_2.png) [@sbhuey](https://discourse.mc-stan.org/u/sbhuey)\
**Post date:** [May 23, 2021, 9:27pm UTC](https://discourse.mc-stan.org/t/rstanarm-survival-random-effects/22662/1 "2021-05-23T21:27:55Z")

</div>

The survival branch of rstanarm is excellent and the Brilleman et al paper has really accelerated my ability and understanding of survival models, so thanks to all who worked so hard on this branch.

My survival model is estimating a right censored response based on several fixed effects and a single random effect. According to the Brilleman paper, the `ranef()` function should return the random effects from the model object, but when I try to do this I receive: `Error: This method is for stan_glmer and stan_lmer models only.`

One other question I have pertains to using `posterior_survfit()` with newdata that contains a random effect level that wasn’t present in the training set. I know in lme4 the `predict()` function has `allow.new.levels` which if set to TRUE can handle this case. Is there an analogous argument that can be passed, or another way to handle? Thanks!

- Operating System: ubuntu 18.04
- rstanarm Version: feature/survival

---

<div class="post-metadata">

**Author:** ![sambrilleman](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sambrilleman/32/1703_2.png) [@sambrilleman](https://discourse.mc-stan.org/u/sambrilleman)\
**Post date:** [May 24, 2021, 2:46am UTC](https://discourse.mc-stan.org/t/rstanarm-survival-random-effects/22662/2 "2021-05-24T02:46:24Z")

</div>

> [@sbhuey](#):
>
> According to the Brilleman paper, the `ranef()` function should return the random effects from the model object, but when I try to do this I receive: `Error: This method is for stan_glmer and stan_lmer models only.`

Yeah I think this was a minor bug that should hopefully have been fixed on a [recent PR](https://github.com/stan-dev/rstanarm/pull/526).

But we would need to rebuild the hosted binary for you to be able to easily install the version with that fix.

@rok_cesnovar @jonah … do you know if it is easy for us to rebuild and host an updated binary for the feature/survival branch? Or is it a manual task?

> [@sbhuey](#):
>
> I know in lme4 the `predict()` function has `allow.new.levels` which if set to TRUE can handle this case. Is there an analogous argument that can be passed, or another way to handle?

I don’t think you need to specify an additional argument in posterior\_survfit. Rather, I think you just need to make sure the prediction data (i.e. newdata) uses group id values that weren’t present in the original estimation data. For example you could call your new IDs / groups something like `new_1`, `new_2` etc in the newdata. Then the predictions for each new group should be marginalised over the dist of random effects. That behaviour should be similar to what is described for stan\_jm’s posterior\_survfit method in the `?posterior_survfit` docs I think.

---

<div class="post-metadata">

**Author:** ![sbhuey](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sbhuey/32/7759_2.png) [@sbhuey](https://discourse.mc-stan.org/u/sbhuey)\
**Post date:** [May 24, 2021, 3:24pm UTC](https://discourse.mc-stan.org/t/rstanarm-survival-random-effects/22662/3 "2021-05-24T15:24:22Z")

</div>

Thanks for the info on the first issue.

On the second issue, I initially tried to pass data with a new level (1248 in agent\_id) but the function threw this error. The warning is also confusing because I explicitly convert agent\_id to a factor in the training and prediction sets:

````
> posterior_survfit(fit, newdata=pred_data)
Error in model.frame.default(Terms, data, xlev = xlevs, drop.unused.levels = drop.unused.levels, : 
  factor agent_id has new level 1248
In addition: Warning messages:
1: In model.frame.default(Terms, data, xlev = xlevs, drop.unused.levels = drop.unused.levels, :
  variable 'agent_id' is not a factor```
````

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [May 24, 2021, 7:59pm UTC](https://discourse.mc-stan.org/t/rstanarm-survival-random-effects/22662/4 "2021-05-24T19:59:17Z")

</div>

> [@sambrilleman](#):
>
> @rok_cesnovar @jonah … do you know if it is easy for us to rebuild and host an updated binary for the feature/survival branch? Or is it a manual task?

Should be doable. Will give it a try this week.

---

<div class="post-metadata">

**Author:** ![sambrilleman](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sambrilleman/32/1703_2.png) [@sambrilleman](https://discourse.mc-stan.org/u/sambrilleman)\
**Post date:** [June 11, 2021, 3:09am UTC](https://discourse.mc-stan.org/t/rstanarm-survival-random-effects/22662/5 "2021-06-11T03:09:13Z")

</div>

> [@sbhuey](#):
>
> On the second issue, I initially tried to pass data with a new level (1248 in agent\_id) but the function threw this error. The warning is also confusing because I explicitly convert agent\_id to a factor in the training and prediction sets:
> 
> ````no-highlight
> > posterior_survfit(fit, newdata=pred_data)
> Error in model.frame.default(Terms, data, xlev = xlevs, drop.unused.levels = drop.unused.levels, : 
> factor agent_id has new level 1248
> In addition: Warning messages:
> 1: In model.frame.default(Terms, data, xlev = xlevs, drop.unused.levels = drop.unused.levels, :
> variable 'agent_id' is not a factor```
> 
> ````

Yeah this sounds buggy. Also, sorry that I’m only getting to reply now!

It seems that there are some issues around handling the grouping variable when it was a `factor` data type in the training fitting. I have got tests [here](https://github.com/stan-dev/rstanarm/blob/dab004fa4b5b18f496f8d3f7dc7936375348b79e/tests/testthat/test_stan_surv.R#L315) that check `posterior_survfit` but they only test it with numeric data types for the grouping variable, and not factors.

After a bit of digging I’ve actually found a related [Github issue](https://github.com/stan-dev/rstanarm/issues/457), that discussed the same problem for `character` variables. I’ve updated it to also include `factor` variables in the example.

Unfortunately it is from August 2020 and still hasn’t been fixed!! :-(

The quick solution seems to be to use `numeric` data type for your agent\_id variable. Sorry that I wasn’t able to reply with that quick fix earlier for you! (hopefully you managed to find it)

Longer term, I hope we can fix up the handling of factors/characters. But not sure when that work will get done.

---

<div class="post-metadata">

**Author:** ![sbhuey](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/sbhuey/32/7759_2.png) [@sbhuey](https://discourse.mc-stan.org/u/sbhuey)\
**Post date:** [June 13, 2021, 3:40am UTC](https://discourse.mc-stan.org/t/rstanarm-survival-random-effects/22662/6 "2021-06-13T03:40:52Z")

</div>

Thanks Sam, converting to numeric worked. Appreciate the help!