Predict error with nested random effect structure

Interfaces brms
1

Hi,

I am having issues using the predict.brmsfit function when I have new random effect levels. I have a nested random effect structure of (1|Region/Species). From the R documentation, I have tried setting allow_new_levels=TRUE as well as re_formula=NA however, in both cases I get the following error:
Error: New factor levels are not allowed. followed by a list of all of contained species incorporated in the analysis.

I have tried to troubleshoot this and I have found if I remove the nestedness and re-fit the model with only (1|Species) setting either allow_new_levels = TRUE or re_formula=NA results in a normal predict output.

Is there something else I need to specify in order for this to work with my nested structure?

Thanks in advance - Liam

Please also provide the following information in addition to your question:

  • Operating System: Mac OS High Sierra
  • brms Version: 2.31
2

Strange, I would expect allow_new_levels = TRUE. Let me see if I can reproduce this problem.

3

Thanks, if not I will make a reproducible example and post it on here

4

If it is not too much work, I would prefer a reprex from you, of course. :-)

5

no problem, iā€™m on it

6

Here you go - i fitted the model with only a subset and then tried to use predict using the entire data frame

df.csv (2.1 KB)

forum_example.R (205 Bytes)

1 Like
7

Thanks! It should be fixed now in the dev version of brms. You can install it via

if (!requireNamespace("devtools")) {
  install.packages("devtools")
}
devtools::install_github("paul-buerkner/brms")
2 Likes
8

fantastic, thank you! it works perfectly

9

I just downloaded the dev version, but Iā€™m still getting the same issue. For what itā€™s worth, the help page for predict.brmsfit also no longer mentions the allow_new_levels argument or the sample_new_levels argument. Are those arguments no longer supported?

  • Operating System: Mac OS High Sierra
  • brms Version: 2.7.1
10

Can you provide a reproducible example?

The arguments you are mentioning are now documented under ?extract_draws.brmsfit but still function as before.

11

Here is a minimally modified example based on liamkendallā€™s example in this thread (using the same data set). It seems what breaks things is the use of random effects specified as in mgcv (which I need because I want to specify random slopes for a smooth)

#read df
df=read.csv("df.csv",header=T)

library(brms)
library(dplyr)
#Create model with subset
mod1=brm(Spec.wgt~s(IT)+s(Genus, bs = "re"),df[1:10,])
mod2=brm(Spec.wgt~s(IT)+(1 | Genus),df[1:10,])

#try to predict with full df. The first one does not work, the second works.
predict(mod1,newdata=df,
        allow_new_levels = TRUE, 
        sample_new_levels = "uncertainty")

predict(mod2,newdata=df,
        allow_new_levels = TRUE, 
        sample_new_levels = "uncertainty")

df.csv (2.1 KB)

12

Thanks for this example!

brms does not offer the same amount of support for random effects via splines as for ā€œstandardā€ random effects. I can confirm, the first wonā€™t allow new levels, but I donā€™t see this can be fixed without considerable amount of special case coding. I donā€™t think itā€™s worth it to be honest, since brms already offers such a flexibel multilevel syntax that one does not need to specify random effects via splines.

13

Yes, that makes sense. When I originally posted the question, I hadnā€™t noticed that the problem was the random effect smooths. The only functionality that is lost this way are random slopes (incl. factor smooths) for non-linear effects, for whichā€“I thinkā€“thereā€™s not alternative in brms?

14

Yes indeed, non-linear random smooths are possibly the only thing brms currently can predict values for new levels.

15

Hi Paul,

Iā€™m surfing through the forum looking for a thread I had seen before about random effects in mgcv vs. brms and ran across this one. Quick question regarding your comment above:

If fitting a multilevel model such as: response ~ s(var1) + (1 | group1) , do I also need to specify the random effect in the spline, so that it would read: response ~ s(var1, by= group1) + (1 | group1)? Iā€™m assuming from the comment above that the answer is no. If this is the case, when does one use the argument ā€œby=ā€ in a brms spline (if ever)?

Thank you!
Denise Colombano

16

ā€œNeedā€ is perhaps not the right question. This will depend on how you want to model your data. Do you expect the spline to vary over groups? If yes, it can be sensible to add a s(var1, by = group1) term, but be aware that this is not a ā€œrandomā€ spline in the sense that there is no average spline and no shrinkage of the indiviudal splines will happen. instead, itā€™s just one spline per level of group1. You can have ā€˜actualā€™ random splines via the bs = "fs" type. For more details, see ?mgcv::s

As brmsā€™ splines are based on mgcv, most of what mgcv supports works with brms as well.

2 Likes