Problem with categorical response model

I’m trying to run this code:

bform1 ← bf(y1 | subset(sub1) ~ pred1) + categorical() +
bf(y2 | subset(sub2) ~ pred2) + categorical() + set_rescor(F)

m1 ← brm(bform1,
prior = prior(normal(0, 5), class = “b”),
data = d2,
warmup = 1000,
iter = 3500,
chains = 4,
cores = 4,
file = “m1_brms”)

and then I get this error message:

Error: At least two response categories are required.
In addition: Warning messages:
1: Rows containing NAs were excluded from the model.
2: In max(out$Y) : no non-missing arguments to max; returning -Inf

Both y1 and y2 are copies of the same integer-valued variable y (I was not allowed to use the same response variable twice, even though I use it on different subsets). On both subsets sub1 and sub2 they take values in {1,2,3}, so it seems that there are at least two response categories.

What I am doing wrong?

Please also provide the following information in addition to your question:

  • Operating System: Ubuntu 18.04
  • brms Version: 2.7.3
1 Like

Can you provide minimal reproducible example for me to try out?

It’s hard to provide a minimal reproducible example, so here you can find the data plus the script:

Thanks!

The problem is that brms will still omits all rows with NAs in variables used anywhere in the model.
In other words, your predictor variables should not be NA even for observations not actually used for the sub-model. I will add an informative error message for this.

Aha, thank you very much! That’s easy to fix then.

I managed to fit the models, but now I am having trouble computing waic/loo for them.

Here’s the code again:

d2$y1 ← d2$y
d2$y2 ← d2$y
d2$pred1 ← d2$species:d2$infected:d2$test
d2$pred2 ← d2$pred1:d2$y_prev:d2$time
d2$sub1 ← ifelse(d2$time==“2”,1,0)
d2$sub2 ← ifelse(d2$sub1==0,1,0)

bform1 ← bf(y1 | subset(sub1) ~ pred1) + categorical() +
bf(y2 | subset(sub2) ~ pred2) + categorical() + set_rescor(F)

m1 ← brm(bform1,
prior = prior(normal(0, 5), class = “b”),
data = d2,
warmup = 1000,
iter = 3500,
chains = 4,
cores = 4,
file = “m1_brms”)

If I now run WAIC(m1) I get this error message:

Error: Argument ‘resp’ must be a single variable name for models using addition argument ‘subset’.

From elsewhere in the forum I understood that it should not be problematic to compute waic/loo for multivariate models.

The error message is pretty informative on what is the problem. For models including subsets, you need to compute loo/waic per response variable and can’t do it jointly (because there are not based on the same observations).

Thanks for the swift response! I would have thought that a “joint” loo/waic could be computed anyway by simply adding the log-likelihoods of the 2 submodels. If not, how would I compute loo/waic per response from the multivariate model, without having to run separate models again?

WAIC and LOO are not just about adding log-likelihoods, but work on a pointwise manner.

Error: Argument ‘resp’ must be a single variable name for models using addition argument ‘subset’.

The error is pretty specific on what you need to do to get univariate predictions out of a multivariate model.

Thank you. Yes, I could figure out what to do from the error message and consulting brms.pdf, so there was no need for me to take up your time. Sorry!

Having looked up the definitions again (in Gelman, Hwang & Vehtari 2013), waics are just point-wise sums of log-“average-likelihoods”, where the averages are taken over the posterior. Then an “effective number of parameters” is subtracted and the whole thing is multiplied by -2.

It then seems reasonable to me to simply add the waics of the two sub-models and call the result the “joint” waic and use that to compare multivariate models with different sets of predictors (but the same y-values). Wouldn’t you agree?

If you are careful with adding the right things, it can be meaningful, but I don’t want to automate this in brms (yet). You will have to work on the (pointwise) results returned by loo/waic etc. yourself to get the desired results.

1 Like

Will do. Thanks again for your 24/7 availability to help!