Conditional_effects() producing different Expected Values than fitted()

Interfaces brms
1

I’m debating whether to plot predictor effects manually or with conditional_effects(), and I’ve made the disturbing discovery that fitted.brmsfit(), which I use to calculate predictor effects manually, produces different expected values than conditional_effects() on the default setting of method = "fitted". It is disturbing because now I don’t know which expected values are correct. It’s a categorical model. Here’s how to reproduce the phenomenon with simple categorical data:

data(iris)

Cores = 4

model <- brm(Species ~ ., family = categorical, seed = 2023,
  prior = prior(normal(0, 1)), data = iris, backend = "cmdstanr", 
  cores = Cores, chains = Cores/2, threads = threading(Cores/2))

NewData <- as.data.frame(
  lapply(lapply(iris[,names(iris) != "Species"], mean), rep, 2))

Predictor <- "Sepal.Length"
NewData[,Predictor] <- range(iris[,Predictor])

Fitted <- fitted(model, newdata = NewData)[,1,]

CondEff <- conditional_effects(model, Predictor, conditions = NewData[1,], categorical = T)

Fitted.CondEff <- with(CondEff[[1]], sapply(levels(cats__), function(x) estimate__[cats__ == x][c(1, 100)]))

list(Fitted, Fitted.CondEff)

[[1]]
     P(Y = setosa) P(Y = versicolor) P(Y = virginica)
[1,]    0.33536731         0.6051630       0.05946970
[2,]    0.04619066         0.9315982       0.02221116

[[2]]
         setosa versicolor   virginica
[1,] 0.30030614  0.6249712 0.037091023
[2,] 0.01700964  0.9663580 0.008161822

Hence, the mind boggles. Which set of expected values can I trust?

2

Just a quick thought:
My understanding is that fitted gives you model predictions, given input data, while conditional_effects gives you conditional or marginal effects. The difference is that for the conditional effects all parameters not part of the condition you investigate are set to their mean/reference category.

Besides that, the docu of conditional_effects’ method parameter reads

Method used to obtain predictions. Can be set to "posterior_epred" (the default), "posterior_predict" , or "posterior_linpred" . For more details, see the respective function documentations.

Such that “fitted” doesn’t seem to be a valid option.

I don’t think they are supposed to do the same thing.

ps. Your code doesn’t run:

Error: The following priors do not correspond to any model parameter:
b ~ normal(0, 1)

I also think you can’t use threading and multiple cores per chain simultaneously.

3

just a quick addition: fitted is an outdated name for posterior_epred actually. however conditional effects uses robust = TRUE so computes the posterior median as point summary by default instead of the posterior mean.

2 Likes
4

Oh! It was just because of mean vs median. Now everything replicates again. Thanks!

5

Afterthought: A major source of my confusion was that when I Google conditional_effects(), the first search result (which I mistook for accurate) was this outdated helpfile for the function. In that version, method = "fitted" is specified as the alternative to “predict”…

Not sure why the replication code wouldn’t work though. I’m using brms 2.19.0 with cmdstan 2.32.0 and cmrstanr 0.5.3, and everything runs without issues.