Conditional_effects() plots from brms model has a response category with consistent 0 probability (flat plot)

General
, , ,
1

Hello,

I have a multinomial logistic regression model (code below) using brm() with the response variable is six different species. Model converges nicely and the summary output (see below) shows that one of the six species, D. brevicaulis, has a few significant predictors. When calculating average marginal effects with avg_slopes(), there are also clearly a few of the predictors of this species that are also significant.

My model:

model1_07_test3 <- brm(Species ~ 
                   Density_1 +
                   Density_2 +
                   Canopy_Height +
                   Soil_texture +
                   pH
                 + (1 | Fragment)
                 , data = df_scaled_use, 
                 family = categorical(), 
                 iter = 30000,
                 thin=1,
                 save_pars = save_pars(all = TRUE))

Model summary - fixed effect coefficients

> summary(model1_07_test3)
 Family: categorical 
  Links: muCprestonianus = logit; muCpsammophilus = logit; muCsaintelucei = logit; muDbrevicaulis = logit; muDscottiana = logit 
Formula: Species ~ Density_1 + Density_2 + Canopy_Height + Soil_texture + pH + (1 | Fragment) 
   Data: df_scaled_use (Number of observations: 120) 
  Draws: 4 chains, each with iter = 30000; warmup = 15000; thin = 1;
         total post-warmup draws = 60000

Multilevel Hyperparameters:
~Fragment (Number of levels: 5) 
                              Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(muCprestonianus_Intercept)     4.83      2.59     1.72    11.28 1.00    29867    28402
sd(muCpsammophilus_Intercept)     1.18      1.07     0.05     3.92 1.00    22602    29826
sd(muCsaintelucei_Intercept)      0.98      0.85     0.04     3.15 1.00    24456    29463
sd(muDbrevicaulis_Intercept)      4.81      2.85     1.63    12.47 1.00    10697     4711
sd(muDscottiana_Intercept)        1.46      0.97     0.12     3.83 1.00    15594     9180

Regression Coefficients:
                              Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
muCprestonianus_Intercept        -1.21      1.89    -5.34     2.21 1.00    29994    29220
muCpsammophilus_Intercept        -0.24      0.93    -2.25     1.44 1.00    34371    29229
muCsaintelucei_Intercept         -0.17      0.83    -1.91     1.42 1.00    35543    36690
muDbrevicaulis_Intercept         -2.77      2.45    -8.53     1.19 1.00    15582     9309
muDscottiana_Intercept            0.15      0.94    -1.68     2.07 1.00    34874    35114
muCprestonianus_Density_1        -1.75      1.08    -4.06     0.16 1.00    42367    40508
muCprestonianus_Density_2         0.72      0.84    -0.77     2.56 1.00    36104    31582
muCprestonianus_Canopy_Height    -0.45      0.57    -1.55     0.71 1.00    44925    35058
muCprestonianus_Soil_texture     -0.63      0.74    -2.13     0.78 1.00    29699    36874
muCprestonianus_pH                1.96      0.75     0.58     3.53 1.00    35279    33854
muCpsammophilus_Density_1        -1.91      0.77    -3.49    -0.47 1.00    27776    34416
muCpsammophilus_Density_2        -2.69      0.80    -4.33    -1.21 1.00    37758    38340
muCpsammophilus_Canopy_Height    -1.29      0.58    -2.46    -0.20 1.00    38060    43157
muCpsammophilus_Soil_texture      0.24      0.57    -0.86     1.38 1.00    25039    36471
muCpsammophilus_pH                0.28      0.66    -1.00     1.61 1.00    27993    15344
muCsaintelucei_Density_1          0.91      0.60    -0.23     2.12 1.00    33805    39178
muCsaintelucei_Density_2         -1.92      0.63    -3.23    -0.76 1.00    26457    26945
muCsaintelucei_Canopy_Height     -1.02      0.54    -2.12     0.02 1.00    38513    44259
muCsaintelucei_Soil_texture      -1.77      0.61    -3.04    -0.63 1.00    32703    39335
muCsaintelucei_pH                 0.38      0.59    -0.76     1.55 1.00    38464    40821
muDbrevicaulis_Density_1         -1.61      0.74    -3.14    -0.24 1.00    28144    26978
muDbrevicaulis_Density_2         -1.88      0.74    -3.41    -0.52 1.00    40920    42565
muDbrevicaulis_Canopy_Height     -0.85      0.57    -1.96     0.26 1.00    40156    42374
muDbrevicaulis_Soil_texture       1.31      0.65     0.09     2.65 1.00    25644    27220
muDbrevicaulis_pH                 1.70      0.71     0.34     3.14 1.00    35620    39138
muDscottiana_Density_1           -0.85      0.69    -2.25     0.47 1.00    27250    32675
muDscottiana_Density_2           -2.99      0.79    -4.63    -1.56 1.00    30420    36972
muDscottiana_Canopy_Height       -2.72      0.70    -4.17    -1.42 1.00    36353    38737
muDscottiana_Soil_texture         0.57      0.59    -0.58     1.76 1.00    23131    34495
muDscottiana_pH                   1.31      0.63     0.11     2.59 1.00    33264    38125

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

Here are the average marginal effects :

> slopes_all

               Group          Term    Contrast Estimate     2.5 %   97.5 %
 B. madagascariensis Canopy_Height mean(dY/dX)  0.07245  0.025906  0.11968
 B. madagascariensis Density_1     mean(dY/dX)  0.04051 -0.018407  0.10549
 B. madagascariensis Density_2     mean(dY/dX)  0.10778  0.060332  0.15748
 B. madagascariensis pH            mean(dY/dX) -0.06318 -0.124864 -0.00596
 B. madagascariensis Soil_texture  mean(dY/dX)  0.02362 -0.025333  0.07541
 C. prestonianus     Canopy_Height mean(dY/dX)  0.02802 -0.005522  0.06684
 C. prestonianus     Density_1     mean(dY/dX) -0.03808 -0.117040  0.02202
 C. prestonianus     Density_2     mean(dY/dX)  0.08855  0.046807  0.14062
 C. prestonianus     pH            mean(dY/dX)  0.04846  0.006828  0.08902
 C. prestonianus     Soil_texture  mean(dY/dX) -0.02229 -0.066118  0.01676
 C. psammophilus     Canopy_Height mean(dY/dX)  0.01546 -0.066832  0.08721
 C. psammophilus     Density_1     mean(dY/dX) -0.11048 -0.203426 -0.02040
 C. psammophilus     Density_2     mean(dY/dX) -0.07297 -0.193551  0.03208
 C. psammophilus     pH            mean(dY/dX) -0.08623 -0.175233 -0.00118
 C. psammophilus     Soil_texture  mean(dY/dX)  0.00278 -0.049020  0.05764
 C. saintelucei      Canopy_Height mean(dY/dX)  0.00399 -0.060661  0.06840
 C. saintelucei      Density_1     mean(dY/dX)  0.13566  0.064729  0.21109
 C. saintelucei      Density_2     mean(dY/dX) -0.02852 -0.093124  0.03062
 C. saintelucei      pH            mean(dY/dX) -0.02405 -0.097013  0.04178
 C. saintelucei      Soil_texture  mean(dY/dX) -0.14656 -0.207669 -0.08945
 D. brevicaulis      Canopy_Height mean(dY/dX)  0.03213 -0.030350  0.09041
 D. brevicaulis      Density_1     mean(dY/dX) -0.04760 -0.115118  0.02070
 D. brevicaulis      Density_2     mean(dY/dX)  0.01340 -0.069827  0.09330
 D. brevicaulis      pH            mean(dY/dX)  0.08010  0.007546  0.14689
 D. brevicaulis      Soil_texture  mean(dY/dX)  0.09014  0.036158  0.13609
 D. scottiana        Canopy_Height mean(dY/dX) -0.15071 -0.234311 -0.06834
 D. scottiana        Density_1     mean(dY/dX)  0.02168 -0.050046  0.09221
 D. scottiana        Density_2     mean(dY/dX) -0.10522 -0.205225 -0.01735
 D. scottiana        pH            mean(dY/dX)  0.04763 -0.025757  0.11837
 D. scottiana        Soil_texture  mean(dY/dX)  0.05446 -0.000908  0.10753

Columns: term, group, contrast, estimate, conf.low, conf.high, predicted_lo, predicted_hi, predicted, tmp_idx 
Type:  response

The conditional effects plots that are automatically generated from the conditional_effects() function show me the probability of each species outcome from the model for each value along the range of the predictor. This looks good for all my species in each of the plots, except for D. breicaulis which is ‘flatlining’ in each plot. The probability of that species outcome is near 0 across the whole range, and if makes no sense to me. I would be very grateful for anyone to explain this and if this may be due to an initial convergence issue (although my model is converging) or an issue with the conditional effects calculation. See the conditional effects plots below:

image
image891×594 49.3 KB

image
image853×589 50.6 KB

image
image863×594 49.5 KB

image
image855×598 47.1 KB

image
image861×595 44.9 KB

The blue line that runs along the bottom of the plot is D. brevicaulis each time. And I see no reason for this to be happening?
Thanks very much!

2

hiya @LeoJha ,
I’m no expert with multinomial/categorical models. But I did notice that the intercept for D. brevicaulis is around -2.77, which on response (probability) scale is inv_logit_scaled(-2.77) = 0.0590

So your conditional effects are probably right in that that species seems to hug the bottom. Even strong positive effects like pH (1.7) are going to struggle to overcome the negative effects of the two density predictors and canopy height (I think conditional effects fixes those other predictors at their mean, so you’re likely starting even lower than 0.059).

How does the raw data look for that species?

Hope that helps

(p.s. the plots are pretty, but maybe you could turn off the ribbon fill or something as you diagnose things. It’s hard to keep track of the colored lines through multiple colored lenses… it reminds me of some of the puzzles in The Witness!)