Divergent transitions - brms IRT / ordinal regression

Hi,

I’m new to bayesian modelling and brms. When specifiying a model for ordinal regression with multiple Likert items and IRT model they result in divergent transitions. Having trouble to find what is causing this issue. Any help on this topic is much appreciated.

Data generated using package LikertMakeR.

Sessioninfo
sessioninfo::session_info()
─ Session info ────────────────────────────────────────────────────────────────
 setting  value
 version  R version 4.4.1 (2024-06-14)
 os       macOS Sonoma 14.5
 system   aarch64, darwin20
 ui       RStudio
 language (EN)
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 tz       Europe/Stockholm
 date     2024-07-10
 rstudio  2024.04.2+764 Chocolate Cosmos (desktop)
 pandoc   NA

─ Packages ────────────────────────────────────────────────────────────────────
 package        * version  date (UTC) lib source
 abind            1.4-5    2016-07-21 [1] CRAN (R 4.4.0)
 backports        1.5.0    2024-05-23 [1] CRAN (R 4.4.0)
 bayesplot        1.11.1   2024-02-15 [1] CRAN (R 4.4.0)
 bridgesampling   1.1-2    2021-04-16 [1] CRAN (R 4.4.0)
 brms             2.21.0   2024-03-20 [1] CRAN (R 4.4.0)
 Brobdingnag      1.2-9    2022-10-19 [1] CRAN (R 4.4.0)
 checkmate        2.3.1    2023-12-04 [1] CRAN (R 4.4.0)
 cli              3.6.3    2024-06-21 [1] CRAN (R 4.4.0)
 coda             0.19-4.1 2024-01-31 [1] CRAN (R 4.4.0)
 codetools        0.2-20   2024-03-31 [1] CRAN (R 4.4.1)
 colorspace       2.1-0    2023-01-23 [1] CRAN (R 4.4.0)
 curl             5.2.1    2024-03-01 [1] CRAN (R 4.4.0)
 distributional   0.4.0    2024-02-07 [1] CRAN (R 4.4.0)
 dplyr            1.1.4    2023-11-17 [1] CRAN (R 4.4.0)
 emmeans          1.10.3   2024-07-01 [1] CRAN (R 4.4.0)
 estimability     1.5.1    2024-05-12 [1] CRAN (R 4.4.0)
 fansi            1.0.6    2023-12-08 [1] CRAN (R 4.4.0)
 generics         0.1.3    2022-07-05 [1] CRAN (R 4.4.0)
 ggplot2          3.5.1    2024-04-23 [1] CRAN (R 4.4.0)
 glue             1.7.0    2024-01-09 [1] CRAN (R 4.4.0)
 gridExtra        2.3      2017-09-09 [1] CRAN (R 4.4.0)
 gtable           0.3.5    2024-04-22 [1] CRAN (R 4.4.0)
 hypr           * 0.2.8    2023-11-09 [1] CRAN (R 4.4.0)
 inline           0.3.19   2021-05-31 [1] CRAN (R 4.4.0)
 jsonlite         1.8.8    2023-12-04 [1] CRAN (R 4.4.0)
 lattice          0.22-6   2024-03-20 [1] CRAN (R 4.4.1)
 lifecycle        1.0.4    2023-11-07 [1] CRAN (R 4.4.0)
 loo              2.8.0    2024-07-03 [1] CRAN (R 4.4.0)
 magrittr         2.0.3    2022-03-30 [1] CRAN (R 4.4.0)
 MASS             7.3-60.2 2024-04-26 [1] CRAN (R 4.4.1)
 Matrix           1.7-0    2024-04-26 [1] CRAN (R 4.4.1)
 matrixStats      1.3.0    2024-04-11 [1] CRAN (R 4.4.0)
 munsell          0.5.1    2024-04-01 [1] CRAN (R 4.4.0)
 mvtnorm          1.2-5    2024-05-21 [1] CRAN (R 4.4.0)
 nlme             3.1-164  2023-11-27 [1] CRAN (R 4.4.1)
 pillar           1.9.0    2023-03-22 [1] CRAN (R 4.4.0)
 pkgbuild         1.4.4    2024-03-17 [1] CRAN (R 4.4.0)
 pkgconfig        2.0.3    2019-09-22 [1] CRAN (R 4.4.0)
 posterior        1.6.0    2024-07-03 [1] CRAN (R 4.4.0)
 pracma           2.4.4    2023-11-10 [1] CRAN (R 4.4.0)
 QuickJSR         1.2.2    2024-06-07 [1] CRAN (R 4.4.0)
 R6               2.5.1    2021-08-19 [1] CRAN (R 4.4.0)
 Rcpp             1.0.12   2024-01-09 [1] CRAN (R 4.4.0)
 RcppParallel     5.1.7    2023-02-27 [1] CRAN (R 4.4.0)
 rlang            1.1.4    2024-06-04 [1] CRAN (R 4.4.0)
 rstan            2.32.6   2024-03-05 [1] CRAN (R 4.4.0)
 rstantools       2.4.0    2024-01-31 [1] CRAN (R 4.4.0)
 rstudioapi       0.16.0   2024-03-24 [1] CRAN (R 4.4.0)
 scales           1.3.0    2023-11-28 [1] CRAN (R 4.4.0)
 sessioninfo      1.2.2    2021-12-06 [1] CRAN (R 4.4.0)
 StanHeaders      2.32.9   2024-05-29 [1] CRAN (R 4.4.1)
 stringi          1.8.4    2024-05-06 [1] CRAN (R 4.4.0)
 stringr          1.5.1    2023-11-14 [1] CRAN (R 4.4.0)
 tensorA          0.36.2.1 2023-12-13 [1] CRAN (R 4.4.0)
 tibble           3.2.1    2023-03-20 [1] CRAN (R 4.4.0)
 tidyselect       1.2.1    2024-03-11 [1] CRAN (R 4.4.0)
 utf8             1.2.4    2023-10-22 [1] CRAN (R 4.4.0)
 V8               4.4.2    2024-02-15 [1] CRAN (R 4.4.0)
 vctrs            0.6.5    2023-12-01 [1] CRAN (R 4.4.0)
 xtable           1.8-4    2019-04-21 [1] CRAN (R 4.4.0)

 [1] /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library

───────────────────────────────────────────────────────────────────────────────
#Model 1
fit_cu2 <- brm(
  formula = Loy ~ 1 + cs(Country),
  data = rdatLong,
  family = acat("probit")
)

#Model 2 with priors from Bayesian Item Response Modeling in R with brms and Stan by Bürkner

irt1PL <- bf(Loy ~ 1 + (1 | Item) + (1 | Ind))

prior_irt1PL <- 
  prior("normal(0, 3)", class = "sd", group = "Ind") +
  prior("normal(0, 3)", class = "sd", group = "Item")

fit_irt1PL <- brm(
  formula = irt1PL,
  data = rdatLong2,
  family = brmsfamily("cumulative", "logit"),
  prior = prior_irt1PL
)

Code for simulated data

C1 <- lfast(
  n = 25,
  mean = 6,
  sd = 1.0,
  lowerbound = 1,
  upperbound = 7,
  items = 1
)

C2 <- lfast(
  n = 25,
  mean = 6,
  sd = 1.0,
  lowerbound = 1,
  upperbound = 7,
  items = 1
)

C3 <- lfast(
  n = 25,
  mean = 4,
  sd = 2.0,
  lowerbound = 1,
  upperbound = 7,
  items = 1
)

C4 <- lfast(
  n = 25,
  mean = 3,
  sd = 2.0,
  lowerbound = 1,
  upperbound = 7,
  items = 1
)

The data is compiled to long format before trying to fit the model.

How many items?

In the example used above I had 4 items.

I’ve also tried using the cumulative probit model with 8 items.

This works:
fit_cuML ← brm(
formula = Lo ~ 1 + Co + Ty + Ty:Co + (1 | Ind),
data = dat,
family = cumulative(“probit”)
)

This does not work:
fit_cuML2 ← brm(
formula = Lo ~ 1 + Co + Ty + Ty:Co + (1 | Ind) + (1 | Item),
data = dat,
family = cumulative(“probit”)
)

Results in:
Warning messages:
1: There were 152 divergent transitions after warmup. See
https://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
to find out why this is a problem and how to eliminate them.
2: Examine the pairs() plot to diagnose sampling problems

3: The largest R-hat is NA, indicating chains have not mixed.
Running the chains for more iterations may help. See
https://mc-stan.org/misc/warnings.html#r-hat
4: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
Running the chains for more iterations may help. See
https://mc-stan.org/misc/warnings.html#bulk-ess
5: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
Running the chains for more iterations may help. See
https://mc-stan.org/misc/warnings.html#tail-ess

The model aims to assess how much each respondents assessments (Lo) vary given category Co, Ty and their interaction in a 4 x 3 repeated measures design. Would this model be a good way to achieve that?

Thanks for your help on this.

In my experience, it’s not unusual to have problems like divergent transitions when you set random effects to groups with few numbers, such as for just 4 items. When I run into this, I often increase the adapt_delta setting closer to 1, something like adapt_delta = 0.99 or even higher. It’s also my experience that cumulative(“probit”) models in brms tend to have difficulties with the init values, which you can often improve by setting something like init_r = 0.2. Also, in the example code you’re showing, it looks like you’re using the defaults for the upper-level \sigma parameters. With so few items, I’d use something stronger. My default for a cumulative-probit model is prior(exponential(1), class = sd).

2 Likes

Thanks for your help. Much appreciated. It worked when adding init_r and adapt_delta.

Would the model be a good fit for answering the research question? How much does each responds assessment of Lo vary given category Co, Ty and their interaction?

Thanks!

I don’t know enough about your data and research context to give you a strong answer, but your model

Lo ~ 1 + Co + Ty + Ty:Co + (1 | Ind) + (1 | Item)

seems like it’s at least a reasonable starting point for answering your research question.