Low ESS & R-hat issues w/ 2-parameter IRT

Hi all,

I am trying to fit a 2-parameter IRT model to estimate the ideological positioning of elected officials using survey responses from some non-elected political elites. The non-elected political elites indicated the perceived ideological positions of the elected officials on a continuous 1-5 scale (1 as conservative, 5 as liberal; the survey used a slider so values are truly continuous).

I’ve been following @paul.buerkner’s helpful forthcoming JSS on implementing IRTs using brms, and I think I’ve gotten most of the way there but am stuck on some model fit issues. Even after tweaking some estimation options, I am getting the following warnings post-estimation:

1: The largest R-hat is NA, indicating chains have not mixed.
Running the chains for more iterations may help. See
2: Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
Running the chains for more iterations may help. See
3: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
Running the chains for more iterations may help. See

For warning 1, the only parameter with an NA R-hat is the standard deviation for the rater intercept in the eta term, which I fix as a constant for identifiability (as per @paul.buerkner). This might be a silly question, but is this R-hat coming back as NA because the parameter is fixed as 1? If so, can I safely ignore this warning?

For warnings 2 and 3, some parameters have stubbornly low bulk and tail ESS despite increasing the number of iterations. How should I adjust the model to try to address this issue?

The code I am currently using is as follows:

model_formula <- bf(rating ~ exp(logalpha)*eta,
                    eta ~ 1 + (1|d|rater) + (1|elected_official),
                    logalpha ~ 1 + (1|d|rater),
                    nl = TRUE)

model_priors <- prior("normal(0, 3)", class = "b", nlpar = "eta") +
                  prior("normal(0, 1)", class = "b", nlpar = "logalpha") +
                  prior("constant(1)", class = "sd", group = "elected_official", nlpar = "eta") +
                  prior("normal(0, 3)", class = "sd", group = "rater", nlpar = "eta") +
                  prior("normal(0, 1)", class = "sd", group = "rater", nlpar = "logalpha")

rating_model <- brm(formula = model_formula,
                    data = data,
                    family = brmsfamily(family = "gaussian"),
                    prior = model_priors, 
                                 max_treedepth = 20),
                    refresh = 10,
                    iter = 10000, warmup = 8000,
                    chains = 4, cores = 4, seed = 1989)


  • Operating System: Windows 10 Pro
  • brms Version: 2.14.5

Just a quick comment on the NA Rhat for parameters constant by design (fixation). This is a false positive that you can safely ignore. In the future, I might be able to selectively turn off this warning for parameters manually set to constants.

1 Like

That was what I thought but just wanted to make sure, thanks for confirming!