I’m trying to understand what could cause a GLMM (with bernoulli link) to result in NAs in the Cholesky factor for the (by-subject and/or by-item) random effects without also resulting in any such problem with regard to the random effect SDs and CORs. I’ve seen that case a number of times now.
For example, here’s a model output, with all R-hats ~ 1 and none of them NA for which I got a warning that some of the R-hats are NA indicating that the chains might not have mixed. There are also warning about low Bulk and Tail ESS. (bias and dist are binary predictors; step is a continuous predictor for which I’m including orthogonal linear & quadratic components).
Family: bernoulli
Links: mu = logit
Formula: response ~ 1 + bias * dist * (step.1 + step.2) + (1 + bias * dist * (step.1 + step.2) | subject) + (1 + dist * (step.1 + step.2) | item)
Data: d (Number of observations: 2469)
Samples: 4 chains, each with iter = 22500; warmup = 7500; thin = 5;
total post-warmup samples = 12000
Group-Level Effects:
~item (Number of levels: 16)
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept) 0.49 0.20 0.11 0.93 1.00 9241 7460
sd(distfarvs.near) 1.45 0.55 0.46 2.65 1.00 8755 9475
sd(step.1) 1.05 0.74 0.05 2.77 1.00 8892 10139
sd(step.2) 0.75 0.52 0.03 1.92 1.00 8397 9350
sd(distfarvs.near:step.1) 4.70 1.90 1.25 8.85 1.00 8208 9033
sd(distfarvs.near:step.2) 4.43 1.43 2.03 7.62 1.00 8420 9992
cor(Intercept,distfarvs.near) 0.17 0.31 -0.46 0.72 1.00 10870 11175
cor(Intercept,step.1) -0.08 0.36 -0.74 0.63 1.00 11888 11202
cor(distfarvs.near,step.1) -0.10 0.36 -0.74 0.63 1.00 11279 11715
cor(Intercept,step.2) 0.10 0.38 -0.64 0.76 1.00 11133 11298
cor(distfarvs.near,step.2) 0.04 0.36 -0.67 0.71 1.00 11826 11366
cor(step.1,step.2) 0.08 0.37 -0.65 0.75 1.00 11051 11843
cor(Intercept,distfarvs.near:step.1) -0.15 0.31 -0.71 0.48 1.00 10575 11414
cor(distfarvs.near,distfarvs.near:step.1) -0.03 0.31 -0.63 0.55 1.00 11354 11214
cor(step.1,distfarvs.near:step.1) 0.10 0.36 -0.62 0.75 1.00 9631 11170
cor(step.2,distfarvs.near:step.1) -0.06 0.37 -0.72 0.64 1.00 9942 11119
cor(Intercept,distfarvs.near:step.2) -0.03 0.30 -0.60 0.56 1.00 10448 11401
cor(distfarvs.near,distfarvs.near:step.2) 0.13 0.32 -0.54 0.68 1.00 10016 11414
cor(step.1,distfarvs.near:step.2) -0.19 0.36 -0.79 0.56 1.00 8898 10181
cor(step.2,distfarvs.near:step.2) 0.08 0.35 -0.62 0.72 1.00 10285 11079
cor(distfarvs.near:step.1,distfarvs.near:step.2) -0.14 0.31 -0.73 0.47 1.00 10651 11765
~subject (Number of levels: 20)
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sd(Intercept) 1.13 0.25 0.72 1.70 1.00 10652 11489
sd(biasS.vs.SH) 0.74 0.31 0.11 1.36 1.00 9291 8438
sd(distfarvs.near) 0.32 0.21 0.02 0.78 1.00 10670 11345
sd(step.1) 2.18 0.73 0.98 3.85 1.00 10761 11029
sd(step.2) 2.16 0.62 1.07 3.50 1.00 10909 11228
sd(biasS.vs.SH:distfarvs.near) 0.46 0.35 0.02 1.29 1.00 11482 11555
sd(biasS.vs.SH:step.1) 0.98 0.77 0.04 2.90 1.00 10908 11644
sd(biasS.vs.SH:step.2) 1.76 0.95 0.14 3.75 1.00 8580 9191
sd(distfarvs.near:step.1) 1.31 0.96 0.05 3.62 1.00 11588 11515
sd(distfarvs.near:step.2) 0.57 0.45 0.02 1.68 1.00 11302 11260
sd(biasS.vs.SH:distfarvs.near:step.1) 2.00 1.52 0.08 5.63 1.00 11648 11528
sd(biasS.vs.SH:distfarvs.near:step.2) 1.22 0.92 0.05 3.45 1.00 11438 11047
cor(Intercept,biasS.vs.SH) -0.11 0.24 -0.55 0.38 1.00 11551 11973
cor(Intercept,distfarvs.near) -0.15 0.27 -0.63 0.40 1.00 11629 11135
cor(biasS.vs.SH,distfarvs.near) 0.01 0.27 -0.50 0.52 1.00 12359 11457
cor(Intercept,step.1) -0.24 0.23 -0.65 0.22 1.00 12110 11282
cor(biasS.vs.SH,step.1) 0.05 0.25 -0.44 0.53 1.00 11447 11878
cor(distfarvs.near,step.1) 0.16 0.27 -0.39 0.65 1.00 11217 11090
cor(Intercept,step.2) 0.12 0.22 -0.31 0.52 1.00 11399 11553
cor(biasS.vs.SH,step.2) -0.03 0.24 -0.49 0.45 1.00 10698 11681
cor(distfarvs.near,step.2) -0.03 0.26 -0.53 0.48 1.00 10442 11566
cor(step.1,step.2) -0.20 0.24 -0.64 0.29 1.00 11012 11649
cor(Intercept,biasS.vs.SH:distfarvs.near) 0.01 0.27 -0.51 0.53 1.00 12058 11927
cor(biasS.vs.SH,biasS.vs.SH:distfarvs.near) -0.04 0.27 -0.55 0.50 1.00 11937 11754
cor(distfarvs.near,biasS.vs.SH:distfarvs.near) -0.01 0.28 -0.53 0.52 1.00 12309 11848
cor(step.1,biasS.vs.SH:distfarvs.near) 0.02 0.27 -0.50 0.53 1.00 12206 11861
cor(step.2,biasS.vs.SH:distfarvs.near) 0.06 0.27 -0.47 0.57 1.00 11380 11891
cor(Intercept,biasS.vs.SH:step.1) 0.01 0.27 -0.50 0.54 1.00 11233 11163
cor(biasS.vs.SH,biasS.vs.SH:step.1) 0.02 0.27 -0.51 0.54 1.00 12199 12050
cor(distfarvs.near,biasS.vs.SH:step.1) -0.02 0.28 -0.54 0.52 1.00 11763 11261
cor(step.1,biasS.vs.SH:step.1) -0.02 0.27 -0.54 0.51 1.00 12374 11495
cor(step.2,biasS.vs.SH:step.1) -0.06 0.28 -0.57 0.48 1.00 12431 10555
cor(biasS.vs.SH:distfarvs.near,biasS.vs.SH:step.1) -0.02 0.28 -0.55 0.52 1.00 12033 11888
cor(Intercept,biasS.vs.SH:step.2) 0.18 0.25 -0.34 0.62 1.00 11501 11923
cor(biasS.vs.SH,biasS.vs.SH:step.2) 0.07 0.27 -0.48 0.56 1.00 11773 11684
cor(distfarvs.near,biasS.vs.SH:step.2) 0.05 0.27 -0.48 0.56 1.00 11604 11807
cor(step.1,biasS.vs.SH:step.2) 0.03 0.25 -0.46 0.52 1.00 11435 11946
cor(step.2,biasS.vs.SH:step.2) -0.19 0.26 -0.65 0.36 1.00 11632 11189
cor(biasS.vs.SH:distfarvs.near,biasS.vs.SH:step.2) -0.01 0.27 -0.53 0.53 1.00 11366 11767
cor(biasS.vs.SH:step.1,biasS.vs.SH:step.2) 0.05 0.28 -0.50 0.56 1.00 10864 11161
cor(Intercept,distfarvs.near:step.1) 0.03 0.27 -0.49 0.53 1.00 11521 12101
cor(biasS.vs.SH,distfarvs.near:step.1) 0.03 0.27 -0.50 0.54 1.00 11963 11472
cor(distfarvs.near,distfarvs.near:step.1) 0.01 0.27 -0.53 0.54 1.00 11526 11899
cor(step.1,distfarvs.near:step.1) -0.03 0.27 -0.55 0.49 1.00 11938 11846
cor(step.2,distfarvs.near:step.1) -0.13 0.27 -0.62 0.42 1.00 11726 11764
cor(biasS.vs.SH:distfarvs.near,distfarvs.near:step.1) -0.06 0.28 -0.58 0.49 1.00 11669 11725
cor(biasS.vs.SH:step.1,distfarvs.near:step.1) 0.00 0.28 -0.54 0.53 1.00 12372 11802
cor(biasS.vs.SH:step.2,distfarvs.near:step.1) 0.02 0.27 -0.50 0.55 1.00 11870 11405
cor(Intercept,distfarvs.near:step.2) -0.00 0.27 -0.52 0.52 1.00 11415 11497
cor(biasS.vs.SH,distfarvs.near:step.2) 0.05 0.28 -0.49 0.57 1.00 12091 11806
cor(distfarvs.near,distfarvs.near:step.2) 0.07 0.29 -0.48 0.60 1.00 12138 11917
cor(step.1,distfarvs.near:step.2) -0.00 0.28 -0.54 0.52 1.00 11481 11338
cor(step.2,distfarvs.near:step.2) -0.03 0.27 -0.56 0.50 1.00 11335 11604
cor(biasS.vs.SH:distfarvs.near,distfarvs.near:step.2) -0.00 0.28 -0.54 0.53 1.00 11848 11196
cor(biasS.vs.SH:step.1,distfarvs.near:step.2) 0.01 0.28 -0.53 0.54 1.00 11494 11496
cor(biasS.vs.SH:step.2,distfarvs.near:step.2) -0.03 0.28 -0.55 0.51 1.00 11694 11887
cor(distfarvs.near:step.1,distfarvs.near:step.2) 0.01 0.28 -0.53 0.54 1.00 12084 11705
cor(Intercept,biasS.vs.SH:distfarvs.near:step.1) -0.08 0.27 -0.59 0.46 1.00 11486 11207
cor(biasS.vs.SH,biasS.vs.SH:distfarvs.near:step.1) 0.06 0.27 -0.48 0.57 1.00 11519 11873
cor(distfarvs.near,biasS.vs.SH:distfarvs.near:step.1) 0.01 0.28 -0.53 0.55 1.00 12364 11726
cor(step.1,biasS.vs.SH:distfarvs.near:step.1) 0.06 0.27 -0.47 0.57 1.00 12151 11849
cor(step.2,biasS.vs.SH:distfarvs.near:step.1) -0.05 0.27 -0.56 0.49 1.00 11994 11404
cor(biasS.vs.SH:distfarvs.near,biasS.vs.SH:distfarvs.near:step.1) 0.01 0.28 -0.53 0.53 1.00 11923 12128
cor(biasS.vs.SH:step.1,biasS.vs.SH:distfarvs.near:step.1) 0.00 0.28 -0.53 0.53 1.00 11566 11563
cor(biasS.vs.SH:step.2,biasS.vs.SH:distfarvs.near:step.1) -0.05 0.28 -0.58 0.50 1.00 11956 11888
cor(distfarvs.near:step.1,biasS.vs.SH:distfarvs.near:step.1) 0.02 0.28 -0.52 0.54 1.00 12302 11848
cor(distfarvs.near:step.2,biasS.vs.SH:distfarvs.near:step.1) 0.01 0.27 -0.52 0.54 1.00 11551 11210
cor(Intercept,biasS.vs.SH:distfarvs.near:step.2) -0.08 0.27 -0.59 0.46 1.00 12252 11877
cor(biasS.vs.SH,biasS.vs.SH:distfarvs.near:step.2) 0.02 0.27 -0.51 0.55 1.00 11815 11675
cor(distfarvs.near,biasS.vs.SH:distfarvs.near:step.2) 0.04 0.28 -0.50 0.56 1.00 11271 11635
cor(step.1,biasS.vs.SH:distfarvs.near:step.2) -0.02 0.27 -0.53 0.51 1.00 11690 11755
cor(step.2,biasS.vs.SH:distfarvs.near:step.2) -0.00 0.28 -0.53 0.54 1.00 12029 11386
cor(biasS.vs.SH:distfarvs.near,biasS.vs.SH:distfarvs.near:step.2) 0.04 0.28 -0.51 0.58 1.00 12180 11682
cor(biasS.vs.SH:step.1,biasS.vs.SH:distfarvs.near:step.2) -0.01 0.28 -0.54 0.52 1.00 11683 11889
cor(biasS.vs.SH:step.2,biasS.vs.SH:distfarvs.near:step.2) -0.01 0.27 -0.53 0.53 1.00 12025 11766
cor(distfarvs.near:step.1,biasS.vs.SH:distfarvs.near:step.2) 0.02 0.28 -0.52 0.54 1.00 11973 11687
cor(distfarvs.near:step.2,biasS.vs.SH:distfarvs.near:step.2) -0.01 0.28 -0.54 0.51 1.00 11939 11569
cor(biasS.vs.SH:distfarvs.near:step.1,biasS.vs.SH:distfarvs.near:step.2) 0.01 0.27 -0.51 0.53 1.00 11589 10821
Population-Level Effects:
Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept 0.50 0.39 -0.26 1.27 1.00 9264 9988
biasS.vs.SH 1.81 0.47 0.89 2.77 1.00 11466 11608
distfarvs.near 0.64 0.51 -0.35 1.65 1.00 11278 11798
step.1 -9.86 1.08 -12.10 -7.89 1.00 10577 11608
step.2 2.81 0.89 1.02 4.58 1.00 11005 11047
biasS.vs.SH:distfarvs.near -0.60 0.88 -2.36 1.14 1.00 11599 11357
biasS.vs.SH:step.1 0.95 1.26 -1.42 3.56 1.00 11597 11526
biasS.vs.SH:step.2 0.80 1.12 -1.37 3.07 1.00 11677 11724
distfarvs.near:step.1 -0.96 1.45 -3.84 1.86 1.00 11740 11198
distfarvs.near:step.2 1.64 1.36 -0.96 4.38 1.00 11776 11812
biasS.vs.SH:distfarvs.near:step.1 1.79 2.31 -2.49 6.78 1.00 12002 11286
biasS.vs.SH:distfarvs.near:step.2 0.33 1.97 -3.52 4.30 1.00 12009 11453
Samples were drawn using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).
When I look into what caused the warning, I find that a few components of the lower triangular of L have NA Rhats. Looking further into it, it seems that they have constant estimates. Shinystan reports (L1 should be the Cholesky factor of the by-item effects reported in the above output):
N_eff mean se sd
L_1[1,1] 12,000 1 0 0
I am not sure what to make out of this. None of the estimates of the cov for the random effects have problematic R-hats (all chains look like they mix well) and none of them seem to have estimates close to any boundary condition. What does it mean that samples of L_1[1,1] seems to be constant across and within chains? And does this indicate a problem in the model’s assumptions? Thank you for any pointers and apologies if this is a naive question. (Fwiw, the data are approximately balanced with 24+ data points in each of the 2x2 cells defined by bias and dist).