Dear all,
I ran this multilevel logit model using rstanarm with 10 groups at level-2. My focal variable is gender (male). The thing that puzzles me is that the overall/grand effect (by looking at the 95% credible intervals) for male is positive and statistically significant (in a Bayesian sense) without containing zeros. But when I examine group-specific coeffcients for male, it turns out none of them are statistically significant. My question is how to make sense of that. My initial response is that maybe the group-specific sub-sample size is small, but the overall sample size is large enough? Or, this is just simply a coincidence. How to make sense of such findings?
Thanks a lot in advance!
Jun Xu, PhD
Professor
Department of Sociology
University of Macau
Macau SAR, China
Email: xujun@um.edu.mo
Web: Jun Xu
So the specific level-2 equation for \beta_{male} is as follows,
\beta_{male} = \gamma_{male} + u_g
So don’t the results indicate that \hat{\gamma}_{male} has positive significant effects, but \hat{\gamma}_{male}+\hat{u}_g largely deviates from \hat{\gamma}_{male}, the distributions of which all contain zeros?
> summary(mod01, digits=3, probs=c(0.025, 0.5, 0.975))
Model Info:
function: stan_glmer
family: binomial [logit]
formula: micrcall ~ male + age35 + marry + employed + usborn + useduc +
usnoed + engprof + inc02 + inc05 + inc75 + inc10 + incmis +
edlths + edhscl + edscol + edcolg + poldem + poloth + (1 +
male || RETHNICS)
algorithm: sampling
sample: 20000 (posterior sample size)
priors: see help('prior_summary')
observations: 4198
groups: RETHNICS (10)
Estimates:
mean sd 2.5%
(Intercept) -2.821 1.197 -5.457
male 0.219 0.103 0.019
age35 -0.119 0.126 -0.365
marry 0.000 0.112 -0.222
employed 0.310 0.101 0.113
usborn 0.437 1.172 -1.567
useduc 0.467 1.172 -1.533
usnoed 0.085 1.171 -1.918
engprof 0.138 0.144 -0.147
inc02 0.501 0.175 0.161
inc05 0.400 0.146 0.114
inc75 0.207 0.156 -0.100
inc10 -0.204 0.181 -0.563
incmis 0.050 0.163 -0.265
edlths 0.195 0.199 -0.190
edhscl 0.135 0.166 -0.189
edscol 0.146 0.168 -0.181
edcolg 0.129 0.132 -0.133
poldem 0.352 0.148 0.067
poloth 0.071 0.150 -0.223
b[(Intercept) RETHNICS:1] 0.253 0.185 -0.097
b[(Intercept) RETHNICS:2] -0.423 0.203 -0.853
b[(Intercept) RETHNICS:3] -0.194 0.181 -0.570
b[(Intercept) RETHNICS:4] 0.316 0.165 0.009
b[(Intercept) RETHNICS:5] 0.324 0.176 -0.006
b[(Intercept) RETHNICS:6] 0.011 0.173 -0.333
b[(Intercept) RETHNICS:7] -0.182 0.175 -0.540
b[(Intercept) RETHNICS:8] -0.012 0.164 -0.340
b[(Intercept) RETHNICS:10] -0.049 0.181 -0.417
b[(Intercept) RETHNICS:11] -0.089 0.175 -0.451
b[male RETHNICS:1] 0.039 0.117 -0.159
b[male RETHNICS:2] -0.006 0.111 -0.251
b[male RETHNICS:3] -0.014 0.108 -0.268
b[male RETHNICS:4] -0.029 0.107 -0.289
b[male RETHNICS:5] -0.017 0.108 -0.277
b[male RETHNICS:6] 0.018 0.107 -0.187
b[male RETHNICS:7] 0.029 0.108 -0.164
b[male RETHNICS:8] -0.022 0.107 -0.278
b[male RETHNICS:10] -0.035 0.115 -0.323
b[male RETHNICS:11] 0.031 0.110 -0.165
Sigma[RETHNICS:(Intercept),(Intercept)] 0.115 0.096 0.023
Sigma[RETHNICS:male,male] 0.017 0.031 0.000
50% 97.5%
(Intercept) -2.716 -0.759
male 0.219 0.420
age35 -0.119 0.130
marry 0.000 0.219
employed 0.309 0.506
usborn 0.318 3.056
useduc 0.348 3.079
usnoed -0.031 2.702
engprof 0.139 0.419
inc02 0.503 0.839
inc05 0.399 0.686
inc75 0.208 0.508
inc10 -0.203 0.149
incmis 0.050 0.369
edlths 0.193 0.586
edhscl 0.135 0.457
edscol 0.148 0.476
edcolg 0.129 0.388
poldem 0.351 0.648
poloth 0.070 0.367
b[(Intercept) RETHNICS:1] 0.248 0.628
b[(Intercept) RETHNICS:2] -0.411 -0.058
b[(Intercept) RETHNICS:3] -0.185 0.141
b[(Intercept) RETHNICS:4] 0.310 0.664
b[(Intercept) RETHNICS:5] 0.318 0.681
b[(Intercept) RETHNICS:6] 0.012 0.355
b[(Intercept) RETHNICS:7] -0.177 0.148
b[(Intercept) RETHNICS:8] -0.010 0.314
b[(Intercept) RETHNICS:10] -0.048 0.303
b[(Intercept) RETHNICS:11] -0.084 0.245
b[male RETHNICS:1] 0.012 0.343
b[male RETHNICS:2] -0.001 0.230
b[male RETHNICS:3] -0.004 0.201
b[male RETHNICS:4] -0.009 0.165
b[male RETHNICS:5] -0.004 0.193
b[male RETHNICS:6] 0.005 0.276
b[male RETHNICS:7] 0.009 0.298
b[male RETHNICS:8] -0.006 0.178
b[male RETHNICS:10] -0.011 0.166
b[male RETHNICS:11] 0.011 0.308
Sigma[RETHNICS:(Intercept),(Intercept)] 0.091 0.351
Sigma[RETHNICS:male,male] 0.007 0.098
Fit Diagnostics:
mean sd 2.5% 50% 97.5%
mean_PPD 0.149 0.008 0.134 0.149 0.164
The mean_ppd is the sample average posterior predictive distribution of the outcome variable (for details see help('summary.stanreg')).
MCMC diagnostics
mcse Rhat n_eff
(Intercept) 0.012 1.000 9365
male 0.001 1.000 21198
age35 0.001 1.000 21033
marry 0.001 1.000 21725
employed 0.001 1.000 23647
usborn 0.012 1.000 9189
useduc 0.012 1.000 9197
usnoed 0.012 1.000 9181
engprof 0.001 1.000 19222
inc02 0.002 1.001 12719
inc05 0.001 1.001 12074
inc75 0.001 1.000 15445
inc10 0.001 1.000 19618
incmis 0.001 1.001 14736
edlths 0.002 1.000 14272
edhscl 0.001 1.000 13619
edscol 0.001 1.000 15258
edcolg 0.001 1.000 14814
poldem 0.001 1.000 17191
poloth 0.001 1.000 17101
b[(Intercept) RETHNICS:1] 0.002 1.000 12958
b[(Intercept) RETHNICS:2] 0.002 1.000 13068
b[(Intercept) RETHNICS:3] 0.002 1.000 14121
b[(Intercept) RETHNICS:4] 0.002 1.000 11108
b[(Intercept) RETHNICS:5] 0.002 1.000 13648
b[(Intercept) RETHNICS:6] 0.002 1.000 13134
b[(Intercept) RETHNICS:7] 0.002 1.000 13159
b[(Intercept) RETHNICS:8] 0.001 1.000 12618
b[(Intercept) RETHNICS:10] 0.001 1.000 14856
b[(Intercept) RETHNICS:11] 0.002 1.000 13332
b[male RETHNICS:1] 0.001 1.000 16882
b[male RETHNICS:2] 0.001 1.000 21886
b[male RETHNICS:3] 0.001 1.000 20968
b[male RETHNICS:4] 0.001 1.000 18639
b[male RETHNICS:5] 0.001 1.000 20893
b[male RETHNICS:6] 0.001 1.000 18550
b[male RETHNICS:7] 0.001 1.000 19129
b[male RETHNICS:8] 0.001 1.000 20084
b[male RETHNICS:10] 0.001 1.000 17611
b[male RETHNICS:11] 0.001 1.000 17784
Sigma[RETHNICS:(Intercept),(Intercept)] 0.001 1.000 8908
Sigma[RETHNICS:male,male] 0.000 1.000 12294
mean_PPD 0.000 1.000 19746
log-posterior 0.073 1.000 5605
For each parameter, mcse is Monte Carlo standard error, n_eff is a crude measure of effective sample size, and Rhat is the potential scale reduction factor on split chains (at convergence Rhat=1).