I’m learning hierarchical Bayesian modeling and having a hard time finding some formal results on the asymptotic behavior of hierarchical models—their consistency and limiting distribution—when the group sizes are fixed and small (say, 3 students in each class/group) while the number of groups goes to infinity.
Gelman et. al’s Bayesian Data Analysis has proofs for the consistency and asymptotic normality for general posterior mode estimators. Their proof requires the number of observations to go to infinity, which I’m not sure if it directly applies to my question, as I need the group sizes to be fixed at quite small numbers (e.g., 3) and only allow the number of groups to go to infinity.
Let me try to be a bit more specific about my question. Say that we have the following hierarchical model:
where y_i and x_i are data from student i, who is in some class j\in\{1,\cdots,J\}. Denote the size of class j by n_j. I’m interested in the posterior mode estimates of \mu, \sigma, and \omega. Because this is a hierarchical model, b_j's will also be estimated but are not of primary interest. Suppose all the parameters are identifiable in the data (and the b_j's are not exchangeable).
If we let J\rightarrow\infty and require 1\leq n_j \leq3 to be fixed, under certain regularity conditions, do we have consistency and asymptotic normality for the two hyperparameters \mu and \sigma?
(I believe the \beta_j's won’t be estimated accurately given the tiny class sizes, but will their inaccuracy affect the consistency and asymptotic normality of the two hyperparameters?)
Gelman and Hill (2007) have some comments that seem to be in line with that we still have the consistency and asymptotic normality. I would like to read some more formal results/proofs if they exist.
Any suggestion is greatly appreciated! Thank you!