Hello!
I’m working on an item response theory (IRT) application and have a general Stan modeling question about “carrying forward” a posterior distribution from one model into a second model.
Use case: In a typical calibration-and-field-test workflow, we first fit an IRT model to estimate both person parameters (θ) and item parameters (a, b, etc.) on an operational item set. Later, we field-test new items and want to estimate their item parameters using the same person parameter estimates from the first calibration — essentially holding the person parameter posteriors fixed rather than re-estimating them jointly with the new item parameters.
My question: Is there a standard/recommended way in Stan to pass a previously-estimated posterior distribution in as data so that it acts as a fixed (not re-estimated) distribution for those parameters in a second model?
The classic way to do this is via multiple imputation. Fit the model many times with different joint posterior iterations of the values that you desire to fix. Ensure that convergence diagnostics are ok in each of the multiple fits, then combine the chains across fits for posterior inference.
See discussion here Models with parameters with known posterior
Thank you for the link to the other discussion thread! That was really interesting and informative!
I’m pretty sure the answer is no, but I’ll ask just for clarity. Is there any sort of nontraditional method that would avoid the computation requirements of multiple imputation or the errors that were encountered with cut()? I ask because we have a fairly nontraditional modeling setup. We have 1,000+ models that we have to estimate for our operational application, and estimating all of the models usually takes 4-6 weeks with the computing power that we have available. While I certainly see the point about multiple imputation being the ideal solution, the problem we run into is that even running 100 iterations for multiple imputation (which admittedly is probably not enough iterations) would take several months, which is just not feasible.
Based on the other thread, I imagine that multiple imputation is really the only viable approach, but I wanted to ask to make sure I’m being thorough in my search for potential workarounds.
Thanks again for all of your help and insight!
No, in the sense that I don’t know anything we’ve published on this as part of our documentation.
In addition to @jsocolar’s suggestion, which does what you are asking for (i.e., cut), another approach is to approximate your posterior with a multivariate normal (with link functions if not all variables are unconstrained) and just plug that in as the prior in the next round.
But IRT models are really fast to fit if they’re parameterized in an identifiable way, so this may not all be necessary. If you’re trying to fit an IRT-2PL model, you have to (a) identify the location, (b) identify the scale, (c) deconvolve discrimination parameterization, and (d) non-center if it’s hierarchical with low data or center if it’s hierarchical with high data. For (c), the typical formulation of the likelihood would involve student ability parameters \alpha_j \in \mathbb{R}, item difficulty parameters \beta_i \in \mathbb{R} and item discrimination parameters \delta_i \in (0, \infty) [the sign here might not be known in ideal-point applications].
y_n \sim \textrm{bernoulli}(\textrm{logit}^{-1}(\delta_i \cdot (\alpha_j - \beta_i)))
This turns out to be hard to sample. A better parameterization is
y_n \sim \textrm{bernoulli}(\textrm{logit}^{-1}(\delta_i \cdot \alpha_j - \gamma_i))),
and then you can define \beta_i = \gamma_i / \delta_i to recover item difficulties. Otherwise, you get a nasty \delta_i \cdot \beta_i term that leads to bananas in the posterior due to multiplicative non-identifiability.