Those questions on “incremental fit” arise from time to time. The short answer as far as I understand this topic is: there are many ways to approximate this, none of them particularly great. It is an open research problem.
I recently responded to a somewhat similar question: Estimating Subject Level Effects for New Subjects in IRT, or multilevel models in general Would that be a similar problem to what you are facing?
Best of luck with your model