I am working on a project where respondents rate how characteristic different adjectives are for a generalized “man” “woman” or “leader”. Past studies have used ICC to analyze this data and determine that respondents tend to use the same adjectives for men as they do for leaders, but not the same adjectives for women as they use for leaders. But they have also found that this effect varies by country and cultural context. We want to suggest that political representation can be predicted by the ICC of men / leaders and women / leaders. I want to use a multilevel Stan model to do this work.
Is there a way I can, at the individual level, compute the ICC, then use the ICC as a predictor at the country level? Has something similar been done that I can refer to?
Maybe I don’t understand your question well, but assuming what you want to do is to have model that fits the adjective data while also fitting the political representation data and tying it together, this should be relatively straightforward as you should be able to compute the ICC directly from the parameters of the adjective model (all the sigmas should already be there).
So maybe I am not understanding your question well?
I am however unaware whether something similar has already been done, but I am not well read in this kind of literature.
@martinmodrak Thanks for the reply, and sorry for the vagueness.
I’m still very new this approach. I’ve previously only used ICC to assess the quality of measurements. I’m not sure how to compute it from within Stan and use it as a predictor at a higher level. This would be done in the “transformed variables” part of the Stan specification, I suppose?
I will do some more practice with the software to gain a better understanding and come back with a more informed question.
Yes, once you have the model for the adjective data, you can compute the ICC in
transformed parameters - the formula at https://en.wikipedia.org/wiki/Intraclass_correlation#Modern_ICC_definitions:_simpler_formula_but_positive_bias is probably what you are after.
I should also note that I am unsure that fitting the adjective data AND political representation in a single model (if that is what you are after) might be problematic as you would be letting the information from political representation to leak into your estimates for the adjective model. I.e. if there is a suboptimal fit to
ajdective data that greatly improves the fit for the political representation data, it might be preferred. So not sure if you want that.
Anyway, best of luck with learning Stan!