I am trying to figure out whether for my current project there could be a reason not to model a hierarchy:
I have collected data from 400 humans on a computer task. I have analysed the data in a non-hierarchical model (logistic regression reparameterized to have the ‘noisiness’/‘inverse temperature’ separately for each person from the regression weights) that suggested that, given appropriate transformation the parameter estimates are more or less normally distributed.
In a next step I want to link these parameter estimates to responses participants have made on questionnaires about psychiatric symptoms. For this I have used a regression analysis predicting the behavioural measures based on the questionnaire scores.
My question now is: Should include the behavioural measures that I obtain from the hierarchical or the non-hierarchical model? They are correlated, but not perfectly, r ~0.8. And how would I decide? One hesitation I have about the hierarchical model is that for each individual person, their estimate will be influenced to an extent by the behaviour of other people, which might not be a desirable feature.
I do realise that an alternative would be - instead of running two types of models - to instead make one large model that explains both the behaviour in the computer task and the questionnaires at the same time. I’m hesitant to do that as a first step because it would be quite unusual for the field. Also, because this now becomes a very large model, it seems to take very long to fit. Ideally I’d do it as well as the approach above and find the same.
I’d be very grateful for any input on this