Hierarchical models for heterogenous groups?

heya! i’ve written a few custom stan models, and i can simulate/generate data and recover the parameters used for the simulation by fitting my stan models.

i’m keen on hierarchical models, since data can be pooled and thus (hopefully) improve the model fits. this works okay when fitting model M on data that i know is generated by model M. but i’m struggling how to handle heterogenous data.

as an example, say i have some psychology experiment where participants perform some task that generate data in the form of choices on each trial for 100 trials.

now, say i suspect some participants use strategy A for their choices and some participants use strategy B. i don’t know who uses what strategy, but i’d like to find out.

i can fit models A and B individually for each subject, but 100 trials for each subject isn’t much data, so is there a way to use hierarchical modelling for this?

i can come up with a brute force way: select x subjects, try to fit strategy A and B for that subgroup, and go through all permutations of selecting x subjects. but that seems extremely inefficient.

is there’s some “smart” way to do this?

What you describe sounds like it’d be covered by a mixture model.

1 Like

I have no formal solutions but references if you are so inclined. My familiarity with the topic is via the cognitive skill learning literature and trying to discern when people transition from using one strategy to another. I think the model described in Williams et al (2019; “A Bayesian nonlinear mixed-effects location scale model for learning,” doi: 10.3758/s13428-019-01255-9) would be relevant (and easily ‘portable’). Their model allegedly incorporates the model described in Evans et al. (2018; ‘refining the law of practice,’ doi: 10.1037/rev0000105), which offers a similar, but different perspective. From an experimental perspective, I have had success with post-trial strategy probes, where the subject can pick which strategy they used after each trial, but that won’t help you in this case :-p

2 Likes

Mixture model does indeed sound like what I was after, thank you @mike-lawrence! I had to first read the wikipedia page in order to parse what the stan documentation says, as the language there is incredibly dense.

@jack_kuhns that paper looks interesting, is mixed-effects another name for mixture model or are they separate? I ask because “mixture model” and “mixed model” are apparently very different, so I’m learning that statistics terminology is… confusing… :)