Hierarchical Behavioural Model Specification- hyperparameters/ priors on groups or subjects

Modeling
, , ,
1

Hello!
I’m seeking guidance in how to specifiy a hierarchical approach to modelling RL behavioural data from patient groups.

I don’t have the raw data to share; however for each participant, we have two data aquisitions- pre-treatment & post-treatment data. Each participant gets either one of two types of treatment (never neither, never both). We will have at the end about 15-20 participants per group so ~30 pre and 15 treat_1, 15 treat_2.

I’ll use the Rescorla Wagner delta rule as an example here of the type of model we’ve used before (but also use kalman filter, plus exploration & perseveration bonuses). The approach the group has taken before is to run the model on 3 groups separately- pretreat and e.g. posttreat_1, posttreat_2. So the priors are effectively set on the groups; as each group’s data is fitted separately.

// Defining the data that is inputted into the model
data {
  int<lower=1> nSubjects; // number of subjects
  int<lower=1> nTrials; // number of trials  
  array[nSubjects, nTrials] int<lower=0, upper=4> choice; // Choice data for each subject in each trial
  array[nSubjects, nTrials] real<lower=0, upper=100> reward; // Reward data for each subject in each trial
}

// Defining transformed data
transformed data {
  real<lower=0, upper=100> v1; // prior belief mean reward value trial 1
  
  v1 = 50.0;
}

// Defining parameters of the model
parameters {
  real<lower=0,upper=1> alpha_mu; // mean of learning rate
  real<lower=0,upper=3> beta_mu; // mean of inverse temperature

  real<lower=0> alpha_sd; // standard deviation of learning rate
  real<lower=0> beta_sd; // standard deviation of inverse temperature
  array[nSubjects] real<lower=0, upper=1> alpha; // learning rate for each subject
  array [nSubjects] real<lower=0,upper=3> beta; // inverse temperature for each subject
}

// Model specifications
model {

    alpha_sd ~ cauchy(0,1);
    beta_sd ~ cauchy(0,1);

  for (s in 1:nSubjects) {
    vector[4] v; // value (mu)
    real pe; // prediction error

    v = rep_vector(v1, 4); // initialize value vector with prior belief mean reward


    // Prior distributions for parameters
    alpha[s] ~ normal(alpha_mu, alpha_sd);  
    beta[s] ~ normal(beta_mu, beta_sd);        

    for (t in 1:nTrials) {
      if (choice[s,t] != 0) {
        // Calculate action probabilities
        choice[s,t] ~ categorical_logit(beta[s] * (v));
  
        // Calculate prediction error 
        pe = reward[s,t] - v[choice[s,t]];
  
        // Value/mu updating (learning)
        v[choice[s,t]] = v[choice[s,t]] + alpha[s] * pe;
      }
    }
  }
}

So my question is- if i want to reproduce the above I could alter the shape of the data coming in and re-structure the model to be per group too somehow (I’d need to figure out how to split post treat into two groups so I’d need n=3 hyperparameters I think). I think this is arguably looking at a group level change.

However if I wanted pulling out a given subjects likelihood of a given model fit pre and post (and then seeing how that fits within a given group) would I need to specify all the priors at a per subject level for each group? would this be a valid approach; or is group-level best.

I suppose I’m asking for hierarchical modelling of this type I’m seeking guidance on how to choose over which levels I want to pool (I may have answered my own question as I probably want to do both… as the real world impact of this will eventually be ‘will this help an individual’). I have also heard part-pooling discussed; would this be an approach to follow?

Thanks in advance.

I’ve been here before with this: Hierarchical Model - behavioural data: should param standard deviation priors be group level or inside per subject loop?; I cannot thank you enough as a community & @amas0 specifically- I was able to show the forum post to supervisor as justification to make changes and the resulting findings put us back on solid ground.

2

Hi,

What do you mean by “pulling out a given subject’s change”? If you are doing partial pooling over individuals by fitting them as part of a larger group, you unfortunately lose the ability to make inferences on individual participants independent of the group they were fitted as a part of. This is a challenge in clinical research that we haven’t solved yet, though there are theoretically ways around it (c.f. two-stage hierarchical modeling in fMRI research - but that also has caveats).

Also, if you have both between- and within-subject variation (pre- and post-treatment in different groups), you may not want to treat those as independent groups. We attempted stan code to do this here: OSF (very messy sorry) but you could also look at how brms sets up nested data in stan to get an idea of what you want to do.

Vanessa

3

Thanks a lot for your reply- I edited my question to make that clearer (from ‘a given subjects change’ to ‘likelihood of a given model fit pre and post’.
You are right, I think I need to take account of both between & and within subject variation in order to infer anything of value from the behavioural data. Thanks so much for the link; I will give it a read & post approaches back here. Much appreciated!