Custom Item-Response Theory model in brms

Modeling
1

Hi, I’m trying to implement this model from Morey et al. ( 2009) in brms, to have more flexibility given that I am not an expert in raw STAN. I have read about brms non-linear models and the possibility to insert transformed parameters and other more sophisticated stuff. My question is whether could be possible to implement this model in brms. Given that I have never used other than the standard lme4-like syntax in brms I am not sure if brms can handle this.
This is the Github repo https://github.com/richarddmorey/mass-at-chance and this is the raw STAN code:

data {
  
  int<lower=1> I;
  int<lower=1> J;
  int N[I,J];
  int y[I,J];
  real<lower=0,upper=1> chance_p;
}

parameters {
  vector[I] alpha;
  vector<lower=0>[I] theta;
  vector[J] beta;
  
  real<lower=0> sigma_alpha;
  real<lower=0> sigma_theta;
}

transformed parameters{
  real p[I,J];
  real x[I,J];
  real q[I,J];
  for(i in 1:I)
    for(j in 1:J){
      x[i,j] = theta[i] * (alpha[i] - beta[j]);
      q[i,j] = x[i,j] > 0 ? Phi( x[i,j] ) : 0.5 ;
      p[i,j] = (q[i,j] - 0.5) * 2 * (1 - chance_p) + chance_p;
    }
}

model {
  
  alpha ~ normal(0, sigma_alpha);
  theta ~ normal(0, sigma_theta);
  beta  ~ normal(0, 1);
  
  // I chose the parameters to approximately match
  // the medians of the gamma priors on precision 
  // in the JAGS model
  sigma_alpha ~ cauchy(0, 0.775);
  sigma_theta ~ cauchy(0, 0.775);
  
  for(i in 1:I)
    for(j in 1:J)
      y[i,j] ~ binomial(N[i,j], p[i,j]);
}

Reading the paper I think that the most critical parts are:

  • the truncated-probit link function
  • the latent parameter estimated from data
2

I think this is not easily done in brms, because you need to implement this line:

q[i,j] = x[i,j] > 0 ? Phi( x[i,j] ) : 0.5 ;

One possibility could be to use brms custom families.

Then you could do these things:

x = theta * (alpha - beta); 
q = x > 0 ? Phi(x) : 0.5 ;
p = (q - 0.5) * 2 * (1 - chance_p) + chance_p;
binomial(N, p)
return binomial_lpmf(y | N, p)

in your custom likelihood function. (this ommits the declaration of the custom function, and this simple version would accept response by response data)

More specifically, you could first make a simple binary IRT model in brms (basically a logistic regression with random effects for individuals and items, more here), look at the generated stan-code to see what the parameter names for the discrimination parameter (theta in the original model) and random effects for individuals and items are (alpha and beta in the original model), and continue from there.

I do not know of a way to implement this in brms without writing some stan-code. So it might be easier to just use the Stan model as set up by @richarddmorey.

As an aside: The expression q[i,j] = x[i,j] > 0 ? Phi( x[i,j] ) : 0.5 ; indicates to me that the likelihood can change in a non-smooth manner, which is something Stan is not made for (I am not 100% sure, and it might still work).

1 Like