Modeling number of trials until k (6) consecutive successes with right censorship

Modeling
,
#1

I have a dataset in which each participant has to repeatedly solve a task (2 alternative forced choice, participants have to infer a rule to discriminate between them).
In order to finish a given “stage”, they have to have 6 consecutive successes .
If they manage, they enter a new stage (different rule determining which choice is right).
If they don’t manage to get 6 consecutive within 50 trials the experiment stops.

This is tricky to model (for me) because:

  • the k “consecutive” successes: not a geometric (k successes), nor a negative binomial (“successive”). And ideed the fit is terrible. I’d need something more like this: https://www.ism.ac.jp/editsec/aism/pdf/046_1_0193.pdf
  • the data is censored within “stage” (right censoring when the participants reach 50 trials)
  • the data is censored between “stages” (some participants don’t reach the higher stages).

Are there similar worked out examples?

2 Likes
#2

Oh, that sounds tough. I fear the distributions given in the reference you gave would not be super useful as they assume that the success/failures come from some sort of fixed process, but you would (if I understand it correctly) actually expect the process to be changing because learning is taking place.

One way could be that you would fit a (monotonic) spline/Gaussian process representing “understanding” over the trials (or maybe linear model would be enough) and put a bernoulli likelihood on top of this for each trial. Possibly a per-participant spline would make sense but that gets a bit tough to fit - maybe a global spline and varying intercept + time per participant could be workable?

I unfortunately don’t have a good idea how to model the censoring between stages… Maybe a per-participant varying intercept and varying effect of stage could take care of most of this? So you could in-principle simulate new data from participants that didn’t progress - if your simulation ends up with less than 6 consecutive success in a stage given the fitted predictors for the stage, you stop. If not, you continue. If the participant has not actually seen that stage, you pick a new value for the participant-stage effect from the fitted population distribution… You would then interpret all predictions for the later stages as conditional on the participant reaching the stage…

Hope that makes sens and helps at least a little…

1 Like
#3

Thanks!
Yes, you are correct, the generative (cognitive) model involves learning, and I’m leaning now towards an explicit model of that, e.g. relying on reinforcement learning models.

My original plan was to first figure out how to model the structure of the task without learning, and then move to learning. I added the implied stages (stage level censoring), and still predictive checks are crap.

oh well, onto the learning model.

1 Like