Modelling two-choice decision from time-series data

I have time-series sensor data from different participants in a repeated experiment. In each run of the experiment, there is either a specific time when the participant has done a certain decision or not (at all). I can read this directly from the data. Now, given that I take all of this data for training and test, I would like to build a time-series dependent model. The model shall give at any time (and with the sensor data at that time) a prediction between 0 and 1, where 0 would mean that the person would not do take decision and 1 that the person takes the decision. I also want to investigate if the model can be improved by including past measurement samples.

What would you recommend for tackling this modeling problem? My first try would be some sort of logistic regression which I train on every time-sample, either with response 1 (after the decision was made) or 0 (before the decision was made). The sensor readings at that time (and possibly in the past) would be the covariates of the linear predictor. I have so far mainly encountered usage of logistic regression in “static” rather than time-series problems. Furthermore, how could I do this in a Bayesian setting? Are there already functions for instance in brms?

1 Like

I think what you describe is a survival model. There is a lot of discussion in the forums here if you look for survival model or proportional hazard models. I think the more robust Stan implementation is in rstanarm::surv_jm. There is a more experimental implementation in brms. See this Github discussion.

PS: I never worked with any of these models. So take this with a grain of salt.

3 Likes

If your trials are set up such that the trial ends when the participant responds and your outcome is the time-to-respond, then a survival model might make sense. It’s also common to have a timeout in such experiments such that the trial ends after, say 2minutes if the participant hasn’t responded yet, in which case you’d be dealing with a censored survival model.

If, on the other hand, your trials are set up with some sort of probe stimulus that occurs at different times from trial-to-trial, and you’re measuring whether the participant responds to the probe or not, then your original idea of modelling the outcome as binomial predicted by time-of-stimulus-onset is more appropriate. You can model the effect of time as linear very easily, but you should also consider methods that permit discerning of non-linear effects as well, including Generalized Additive Models and Gaussian Processes (the former being a computationally faster approximation of the latter).

3 Likes

I should also say that if the trials are time-to-decision and said times are relatively rapid (<2s), you should look at this resource: https://lindeloev.github.io/shiny-rt/

3 Likes

Thanks, I’ll definitely take a look into survival models, the Cox Proportional Hazards model might be the most appropriate one on the first view.

Thanks for the excellent link first. I shall also check the Generative Additive Models.

Actually, I would say that I am doing some sort of probe stimulus which occurs in different variations according to protocol and the participants need to evaluate between two options. Option 1 is connected to an increased risk to the participant while option 2 is to not take that risk and go for a safer option. So I would say that a participant must have responded within couple of seconds roughly, for either of the two options. The issue I found with this logistic regression is how I set up the response vector. For the participants that chose the riskier option 1, I set the response to 0 until they took the decision and then 1 for some time, which I have a hard time to define. So the 0’s are currently quite over-represented in the response vector. Also, most participants went for the riskier decision, so that is also over-represented by a factor of 3 roughly. Can those imbalances cause certain issues?

Another follow-up question: my logistic regression fit seems to be “too good”, which I assume is due to the high time resolution, i.e. high sampling frequency. I have few trials but within each trial many samples. Is this an issue that affects the overall model?

Can you clarify on how your observed/quantified this?

When I look at the confidence interval of the model estimates (I did frequentist GLM for debugging for now), the interval is extremely narrow, indicating to me that the model is extremely certain about the estimate. When I downsample the signals, the confidence interval understandably gets wider.