Fitting brm shifted lognormal model for reaction times

Modeling
,
1

Hi. I’m fairly new to Bayesian analaysis, brms, etc., and I’ve been trying to fit a brm shifted lognormal model for about two weeks now, but I’m having some issues (from what I understand about the model checks…). Please forgive me for any basic or ignorant questions on this.

My experiment was linguistic/psycholinguistic: participants were exposed to a noun phrase, and then they had to determine the correct adjective. For example “la mesa [roja/*rojo]” (the red table). So they heard “la mesa”, they simultaneously saw “la mesa”, and then “rojo/roja” showed up and they clicked a button to choose the correct one. They are allowed to respond as soon as the noun “mesa” audio ends. I measured reaction time, and there are no negative values.

They progressed through 8 levels linearly over 8 days. They were exposed to four conditions in each level. Notably, in two conditions, the determiner (“la” in the above example) allows them to predict the adjective, whereas in the other two conditions, they have to wait to process the noun to get the gender information. I point this out for a later question about ndt.

One group was exposed to natural voice, a second group was exposed to AI voice.

I decided to use a shifted lognormal based on this guide.

I’m having a really hard time understanding priors, and I’m having an even harder time finding resources that explain them in a way I understand. I’ve been studying with Mcelreath’s Statistical Rethinking, but any other resources would be greatly appreciated.

I based my priors off of the guide I linked above, and then modified them based on my data’s mean and standard deviation:

rt_priors_shiftedln ← c(
set_prior(‘normal(0.1, 0.1)’, class = ‘Intercept’),
set_prior(‘normal(-0.4, 0.2)’, class = ‘sigma’),
set_prior(‘normal(0, 0.3)’, class = ‘b’),
set_prior(‘normal(0.3, 0.1)’, class = ‘sd’),
set_prior(‘normal(0.2, 0.05)’, class = “ndt”)
)

I did a priors only model:

rt_prior_model ← brm(
formula =
reaction_time ~ game_level * condition + group +
(1 | player_id) +
(1 | item),
data = nat_and_ai_rt_tidy,
warmup = 1000, iter = 2000, chains = 4,
family = shifted_lognormal(),
prior = rt_priors_shiftedln,
sample_prior = “only”,
cores = parallel::detectCores()
)

image
image798×552 4.96 KB

From what I understand, the pp_check() for the priors looks okay… but I’m not 100% certain. It seems like it’s producing some extreme values up to 20ish seconds.

Then I fit the actual model using the same formula and the pp_check() looks bad (from what I understand):

image
image790×555 4.41 KB

Everything converged, the rhat values are all 1.00.

So my actual questions:

  1. Is the pp_check() for the priors what is expected? Is there something else I can check about the priors only model to determine that the priors are okay?
  2. Is the pp_check() for the actual model as problematic as I’m understanding? Should I be looking at something else before deciding the model as it stands is problematic?
  3. Since I would expect some very fast responses to 2 conditions, whereas I know very fast responses to the 2 other conditions are highly unlikely (almost impossible), does the ndt as it is now allow for that variability across conditions? I have a feeling I did something wrong with the ndt, because right now, in “Further Distributional Parameters”, the estimate and CIs are 0.00.
  4. On the same ndt topic, I saw in the link above I can do something like “ndt ~ participant”, and I tried doing “ndt ~ condition”, assuming this would allow the ndt of each condition to vary, but the pp_check() came out worse than what I showed above. I’m not sure if that’s because I did something ELSE wrong in the model or because ndt ~ condition just isn’t appropiate here.
  5. Should I be including random slopes? If I include a random slope for player_id, is it recommended that I do the interaction game_level * condition?

Thank you for any advice or resources at all for any of these questiosn!! If any further information is needed, please let me know.

2

Hi,
I do believe the mismatch between model prediction and observed data was caused by no random slopes.
I had similar problem (but for the choice logistic model): model prediction of the random intercept model did not capture the pattern of the observed data whereas the model with random slope can well describe the data.
Also, although there is controversy over whether to include the random slope, but simulations show adding random slope can decrease the type I error (see, Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3)).
And if you follow the Barr paper’s suggestion, all predictors including the interaction term should be treated as random slopes.

3

I think you probably don’t want the prior on sigma to be a normal with a negative mean, since the sigma of a lognormal can’t be negative (it would be different if there were predictors on sigma, in which case sigma would be modelled with a log link, and you might want such a negative value).

Also, I would probably use an exponential distribution as priors on “sigma” and “sd”.

(here are some examples from my own work of priors for lognormal/shifted-lognormal models for RTs and naming times:
https://osf.io/preprints/psyarxiv/7wfus_v1
Investigating variability in morphological processing with Bayesian distributional models | Psychonomic Bulletin & Review )

4

Likewise, you can have a predictor on ndt (e.g., ndt ~ condition), but then ndt will be modelled with a log link and expressed in log-seconds (rather than seconds).

So if you’ve kept the same prior for that model, then the prior on ndt would now have a mean (in seconds) of 1.2 seconds (i.e., exp(0.2)), which is certainly inappropriate (and would lead to much worse prior predictive checks, as you’ve obtained).