Accuracy of my predictions for one factor

Interfaces brms
1

Hello,
I would like to know, how to improve the predictive accuracy of my model for one factor. When I plot the posterior draws against all data, then the model seems to fit quite well:

pp_check_allData
pp_check_allData796×719 12.9 KB

However, when I take out the levels of one factor, the accuracy of predictions is quite weak:
This is position == “Initial”:

pp_check_positionInitial
pp_check_positionInitial796×719 17.5 KB

And this is position == “Final”

pp_check_positionFinal
pp_check_positionFinal796×719 14.4 KB

Any ideas, how to improve my model fit?

I am modeling the absolute duration of the eye gaze with a skew_normal family. I have three two-level factors load, length and position and I would like to examine the impact of the three-way interaction between the factors on the duration of the eye gaze. This was a repeated measures design where a number of participants saw sets of items in between conditions. The trial index track the trials as the participants proceeded with the experiment. This is how my model looks

myAbsModel <- brm(GazeDurAbs ~  sentenceLength * position * load  +
                    (1 + sentenceLength * position * load + scaled_trialIndex | participantID) +
                    (1 + sentenceLength * position * load + scaled_trialIndex | item),
                  data = test,
                  family = "skew_normal",
                    iter = 10000,
                  thin = 5)

Great thanks for your help!

2

Your model only allows the location parameter to be influenced by the predictor variables. Your standard deviation and skew parameters will default to be intercept-only (i.e., sigma ~ 1 and alpha ~ 1). The vignette on distributional models provides the general approach for adding predictors to these other parameters.

You might also consider using a different response distribution family that better matches your response variable constraints. If gaze duration can’t be less than or equal to zero, you may try a bounded distribution like lognormal, gamma, or another one better justified by your understanding of the process.

edit: fix typo

1 Like
3

+1 to both of @wpetry’s points.

1 Like
4

Thanks! The gamma distribution did the trick. I feel that the distributional models are also relevant to my data but I still need to think about them.

2 Likes
5

They take practice, that’s for sure.

1 Like