Credible intervals changing after running the model again

Interfaces brms
1

I ran a model to find whether there was evidence of an interaction or not between two variables. According to the credible intervals, there was evidence of just one out of three possible interactions. I ran the model again and now two out of three seem to have evidence in favour. How come an interaction can appear and dissapear? Here are the last results I have after running the model again.

one
( b = -0.19, SD = 0.47, CI = [-1.12, 0.75])

two
( b = -1.18, SD = 0.54, CI = [-2.21, -0.13])

three
( b = -1.37, SD = 0.53, CI = [-2.42, -0.35])

  • Operating System: Windows 10
  • brms Version: 2.15
2

Hi there,

Could you provide a bit more information?

  • Are those 0.95 intervals?
  • What are the ESS values like for the models? For 0.95 you need a pretty high ESS to have reliable estimates of the tails
  • On the run where (presumably) the second interaction wasn’t ‘’’’’‘significant’’’’’’, was it the case that the upper bound for the interval was only barely above zero?

Hugo

1 Like