I want to know the Bayes Factors in favour of the lack of interaction in a model. I ran two models, one with interaction and one without it. Then compared them with bayes_factor(inter, nointer) and got this: Estimated Bayes factor in favor of inter over nointer: 0.50862 How do I interpret this? My…

I don’t know if this is the right place to ask about Bayes factors, since afaik most of the senior developers on here tend to work with approximate out-of-sample predictive accuracy using Pareto-smoothed importance sampling leave-one-out cross-validation (PSIS-LOO cv) for model comparison, rather th…

Hi, and thank you for your response. I do use PSIS-LOO cv for model comparison. The message I received asking me to explore BF said: “It would be worth adding Bayesian stats to have an idea of how solid is the negative result”. When I asked them what did they want (as I have done the same analysis …

They can get it correct by just guessing, 50% of the times. Do I need to specify this somehow in the model or is that irrelevant? Depending on how your response format is setup, you could use a metric called D-prime, which factors in chance rates of being correct dprime analysis Your BF for the …

One small further thing to note. It seems like there is something different in your models besides the presence of an interaction? One has ( 1 + delay | id ) the other has ( 1 + time | id ) If you wish to make an interence about how one aspect such as the interaction changes your inferences, the…

I think both @abartonicek ’s @JimBob ’s suggestions are reasonable first steps, but I would like to add a bit of broader context. I would be very cautious about claiming “lack of interaction” from just BF or LOO results. Remember that LOO is really just a guess at the model’s ability to predict new …

Thank you for such a thoughtful response. This was a small side experiment and they wanted to investigate if it made sense to do more experiments in the same topic and try to publish them, or if we just drop it. Because there was no evidence for the interaction, they wanted to know if we had strong …

Martin raised some great points. The way I understood it, the main take-away is that not concluding you need an interaction (to predict out-of-sample data) is not the same as concluding you don’t need an interaction. Absence of evidence is not evidence of absence. [image] BigFish_Artwire: If I …

[image] abartonicek: The way I understood it, the main take-away is that not concluding you need an interaction (to predict out-of-sample data) is not the same as concluding you don’t need an interaction . Absence of evidence is not evidence of absence. This is important, but what I wanted to…

Edit: I think Martin answered this question whilst I was typing it! I haven’t thought about it! It’s a pretty awesome example but I am not sure how to apply it to my dependent variable. I have used Bernoulli family because my dependent variable are 1 (correct) and 0s (incorrect). I thought that the…

Ok, noted! But I thought that if the 95% CI contained a 0 that means no substantial effect! [image] martinmodrak: Assuming the covarietes are around +/- 1 (which would be true if your covariates are factors), changing the log-odds (the unit the model works with) by 0.42 * 1 means multiplying th…

Tiny Bayes Factor for a model with a non substantial interaction vs one without interaction

Interfaces brms

BigFish_Artwire May 6, 2020, 12:40pm 17

Like this? I think the points are the posterior probability and the bars are standard deviations from the mean?

Topic		Replies	Views
Bayes factors for a moderating effect brms rstan	11	1261	July 5, 2020
Proper interpretation of Bayes factor (from rstanarm + bridgesampling) General bayes-factor , interpret-results	5	3060	December 14, 2018
Bayes_factor() Warning messages: 1: logml could not be estimated within maxiter, Interfaces brms	15	1291	July 28, 2022
Reporting evidence in favot of excluding a predictor Modeling loo , brms	0	355	December 18, 2022
Bug in brms bayes_factor for model with me() term? brms loo , bug , bayes-factor	5	916	January 27, 2020

Tiny Bayes Factor for a model with a non substantial interaction vs one without interaction

Related topics