I am trying to compute linear models and Bayes factors for a categorical predictor variable. Both, the frequentist lmer and brm model summary show significant effects of the predictor, however when computing the BF for the predictor versus a null model (intercept only), it indicates no effect (BF = 0.017).
Puzzled by this non-convergence, I changed the same predictor to numeric and now the effects converge between the models and the BF (219.07). I would prefer to keep the categorical predictor though for theoretical reasons. Can you help me understand where it goes wrong?
Many thanks in advance
**With position as a categorical predictor:**
res_RP_5 <- lmer(RP ~ position + (1|subj),
data = SUMMDATA_AVG_5items)
print(anova(res_RP_5))
> Type III Analysis of Variance Table with Satterthwaite's method
> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
> position 0.0928016 0.0232004 4 72 7.12666 0.000068326 ***
> ---
> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
pr = prior(normal(0, 1), class = 'b')
res_CV_B_5_position <- brm(CV ~ position + (1|subj),
data = SUMMDATA_AVG_5items,
prior = pr,
cores = 3, save_pars = save_pars(all=TRUE))
print(summary(res_CV_B_5_position))
> Family: gaussian
> Links: mu = identity; sigma = identity
> Formula: CV ~ position + (1 | subj)
> Data: SUMMDATA_AVG_5items (Number of observations: 95)
> Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
> total post-warmup draws = 4000
>
> Group-Level Effects:
> ~subj (Number of levels: 19)
> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
> sd(Intercept) 0.05 0.01 0.03 0.08 1.00 1080 1400
>
> Population-Level Effects:
> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
> Intercept 0.31 0.02 0.28 0.35 1.00 1939 2294
> position2 0.03 0.02 -0.00 0.07 1.00 3387 2987
> position3 0.07 0.02 0.04 0.11 1.00 3182 2872
> position4 0.07 0.02 0.03 0.11 1.00 3444 3028
> position5 0.09 0.02 0.05 0.13 1.00 3669 3032
>
> Family Specific Parameters:
> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
> sigma 0.06 0.01 0.05 0.07 1.00 2749 3091
res_CV_B_5_null = update(res_CV_B_5_position, formula = ~ .-position)
BF_5_position <- bayes_factor(res_CV_B_5_position, res_CV_B_5_null)
print(paste('Effect of position:', BF_5_position))
> [1] "Effect of position: 0.0166357493642797" "Effect of position: FALSE"
**With position as a numerical predictor:**
res_RP_5 <- lmer(RP ~ position + (1|subj),
data = SUMMDATA_AVG_5items)
print(anova(res_RP_5))
> Type III Analysis of Variance Table with Satterthwaite's method
> Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
> position 0.0824642 0.0824642 1 75 25.2721 0.0000032914 ***
> ---
> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
pr = prior(normal(0, 1), class = 'b')
res_CV_B_5_position <- brm(CV ~ position + (1|subj),
data = SUMMDATA_AVG_5items,
prior = pr,
cores = 3, save_pars = save_pars(all=TRUE))
print(summary(res_CV_B_5_position))
> Family: gaussian
> Links: mu = identity; sigma = identity
> Formula: CV ~ position + (1 | subj)
> Data: SUMMDATA_AVG_5items (Number of observations: 95)
> Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
> total post-warmup draws = 4000
>
> Group-Level Effects:
> ~subj (Number of levels: 19)
> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
> sd(Intercept) 0.05 0.01 0.03 0.08 1.00 1124 1665
>
> Population-Level Effects:
> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
> Intercept 0.30 0.02 0.27 0.34 1.00 2212 2724
> position 0.02 0.00 0.01 0.03 1.00 6785 3076
>
> Family Specific Parameters:
> Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
> sigma 0.06 0.01 0.05 0.07 1.00 3608 2748
res_CV_B_5_null = update(res_CV_B_5_position, formula = ~ .-position)
BF_5_position <- bayes_factor(res_CV_B_5_position, res_CV_B_5_null)
print(paste('Effect of position:', BF_5_position))
> [1] "Effect of position: 219.065334966854" "Effect of position: FALSE"