Let’s say we run an experiment with two conditions. In one condition participants all agree on the rating which is typically 3 on a scale of 1-5 but in the other, there is disagreement and half the people will typically rate 2 and the other half will rate 4. I have created a simulated dataset which yields this hypothetical scenario but I a bit in doubt of how to quantify this difference.

```
# Inter and intra variance simulated data
# Simulate data with the below functions
simulate_data <- function(mean_value, sd_value, draws) {
random_numbers <- rnorm(draws, mean = mean_value, sd = sd_value)
break_points <- c(-Inf, 1, 2, 3, 4, Inf)
category_ACR <- cut(random_numbers, breaks = break_points, labels = c("1", "2", "3", "4", "5"))
num_ACR <- as.numeric(category_ACR)
hist(num_ACR)
print(paste("Mean: ", mean(num_ACR)))
print(paste("SD: ", sd(num_ACR)))
return(num_ACR)
}
generate_ratings <- function(norm, n) {
return(sample(norm, n))
}
generate_data <- function() {
data <- data.frame(Participant = integer(), Condition = character(), Rating = numeric())
for (i in 1:50) {
ratings1 <- generate_ratings(norm2.5, 20)
ratings2 <- if (i <= 25) generate_ratings(norm2, 20) else generate_ratings(norm3, 20)
data <- rbind(data, data.frame(Participant = rep(i, 40), Condition = rep(c("Condition1", "Condition2"), each = 20), Rating = c(ratings1, ratings2)))
}
return(data)
}
mean_value <- 2
sd_value <- 1
draws <- 10000
norm2.5 <- simulate_data(2.5, sd_value, draws*2)
norm2 <- simulate_data(2, 0.85, draws)
norm3 <- simulate_data(3, 0.85, draws)
data <- generate_data()
# BRMS
GausIndCon <- brm(
formula = bf(Rating ~ -1 + Condition+ (Condition|Participant),
sigma ~ Condition),
data = data,
family = gaussian(),
chains = 2,
cores = 4,
iter = 4000,
warmup = 1000
)
consub <- make_conditions(data, vars =c("Condition"))
conditional_effects(
GausIndCon,
"Participant",
conditions = consub,
re_formula = NULL, ordinal = F
)
```

When I model this data I see that the mean rating and the variance across conditions are more or less the same:

However, when I plot each participant I can visually see the difference between the conditions easily:

I am in doubt how to quantify this difference, and in my attempt to model the sigma directly I am surprisingly being told that condition 2 has a lower sigma:

sigma_ConditionCondition2 -0.17

In the cases I wish to model I have more than 2 conditions and there is not so obvious a difference, but I believe some conditions will have more individual disagreements than others.

All tips and help on how to quantify/test this will be extremely welcome. Thanks for reading.