Another problem with "stan_clogit" and "posterior predict"

Interfaces RStan
1

Hi all!! I am still having problem with rstanarm and postestimation after “stan_clogit”. “loo”, waic etc. did not work for model comparison and produced strange results. Therefore, I tried to calculate the rates of successful prediction with the original data using “posterior_predict”. This also did not work and I do not get why.

I have three conditional logits: a simple, bivariate model, an additive model with two predictors, and an interactive model with two predictors. The interactive model should outperform the additive, and the additive model should outperform the simple, bivariate specification. This is also what I get using “mlogit”. However, some simple calculations using “posterior_predict” tell me the simple model gets it right for about 35%, while the additive (that merely ADDS another predictor) is at about 5%, and the interactive specification is at 7%.

I really do not get why models with additional predictors can produce far worse predictions than simple, bivariate baseline models. Obviously, I must have made some very foolish error. Any hints??

Best,
Guido

library(tidyverse)
library(rstanarm)

srv_nol <- read_csv("REC_SRV_NOL_ij.csv", show_col_types = FALSE) 

# THREE MODELS ;
unc_clogit     <- stan_clogit(vote_ij ~ ut_ij + (1|j), strata=i, data=srv_nol) 
unc_clogit_add <- stan_clogit(vote_ij ~ ut_ij + lr_ij_unc + (1|j), strata=i, data=srv_nol) 
unc_clogit_int <- stan_clogit(vote_ij ~ ut_ij * lr_ij_unc + (1|j), strata=i, data=srv_nol) 

# THREE MODEL PREDICTIONS ;
unc_clogit_pr      <- posterior_predict(unc_clogit)
unc_clogit_add_pr  <- posterior_predict(unc_clogit_add)
unc_clogit_int_pr  <- posterior_predict(unc_clogit_int)

# COMPIUTE SUCCESSFUL PREDICTION RATES ;
# simple model ;
unc_clogit_fit     <- data.frame(srv_nol$vote_ij, t(unc_clogit_pr)) 
unc_clogit_fit     <- filter(unc_clogit_fit, srv_nol.vote_ij==1)  
mean(as.matrix(unc_clogit_fit[,-1]))

# additive model ;
unc_clogit_add_fit <- data.frame(srv_nol$vote_ij, t(unc_clogit_add_pr)) 
unc_clogit_add_fit <- filter(unc_clogit_add_fit, srv_nol.vote_ij==1)  
mean(as.matrix(unc_clogit_add_fit[,-1]))

# interactive model ;
unc_clogit_int_fit <- data.frame(srv_nol$vote_ij, t(unc_clogit_int_pr)) 
unc_clogit_int_fit <- filter(unc_clogit_int_fit, srv_nol.vote_ij==1)  
mean(as.matrix(unc_clogit_int_fit[,-1]))

Setup: OS is Windows 11, R 4.2.3; all packages up to date.

2

REC_SRV_NOL_ij.csv (630.8 KB)