I implemented a RL model in Stan. The model uses a TD-learning mechanism to compute a choice probability theta for each trial. Observations (choices) are in turn sampled with y ~ bernoulli(theta). MCMC diagnostics look good and parameter estimates look reasonable. However, when I check the model u…

Yes, the current implementation of LOO-PIT is not correct for discrete and we’re working on a correct implementation. In addition even the discrete LOO-PIT is not very good for binary target. See instead Section 4.4 in https://avehtari.github.io/modelselection/diabetes.html for how to evaluate cali…

Bernoulli model with many LOO-PIT values at 1

Modeling

adkinsty November 13, 2020, 1:42am 2

However, a comment from @avehtari on a related post suggests that these plots are not valid for discrete models. So, perhaps I should ignore this.

Topic		Replies	Views
Error in ll_args.stanreg(x) : all(y %in% c(0, 1)) is not TRUE Modeling loo , rstanarm	1	300	July 28, 2023
Help me understand ppc_loo_pit_overlay from bayesplot v1.5 Modeling bayesplot , loo	1	1832	April 6, 2018
LOO Visualization / Stacking Modeling bayesplot , loo	4	791	March 24, 2020
LOO on large ODE fit: Am I interpreting it correctly? Modeling loo	1	387	July 25, 2023
Alternative to LOO for simulation studies Modeling loo , posterior-predictive , model-comparison	14	1163	November 17, 2020

Bernoulli model with many LOO-PIT values at 1

Related topics