# AIC, BIC, DIC in Stan

#1

Hi everyone, I am a Stan newbie.
Are there any functions for calculating AIC, BIC and DIC values for fitted model?

#2

This isn’t a great answer to the question you actually asked, but this paper from @avehtari talks about WAIC as well as reason why you might not want to do it. You can compute WAIC using the `waic` function in the `loo` package (in R).

As for DIC, you can compute that directly from samples (although I don’t have a link to hand).

Hope this helps at least a little.

(Edit because i forgot the link)

#3

I saw loo package but I forgot to mention that I’m using PyStan. Is there anything like `loo` for python?

#4

There is a reference code Python https://github.com/avehtari/PSIS
@ahartikainen wrote “I think ArviZ library will include PSIS-loo code.” https://github.com/stan-dev/pystan/issues/278#issuecomment-405917515

#5

@avehtari Thank you, I’ll take a look.

#6

If you can find a copy of Bayesian Data Analysis (BDA3, Gelman et al.), there’s some in-detail description of both LOO-CV and WAIC (as well as DIC) around pages 172-174.
I used that to write a Python script for computing the WAIC a while ago and if you’re willing to go there it’s fairly straightforward, although tedious and requiring a lot more calculations than the DIC (the latter requires only computing the logp at the posterior mean and the expected value of logp, while the former requires computing a version of that for each data point and mcmc sample).

That said, I would find it helpful if there was a pystan function to compute LOO and WAIC, since Stan developers seem to feel strongly against DIC and (god forbid) BIC.

#7

This would be a great project for someone to develop in Python. In R, the loo package uses the output of RStan, and has special hooks for RStan, but is otherwise pretty general I believe given the right form of MCMC output.

Right—it’s doing the equivalent of a posterior predictive calculation using full Bayes from the sample rather than a point estimate. That’s also why you’ll find