# AIC, BIC, DIC in Stan

Hi everyone, I am a Stan newbie.
Are there any functions for calculating AIC, BIC and DIC values for fitted model?

This isnâ€™t a great answer to the question you actually asked, but this paper from @avehtari talks about WAIC as well as reason why you might not want to do it. You can compute WAIC using the `waic` function in the `loo` package (in R).

As for DIC, you can compute that directly from samples (although I donâ€™t have a link to hand).

Hope this helps at least a little.

(Edit because i forgot the link)

1 Like

I saw loo package but I forgot to mention that Iâ€™m using PyStan. Is there anything like `loo` for python?

There is a reference code Python https://github.com/avehtari/PSIS
@ahartikainen wrote â€śI think ArviZ library will include PSIS-loo code.â€ť https://github.com/stan-dev/pystan/issues/278#issuecomment-405917515

@avehtari Thank you, Iâ€™ll take a look.

If you can find a copy of Bayesian Data Analysis (BDA3, Gelman et al.), thereâ€™s some in-detail description of both LOO-CV and WAIC (as well as DIC) around pages 172-174.
I used that to write a Python script for computing the WAIC a while ago and if youâ€™re willing to go there itâ€™s fairly straightforward, although tedious and requiring a lot more calculations than the DIC (the latter requires only computing the logp at the posterior mean and the expected value of logp, while the former requires computing a version of that for each data point and mcmc sample).

That said, I would find it helpful if there was a pystan function to compute LOO and WAIC, since Stan developers seem to feel strongly against DIC and (god forbid) BIC.

This would be a great project for someone to develop in Python. In R, the loo package uses the output of RStan, and has special hooks for RStan, but is otherwise pretty general I believe given the right form of MCMC output.

Rightâ€”itâ€™s doing the equivalent of a posterior predictive calculation using full Bayes from the sample rather than a point estimate. Thatâ€™s also why youâ€™ll find

1 Like

I might have to compute a DIC myself (hereâ€™s the forum post). If we need to implement the DIC ourselves, then if anyoneâ€™s got a link to a sample data set and correct DIC values as a check on our work, Iâ€™d appreciate the post!

I defer to Aki Vehtari et al., who strongly discourage the use of DIC.

If you need it for some kind of reason you canâ€™t control, you might try the Coda package in R. Stanâ€™s output can be converted to Codaâ€™s input. But I donâ€™t know what it requires that might not be available in the RStan/Stan output. (Ack, canâ€™t put R in parens or I get Â®).