Using loo with cmdstanr

I want to use functionality from the loo package with output from cmdstanr. I’ve got log_lik in the generated quantities block of the model but as the output from cmdstanr sampling is not a stanfit object I can’t use extract_log_lik or similar functions it seems. Is there a straightforward way of using the loo functionality with cmdstanr output or any guidance or examples anywhere?

Yeah it should just be

# assuming 'fit' is your fitted model object and you used the name 'log_lik' in generated quantities  
loo(fit$draws("log_lik"))

Does that work for you?

3 Likes

Ah yes it does! That was simpler than I thought. Thanks!

2 Likes

I added an issue for cmdstanr https://github.com/stan-dev/cmdstanr/issues/366
as it would be quite easy to add support for loo(fit) that would also take into account r_eff part missing in the above answer.

1 Like