Improve PSIS diagnostic plot

General
1

I was plotting the PSIS diagnostic plot with the plot method. However, it automatically generated thresholds at 0 and 0.5. I want to revise the threshold to 0.7, and also change the font of the labels and axis labels so that it would be easier to read on publication. I did “plot(x, threshold = 0.7, font.lab=2, cex.lab=1.2)”, but it neither changes the threshold nor the font. Any idea here?

See current figure here: k_diagnostics_2pc_1knots.pdf (6.9 KB)

1 Like
2

Thanks for the suggestion. We can definitely add some more options to control this. If you don’t mind, can you open an issue for this at https://github.com/stan-dev/loo/issues? Thanks!

3

Meanwhile you could use something like this

loo_1 <- loo(...)
pkdf<-data.frame(pk=loo_1$diagnostics$pareto_k, id=1:n)
ggplot(pkdf, aes(x=id,y=pk)) + geom_point(shape=3, color="blue") +
    labs(x="Observation left out", y="Pareto k") +
    geom_hline(yintercept = 0, linetype=3, color="red", size=0.2) +
    geom_hline(yintercept = 0.5, linetype=4, color="red", size=0.2) +
    geom_hline(yintercept = 0.7, linetype=2, color="red", size=0.2) +
    scale_y_continuous(breaks=c(0, 0.25, 0.5, 0.7), limits=c(-0.1,0.7)) +
    ggtitle("PSIS-LOO diagnostics")
1 Like
4

Or just pareto_k_values(loo_1)

Also I think doing plot(...) and then abline(h=0.7), possibly with additional style arguments, probably works as a short term option until we get more functionality into the next official release.

1 Like