Loo 2.0: error for stanreg object

Interfaces rstanarm
1

Dear Stan users,
I’m looking at the latest version of the loo package (2.0.0 - thank you Stan developers!) and I’m getting the following error when computing PSIS-LOO for an rstanarm object:

loo1 <- loo(fit1, save_psis = TRUE)
Error: is.data.frame(data) || is.matrix(data) is not TRUE

Below there is a complete example (following http://mc-stan.org/loo/articles/loo2-example.html).
Note that it works correctly using log_lik, like here:

loo1 <- loo(log_lik(fit1), save_psis = TRUE)

In addition, loo works fine with rstan objects (eg, following http://mc-stan.org/loo/articles/loo2-with-rstan.html).

I wanted to check if this is a general issue to be fixed or there is something specific about my machine.

Thank you.
Luca

library("rstanarm")
library("bayesplot")
library("loo")

options(mc.cores = parallel::detectCores())

data(roaches)
str(roaches)

roaches$roach1 <- roaches$roach1 / 100

fit1 <-
  stan_glm(
    formula = y ~ roach1 + treatment + senior,
    offset = log(exposure2),
    data = roaches,
    family = poisson(link = "log"),
    prior = normal(0, 2.5, autoscale = TRUE),
    prior_intercept = normal(0, 5, autoscale = TRUE),
    seed = 12345
  )

loo1 <- loo(fit1, save_psis = TRUE)
# Error: is.data.frame(data) || is.matrix(data) is not TRUE

loo1 <- loo(log_lik(fit1), save_psis = TRUE)

print(loo1)

# Computed from 4000 by 262 log-likelihood matrix
# 
#          Estimate     SE
# elpd_loo  -6244.5  726.8
# p_loo       292.5   72.9
# looic     12489.1 1453.7
# ------
#   Monte Carlo SE of elpd_loo is NA.
# 
# Pareto k diagnostic values:
#                          Count Pct.    Min. n_eff
# (-Inf, 0.5]   (good)     239   91.2%   399       
#  (0.5, 0.7]   (ok)         8    3.1%   84        
#    (0.7, 1]   (bad)        8    3.1%   13        
#    (1, Inf)   (very bad)   7    2.7%   1         
# See help('pareto-k-diagnostic') for details.

sessionInfo()

# R version 3.4.4 (2018-03-15)
# Platform: x86_64-pc-linux-gnu (64-bit)
# Running under: Ubuntu 16.04.4 LTS
# 
# Matrix products: default
# BLAS: /usr/lib/libblas/libblas.so.3.6.0
# LAPACK: /usr/lib/lapack/liblapack.so.3.6.0
# 
# locale:
#   [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
# [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
# [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
# 
# attached base packages:
#   [1] stats     graphics  grDevices utils     datasets  methods   base     
# 
# other attached packages:
#   [1] loo_2.0.0       bayesplot_1.5.0 rstanarm_2.17.3 Rcpp_0.12.16   
# 
# loaded via a namespace (and not attached):
#   [1] lattice_0.20-35    zoo_1.8-1          gtools_3.5.0       assertthat_0.2.0   digest_0.6.15      mime_0.5           R6_2.2.2          
# [8] plyr_1.8.4         ggridges_0.5.0     stats4_3.4.4       ggplot2_2.2.1      colourpicker_1.0   pillar_1.2.1       rlang_0.2.0       
# [15] lazyeval_0.2.1     minqa_1.2.4        miniUI_0.1.1       nloptr_1.0.4       Matrix_1.2-14      DT_0.4             shinythemes_1.1.1 
# [22] splines_3.4.4      shinyjs_1.0        lme4_1.1-17        stringr_1.3.0      htmlwidgets_1.0    igraph_1.2.1       munsell_0.4.3     
# [29] shiny_1.0.5        compiler_3.4.4     httpuv_1.3.6.2     rstan_2.17.3       pkgconfig_2.0.1    base64enc_0.1-3    rstantools_1.4.0  
# [36] htmltools_0.3.6    tibble_1.4.2       gridExtra_2.3      codetools_0.2-15   matrixStats_0.53.1 threejs_0.3.1      dplyr_0.7.4       
# [43] MASS_7.3-49        grid_3.4.4         nlme_3.1-137       xtable_1.8-2       gtable_0.2.0       magrittr_1.5       StanHeaders_2.17.2
# [50] scales_0.5.0       stringi_1.1.7      reshape2_1.4.3     bindrcpp_0.2.2     dygraphs_1.1.1.4   xts_0.10-2         tools_3.4.4       
# [57] glue_1.2.0         shinystan_2.4.0    markdown_0.8       crosstalk_1.0.0    survival_2.41-3    parallel_3.4.4     rsconnect_0.8.8   
# [64] yaml_2.1.18        inline_0.3.14      colorspace_1.3-2   bindr_0.1.1
2

Sorry about this. We tried to avoid this, but we’re waiting on CRAN to accept the rstanarm update that makes sure this error doesn’t happen (that’s why we haven’t announced loo 2.0 officially yet).

Until the rstanarm update installable from CRAN, In the meantime, assuming your version of loo is 2.0.0 and your version of rstanarm is 2.17.3 then the easiest way to fix this issue until the rstanarm update is on CRAN is to keep your rstanarm version but install the previous release of the loo package, which you can do with:

# first start fresh R session, then:
library("devtools") # install.packages("devtools") if necessary
remove.packages("loo")
install_github("stan-dev/loo", ref = "v1.1.0")

Alternatively, you could also install the new rstanarm from GitHub but that’s more of a pain because of compilation.

We’ll officially announce the release of loo 2 once rstanarm also makes it through the process.

3

Thank you, Jonah. I appreciate your detailed answers.

4

No problem, and sorry for the hassle! There shouldn’t be any backwards compatibility issues once both packages can actually be installed together.

6

@Luca_Campanelli Just following up to let you know that the loo and rstanarm versions on CRAN are now in sync.

7

thank you, Jonah, and thanks to all Stan developers for your work.