Error in unserialize(socklist[[n]]), running sampling on rstudio server on AWS


#1

I’m running rstudioserver using the AMI: ami-3b0c205e on an EC2 AWS instance to run a Stan model that I developed locally on my OSX machine.

Everything seems to work fine and optimizing seems to work fine on the model as well. However, when I run my sampling call, I get the error Error in unserialize(socklist[[n]]) : error reading from connection. I considered that this might be a memory issue and have tried the xlarge (4 cores, 16gb RAM) and 2xlarge (8 cores, 32gb RAM) instances and got the same error both times. When I run this on my mac, which has 16gb of RAM, sampling is slow but works fine.

The code I’m using to run the model is here, though it requires data not on github to run. The model itself is here.

Does anyone know what might be going on?

Below is the session info from the aws rstudioserver session:

sessionInfo()
R version 3.4.2 (2017-09-28)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.3 LTS

Matrix products: default
BLAS: /usr/lib/openblas-base/libblas.so.3
LAPACK: /usr/lib/libopenblasp-r0.2.18.so

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] rstan_2.16.2 StanHeaders_2.16.0-1 bindrcpp_0.2.2 forcats_0.3.0 stringr_1.2.0 dplyr_0.7.4
[7] purrr_0.2.4 readr_1.1.1 tidyr_0.8.0 tibble_1.4.2 ggplot2_2.2.1 tidyverse_1.2.1

loaded via a namespace (and not attached):
[1] Rcpp_0.12.16 cellranger_1.1.0 pillar_1.2.2 compiler_3.4.2 plyr_1.8.4 bindr_0.1.1 tools_3.4.2
[8] lubridate_1.7.4 jsonlite_1.5 gtable_0.2.0 nlme_3.1-131 lattice_0.20-35 pkgconfig_2.0.1 rlang_0.2.0
[15] psych_1.8.4 cli_1.0.0 rstudioapi_0.7 yaml_2.1.14 parallel_3.4.2 haven_1.1.1 gridExtra_2.3
[22] xml2_1.2.0 httr_1.3.1 hms_0.4.2 stats4_3.4.2 grid_3.4.2 glue_1.2.0 inline_0.3.14
[29] R6_2.2.2 readxl_1.1.0 foreign_0.8-69 modelr_0.1.2 reshape2_1.4.2 magrittr_1.5 codetools_0.2-15
[36] scales_0.5.0 rvest_0.3.2 assertthat_0.2.0 mnormt_1.5-5 colorspace_1.3-2 stringi_1.1.5 lazyeval_0.2.0
[43] munsell_0.4.3 broom_0.4.4 crayon_1.3.4 zoo_1.8-1


#2

Does it work on AWS with chains = 1? You can just do a few iterations to find out.


#3

I also ran into this error. In my case it was due to running out of memory. I was using the transformed parameters block and that really increased memory usage. Bob had suggested doing the transformation in the model block and that pratically helped me solve the problem without having to upsize the instance.


#4

Thanks for the replies, guys! I was using the transformed params block excessively too, so following @statwonk’s advice I moved most of the heavy memory stuff out. I then tried out running 20 iterations with chains = 1 and my rsession failed.

Perhaps there’s a deeper problem with my Stan model?


#5

Possibly. Are there logs / messages as the R session failed?


#6

Just this
image


#7

I would try starting an R session and noting what tempdir() is. Then, in another shell go to that temporary directory and watch the log file (the one without an extension) until R crashes. My guess is stack overflow.