Error in unserialize(socklist[[n]]) : error reading from connection when specifiying adapt_delta

Interfaces RStan
1

Hello, I’ve encountered some strange behavior wherein stan throws the error ## Error in unserialize(socklist[[n]]) : error reading from connection upon running a stan program in parallel (or Rstudio crashes upon running sequentially). I am aware that other uses have encountered this issue but in my case the cause seems unrelated. Specifically, if I leave control at its default value, the program runs fine, but if I explicitly set control = list(adapt_delta = 0.8), the aforementioned error gets thrown. A reprex is below along with system information. Any thoughts?
Thanks, Phil

library(rstan)
#> Loading required package: StanHeaders
#> Loading required package: ggplot2
#> rstan (Version 2.21.2, GitRev: 2e1f913d3ca3)
#> For execution on a local, multicore CPU with excess RAM we recommend calling
#> options(mc.cores = parallel::detectCores()).
#> To avoid recompilation of unchanged Stan programs, we recommend calling
#> rstan_options(auto_write = TRUE)
options(mc.cores = parallel::detectCores())
rstan_options(auto_write = TRUE)
stan_normal <- 
  "
data {
  int<lower=0> N;
  vector[N] y;
}
parameters {
  real mu;
  real<lower=0> sigma;
}
model {
  y ~ normal(mu, sigma);
}
"

fit <- 
  stan(model_code = stan_normal,
       data = list(N  = as.integer(100), 
                   y = rnorm(100)))
#> Trying to compile a simple C file
#> Running /Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB foo.c
#> clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I"/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include/"  -I"/Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library/RcppEigen/include/"  -I"/Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library/RcppEigen/include/unsupported"  -I"/Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library/BH/include" -I"/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/src/"  -I"/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/"  -I"/Library/Frameworks/R.framework/Versions/4.1/Resources/library/RcppParallel/include/"  -I"/Library/Frameworks/R.framework/Versions/4.1/Resources/library/rstan/include" -DEIGEN_NO_DEBUG  -DBOOST_DISABLE_ASSERTS  -DBOOST_PENDING_INTEGER_LOG2_HPP  -DSTAN_THREADS  -DBOOST_NO_AUTO_PTR  -include '/Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp'  -D_REENTRANT -DRCPP_PARALLEL_USE_TBB=1   -I/usr/local/include   -fPIC  -Wall -g -O2  -c foo.c -o foo.o
#> In file included from <built-in>:1:
#> In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
#> In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library/RcppEigen/include/Eigen/Dense:1:
#> In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library/RcppEigen/include/Eigen/Core:88:
#> /Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library/RcppEigen/include/Eigen/src/Core/util/Macros.h:628:1: error: unknown type name 'namespace'
#> namespace Eigen {
#> ^
#> /Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library/RcppEigen/include/Eigen/src/Core/util/Macros.h:628:16: error: expected ';' after top level declarator
#> namespace Eigen {
#>                ^
#>                ;
#> In file included from <built-in>:1:
#> In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
#> In file included from /Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library/RcppEigen/include/Eigen/Dense:1:
#> /Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library/RcppEigen/include/Eigen/Core:96:10: fatal error: 'complex' file not found
#> #include <complex>
#>          ^~~~~~~~~
#> 3 errors generated.
#> make: *** [foo.o] Error 1

fit2 <- 
  stan(model_code = stan_normal,
       data = list(N  = as.integer(100), 
                   y = rnorm(100)), 
       control = list(adapt_delta = 0.8))
#> Warning in mccollect(jobs): 4 parallel jobs did not deliver results
#> Error in FUN(X[[i]], ...): trying to get slot "mode" from an object of a basic class ("NULL") with no slots

Created on 2021-07-26 by the reprex package (v2.0.0)

Other information:

  • Operating system: Mac OS Catalina 10.15.7
  • Rstan version: 2.21.2
writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))

CXX14FLAGS += -O3 -mtune=native -arch x86_64 -ftemplate-depth-256

devtools::session_info("rstan")
─ Session info ───────────────────────────────────────────────────────────
 setting  value                       
 version  R version 4.1.0 (2021-05-18)
 os       macOS Catalina 10.15.7      
 system   x86_64, darwin17.0          
 ui       RStudio                     
 language (EN)                        
 collate  en_US.UTF-8                 
 ctype    en_US.UTF-8                 
 tz       America/Detroit             
 date     2021-07-26                  

─ Packages ───────────────────────────────────────────────────────────────
 package      * version   date       lib source        
 backports      1.2.1     2020-12-09 [2] CRAN (R 4.1.0)
 BH             1.75.0-0  2021-01-11 [2] CRAN (R 4.1.0)
 callr          3.7.0     2021-04-20 [2] CRAN (R 4.1.0)
 checkmate      2.0.0     2020-02-06 [2] CRAN (R 4.1.0)
 cli            2.5.0     2021-04-26 [2] CRAN (R 4.1.0)
 colorspace     2.0-1     2021-05-04 [2] CRAN (R 4.1.0)
 crayon         1.4.1     2021-02-08 [2] CRAN (R 4.1.0)
 curl           4.3.1     2021-04-30 [2] CRAN (R 4.1.0)
 desc           1.3.0     2021-03-05 [1] CRAN (R 4.1.0)
 digest         0.6.27    2020-10-24 [2] CRAN (R 4.1.0)
 ellipsis       0.3.2     2021-04-29 [2] CRAN (R 4.1.0)
 fansi          0.5.0     2021-05-25 [2] CRAN (R 4.1.0)
 farver         2.1.0     2021-02-28 [2] CRAN (R 4.1.0)
 ggplot2      * 3.3.3     2020-12-30 [2] CRAN (R 4.1.0)
 glue           1.4.2     2020-08-27 [2] CRAN (R 4.1.0)
 gridExtra      2.3       2017-09-09 [2] CRAN (R 4.1.0)
 gtable         0.3.0     2019-03-25 [2] CRAN (R 4.1.0)
 inline         0.3.19    2021-05-31 [1] CRAN (R 4.1.0)
 isoband        0.2.4     2021-03-03 [2] CRAN (R 4.1.0)
 jsonlite       1.7.2     2020-12-09 [2] CRAN (R 4.1.0)
 labeling       0.4.2     2020-10-20 [2] CRAN (R 4.1.0)
 lattice        0.20-44   2021-05-02 [1] CRAN (R 4.1.0)
 lifecycle      1.0.0     2021-02-15 [2] CRAN (R 4.1.0)
 loo            2.4.1     2020-12-09 [1] CRAN (R 4.1.0)
 magrittr       2.0.1     2020-11-17 [2] CRAN (R 4.1.0)
 MASS           7.3-54    2021-05-03 [1] CRAN (R 4.1.0)
 Matrix         1.3-4     2021-06-01 [1] CRAN (R 4.1.0)
 matrixStats    0.58.0    2021-01-29 [2] CRAN (R 4.1.0)
 mgcv           1.8-35    2021-04-18 [1] CRAN (R 4.1.0)
 munsell        0.5.0     2018-06-12 [2] CRAN (R 4.1.0)
 nlme           3.1-152   2021-02-04 [1] CRAN (R 4.1.0)
 pillar         1.6.1     2021-05-16 [2] CRAN (R 4.1.0)
 pkgbuild       1.2.0     2020-12-15 [1] CRAN (R 4.1.0)
 pkgconfig      2.0.3     2019-09-22 [2] CRAN (R 4.1.0)
 prettyunits    1.1.1     2020-01-24 [2] CRAN (R 4.1.0)
 processx       3.5.2     2021-04-30 [2] CRAN (R 4.1.0)
 ps             1.6.0     2021-02-28 [2] CRAN (R 4.1.0)
 R6             2.5.0     2020-10-28 [2] CRAN (R 4.1.0)
 RColorBrewer   1.1-2     2014-12-07 [2] CRAN (R 4.1.0)
 Rcpp           1.0.7     2021-07-07 [1] CRAN (R 4.1.0)
 RcppEigen      0.3.3.9.1 2020-12-17 [2] CRAN (R 4.1.0)
 RcppParallel   5.1.4     2021-05-04 [1] CRAN (R 4.1.0)
 rlang          0.4.11    2021-04-30 [2] CRAN (R 4.1.0)
 rprojroot      2.0.2     2020-11-15 [1] CRAN (R 4.1.0)
 rstan        * 2.21.2    2020-07-27 [1] CRAN (R 4.1.0)
 scales         1.1.1     2020-05-11 [2] CRAN (R 4.1.0)
 StanHeaders  * 2.21.0-7  2020-12-17 [1] CRAN (R 4.1.0)
 tibble         3.1.2     2021-05-16 [2] CRAN (R 4.1.0)
 utf8           1.2.1     2021-03-12 [2] CRAN (R 4.1.0)
 V8             3.4.2     2021-05-01 [1] CRAN (R 4.1.0)
 vctrs          0.3.8     2021-04-29 [2] CRAN (R 4.1.0)
 viridisLite    0.4.0     2021-04-13 [2] CRAN (R 4.1.0)
 withr          2.4.2     2021-04-18 [2] CRAN (R 4.1.0)

[1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library
[2] /Library/Frameworks/R.framework/Versions/4.1/Resources/miworkspace-library
1 Like
2

I should note that the output from the reprex is a little misleading, as it does not report the exact error. When I run the code in my above reprex ‘live’ I get the following:

> fit2 <- 
+   stan(model_code = stan_normal,
+        data = list(N  = as.integer(100), 
+                    y = rnorm(100)), 
+        control = list(adapt_delta = 0.8))
starting worker pid=16424 on localhost:11995 at 15:32:40.165
starting worker pid=16438 on localhost:11995 at 15:32:40.537
starting worker pid=16452 on localhost:11995 at 15:32:40.870
starting worker pid=16466 on localhost:11995 at 15:32:41.240

SAMPLING FOR MODEL 'b968140be90012eecf61d55b786a5276' NOW (CHAIN 1).
Error in unserialize(socklist[[n]]) : error reading from connection

SAMPLING FOR MODEL 'b968140be90012eecf61d55b786a5276' NOW (CHAIN 3).

SAMPLING FOR MODEL 'b968140be90012eecf61d55b786a5276' NOW (CHAIN 2).
1 Like
3

Would trying with cmdstanr be an option for you?

1 Like
4

Hi and thank you for the response and suggestion. It would indeed be an option and it works fine in this particular example. A student I’m working with has made the switch to cmdstan from rstan, and thus far I’ve been putting off making the switch myself, but this may be the impetus to do so. I’m convinced that this error – which I’ve not been able to recreate on any other machine and which only recently cropped up seemingly out of the blue – is stemming from some clash of versions in my set up. Thanks again

1 Like