Double free or corruption (out) Error in unserialize(socklist[[n]]) : error reading from connection

Caio_Goncalves · August 16, 2022, 11:09pm

Performing a simple regression, the following error message appears:
I couldn’t identify the problem. Could it be something from the installation?

# options
options(mc.cores = parallel::detectCores()) 
rstan_options(auto_write = TRUE) 

# configurações para mcmc
chains <- 4
iter <- 1500 
warmup <- 700 
seed <- 12345677

# input data
input_data<- list(
  N = 100,
  x = rnorm(100),
  y = rnorm(100,2,3)
)

# parametrs
pars_model_1 <- c('alpha', 'sigma','beta')
# model
fit_model_1 <- stan(
  file= "linear_regression.stan",
  data= input_data,
  iter = iter + warmup,
  warmup = warmup,
  seed = seed,
  pars = pars_model_1
)

linear_regression.stan (198 Bytes)

Output:
starting worker pid=2626078 on localhost:11429 at 18:56:29.241
starting worker pid=2626091 on localhost:11429 at 18:56:29.446
starting worker pid=2626104 on localhost:11429 at 18:56:29.652
starting worker pid=2626117 on localhost:11429 at 18:56:29.864

SAMPLING FOR MODEL ‘linear_regression’ NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 3e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.3 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:
double free or corruption (out)

SAMPLING FOR MODEL ‘linear_regression’ NOW (CHAIN 2).
Chain 2:
Chain 2: Gradient evaluation took 4.5e-05 seconds
Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.45 seconds.
Chain 2: Adjust your expectations accordingly!
Chain 2:
Chain 2:
double free or corruption (out)
Error in unserialize(socklist[[n]]) : error reading from connection

Operating System
platform x86_64-redhat-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 4
minor 0.2
year 2020
month 06
day 22
svn rev 78730
language R
version.string R version 4.0.2 (2020-06-22)
nickname Taking Off Again
RStan Version
2.21.5
Output of writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))
Error in file(con, “r”) : cannot open the connection
In addition: Warning message:
In file(con, “r”) :
cannot open file ‘/home/ibge.gov.br/mbueno/.R/Makevars’: No such file or directory
Output of devtools::session_info("rstan")
─ Session info ──────────────────────────────────────────────────────────────────────────────────
setting value
version R version 4.0.2 (2020-06-22)
os CentOS Linux 8 (Core)
system x86_64, linux-gnu
ui RStudio
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz America/New_York
date 2022-08-16

─ Packages ──────────────────────────────────────────────────────────────────────────────────────
package * version date lib source
assertthat 0.2.1 2019-03-21 [2] CRAN (R 4.0.2)
backports 1.1.10 2020-09-15 [2] CRAN (R 4.0.2)
BH 1.72.0-3 2020-01-08 [2] CRAN (R 4.0.2)
brio 1.1.3 2021-11-30 [1] CRAN (R 4.0.2)
callr 3.7.0 2021-04-20 [1] CRAN (R 4.0.2)
checkmate 2.0.0 2020-02-06 [1] CRAN (R 4.0.2)
cli 3.3.0 2022-04-25 [1] CRAN (R 4.0.2)
colorspace 1.4-1 2019-03-18 [2] CRAN (R 4.0.2)
crayon 1.3.4 2017-09-16 [2] CRAN (R 4.0.2)
desc 1.4.1 2022-03-06 [1] CRAN (R 4.0.2)
diffobj 0.3.5 2021-10-05 [1] CRAN (R 4.0.2)
digest 0.6.25 2020-02-23 [2] CRAN (R 4.0.2)
ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.0.2)
evaluate 0.15 2022-02-18 [1] CRAN (R 4.0.2)
fansi 0.4.1 2020-01-08 [2] CRAN (R 4.0.2)
farver 2.0.3 2020-01-16 [2] CRAN (R 4.0.2)
ggplot2 * 3.3.6 2022-05-03 [1] CRAN (R 4.0.2)
glue 1.6.2 2022-02-24 [1] CRAN (R 4.0.2)
gridExtra 2.3 2017-09-09 [2] CRAN (R 4.0.2)
gtable 0.3.0 2019-03-25 [2] CRAN (R 4.0.2)
inline 0.3.19 2021-05-31 [1] CRAN (R 4.0.2)
isoband 0.2.2 2020-06-20 [2] CRAN (R 4.0.2)
jsonlite 1.8.0 2022-02-22 [1] CRAN (R 4.0.2)
labeling 0.3 2014-08-23 [2] CRAN (R 4.0.2)
lattice 0.20-41 2020-04-02 [2] CRAN (R 4.0.2)
lifecycle 1.0.1 2021-09-24 [1] CRAN (R 4.0.2)
loo 2.4.1 2020-12-09 [1] CRAN (R 4.0.2)
magrittr 1.5 2014-11-22 [2] CRAN (R 4.0.2)
MASS 7.3-51.6 2020-04-26 [2] CRAN (R 4.0.2)
Matrix 1.2-18 2019-11-27 [2] CRAN (R 4.0.2)
matrixStats 0.59.0 2021-06-01 [1] CRAN (R 4.0.2)
mgcv 1.8-31 2019-11-09 [2] CRAN (R 4.0.2)
munsell 0.5.0 2018-06-12 [2] CRAN (R 4.0.2)
nlme 3.1-148 2020-05-24 [2] CRAN (R 4.0.2)
pillar 1.4.6 2020-07-10 [2] CRAN (R 4.0.2)
pkgbuild 1.3.1 2021-12-20 [1] CRAN (R 4.0.2)
pkgconfig 2.0.3 2019-09-22 [2] CRAN (R 4.0.2)
pkgload 1.2.4 2021-11-30 [1] CRAN (R 4.0.2)
praise 1.0.0 2015-08-11 [2] CRAN (R 4.0.2)
prettyunits 1.1.1 2020-01-24 [2] CRAN (R 4.0.2)
processx 3.7.0 2022-07-07 [1] CRAN (R 4.0.2)
ps 1.3.4 2020-08-11 [2] CRAN (R 4.0.2)
R6 2.4.1 2019-11-12 [2] CRAN (R 4.0.2)
RColorBrewer 1.1-2 2014-12-07 [2] CRAN (R 4.0.2)
Rcpp 1.0.5 2020-07-06 [2] CRAN (R 4.0.2)
RcppEigen 0.3.3.9.1 2020-12-17 [1] CRAN (R 4.0.2)
RcppParallel 5.1.4 2021-05-04 [1] CRAN (R 4.0.2)
rematch2 2.1.2 2020-05-01 [2] CRAN (R 4.0.2)
rlang 1.0.2 2022-03-04 [1] CRAN (R 4.0.2)
rprojroot 2.0.2 2020-11-15 [1] CRAN (R 4.0.2)
rstan * 2.21.5 2022-04-11 [1] CRAN (R 4.0.2)
rstudioapi 0.11 2020-02-07 [2] CRAN (R 4.0.2)
scales 1.1.1 2020-05-11 [2] CRAN (R 4.0.2)
StanHeaders * 2.21.0-7 2020-12-17 [1] CRAN (R 4.0.2)
testthat 3.1.4 2022-04-26 [1] CRAN (R 4.0.2)
tibble 3.0.3 2020-07-10 [2] CRAN (R 4.0.2)
utf8 1.1.4 2018-05-24 [2] CRAN (R 4.0.2)
vctrs 0.4.1 2022-04-13 [1] CRAN (R 4.0.2)
viridisLite 0.3.0 2018-02-01 [2] CRAN (R 4.0.2)
waldo 0.4.0 2022-03-16 [1] CRAN (R 4.0.2)
withr 2.5.0 2022-03-03 [1] CRAN (R 4.0.2)

[1] /home/shared/DPE/gradepop/library
[2] /usr/lib64/R/library
[3] /usr/share/R/library

andrjohns · August 17, 2022, 8:01am

Can you try the preview of the next rstan version:

remove.packages(c("StanHeaders", "rstan"))
install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))

Caio_Goncalves · August 17, 2022, 9:34pm

The following outputs appear:

Removing packages from ‘/home/shared/DPE/gradepop/library’
(as ‘lib’ is unspecified)

install.packages(“rstan”, repos = c(“Repository for distributing (some) stan-dev R packages | r-packages”, getOption(“repos”)))
Error in install.packages : Updating loaded packages

Restarting R session…

install.packages(“rstan”, repos = c(“Repository for distributing (some) stan-dev R packages | r-packages”, getOption(“repos”)))
Loading required package: rstan
Error in .requirePackage(package) :
unable to find required package ‘rstan’
Loading required package: rstan
Error in .requirePackage(package) :
unable to find required package ‘rstan’
Loading required package: rstan
…

installing source package ‘rstan’ …
** package ‘rstan’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
g++ -m64 -std=gnu++14 -I"/usr/include/R" -DNDEBUG -I"…/inst/include" -I"…/inst/include/boost_not_in_BH" -I"." -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -DBOOST_NO_AUTO_PTR -D_REENTRANT -DSTAN_THREADS -I’/usr/lib64/R/library/Rcpp/include’ -I’/home/shared/DPE/gradepop/library/RcppEigen/include’ -I’/usr/lib64/R/library/BH/include’ -I’/home/shared/DPE/gradepop/library/StanHeaders/include’ -I’/home/shared/DPE/gradepop/library/RcppParallel/include’ -I/usr/local/include -fpic -O3 -mtune=native -arch x86_64 -ftemplate-depth-256 -c chains.cpp -o chains.o
g++: error: x86_64: No such file or directory
g++: error: unrecognized command line option ‘-arch’; did you mean ‘-march=’?
make: *** [/usr/lib64/R/etc/Makeconf:176: chains.o] Error 1
ERROR: compilation failed for package ‘rstan’
removing ‘/home/shared/DPE/gradepop/library/rstan’
Warning in install.packages :
installation of package ‘rstan’ had non-zero exit status

I can’t install with this command.

andrjohns · August 18, 2022, 12:55pm

The error there is:

-fpic -O3 -mtune=native -arch x86_64 -ftemplate-depth-256 -c chains.cpp -o chains.o
g++: error: x86_64: No such file or directory
g++: error: unrecognized command line option ‘-arch’; did you mean ‘-march=’?

Which indicates that you’ve followed the Mac instructions instead of the Linux instructions when configuring your installation. Run the following command to reconfigure this for Linux:

dotR <- file.path(Sys.getenv("HOME"), ".R")
if (!file.exists(dotR)) dir.create(dotR)
M <- file.path(dotR, "Makevars")
if (!file.exists(M)) file.create(M)
cat("\nCXXFLAGS=-O3 -march=native -mtune=native -fPIC",
    file = M, sep = "\n", append = FALSE)

Caio_Goncalves · August 18, 2022, 3:07pm

Thank you.
I used the commands:

dotR ← file.path(Sys.getenv(“HOME”), “.R”)
if (!file.exists(dotR)) dir.create(dotR)
M ← file.path(dotR, “Makevars”)
if (!file.exists(M)) file.create(M)
cat(“\nCXXFLAGS=-O3 -march=native -mtune=native -fPIC”,

```
file = M, sep = "\n", append = FALSE)
```

dotR
[1] “/home/ibge.gov.br/caio.soares/.R”
M
[1] “/home/ibge.gov.br/caio.soares/.R/Makevars”
file.edit(“~/.R/Makevars”)

Open the file with:
CXXFLAGS=-O3 -march=native -mtune=native -fPIC

Running again the model, the problem persists:

fit_model_1 ← stan(

file= “linear_regression.stan”,
data= input_data,
iter = iter + warmup,
warmup = warmup,
seed = seed,
pars = pars_model_1
)
starting worker pid=2700300 on localhost:11958 at 11:06:09.840
starting worker pid=2700313 on localhost:11958 at 11:06:10.041
starting worker pid=2700326 on localhost:11958 at 11:06:10.242
starting worker pid=2700339 on localhost:11958 at 11:06:10.447

SAMPLING FOR MODEL ‘linear_regression’ NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 8.1e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.81 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:
double free or corruption (out)

SAMPLING FOR MODEL ‘linear_regression’ NOW (CHAIN 2).
Chain 2:
Chain 2: Gradient evaluation took 3.5e-05 seconds
Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.35 seconds.
Chain 2: Adjust your expectations accordingly!
Chain 2:
Chain 2:
double free or corruption (out)
Error in unserialize(socklist[[n]]) : error reading from connection

end_time ← Sys.time()
end_time - start_time
Time difference of 5.874819 secs

andrjohns · August 19, 2022, 8:06am

Is this with the preview version installed?

Also, what do you get when you run the example model:

example(stan_model, package="rstan", run.dontrun=TRUE)

Caio_Goncalves · August 22, 2022, 8:34pm

It is the preview version. When I run install.packages(“rstan”, repos = c(“Repository for distributing (some) stan-dev R packages | r-packages”, getOption(“repos”))) it downloaded the 2.21.5 version.

install.packages(“rstan”, repos = c(“Repository for distributing (some) stan-dev R packages | r-packages”, getOption(“repos”)))
Installing package into ‘/home/shared/DPE/gradepop/library’
(as ‘lib’ is unspecified)
Warning in install.packages :
unable to access index for repository https://mc-stan.org/r-packages/src/contrib:
cannot open URL ‘https://mc-stan.org/r-packages/src/contrib/PACKAGES’
trying URL ‘https://cran.fiocruz.br/src/contrib/rstan_2.21.5.tar.gz’
Content type ‘application/x-gzip’ length 1189130 bytes (1.1 MB)

downloaded 1.1 MB
…

Now when I run the example, a new error appears:

example(stan_model, package=“rstan”, run.dontrun=TRUE)
Loading required package: StanHeaders
Loading required package: ggplot2
rstan (Version 2.21.5, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)

stn_md> stancode ← ‘data {real y_mean;} parameters {real y;} model {y ~ normal(y_mean,1);}’

stn_md> mod ← stan_model(model_code = stancode, verbose = TRUE)

TRANSLATING MODEL ‘16a540c6086086816528e4524def24d9’ FROM Stan CODE TO C++ CODE NOW.
successful in parsing the Stan model ‘16a540c6086086816528e4524def24d9’.

stn_md> fit ← sampling(mod, data = list(y_mean = 0))
Error in prep_call_sampler(object) :
could not find function “prep_call_sampler”

andrjohns · August 22, 2022, 8:48pm

The error there is:

Warning in install.packages :
unable to access index for repository https://mc-stan.org/r-packages/src/contrib:
cannot open URL ‘https://mc-stan.org/r-packages/src/contrib/PACKAGES’

I’m guessing that this is a compute cluster or work server with a firewall? An alternative is to download the package sources for rstan and StanHeaders from the repo here and pass them to the cluster to install manually

Caio_Goncalves · August 24, 2022, 7:06pm

Solved. I requested in my institution the release of urls for access and installed version 2.26. The example that I initially presented and the stan example now returns results without an error message. I appreciate all the attention.

Topic		Replies	Views
Some models run without problems, but more complex give error: Error in unserialize(socklist[[n]]) : error reading/writing from connection RStan	13	1346	November 2, 2021
Unserialize(socklist[[1]]) error on Mac Catalina RStan bug	4	563	August 25, 2021
Stan Connection problem Error in unserialize(socklist[[n]]) : error reading from connection General rstan , r , stan , paralellization	1	1219	September 11, 2021
Error in unserialize(socklist[[n]]) : error reading from connection RStan	8	2338	August 8, 2019
Error in unserialize(socklist[[n]]) : error reading from connection Error in serialize(data, node$con, xdr = FALSE) : error writing to connection General	3	3468	August 6, 2019

Double free or corruption (out) Error in unserialize(socklist[[n]]) : error reading from connection

Related topics