Some models run without problems, but more complex give error: Error in unserialize(socklist[[n]]) : error reading/writing from connection

Interfaces RStan
1

I have to use two PC-s for running models, since one of them can not run more complex models; however, simple ones run well. I have googled the error, but haven’t found a solution. Do you have any suggestions for solving such problem?

Such models run like charm:

fit_simple = brm(bf(y ~ p1, hu ~ p1), data = data_nl, family = hurdle_lognormal(), cores = 3, chains = 3)

And more complex ones like

fit_complex = brm(bf(y ~ p1 + p2 + p3 + p4 + p5 + p6 + p7, hu ~ p1 + p2 + p3 + p4 + p5 + p6 + p7), data = data_nl, family = hurdle_lognormal(), cores = 3, chains = 3)

Give the following error:

Error in unserialize(socklist[[n]]) : error reading from connection
16.
unserialize(socklist[[n]])
15.
recvOneData.SOCKcluster(cl)
14.
recvOneData(cl)
13.
recvOneResult(cl)
12.
dynamicClusterApply(cl, fun, length(x), argfun)
11.
clusterApplyLB(cl = cl, x = splitList(X, nchunks), fun = lapply, FUN = fun, ...)
10.
do.call(c, clusterApplyLB(cl = cl, x = splitList(X, nchunks), fun = lapply, FUN = fun, ...), quote = TRUE)
9.
parallel::parLapplyLB(cl, X = 1:chains, fun = callFun)
8.
.local(object, ...)
7.
.fun(object = .x1, data = .x2, pars = .x3, include = .x4, iter = .x5, seed = .x6, init = .x7, warmup = .x8, thin = .x9, control = .x10, show_messages = .x11, chains = .x12, cores = .x13)
6.
.fun(object = .x1, data = .x2, pars = .x3, include = .x4, iter = .x5, seed = .x6, init = .x7, warmup = .x8, thin = .x9, control = .x10, show_messages = .x11, chains = .x12, cores = .x13) at <text>#1
5.
eval(expr, envir, ...)
4.
eval(expr, envir, ...)
3.
eval2(call, envir = args, enclos = parent.frame())
2.
do_call(rstan::sampling, args)
1.
brm(bf(postacute_therapy ~ off_inpatient_rehabilitation + off_inpatient_nursing + off_other_inpatient + off_outpatient_physiotherapy + off_other_outpatient, hu ~ off_inpatient_rehabilitation + off_inpatient_nursing + off_other_inpatient + off_outpatient_physiotherapy + ...
Error in serialize(data, node$con, xdr = FALSE) : error writing to connection
15.
serialize(data, node$con, xdr = FALSE)
14.
sendData.SOCK0node(con, list(type = type, data = value, tag = tag))
13.
sendData(con, list(type = type, data = value, tag = tag))
12.
postNode(n, "DONE")
11.
stopNode(n)
10.
stopCluster.default(cl)
9.
parallel::stopCluster(cl)
8.
.local(object, ...)
7.
.fun(object = .x1, data = .x2, pars = .x3, include = .x4, iter = .x5, seed = .x6, init = .x7, warmup = .x8, thin = .x9, control = .x10, show_messages = .x11, chains = .x12, cores = .x13)
6.
.fun(object = .x1, data = .x2, pars = .x3, include = .x4, iter = .x5, seed = .x6, init = .x7, warmup = .x8, thin = .x9, control = .x10, show_messages = .x11, chains = .x12, cores = .x13) at <text>#1
5.
eval(expr, envir, ...)
4.
eval(expr, envir, ...)
3.
eval2(call, envir = args, enclos = parent.frame())
2.
do_call(rstan::sampling, args)
1.
brm(bf(postacute_therapy ~ off_inpatient_rehabilitation + off_inpatient_nursing + off_other_inpatient + off_outpatient_physiotherapy + off_other_outpatient, hu ~ off_inpatient_rehabilitation + off_inpatient_nursing + off_other_inpatient + off_outpatient_physiotherapy + ...

These solutions did not work out:

rstan_options(auto_write=TRUE)
options(mc.cores=parallel::detectCores())

Other info that may help:

utils::sessionInfo()

R version 3.6.1 (2019-07-05)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=Estonian_Estonia.1257  LC_CTYPE=Estonian_Estonia.1257    LC_MONETARY=Estonian_Estonia.1257 LC_NUMERIC=C                     
[5] LC_TIME=Estonian_Estonia.1257    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] MuMIn_1.43.17        RColorBrewer_1.1-2   survminer_0.4.6      ggeffects_0.15.0     coefplot_1.2.6       lme4_1.1-23         
 [7] Matrix_1.2-17        Hmisc_4.4-0          Formula_1.2-3        lattice_0.20-38      corrplot_0.84        ggalt_0.4.0         
[13] ggforce_0.3.1        factoextra_1.0.7     ggthemes_4.2.0       ggrepel_0.8.2        ggpubr_0.3.0.999     rstan_2.19.3        
[19] StanHeaders_2.21.0-1 tidybayes_2.0.3      pscl_1.5.5           boot_1.3-24          MASS_7.3-51.4        AER_1.2-9           
[25] survival_3.1-11      sandwich_2.5-1       lmtest_0.9-37        zoo_1.8-7            car_3.0-8            carData_3.0-4       
[31] knitr_1.28           bestNormalize_1.4.3  bayesplot_1.7.1      magrittr_1.5         skimr_2.1            brms_2.12.0         
[37] Rcpp_1.0.4.6         treemapify_2.5.3     broom_0.5.6          finalfit_1.0.0       forcats_0.5.0        stringr_1.4.0       
[43] dplyr_1.0.0          purrr_0.3.4          readr_1.3.1          tidyr_1.1.0          tibble_3.0.1         ggplot2_3.3.2       
[49] tidyverse_1.3.0      easypackages_0.1.0  

loaded via a namespace (and not attached):
  [1] tidyselect_1.1.0     htmlwidgets_1.5.1    grid_3.6.1           devtools_2.3.0       munsell_0.5.0        codetools_0.2-16    
  [7] statmod_1.4.34       DT_0.13              miniUI_0.1.1.1       withr_2.2.0          Brobdingnag_1.2-6    colorspace_1.4-1    
 [13] rstudioapi_0.11      stats4_3.6.1         ggsignif_0.6.0       Rttf2pt1_1.3.8       repr_1.1.0           KMsurv_0.1-5        
 [19] polyclip_1.10-0      farver_2.0.3         bridgesampling_1.0-0 rprojroot_1.3-2      coda_0.19-3          vctrs_0.3.1         
 [25] generics_0.0.2       xfun_0.13            R6_2.4.1             markdown_1.1         doParallel_1.0.15    assertthat_0.2.1    
 [31] promises_1.1.0       scales_1.1.1         nnet_7.3-12          gtable_0.3.0         ash_1.0-15           rethinking_1.59     
 [37] processx_3.4.2       rlang_0.4.6          splines_3.6.1        lazyeval_0.2.2       rstatix_0.5.0.999    extrafontdb_1.0     
 [43] acepack_1.4.1        checkmate_2.0.0      inline_0.3.15        yaml_2.2.1           reshape2_1.4.4       abind_1.4-5         
 [49] modelr_0.1.6         threejs_0.3.3        crosstalk_1.1.0.1    backports_1.1.7      useful_1.2.6         httpuv_1.5.2        
 [55] rsconnect_0.8.16     extrafont_0.17       tools_3.6.1          usethis_1.6.0        ellipsis_0.3.1       sessioninfo_1.1.1   
 [61] ggridges_0.5.2       plyr_1.8.6           base64enc_0.1-3      ps_1.3.3             prettyunits_1.1.1    rpart_4.1-15        
 [67] cowplot_1.0.0        cluster_2.1.0        haven_2.3.1          fs_1.4.1             data.table_1.12.8    openxlsx_4.1.5      
 [73] colourpicker_1.0     reprex_0.3.0         mvtnorm_1.0-12       packrat_0.5.0        matrixStats_0.56.0   pkgload_1.1.0       
 [79] evaluate_0.14        hms_0.5.3            shinyjs_1.1          mime_0.9             arrayhelpers_1.1-0   xtable_1.8-4        
 [85] shinystan_2.5.0      jpeg_0.1-8.1         rio_0.5.16           readxl_1.3.1         gridExtra_2.3        rstantools_2.0.0    
 [91] testthat_2.3.2       compiler_3.6.1       mice_3.8.0           maps_3.3.0           KernSmooth_2.23-15   crayon_1.3.4        
 [97] minqa_1.2.4          htmltools_0.4.0      later_1.0.0          lubridate_1.7.8      DBI_1.1.0            sjlabelled_1.1.5    
[103] tweenr_1.0.1         dbplyr_1.4.2         proj4_1.0-10         cli_2.0.2            insight_0.8.5        parallel_3.6.1      
[109] igraph_1.2.5         km.ci_0.5-2          pkgconfig_2.0.3      foreign_0.8-71       plotly_4.9.2.1       xml2_1.3.1          
[115] foreach_1.5.0        svUnit_0.7-12        dygraphs_1.1.1.6     rngtools_1.5         rvest_0.3.5          doRNG_1.8.2         
[121] callr_3.4.3          digest_0.6.25        cellranger_1.1.0     survMisc_0.5.5       htmlTable_1.13.3     curl_4.3            
[127] shiny_1.4.0.2        gtools_3.8.2         nloptr_1.2.2.1       lifecycle_0.2.0      nlme_3.1-140         jsonlite_1.6.1      
[133] viridisLite_0.3.0    desc_1.2.0           fansi_0.4.1          pillar_1.4.4         loo_2.2.0            fastmap_1.0.1       
[139] httr_1.4.1           pkgbuild_1.0.8       glue_1.4.1           xts_0.12-0           remotes_2.1.1        zip_2.0.4           
[145] png_0.1-7            shinythemes_1.1.2    iterators_1.0.12     stringi_1.4.6        ggfittext_0.8.1      latticeExtra_0.6-29 
[151] memoise_1.1.0
2

Need to run it with options(mc.cores = 1) to see the underlying error message.

3

Like running the following code before the model:

options(mc.cores = 1)

Error in unserialize(socklist[[n]]) : error reading from connection
16.
unserialize(socklist[[n]])
15.
recvOneData.SOCKcluster(cl)
14.
recvOneData(cl)
13.
recvOneResult(cl)
12.
dynamicClusterApply(cl, fun, length(x), argfun)
11.
clusterApplyLB(cl = cl, x = splitList(X, nchunks), fun = lapply, FUN = fun, ...)
10.
do.call(c, clusterApplyLB(cl = cl, x = splitList(X, nchunks), fun = lapply, FUN = fun, ...), quote = TRUE)
9.
parallel::parLapplyLB(cl, X = 1:chains, fun = callFun)
8.
.local(object, ...)
7.
.fun(object = .x1, data = .x2, pars = .x3, include = .x4, iter = .x5, seed = .x6, init = .x7, warmup = .x8, thin = .x9, control = .x10, show_messages = .x11, chains = .x12, cores = .x13)
6.
.fun(object = .x1, data = .x2, pars = .x3, include = .x4, iter = .x5, seed = .x6, init = .x7, warmup = .x8, thin = .x9, control = .x10, show_messages = .x11, chains = .x12, cores = .x13) at <text>#1
5.
eval(expr, envir, ...)
4.
eval(expr, envir, ...)
3.
eval2(call, envir = args, enclos = parent.frame())
2.
do_call(rstan::sampling, args)
1.
brm(bf(pt_overall_outpatient ~ off_overall_outpatient + year + (1 | county), hu ~ off_overall_outpatient + year + (1 | county)), data = data, family = hurdle_lognormal(), cores = 3, chains = 3)
Error in serialize(data, node$con, xdr = FALSE) : error writing to connection
15.
serialize(data, node$con, xdr = FALSE)
14.
sendData.SOCK0node(con, list(type = type, data = value, tag = tag))
13.
sendData(con, list(type = type, data = value, tag = tag))
12.
postNode(n, "DONE")
11.
stopNode(n)
10.
stopCluster.default(cl)
9.
parallel::stopCluster(cl)
8.
.local(object, ...)
7.
.fun(object = .x1, data = .x2, pars = .x3, include = .x4, iter = .x5, seed = .x6, init = .x7, warmup = .x8, thin = .x9, control = .x10, show_messages = .x11, chains = .x12, cores = .x13)
6.
.fun(object = .x1, data = .x2, pars = .x3, include = .x4, iter = .x5, seed = .x6, init = .x7, warmup = .x8, thin = .x9, control = .x10, show_messages = .x11, chains = .x12, cores = .x13) at <text>#1
5.
eval(expr, envir, ...)
4.
eval(expr, envir, ...)
3.
eval2(call, envir = args, enclos = parent.frame())
2.
do_call(rstan::sampling, args)
1.
brm(bf(pt_overall_outpatient ~ off_overall_outpatient + year + (1 | county), hu ~ off_overall_outpatient + year + (1 | county)), data = data, family = hurdle_lognormal(), cores = 3, chains = 3)
4

When you call it with cores = 3, that supersedes the options(mc.cores = 1) and uses multiple cores, which unfortunately obscures the true error message. Call it with cores = 1.

5

I am sorry! Tried now as you suggested.

Result: R session aborted, R encoutered a fatal error.

6

That sounds like the -march=native problem. Do you have anything like that in ~/.R/Makevars.win or ~/.R/Makevars ? RStan on CRAN should be ignoring the Makevars file when it finds -march=native but it is possible that it does not catch all patterns.

7

Thanks!

I found Makevars.win file from C:\Program Files\R\R-3.6.1\src\library\windlgs\src

Opened this with notepad and it included:

PKG_LIBS=$(GRAPHAPP_LIB)

Can I do something more?

8

Does anything come up if you check for Makevars and Makevars.win like this?

file.edit(file.path(Sys.getenv("HOME"), ".R", "Makevars.win"))
file.edit(file.path(Sys.getenv("HOME"), ".R", "Makevars"))

Also, as an alternative, brms now supports the new CmdStanR interface as an alternative to RStan (via the new backend argument) and I think using that would probably avoid these problems.

9

Thank you for helping! Yes, this gave me something after running a hurdle-lognormal hierarchical model with “options(mc.cores = 1)” and “cores = 1”.

First, R session aborted!

file.edit(file.path(Sys.getenv("HOME"), ".R", "Makevars.win"))

CXX14FLAGS=-O3 -march=native -mtune=native
CXX11FLAGS=-O3 -march=native -mtune=native

file.edit(file.path(Sys.getenv("HOME"), ".R", "Makevars"))

This code did not give anything. Empty page.

10

Can you try deleting -march=native from both of these lines (keep the rest)? Does that help?

1 Like
11

Jonah, you just bought me a new laptop capable of running brms models! Thank you so much!

1 Like
12

@jonah, I experienced the same error, i.e., “unserialize”, on OS X. Removing -march=native works for me too.

13

@torkar Interesting, I didn’t realize there were cases where removing -march=native was necessary on a Mac. That’s good to know, thanks!

14

for some reason I"m having the same issue and deleting -march=native still doesn’t solve my problem. I’m running MacOS Big Sur 11.1