Can't continue session after stan Error. cannot open file '/tmp/Rtmp1...'

Interfaces brms
, ,
1

If I get a stan error when trying to fit a brms model, I can’t do any more fits due to temp files going missing. Instead, I need to start a new R session.

My current hypothesis is that brms removes the /tmp/Rtmp.... dir, but stan thinks it’s still there. Is there a way to prevent stan for looking for these files?

Here’s some example output.

## Define model
nlform <- bf(song_count ~ rmax*exp(-0.5*(fabs(temp_mean - topt)/a)^b), 
             rmax ~ male,
             topt ~ 1 + (1|male),
             a ~ 1,
             b ~1,
             nl = TRUE)

nlprior <- c(prior(gamma(1., 0.005), nlpar = "rmax", lb = 0),
                 prior(normal(35, 10.), nlpar = "topt", lb = 0),
                 prior(gamma(1., 0.01), nlpar = "a", lb = 0),
                 prior(normal(0, 5), nlpar = "b", lb = 0))


## Fitting leads to an Error

fit_poisson1 <- brm(formula = nlform,
                   data = data,
                   family = poisson(link = "identity"),
                   prior = nlprior,
                   cores = 5,
                   iter = 10000
                   )

Compiling Stan program...
Start sampling

SAMPLING FOR MODEL '05778152ec866a4598b4d03722b38125' NOW (CHAIN 1).
Chain 1: Rejecting initial value:
Chain 1:   Log probability evaluates to log(0), i.e. negative infinity.
Chain 1:   Stan can't start sampling from this initial value.
Chain 1: Rejecting initial value:
Chain 1:   Log probability evaluates to log(0), i.e. negative infinity.
Chain 1:   Stan can't start sampling from this initial value.
Chain 1: 
Chain 1: Gradient evaluation took 9.4e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.94 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1: 
Chain 1: 
Chain 1: Iteration:    1 / 10000 [  0%]  (Warmup)

SAMPLING FOR MODEL '05778152ec866a4598b4d03722b38125' NOW (CHAIN 2).
Chain 2: Rejecting initial value:
Chain 2:   Log probability evaluates to log(0), i.e. negative infinity.
Chain 2:   Stan can't start sampling from this initial value.
Chain 2: Rejecting initial value:
Chain 2:   Log probability evaluates to log(0), i.e. negative infinity.
Chain 2:   Stan can't start sampling from this initial value.
Chain 2: 
Chain 2: Gradient evaluation took 9.5e-05 seconds
Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0.95 seconds.
Chain 2: Adjust your expectations accordingly!
Chain 2: 
Chain 2: 
Chain 2: Iteration:    1 / 10000 [  0%]  (Warmup)

SAMPLING FOR MODEL '05778152ec866a4598b4d03722b38125' NOW (CHAIN 3).
Chain 3: 
Chain 3: Gradient evaluation took 9.6e-05 seconds
Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 0.96 seconds.
Chain 3: Adjust your expectations accordingly!
Chain 3: 
Chain 3: 
Chain 3: Iteration:    1 / 10000 [  0%]  (Warmup)

SAMPLING FOR MODEL '05778152ec866a4598b4d03722b38125' NOW (CHAIN 4).
Chain 4: 
Chain 4: Gradient evaluation took 9.4e-05 seconds
Chain 4: 1000 transitions using 10 leapfrog steps per transition would take 0.94 seconds.
Chain 4: Adjust your expectations accordingly!
Chain 4: 
Chain 4: 
Chain 4: Iteration:    1 / 10000 [  0%]  (Warmup)
Error in FUN(X[[i]], ...) : 
  trying to get slot "mode" from an object of a basic class ("NULL") with no slots
In addition: Warning message:
In mccollect(jobs) : 4 parallel jobs did not deliver results

## Now try to refit model
> 
> fit_poisson1 <- brm(formula = nlform,
+                    data = data,
+                    family = poisson(link = "identity"),
+                    prior = nlprior,
+                    cores = 1,
+                    iter = 10000
+                    )
Compiling Stan program...
Error in file(con, "w") : cannot open the connection
In addition: Warning message:
In file(con, "w") :
  cannot open file '/tmp/Rtmp19xeG2/filec73ba63ded174': No such file or directory

## dir /tmp/Rtmp19... seems to have disappeared
> dir("/tmp/Rtmp*")
character(0)

Here’s my sessionInfo

> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.1 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  utils     datasets  grDevices methods   base     

other attached packages:
 [1] latex2exp_0.9.5      weathermetrics_1.2.2 humidity_0.1.5      
 [4] reshape2_1.4.4       GGally_2.1.2         viridisLite_0.4.1   
 [7] ggpubr_0.4.0         forcats_0.5.2        stringr_1.4.1       
[10] dplyr_1.0.99.9000    purrr_0.3.4          readr_2.1.3         
[13] tidyr_1.2.0.9000     tibble_3.1.8         ggplot2_3.3.6       
[16] tidyverse_1.3.2      brms_2.18.0          Rcpp_1.0.9          
[19] broom_1.0.0          nls.multstart_1.2.0  rTPC_1.0.2          
[22] devtools_2.4.4       usethis_2.1.6        default_1.0.0       

loaded via a namespace (and not attached):
  [1] readxl_1.4.1          backports_1.4.1       plyr_1.8.7           
  [4] igraph_1.3.4          crosstalk_1.2.0       rstantools_2.2.0     
  [7] inline_0.3.19         digest_0.6.29         htmltools_0.5.3      
 [10] fansi_1.0.3           magrittr_2.0.3        checkmate_2.1.0      
 [13] memoise_2.0.1         googlesheets4_1.0.1   tzdb_0.3.0           
 [16] remotes_2.4.2         modelr_0.1.9          RcppParallel_5.1.5   
 [19] matrixStats_0.62.0    xts_0.12.1            prettyunits_1.1.1    
 [22] colorspace_2.0-3      rvest_1.0.3           haven_2.5.1          
 [25] xfun_0.32             callr_3.7.2           crayon_1.5.1         
 [28] jsonlite_1.8.0        zoo_1.8-10            glue_1.6.2           
 [31] gtable_0.3.1          gargle_1.2.1          emmeans_1.8.0        
 [34] distributional_0.3.1  car_3.1-0             pkgbuild_1.3.1       
 [37] rstan_2.21.7          abind_1.4-5           scales_1.2.1         
 [40] mvtnorm_1.1-3         DBI_1.1.3             rstatix_0.7.0        
 [43] miniUI_0.1.1.1        xtable_1.8-4          stats4_4.2.1         
 [46] StanHeaders_2.21.0-7  DT_0.24               profvis_0.3.7        
 [49] htmlwidgets_1.5.4     httr_1.4.4            threejs_0.3.3        
 [52] RColorBrewer_1.1-3    posterior_1.3.1       ellipsis_0.3.2       
 [55] urlchecker_1.0.1      pkgconfig_2.0.3       reshape_0.8.9        
 [58] loo_2.5.1             farver_2.1.1          dbplyr_2.2.1.9000    
 [61] utf8_1.2.2            labeling_0.4.2        tidyselect_1.1.2.9000
 [64] rlang_1.0.5           later_1.3.0           munsell_0.5.0        
 [67] cellranger_1.1.0      tools_4.2.1           cachem_1.0.6         
 [70] cli_3.4.0             generics_0.1.3        ggridges_0.5.4       
 [73] fastmap_1.1.0         yaml_2.3.5            processx_3.7.0       
 [76] knitr_1.40            fs_1.5.2              nlme_3.1-159         
 [79] mime_0.12             xml2_1.3.3            compiler_4.2.1       
 [82] bayesplot_1.9.0       shinythemes_1.2.0     ggsignif_0.6.3       
 [85] reprex_2.0.2          stringi_1.7.8         ps_1.7.1             
 [88] Brobdingnag_1.2-9     lattice_0.20-45       Matrix_1.5-1         
 [91] markdown_1.1          shinyjs_2.1.0         tensorA_0.36.2       
 [94] vctrs_0.4.1.9000      pillar_1.8.1          lifecycle_1.0.1      
 [97] bridgesampling_1.1-2  estimability_1.4.1    httpuv_1.6.5         
[100] R6_2.5.1              promises_1.2.0.1      gridExtra_2.3        
[103] sessioninfo_1.2.2     codetools_0.2-18      colourpicker_1.1.1   
[106] gtools_3.9.3          assertthat_0.2.1      pkgload_1.3.0        
[109] withr_2.5.0           shinystan_2.6.0       parallel_4.2.1       
[112] hms_1.1.2             grid_4.2.1            coda_0.19-4          
[115] carData_3.0-5         googledrive_2.0.0     shiny_1.7.2          
[118] lubridate_1.8.0       base64enc_0.1-3       dygraphs_1.1.1.6     
>

If you need to use math formula, use Latex syntax:

Y \sim N(\mu, \sigma)

Don’t forget to attach tags (top right of this form) for application area/class of models or other general subject areas your topic touches on.

2

It doesn’t look any of your brm runs are working, so I think there might be a bigger issue. Are you able to run any brms models without error?

3

Yes, it does seem to be a bigger problems. Sometimes I can get brms to run, but often it crashed. For example, if I try to run the following code from the brm() help page, I get:

> 
> prior1 <- prior(normal(0,10), class = b) +
+     prior(cauchy(0,2), class = sd)
> fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient),
+             data = epilepsy, family = poisson(),
+             prior = prior1, chains = 1, cores = 1)
Compiling Stan program...
Start sampling

SAMPLING FOR MODEL 'cc0805f6ac2e9417b96c5709bd688d91' NOW (CHAIN 1).
Chain 1: 
Chain 1: Gradient evaluation took 6.2e-05 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.62 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1: 
Chain 1: 
Chain 1: Iteration:    1 / 2000 [  0%]  (Warmup)
Selection:

Note that I can run example(stan_model, package = "rstan", run.dontrun = TRUE) without any problem and have reinstalled both rstan and brms using .deb packages. Here’s my current system state:

> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.1 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  utils     datasets  grDevices methods   base     

other attached packages:
[1] brms_2.18.0          Rcpp_1.0.9           rstan_2.21.7        
[4] ggplot2_3.3.6        StanHeaders_2.21.0-7 devtools_2.4.3      
[7] usethis_2.1.6        default_1.0.0       

loaded via a namespace (and not attached):
 [1] nlme_3.1-160         matrixStats_0.62.0   fs_1.5.2            
 [4] xts_0.12.2           threejs_0.3.3        tensorA_0.36.2      
 [7] tools_4.2.1          backports_1.4.1      utf8_1.2.2          
[10] R6_2.5.1             DT_0.26              colorspace_2.0-3    
[13] withr_2.5.0          tidyselect_1.2.0     gridExtra_2.3       
[16] prettyunits_1.1.1    processx_3.8.0       Brobdingnag_1.2-9   
[19] compiler_4.2.1       cli_3.4.1            shinyjs_2.1.0       
[22] colourpicker_1.1.1   posterior_1.3.1      scales_1.2.1        
[25] dygraphs_1.1.1.6     checkmate_2.1.0      mvtnorm_1.1-3       
[28] ggridges_0.5.4       callr_3.7.2          stringr_1.4.1       
[31] digest_0.6.30        base64enc_0.1-3      pkgconfig_2.0.3     
[34] htmltools_0.5.3      sessioninfo_1.2.2    fastmap_1.1.0       
[37] htmlwidgets_1.5.4    rlang_1.0.6          shiny_1.7.3         
[40] farver_2.1.1         generics_0.1.3       zoo_1.8-11          
[43] crosstalk_1.2.0      gtools_3.9.3         dplyr_1.0.10        
[46] distributional_0.3.1 inline_0.3.19        magrittr_2.0.3      
[49] loo_2.5.1            bayesplot_1.9.0      Matrix_1.5-1        
[52] munsell_0.5.0        fansi_1.0.3          abind_1.4-5         
[55] lifecycle_1.0.3      stringi_1.7.8        pkgbuild_1.3.1      
[58] plyr_1.8.7           grid_4.2.1           parallel_4.2.1      
[61] promises_1.2.0.1     crayon_1.5.2         miniUI_0.1.1.1      
[64] lattice_0.20-45      ps_1.7.1             pillar_1.8.1        
[67] igraph_1.3.5         markdown_1.2         shinystan_2.6.0     
[70] reshape2_1.4.4       codetools_0.2-18     stats4_4.2.1        
[73] pkgload_1.3.0        rstantools_2.2.0     glue_1.6.2          
[76] remotes_2.4.2        RcppParallel_5.1.5   vctrs_0.5.0         
[79] httpuv_1.6.6         gtable_0.3.1         purrr_0.3.5         
[82] cachem_1.0.6         mime_0.12            xtable_1.8-4        
[85] coda_0.19-4          later_1.3.0          tibble_3.1.7        
[88] shinythemes_1.2.0    memoise_2.0.1        ellipsis_0.3.2      
[91] bridgesampling_1.1-2
4

Can you try the rstan preview:

remove.packages(c("StanHeaders", "rstan"))
install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
5

If by preview you mean:

example(stan_model, package = "rstan", run.dontrun = TRUE)

Then, “yes”. Note, I could run it before and after removing the .deb binaries for ‘rstan’ and ‘stanheaders’ and installing from github I am still able to run them.

6

Also worth noting, I am now able to run

prior1 <- prior(normal(0,10), class = b) +
    prior(cauchy(0,2), class = sd)
fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient),
            data = epilepsy, family = poisson(),
            prior = prior1, chains = 1, cores = 1)

fit2 <- brm(count ~ zAge + zBase * Trt + (1|patient),
            data = epilepsy, family = poisson(),
            prior = prior1, chains = 2, cores = 1)

I did have the github versions installed when I first posted this issue, so am at a loss as to which of the tweaks and library removals and/or replacements fixed the issue. Regardless, I am glad it works and greatly appreciate your help.