Not able to run Eight Schools example on rstan 2.21.1 & StanHeaders 2.21.0-5

Hi all,

I was able to successfully install rstan using the binary file from CRAN. However, I have been unable to run the Eight Schools example following the instructions here (https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started).

schools.stan is saved in my working directory.

> schools_dat <- list(J = 8, 
+                     y = c(28,  8, -3,  7, -1,  1, 18, 12),
+                     sigma = c(15, 10, 16, 11,  9, 11, 10, 18))
> fit <- stan(file = 'schools.stan', data = schools_dat)
The NEXT version of Stan will not be able to pre-process your Stan program.
Please open an issue at
 https://github.com/stan-dev/stanc3/issues 
if you can share or at least describe your Stan program. This will help ensure that Stan
continues to work on your Stan programs in the future. Thank you!
This message can be avoided by wrapping your function call inside suppressMessages().
Error in compileCode(f, code, language = language, verbose = verbose) : 
  Compilation ERROR, function(s)/method(s) not created! C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: file18587ca30fc.o:file18587ca30fc.cpp:(.text+0x161c): undefined reference to `rstan::stan_fit::stan_fit(SEXPREC*, int)'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: file18587ca30fc.o:file18587ca30fc.cpp:(.text$_ZN3tbb8internal26task_scheduler_observer_v3D1Ev[_ZN3tbb8internal26task_scheduler_observer_v3D1Ev]+0x14): undefined reference to `tbb::internal::task_scheduler_observer_v3::observe(bool)'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: file18587ca30fc.o:file18587ca30fc.cpp:(.text$_ZN3tbb8internal26task_scheduler_observer_v3D0Ev[_ZN3tbb8internal26task_scheduler_observer_v3D0Ev]+0x1c): undefined reference to `tbb::internal::task_scheduler_observer_v3::observe(bool)'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64
In addition: Warning message:
In file.remove(c(unprocessed, processed)) :
  cannot remove file 'C:\Users\Karthik\AppData\Local\Temp\RtmpQBZTi2\file18582d9e2a83.stan', reason 'No such file or directory'
Error in sink(type = "output") : invalid connection

Here are details on the setup I am using:

  • Operating System: Windows 7 SP 1
  • R Version: 4.0.2
  • RStudio Version: 1.3.959
  • RStan Version: 2.21.1
  • StanHeaders Version: 2.21.0-5
  • Output of writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))
> writeLines(readLines(file.path(Sys.getenv("HOME"), ".R/Makevars")))
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
  cannot open file 'C:\Users\Karthik\Documents/.R/Makevars': No such file or directory
  • Output of devtools::session_info("rstan")
- Session info ---------------------------------------------------------------------------------------
 setting  value                       
 version  R version 4.0.2 (2020-06-22)
 os       Windows 7 x64 SP 1          
 system   x86_64, mingw32             
 ui       RStudio                     
 language (EN)                        
 collate  English_United States.1252  
 ctype    English_United States.1252  
 tz       America/Chicago             
 date     2020-07-10                  

- Packages -------------------------------------------------------------------------------------------
 ! package      * version   date       lib source        
   assertthat     0.2.1     2019-03-21 [1] CRAN (R 4.0.2)
   backports      1.1.7     2020-05-13 [1] CRAN (R 4.0.0)
   BH             1.72.0-3  2020-01-08 [1] CRAN (R 4.0.0)
   callr          3.4.3     2020-03-28 [1] CRAN (R 4.0.2)
   checkmate      2.0.0     2020-02-06 [1] CRAN (R 4.0.2)
   cli            2.0.2     2020-02-28 [1] CRAN (R 4.0.2)
   colorspace     1.4-1     2019-03-18 [1] CRAN (R 4.0.2)
   crayon         1.3.4     2017-09-16 [1] CRAN (R 4.0.2)
   curl           4.3       2019-12-02 [1] CRAN (R 4.0.2)
   desc           1.2.0     2018-05-01 [1] CRAN (R 4.0.2)
   digest         0.6.25    2020-02-23 [1] CRAN (R 4.0.2)
   ellipsis       0.3.1     2020-05-15 [1] CRAN (R 4.0.2)
   evaluate       0.14      2019-05-28 [1] CRAN (R 4.0.2)
   fansi          0.4.1     2020-01-08 [1] CRAN (R 4.0.2)
   farver         2.0.3     2020-01-16 [1] CRAN (R 4.0.2)
   ggplot2      * 3.3.2     2020-06-19 [1] CRAN (R 4.0.2)
   glue           1.4.1     2020-05-13 [1] CRAN (R 4.0.2)
   gridExtra      2.3       2017-09-09 [1] CRAN (R 4.0.2)
   gtable         0.3.0     2019-03-25 [1] CRAN (R 4.0.2)
   inline         0.3.15    2018-05-18 [1] CRAN (R 4.0.2)
   isoband        0.2.2     2020-06-20 [1] CRAN (R 4.0.2)
   jsonlite       1.7.0     2020-06-25 [1] CRAN (R 4.0.2)
   labeling       0.3       2014-08-23 [1] CRAN (R 4.0.0)
   lattice        0.20-41   2020-04-02 [1] CRAN (R 4.0.2)
   lifecycle      0.2.0     2020-03-06 [1] CRAN (R 4.0.2)
   loo            2.3.0     2020-07-07 [1] CRAN (R 4.0.2)
   magrittr       1.5       2014-11-22 [1] CRAN (R 4.0.2)
   MASS           7.3-51.6  2020-04-26 [1] CRAN (R 4.0.2)
   Matrix         1.2-18    2019-11-27 [1] CRAN (R 4.0.2)
   matrixStats    0.56.0    2020-03-13 [1] CRAN (R 4.0.2)
   mgcv           1.8-31    2019-11-09 [1] CRAN (R 4.0.2)
   munsell        0.5.0     2018-06-12 [1] CRAN (R 4.0.2)
   nlme           3.1-148   2020-05-24 [1] CRAN (R 4.0.2)
   pillar         1.4.5     2020-07-09 [1] CRAN (R 4.0.2)
   pkgbuild       1.0.8     2020-05-07 [1] CRAN (R 4.0.2)
   pkgconfig      2.0.3     2019-09-22 [1] CRAN (R 4.0.2)
   pkgload        1.1.0     2020-05-29 [1] CRAN (R 4.0.2)
   praise         1.0.0     2015-08-11 [1] CRAN (R 4.0.2)
   prettyunits    1.1.1     2020-01-24 [1] CRAN (R 4.0.2)
   processx       3.4.3     2020-07-05 [1] CRAN (R 4.0.2)
   ps             1.3.3     2020-05-08 [1] CRAN (R 4.0.2)
   R6             2.4.1     2019-11-12 [1] CRAN (R 4.0.2)
   RColorBrewer   1.1-2     2014-12-07 [1] CRAN (R 4.0.0)
   Rcpp           1.0.5     2020-07-06 [1] CRAN (R 4.0.2)
   RcppEigen      0.3.3.7.0 2019-11-16 [1] CRAN (R 4.0.2)
 D RcppParallel   5.0.2     2020-06-24 [1] CRAN (R 4.0.2)
   rlang          0.4.6     2020-05-02 [1] CRAN (R 4.0.2)
   rprojroot      1.3-2     2018-01-03 [1] CRAN (R 4.0.2)
   rstan        * 2.21.1    2020-07-08 [1] CRAN (R 4.0.2)
   rstudioapi     0.11      2020-02-07 [1] CRAN (R 4.0.2)
   scales         1.1.1     2020-05-11 [1] CRAN (R 4.0.2)
   StanHeaders  * 2.21.0-5  2020-06-09 [1] CRAN (R 4.0.2)
   testthat       2.3.2     2020-03-02 [1] CRAN (R 4.0.2)
   tibble         3.0.2     2020-07-07 [1] CRAN (R 4.0.2)
   utf8           1.1.4     2018-05-24 [1] CRAN (R 4.0.2)
   V8             3.2.0     2020-06-19 [1] CRAN (R 4.0.2)
   vctrs          0.3.1     2020-06-05 [1] CRAN (R 4.0.2)
   viridisLite    0.3.0     2018-02-01 [1] CRAN (R 4.0.2)
   withr          2.2.0     2020-04-20 [1] CRAN (R 4.0.2)

[1] C:/R-4.0.2/library

 D -- DLL MD5 mismatch, broken installation.

A few other things I’ve tried:

  • Installed both R and Rtools in a file path without spaces (directly in C:)

  • Test for Rtools

> pkgbuild::has_build_tools(debug = TRUE)
[1] TRUE

Clearly, the DLL MD5 mismatch also seems like a problem, but I’m not really sure what to do about it (persisted despite uninstalling and reinstalling RcppParallel from CRAN). Sorry if I am missing something basic here, I am very new to RStan. Any help would be greatly appreciated - thank you so much! Please let me know if there’s any other information I can provide.

Try

Thank you for the reply!

I was able to update Rstan without any problems

install.packages( "https://win-builder.r-project.org/8bPDZPDG0522/rstan_2.21.2.zip", repos = NULL, type = "win.binary")

However, running the example gave the following error after Chain 4 of the second model fit:

> example(stan_model, package = "rstan", run.dontrun = TRUE)
# lots of output that I think was related to compiling the code, couldn't really make sense of it
# model started running:
stn_md> fit <- sampling(mod, data = list(y_mean = 0))

SAMPLING FOR MODEL '73fc79f8b1915e8208c736914c86d1a1' NOW (CHAIN 1).
Chain 1: 
Chain 1: Gradient evaluation took 0 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1: 
Chain 1: 
Chain 1: Iteration:    1 / 2000 [  0%]  (Warmup)
Chain 1: Iteration:  200 / 2000 [ 10%]  (Warmup)
Chain 1: Iteration:  400 / 2000 [ 20%]  (Warmup)
Chain 1: Iteration:  600 / 2000 [ 30%]  (Warmup)
Chain 1: Iteration:  800 / 2000 [ 40%]  (Warmup)
Chain 1: Iteration: 1000 / 2000 [ 50%]  (Warmup)
Chain 1: Iteration: 1001 / 2000 [ 50%]  (Sampling)
Chain 1: Iteration: 1200 / 2000 [ 60%]  (Sampling)
Chain 1: Iteration: 1400 / 2000 [ 70%]  (Sampling)
Chain 1: Iteration: 1600 / 2000 [ 80%]  (Sampling)
Chain 1: Iteration: 1800 / 2000 [ 90%]  (Sampling)
Chain 1: Iteration: 2000 / 2000 [100%]  (Sampling)
Chain 1: 
Chain 1:  Elapsed Time: 0.033 seconds (Warm-up)
Chain 1:                0.031 seconds (Sampling)
Chain 1:                0.064 seconds (Total)
Chain 1: 
# ........
Warning message:
In file.remove(c(unprocessed, processed)) :
  cannot remove file 'C:\Users\Karthik\AppData\Local\Temp\RtmpeYROL7\filecfc42006e3d.stan', reason 'No such file or directory'

Running the Eight schools example as above caused the RStudio session to abort:

Does that mean there is some issue with C++ as well?

More likely the toolchain. Can you run the eight schools model from the Terminal app or the Terminal tab of RStudio?

Just tried running the Eight Schools example from Terminal tab of Rstudio. (Basically I entered “R” and then entered the remaining commands as usual - I assume this is what you meant?)

Unfortunately, I got multiple errors saying “R front-end for Windows has stopped working”. Once I closed those windows, the output was similar to before:

starting worker pid=7300 on localhost:11530 at 14:36:10.3s80
starting worker pid=2644 on localhost:11530 at 14:36:10.378
starting worker pid=824 on localhost:11530 at 14:36:10.442
starting worker pid=10068 on localhost:11530 at 14:36:10.512

SAMPLING FOR MODEL 'schools' NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 0 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:
==== C stack trace ===============================

        (No symbol) [0x0x666ed1b0]
        (No symbol) [0x0x50]
        (No symbol) [0x0x666ce389]
        (No symbol) [0x0x43ee220]
        (No symbol) [0x0x666eb85d]
        (No symbol) [0x0x4000000000000000]
# ...........
==== C stack trace ===============================

        (No symbol) [0x0x666ed1b0]
        (No symbol) [0x0x50]
        (No symbol) [0x0x666ce389]
        (No symbol) [0x0x43ee220]
        (No symbol) [0x0x666eb85d]
        (No symbol) [0x0x4000000000000000]
Error in unserialize(socklist[[n]]) : error reading from connection
In addition: Warning messages:
1: In system2(CXX, args = ARGS) :
  '"C:/rtools40/usr/mingw_/bin/g++"' not found
2: In file.remove(c(unprocessed, processed)) :
  cannot remove file 'C:\Users\Karthik\AppData\Local\Temp\RtmpI3X0ii\file2b98587b5a5a.stan', reason 'No such file or directory'
Error in serialize(data, node$con, xdr = FALSE) :
  error writing to connection

The file mentioned truly does not exist.

Is it okay if you run it with cores = 1?

Still didn’t work, unfortunately. This time I only got this much output, and then a window saying that “R for Windows terminal front-end has stopped working”.

SAMPLING FOR MODEL 'schools' NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 0 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:

==== C stack trace ===============================

        (No symbol) [0x0x639ed1b0]
        (No symbol) [0x0x50]
        (No symbol) [0x0x639ce389]
        (No symbol) [0x0x440ae00]
        (No symbol) [0x0x639eb85d]
        (No symbol) [0x0x4000000000000000]

Do you have any other ideas what may be causing these errors? Thank you so much for your continued help, I really appreciate it.

No, but I would guess it has something to do with the CRTP or the TBB. We’ll figure it out.

Thank you! Please let me know if anything comes up that you want me to try out on my computer as you’re investigating.

@wds15 Any idea where to look for these bugs on Windows that look like stack overflows?

Should we consider to avoid pre building the services on windows ? This would increase compilation times, but this is how it worked before.

That seems like a last resort, although one we might have to take if we know the source of these memory exhaustion / stack overflow problems and know that we can’t fix them. I don’t think we know either yet.

Is there anyone who is getting the C stack trace thing with R 4.x? Or is this R 3.x?

Sorry I wasn’t sure if you were asking me or not but I am using R version 4.0.2.

OK. Thanks. Do you have -mtune=native -march=native or similar in your Makevars file?

It’s in the opposite order, but basically yes. This is the entire contents of the Makevars.win file in My Documents/.R. I also have other Makevars files associated with other R packages (including packages for versions of R I thought I had previously uninstalled…)

CXX14FLAGS=-O3 -march=native -mtune=native
CXX11FLAGS=-O3 -march=corei7 -mtune=corei7

CXX14FLAGS=-O3 -march=native -mtune=native
CXX11FLAGS=-O3 -march=corei7 -mtune=corei7

CXX14FLAGS=-O3 -march=native -mtune=native
CXX11FLAGS=-O3 -march=corei7 -mtune=corei7

Can you try setting all of these to

CXX14FLAGS=-O3 -mtune=native -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 
CXX11FLAGS=-O3 -mtune=native -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2

so to avoid the march thing? Apparently there have been issues reported with the minnow compiler and turning on march. Does this resolve the issue?

1 Like

Yes it worked! Basically I copied the code you gave me into the main Makevars.win file 3 times. I still got a warning message (similar to error messages I had received before), but thankfully the program was able to proceed despite this.

1: In file.remove(c(unprocessed, processed)) :
  cannot remove file 'C:\Users\Karthik\AppData\Local\Temp\RtmpQrURDf\file3f98793b5590.stan', reason 'No such file or directory'

I also noticed that when I load the rstan library, I would always follow the suggestion,

For improved execution time, we recommend calling
Sys.setenv(LOCAL_CPPFLAGS = '-march=native -mtune=native')

This time I skipped running this line, which may have helped as well. Thank you for all your help, I really appreciate it! Hopefully things continue to work despite the warning message.

1 Like