Rstan on Windows

Thanks again!

Should be fixed by the StanHeaders on CRAN now.

2 Likes

Set rstan_options("javascript" = FALSE) before trying to compile a Stan model, or just ignore the warning.

4 Likes

Hi @jonah,

I am working on a package that depends on brms to fit various non-linear models. We have set up GitHub Actions to run R CMD check on various OS. It checks fine for osx and linux, however we started getting these error messages when it builds the vignette (which contains multiple independent calls to brms) during check on Windows which I think are related to this thread:

==== C stack trace ===============================
  (No symbol) [0x0x67d2fd80]
  R_RunWeakRefFinalizer [0x0x6c83373f+383]
  R_RunWeakRefFinalizer [0x0x6c8339ed+1069]
  R_initAssignSymbols [0x0x6c7f6ac5+73941]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_has_methods_attached [0x0x6c83a588+920]
  R_has_methods_attached [0x0x6c83b0f4+3844]
  R_initAssignSymbols [0x0x6c7ebdd1+29665]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_has_methods_attached [0x0x6c83a588+920]
  R_has_methods_attached [0x0x6c83b0f4+3844]
  R_initAssignSymbols [0x0x6c7ebdd1+29665]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  Rf_eval [0x0x6c7fbeac+796]
  R_execMethod [0x0x6c800f9a+8378]
  Rf_eval [0x0x6c7fc0e5+1365]
  R_forceAndCall [0x0x6c801f93+3059]
  R_initAssignSymbols [0x0x6c7ebf69+30073]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  Rf_eval [0x0x6c7fc5b9+2601]
  Rf_eval [0x0x6c7fc004+1140]
  R_forceAndCall [0x0x6c802a4a+5802]
  R_BadLongVector [0x0x6c83815f+383]
  R_initAssignSymbols [0x0x6c7ebdd1+29665]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  Rf_eval [0x0x6c7fc5b9+2601]

It seems that the check does fit at least the first model from the vignette because this warning message is spat out as part of the output immediately before the error message:

Start sampling

##[warning]Warning: Found 1 observations with a pareto_k > 0.7 in model 'fit'. It is recommended to set 'moment_match = TRUE' in order to perform moment matching for problematic observations. 

Warning: 

4 (8.3%) p_waic estimates greater than 0.4. We recommend trying loo instead.

Based on your answers, I have tried to bypass this issue manually creating a Makevars.win before running R CMD check on windows by adding this to my Actions yml file:

      - name: Fix Windows Makevars
        if: runner.os == 'Windows'
        run: |
          dir.create("C:\\Users\\runneradmin\\Documents\\.R", recursive = TRUE)
          write("CXX14FLAGS=-O3 -mtune=native", "C:\\Users\\runneradmin\\Documents\\.R\\Makevars.win")
        shell: Rscript {0}

But to no avail. The full error log can be checked here under the Check red cross.

Any ideas how this could be fixed? Thanks!

Hi Diego,

So if Iā€™m reading that right, one of the brms models samples without issue but one of them crashes? That might be a bit different to the issues in this thread, which are more of a wholesale failure to sample at all. Can you provide more information about the models that fail and succeed, respectively?

Also, if you add -Wno-ignored-attributes to your makevars that will drastically cut down on the compiler warnings in the output

2 Likes

Something else to try, weā€™re currently having issues with the inline package on Windows which is causing a lot of failures, so try installing a patched version before running your models:

install.packages("https://github.com/bgoodri/inline/archive/master.tar.gz", 
                 repos = NULL, type = "source")
4 Likes

Hi @andrjohns, thanks very much for the suggestions. I implemented them both but unfortunately it still returns the same error.

I use a mac myself, so I canā€™t verify these errors locally unfortunately. Theyā€™re only occurring on the GitHub Actions Windows VM.

It is interesting though that after implementing both suggestions the check log shows that the first four models of the vignette fit, while it was only fitting the first one previously, so something did change. In the links, bnec is a wrapper function that fit specific non-linear models via brm.

I could force R to spit out info on the log screen via cat statements on the vignette. Is there anything that you think would be useful to know to help debug this? Also, Iā€™d be happy to move this discussion to its own topic you think thatā€™s best.

Thanks again!

1 Like

Would you be able to run the Rstan example model with the verbose=T flag and post the output?

library(rstan)
example(stan_model,run.dontrun=TRUE,verbose=TRUE)

That will give us the arguments being passed to the compiler, so that we can see if thereā€™s a configuration issue somewhere

2 Likes

Thanks @andrjohns, implemented here, output on the GitHub Actions log was:

Loading required package: StanHeaders

Loading required package: ggplot2

rstan (Version 2.21.2, GitRev: 2e1f913d3ca3)

For execution on a local, multicore CPU with excess RAM we recommend calling

options(mc.cores = parallel::detectCores()).

To avoid recompilation of unchanged Stan programs, we recommend calling

rstan_options(auto_write = TRUE)

Do not specify '-march=native' in 'LOCAL_CPPFLAGS' or a Makevars file

make cmd is

  make -f "C:/R/etc/x64/Makeconf" -f "C:/R/share/make/winshlib.mk" CXX='$(CXX14) $(CXX14STD)' CXXFLAGS='$(CXX14FLAGS)' CXXPICFLAGS='$(CXX14PICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX14LDFLAGS)' SHLIB_LD='$(SHLIB_CXX14LD)' SHLIB="file69c424d3ff9.dll" WIN=64 TCLBIN=64 OBJECTS="file69c424d3ff9.o"



make would use

"C:/rtools40/mingw64/bin/"g++  -std=gnu++14 -I"C:/R/include" -DNDEBUG   -I"D:/a/_temp/Library/Rcpp/include/"  -I"D:/a/_temp/Library/RcppEigen/include/"  -I"D:/a/_temp/Library/RcppEigen/include/unsupported"  -I"D:/a/_temp/Library/BH/include" -I"D:/a/_temp/Library/StanHeaders/include/src/"  -I"D:/a/_temp/Library/StanHeaders/include/"  -I"D:/a/_temp/Library/RcppParallel/include/"  -I"D:/a/_temp/Library/rstan/include" -DEIGEN_NO_DEBUG  -DBOOST_DISABLE_ASSERTS  -DBOOST_PENDING_INTEGER_LOG2_HPP  -DSTAN_THREADS  -DBOOST_NO_AUTO_PTR  -include "D:/a/_temp/Library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp"  -std=c++1y         -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c file69c424d3ff9.cpp -o file69c424d3ff9.o

if test "zfile69c424d3ff9.o" != "z"; then \

  if test -e "file69c424d3ff9-win.def"; then \

    echo "C:/rtools40/mingw64/bin/"g++  -shared -s -static-libgcc -o file69c424d3ff9.dll file69c424d3ff9-win.def file69c424d3ff9.o   "D:/a/_temp/Library/rstan/lib/x64/libStanServices.a" -L"D:/a/_temp/Library/StanHeaders/libs/x64" -lStanHeaders -L"D:/a/_temp/Library/RcppParallel/lib/x64" -ltbb  -L"C:/R/bin/x64" -lR ; \

    "C:/rtools40/mingw64/bin/"g++  -shared -s -static-libgcc -o file69c424d3ff9.dll file69c424d3ff9-win.def file69c424d3ff9.o   "D:/a/_temp/Library/rstan/lib/x64/libStanServices.a" -L"D:/a/_temp/Library/StanHeaders/libs/x64" -lStanHeaders -L"D:/a/_temp/Library/RcppParallel/lib/x64" -ltbb  -L"C:/R/bin/x64" -lR ; \

  else \

    echo EXPORTS > tmp.def; \

    "C:/rtools40/mingw64/bin/"nm file69c424d3ff9.o | sed -n 's/^.* [BCDRT] / /p' | sed -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d' | sed 's/[^ ][^ ]*/"&"/g'  >> tmp.def; \

    echo "C:/rtools40/mingw64/bin/"g++  -shared -s -static-libgcc -o file69c424d3ff9.dll tmp.def file69c424d3ff9.o   "D:/a/_temp/Library/rstan/lib/x64/libStanServices.a" -L"D:/a/_temp/Library/StanHeaders/libs/x64" -lStanHeaders -L"D:/a/_temp/Library/RcppParallel/lib/x64" -ltbb  -L"C:/R/bin/x64" -lR ; \

    "C:/rtools40/mingw64/bin/"g++  -shared -s -static-libgcc -o file69c424d3ff9.dll tmp.def file69c424d3ff9.o   "D:/a/_temp/Library/rstan/lib/x64/libStanServices.a" -L"D:/a/_temp/Library/StanHeaders/libs/x64" -lStanHeaders -L"D:/a/_temp/Library/RcppParallel/lib/x64" -ltbb  -L"C:/R/bin/x64" -lR ; \

    rm -f tmp.def; \

  fi \

fi

Hmm, there are no issues that jump out. Are you able to share the brms / stan code and data from one of the failing models? Then I can try and replicate locally on Windows

1 Like

Sure thing.

It fits the first three models, but returns that C stack trace I posted before on the fourth call. You can check the brm fitted models within the objects returned by bnec with out$fit. The actual non-linear brmsformula being fitted in the example below and the vignette can be seen with bayesnec:::bf_nec3param_deflt for non-binomial data, and bayesnec:::bf_nec3param_binom for binomial data.

remotes::install_github("open-AIMS/bayesnec")
library(bayesnec)
library(tidyverse)

binom_data <- "https://pastebin.com/raw/zfrUha88" %>%
  read.table(header = TRUE, dec = ",", stringsAsFactors = FALSE) %>%
  dplyr::rename(raw_x = raw.x) %>%
  dplyr::mutate(raw_x = as.numeric(as.character(raw_x)))

set.seed(333)
out <- bnec(data = binom_data, x_var = "raw_x",
            y_var = "suc", model = "nec3param",
            trials_var = "tot")

prop_data <- "https://pastebin.com/raw/123jq46d" %>%
  read.table(header = TRUE, dec = ",", stringsAsFactors = FALSE) %>%
  dplyr::rename(raw_x = raw.x) %>%
  dplyr::mutate(raw_x = log(as.numeric(as.character(raw_x)) + 1),
                resp = as.numeric(as.character(resp)))

set.seed(333)
out <- bnec(data = prop_data, x_var = "raw_x",
            y_var = "resp", model = "nec3param")

count_data <- "https://pastebin.com/raw/ENgNSgf7" %>%
  read.table(header = TRUE, dec = ",", stringsAsFactors = FALSE) %>%
  dplyr::rename(raw_x = raw.x) %>%
  dplyr::mutate(raw_x = as.numeric(as.character(raw_x)))

set.seed(333)
out <- bnec(data = count_data, x_var = "raw_x",
            y_var = "count", model = "nec3param")

measure_data <- "https://pastebin.com/raw/pWeS6x0n" %>%
  read.table(header = TRUE, dec = ",", stringsAsFactors = FALSE) %>%
  dplyr::rename(raw_x = raw.x) %>%
  dplyr::mutate(dplyr::across(where(is.character), as.numeric))

set.seed(333)
out <- bnec(data = measure_data, x_var = "raw_x",
            y_var = "measure", model = "nec3param")

Iā€™m able to (kind of) replicate the errors locally. Across 2 Windows machines the R session will crash after running three of your models. Oddly enough, it doesnā€™t which three (can be be three runs of the same model).

Iā€™ll keep having a look into this, but will also tag @paul.buerkner since this could be brms related

1 Like

Hey @paul.buerkner, Iā€™ve distilled this down to a ā€˜pureā€™ brms example that crashes.

Here is a list with the arguments to brms: test_brm.R (5.0 KB)
(The file extension is .R, but itā€™s actually an RData file since discourse has restrictions on types of files that can be uploaded).

To replicate the crash, run:

options(mc.cores=4)
library(brms)
load("test_brm.R")

out = do.call(brm,test_brm)
out = do.call(brm,test_brm)
out = do.call(brm,test_brm)

#Crashes on this call:
out = do.call(brm,test_brm)

I had the same behaviour across two Windows machines

@andrjohns can you make an issue with this at the brms Github: https://github.com/paul-buerkner/brms/issues? This is kinda deep in this thread :D.

2 Likes

@martinmodrak can we close this thread with a post advising users to open new threads. These 150+ post threads are impossible to follow and the thread has also gone offtopic I think.

3 Likes

Ah good idea, will do

This thread is now closed. If you need support with installing/operating RStan on Windows, please open a new thread and weā€™ll be happy to help you there!

4 Likes