Rstan on Windows

jmh530 · August 19, 2020, 1:38pm

Thanks again!

bgoodri · August 22, 2020, 5:13pm

Should be fixed by the StanHeaders on CRAN now.

bgoodri · August 22, 2020, 5:15pm

Set rstan_options("javascript" = FALSE) before trying to compile a Stan model, or just ignore the warning.

dbarneche · August 31, 2020, 7:41am

I am working on a package that depends on brms to fit various non-linear models. We have set up GitHub Actions to run R CMD check on various OS. It checks fine for osx and linux, however we started getting these error messages when it builds the vignette (which contains multiple independent calls to brms) during check on Windows which I think are related to this thread:

==== C stack trace ===============================
  (No symbol) [0x0x67d2fd80]
  R_RunWeakRefFinalizer [0x0x6c83373f+383]
  R_RunWeakRefFinalizer [0x0x6c8339ed+1069]
  R_initAssignSymbols [0x0x6c7f6ac5+73941]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_has_methods_attached [0x0x6c83a588+920]
  R_has_methods_attached [0x0x6c83b0f4+3844]
  R_initAssignSymbols [0x0x6c7ebdd1+29665]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_has_methods_attached [0x0x6c83a588+920]
  R_has_methods_attached [0x0x6c83b0f4+3844]
  R_initAssignSymbols [0x0x6c7ebdd1+29665]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  Rf_eval [0x0x6c7fbeac+796]
  R_execMethod [0x0x6c800f9a+8378]
  Rf_eval [0x0x6c7fc0e5+1365]
  R_forceAndCall [0x0x6c801f93+3059]
  R_initAssignSymbols [0x0x6c7ebf69+30073]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  Rf_eval [0x0x6c7fc5b9+2601]
  Rf_eval [0x0x6c7fc004+1140]
  R_forceAndCall [0x0x6c802a4a+5802]
  R_BadLongVector [0x0x6c83815f+383]
  R_initAssignSymbols [0x0x6c7ebdd1+29665]
  Rf_eval [0x0x6c7fbd01+369]
  R_cmpfun1 [0x0x6c7fd9e0+1152]
  Rf_applyClosure [0x0x6c7fec9a+442]
  R_initAssignSymbols [0x0x6c7f2b47+57687]
  Rf_eval [0x0x6c7fbd01+369]
  Rf_eval [0x0x6c7fc5b9+2601]

It seems that the check does fit at least the first model from the vignette because this warning message is spat out as part of the output immediately before the error message:

Start sampling

##[warning]Warning: Found 1 observations with a pareto_k > 0.7 in model 'fit'. It is recommended to set 'moment_match = TRUE' in order to perform moment matching for problematic observations. 

Warning: 

4 (8.3%) p_waic estimates greater than 0.4. We recommend trying loo instead.

Based on your answers, I have tried to bypass this issue manually creating a Makevars.win before running R CMD check on windows by adding this to my Actions yml file:

      - name: Fix Windows Makevars
        if: runner.os == 'Windows'
        run: |
          dir.create("C:\\Users\\runneradmin\\Documents\\.R", recursive = TRUE)
          write("CXX14FLAGS=-O3 -mtune=native", "C:\\Users\\runneradmin\\Documents\\.R\\Makevars.win")
        shell: Rscript {0}

But to no avail. The full error log can be checked here under the Check red cross.

Any ideas how this could be fixed? Thanks!

andrjohns · August 31, 2020, 8:26am

Hi Diego,

So if I’m reading that right, one of the brms models samples without issue but one of them crashes? That might be a bit different to the issues in this thread, which are more of a wholesale failure to sample at all. Can you provide more information about the models that fail and succeed, respectively?

Also, if you add -Wno-ignored-attributes to your makevars that will drastically cut down on the compiler warnings in the output

andrjohns · August 31, 2020, 8:28am

Something else to try, we’re currently having issues with the inline package on Windows which is causing a lot of failures, so try installing a patched version before running your models:

install.packages("https://github.com/bgoodri/inline/archive/master.tar.gz", 
                 repos = NULL, type = "source")

dbarneche · August 31, 2020, 11:14pm

Hi @andrjohns, thanks very much for the suggestions. I implemented them both but unfortunately it still returns the same error.

I use a mac myself, so I can’t verify these errors locally unfortunately. They’re only occurring on the GitHub Actions Windows VM.

It is interesting though that after implementing both suggestions the check log shows that the first four models of the vignette fit, while it was only fitting the first one previously, so something did change. In the links, bnec is a wrapper function that fit specific non-linear models via brm.

I could force R to spit out info on the log screen via cat statements on the vignette. Is there anything that you think would be useful to know to help debug this? Also, I’d be happy to move this discussion to its own topic you think that’s best.

Thanks again!

andrjohns · September 1, 2020, 12:02am

Would you be able to run the Rstan example model with the verbose=T flag and post the output?

library(rstan)
example(stan_model,run.dontrun=TRUE,verbose=TRUE)

That will give us the arguments being passed to the compiler, so that we can see if there’s a configuration issue somewhere

dbarneche · September 1, 2020, 2:28am

Thanks @andrjohns, implemented here, output on the GitHub Actions log was:

Loading required package: StanHeaders

Loading required package: ggplot2

rstan (Version 2.21.2, GitRev: 2e1f913d3ca3)

For execution on a local, multicore CPU with excess RAM we recommend calling

options(mc.cores = parallel::detectCores()).

To avoid recompilation of unchanged Stan programs, we recommend calling

rstan_options(auto_write = TRUE)

Do not specify '-march=native' in 'LOCAL_CPPFLAGS' or a Makevars file

make cmd is

  make -f "C:/R/etc/x64/Makeconf" -f "C:/R/share/make/winshlib.mk" CXX='$(CXX14) $(CXX14STD)' CXXFLAGS='$(CXX14FLAGS)' CXXPICFLAGS='$(CXX14PICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX14LDFLAGS)' SHLIB_LD='$(SHLIB_CXX14LD)' SHLIB="file69c424d3ff9.dll" WIN=64 TCLBIN=64 OBJECTS="file69c424d3ff9.o"



make would use

"C:/rtools40/mingw64/bin/"g++  -std=gnu++14 -I"C:/R/include" -DNDEBUG   -I"D:/a/_temp/Library/Rcpp/include/"  -I"D:/a/_temp/Library/RcppEigen/include/"  -I"D:/a/_temp/Library/RcppEigen/include/unsupported"  -I"D:/a/_temp/Library/BH/include" -I"D:/a/_temp/Library/StanHeaders/include/src/"  -I"D:/a/_temp/Library/StanHeaders/include/"  -I"D:/a/_temp/Library/RcppParallel/include/"  -I"D:/a/_temp/Library/rstan/include" -DEIGEN_NO_DEBUG  -DBOOST_DISABLE_ASSERTS  -DBOOST_PENDING_INTEGER_LOG2_HPP  -DSTAN_THREADS  -DBOOST_NO_AUTO_PTR  -include "D:/a/_temp/Library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp"  -std=c++1y         -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c file69c424d3ff9.cpp -o file69c424d3ff9.o

if test "zfile69c424d3ff9.o" != "z"; then \

  if test -e "file69c424d3ff9-win.def"; then \

    echo "C:/rtools40/mingw64/bin/"g++  -shared -s -static-libgcc -o file69c424d3ff9.dll file69c424d3ff9-win.def file69c424d3ff9.o   "D:/a/_temp/Library/rstan/lib/x64/libStanServices.a" -L"D:/a/_temp/Library/StanHeaders/libs/x64" -lStanHeaders -L"D:/a/_temp/Library/RcppParallel/lib/x64" -ltbb  -L"C:/R/bin/x64" -lR ; \

    "C:/rtools40/mingw64/bin/"g++  -shared -s -static-libgcc -o file69c424d3ff9.dll file69c424d3ff9-win.def file69c424d3ff9.o   "D:/a/_temp/Library/rstan/lib/x64/libStanServices.a" -L"D:/a/_temp/Library/StanHeaders/libs/x64" -lStanHeaders -L"D:/a/_temp/Library/RcppParallel/lib/x64" -ltbb  -L"C:/R/bin/x64" -lR ; \

  else \

    echo EXPORTS > tmp.def; \

    "C:/rtools40/mingw64/bin/"nm file69c424d3ff9.o | sed -n 's/^.* [BCDRT] / /p' | sed -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d' | sed 's/[^ ][^ ]*/"&"/g'  >> tmp.def; \

    echo "C:/rtools40/mingw64/bin/"g++  -shared -s -static-libgcc -o file69c424d3ff9.dll tmp.def file69c424d3ff9.o   "D:/a/_temp/Library/rstan/lib/x64/libStanServices.a" -L"D:/a/_temp/Library/StanHeaders/libs/x64" -lStanHeaders -L"D:/a/_temp/Library/RcppParallel/lib/x64" -ltbb  -L"C:/R/bin/x64" -lR ; \

    "C:/rtools40/mingw64/bin/"g++  -shared -s -static-libgcc -o file69c424d3ff9.dll tmp.def file69c424d3ff9.o   "D:/a/_temp/Library/rstan/lib/x64/libStanServices.a" -L"D:/a/_temp/Library/StanHeaders/libs/x64" -lStanHeaders -L"D:/a/_temp/Library/RcppParallel/lib/x64" -ltbb  -L"C:/R/bin/x64" -lR ; \

    rm -f tmp.def; \

  fi \

fi

andrjohns · September 1, 2020, 2:35am

Hmm, there are no issues that jump out. Are you able to share the brms / stan code and data from one of the failing models? Then I can try and replicate locally on Windows

dbarneche · September 1, 2020, 2:47am

Sure thing.

It fits the first three models, but returns that C stack trace I posted before on the fourth call. You can check the brm fitted models within the objects returned by bnec with out$fit. The actual non-linear brmsformula being fitted in the example below and the vignette can be seen with bayesnec:::bf_nec3param_deflt for non-binomial data, and bayesnec:::bf_nec3param_binom for binomial data.

remotes::install_github("open-AIMS/bayesnec")
library(bayesnec)
library(tidyverse)

binom_data <- "https://pastebin.com/raw/zfrUha88" %>%
  read.table(header = TRUE, dec = ",", stringsAsFactors = FALSE) %>%
  dplyr::rename(raw_x = raw.x) %>%
  dplyr::mutate(raw_x = as.numeric(as.character(raw_x)))

set.seed(333)
out <- bnec(data = binom_data, x_var = "raw_x",
            y_var = "suc", model = "nec3param",
            trials_var = "tot")

prop_data <- "https://pastebin.com/raw/123jq46d" %>%
  read.table(header = TRUE, dec = ",", stringsAsFactors = FALSE) %>%
  dplyr::rename(raw_x = raw.x) %>%
  dplyr::mutate(raw_x = log(as.numeric(as.character(raw_x)) + 1),
                resp = as.numeric(as.character(resp)))

set.seed(333)
out <- bnec(data = prop_data, x_var = "raw_x",
            y_var = "resp", model = "nec3param")

count_data <- "https://pastebin.com/raw/ENgNSgf7" %>%
  read.table(header = TRUE, dec = ",", stringsAsFactors = FALSE) %>%
  dplyr::rename(raw_x = raw.x) %>%
  dplyr::mutate(raw_x = as.numeric(as.character(raw_x)))

set.seed(333)
out <- bnec(data = count_data, x_var = "raw_x",
            y_var = "count", model = "nec3param")

measure_data <- "https://pastebin.com/raw/pWeS6x0n" %>%
  read.table(header = TRUE, dec = ",", stringsAsFactors = FALSE) %>%
  dplyr::rename(raw_x = raw.x) %>%
  dplyr::mutate(dplyr::across(where(is.character), as.numeric))

set.seed(333)
out <- bnec(data = measure_data, x_var = "raw_x",
            y_var = "measure", model = "nec3param")

andrjohns · September 1, 2020, 3:31am

I’m able to (kind of) replicate the errors locally. Across 2 Windows machines the R session will crash after running three of your models. Oddly enough, it doesn’t which three (can be be three runs of the same model).

I’ll keep having a look into this, but will also tag @paul.buerkner since this could be brms related

andrjohns · September 1, 2020, 4:25am

Hey @paul.buerkner, I’ve distilled this down to a ‘pure’ brms example that crashes.

Here is a list with the arguments to brms: test_brm.R (5.0 KB)
(The file extension is .R, but it’s actually an RData file since discourse has restrictions on types of files that can be uploaded).

To replicate the crash, run:

options(mc.cores=4)
library(brms)
load("test_brm.R")

out = do.call(brm,test_brm)
out = do.call(brm,test_brm)
out = do.call(brm,test_brm)

#Crashes on this call:
out = do.call(brm,test_brm)

I had the same behaviour across two Windows machines

bbbales2 · September 1, 2020, 5:17pm

@andrjohns can you make an issue with this at the brms Github: https://github.com/paul-buerkner/brms/issues? This is kinda deep in this thread :D.

rok_cesnovar · September 1, 2020, 5:59pm

@martinmodrak can we close this thread with a post advising users to open new threads. These 150+ post threads are impossible to follow and the thread has also gone offtopic I think.

andrjohns · September 2, 2020, 12:29am

Ah good idea, will do

andrjohns · September 2, 2020, 12:31am

This thread is now closed. If you need support with installing/operating RStan on Windows, please open a new thread and we’ll be happy to help you there!

Topic		Replies	Views
Problems getting Stan to run after upgrading to R 4.0.2 RStan	48	10417	March 20, 2023
Stan just broke after upgrade to R 4.0.2 under windows 10 required reinstallation - Help needed Interfaces rstan , installation	4	820	July 20, 2020
R4.0, rtools 4.0 and rstan 2.19 on windows Interfaces	2	602	April 24, 2020
Rstan install trouble RStan installation	34	1848	December 15, 2020
Stan & R 4.2 on Windows General	16	2436	July 28, 2022

Rstan on Windows

Related topics