R CMD check fails after adding Stan infrastructure to an existing R package

I have created a R library via RStudio and have added Stan infrastructure to it, using

library(rstantools)
rstantools::use_rstan().

The development version of the package is available on Github (here). Before adding the Stan infrastructure, performing a R CMD check would be concluded with no errors.

Having executed rstantools::use_rstan(), I add the following lines to the R/Bernadette-package.R file, as instructed, since I am using roxygen:

#' @useDynLib bernadette, .registration=TRUE
#' @import Rcpp
#' @importFrom rstan sampling vb

Then, I perform a devtools::check() again. Execution fails, here is the error mesage:

i Updating Bernadette documentation
i Loading Bernadette
Re-compiling Bernadette
-  installing *source* package 'Bernadette' ... (349ms)
   ** using staged installation
   ** libs
   no DLL was created
   ERROR: compilation failed for package 'Bernadette'
-  removing 'C:/Users/lbour/AppData/Local/Temp/RtmpAbbTwb/devtools_install_23f468b66d4e/Bernadette'

Error: System command 'Rcmd.exe' failed, exit status: 1, stdout + stderr:
E> * installing *source* package 'Bernadette' ...
E> ** using staged installation
E> ** libs
E> no DLL was created
E> ERROR: compilation failed for package 'Bernadette'
E> * removing 'C:/Users/lbour/AppData/Local/Temp/RtmpAbbTwb/devtools_install_23f468b66d4e/Bernadette'

 Stack trace:

 1. devtools::check(, check_dir = dirname(getwd()))
 2. devtools:::document(pkg, quiet = quiet)
 3. roxygen2::roxygenise(pkg$path, roclets)
 4. roxygen2:::load_code(base_path)
 5. pkgload::load_all(path, helpers = FALSE, attach_testthat = FALSE)
 6. pkgbuild::compile_dll(path, quiet = quiet)
 7. pkgbuild:::install_min(path, dest = install_dir, components = "li ...
 8. pkgbuild:::rcmd_build_tools("INSTALL", c(path, paste("--library=" ...
 9. pkgbuild:::with_build_tools(callr::rcmd_safe(..., env = env,  ...
 10. withr::with_path(rtools_path(), code)
 11. base:::force(code)
 12. callr::rcmd_safe(..., env = env, spinner = FALSE, show = FALSE,  ...
 13. callr:::run_r(options)
 14. base:::with(options, with_envvar(env, do.call(processx::run,  ...
 15. base:::with.default(options, with_envvar(env, do.call(processx::r ...
 16. base:::eval(substitute(expr), data, enclos = parent.frame())
 17. base:::eval(substitute(expr), data, enclos = parent.frame())
 18. callr:::with_envvar(env, do.call(processx::run, c(list(bin, args  ...
 19. base:::force(code)
 20. base:::do.call(processx::run, c(list(bin, args = real_cmdargs,  ...
 21. (function (command = NULL, args = character(), error_on_status = TRUE,  ...
 22. throw(new_process_error(res, call = sys.call(), echo = echo,  ...

 x System command 'Rcmd.exe' failed, exit status: 1, stdout + stderr:
E> * installing *source* package 'Bernadette' ...
E> ** using staged installation
E> ** libs
E> no DLL was created
E> ERROR: compilation failed for package 'Bernadette'
E> * removing 'C:/Users/lbour/AppData/Local/Temp/RtmpAbbTwb/devtools_install_23f468b66d4e/Bernadette' 

Execution halted

Exited with status 1.

I have spent some time to find a solution, both at this forum and elsewhere, but no luck so far. Can you please advise on how to resolve this? Here is my local machine configuration, with RStudio 1.4.1106 and RTools 3.5:

R version 3.6.0 (2019-04-26)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=Greek_Greece.1253  LC_CTYPE=Greek_Greece.1253    LC_MONETARY=Greek_Greece.1253
[4] LC_NUMERIC=C                  LC_TIME=Greek_Greece.1253    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rstantools_2.1.1 Bernadette_0.1.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.6           lubridate_1.7.10     lattice_0.20-44      prettyunits_1.1.1   
 [5] ps_1.6.0             assertthat_0.2.1     rprojroot_2.0.2      utf8_1.2.1          
 [9] V8_3.4.2             R6_2.5.0             stats4_3.6.0         ggplot2_3.3.4       
[13] pillar_1.6.1         rlang_0.4.11         curl_4.3.1           rstudioapi_0.13     
[17] callr_3.7.0          Matrix_1.3-3         desc_1.3.0           devtools_2.4.1      
[21] readr_1.4.0          RcppEigen_0.3.3.9.1  scoringRules_1.0.1   loo_2.4.1           
[25] munsell_0.5.0        compiler_3.6.0       xfun_0.24            rstan_2.21.2        
[29] pkgconfig_2.0.3      pkgbuild_1.2.0       tidyselect_1.1.1     tibble_3.1.2        
[33] gridExtra_2.3        codetools_0.2-18     matrixStats_0.59.0   fansi_0.5.0         
[37] crayon_1.4.1         dplyr_1.0.7          withr_2.4.2          MASS_7.3-54         
[41] BH_1.75.0-0          grid_3.6.0           jsonlite_1.7.2       gtable_0.3.0        
[45] lifecycle_1.0.0      DBI_1.1.1            magrittr_2.0.1       StanHeaders_2.21.0-7
[49] scales_1.1.1         RcppParallel_5.1.4   cli_2.5.0            cachem_1.0.5        
[53] remotes_2.3.0        fs_1.5.0             testthat_3.0.2       ellipsis_0.3.2      
[57] generics_0.1.0       vctrs_0.3.8          tools_3.6.0          glue_1.4.2          
[61] purrr_0.3.4          hms_1.1.0            processx_3.5.2       pkgload_1.2.1       
[65] parallel_3.6.0       fastmap_1.1.0        inline_0.3.19        colorspace_2.0-1    
[69] sessioninfo_1.1.1    memoise_2.0.0        knitr_1.33           usethis_2.0.1 

Thank you in advance,
Lampros

A quick look at the GitHub suggests that you don’t yet have any Stan files. I would recommend adding them and trying again.

Best,
Jeffrey

2 Likes

You probably need to have at least one .stan file in inst/stan, directly, not enought to have something (like licence.stan) in its subfolders. Although rstantools seems to try to also handle the case when no such files are found (rstantools/rstan_config.R at master · stan-dev/rstantools · GitHub)

1 Like

Hi both,

Thank you for your suggestions. I have added some dummy code for a simple linear regression model (inst/stan/dummy_lm_model.stan).

Having done that, first attempt to R CMD check failed, with the error being:

     C++14 standard requested but CXX14 is not defined

Combining the proposed solutions from related questions on this error here and here, I have modified the C:/Users/.../RHome/.R/Makevars and C:/Users/.../RHome/.R/Makevars.win, txt copies of which I attach below:
Makevars.txt (448 Bytes)
Makevars_win.txt (130 Bytes)

Second attempt to compile: after about an hour I receive a bunch of errors, related to Stanheaders (screenshot attached):

Thank you,
Lampros

You need to instruct R to use the c++14 extensions when compiling. Modify the CXX14 setting in your Makevars files to:

CXX14 = C:/RBuildTools/3.5/mingw_64/bin/g++ -std=c++1y

Thanks,

This is exactly what I have in the Makevars file, see the attached file above. I’ve also tried changing to the c++14 extensions to

CXX14=C:/RBuildTools/3.5/mingw_64/bin/g++ -std=c++1y

but the same errors as above appear again. No other changes performed to the Makevars and Makevars.win files.

This is exactly what I have in the Makevars file, see the attached file above.

Yep, but you didn’t have the -std=c++1y extensions. Can you restart R, run the check again and post the output here?

Thanks Andrew,

It looks like you were referring to the contents of the Makevars.win file. I searched more on the forum, since people have been facing similar issues before. I’m taking a step back to summarize the setting, with the hope to provide more information and help resolve this:

  1. Output from running Sys.getenv("PATH") in the console:
[1] "C:\\Program Files\\R\\R-3.6.0\\bin\\x64;C:\\Program Files (x86)\\Intel\\iCLS Client\\;C:\\Program Files\\Intel\\iCLS Client\\;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;c:\\Program Files (x86)\\ATI Technologies\\ATI.ACE\\Core-Static;C:\\Program Files (x86)\\Bitvise SSH Client;C:\\Program Files\\MiKTeX 2.9\\miktex\\bin\\x64\\;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files\\Git\\cmd;C:\\Users\\lbour\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\lbour\\AppData\\Local\\GitHubDesktop\\bin;%USERPROFILE%\\AppData\\Local\\Microsoft\\WindowsApps;"
  1. The contents of the .Renviron file are:
PATH="C:/RBuildTools/3.5/bin;${PATH}"
  1. The contents of the .Rprofile file:
Sys.setenv(BINPREF = "C:/RBuildTools/3.5/mingw_$(WIN)/bin/")

Having checked that, I proceed with the following.

Option 1
4. I modify my Makevars file to:

CXX11 = "C:/RBuildTools/3.5/mingw_64/bin/g++.exe" -std=c++11
CXX11FLAGS = -O3 -Wno-unused-variable -Wno-unused-function

CXX14 = "C:/RBuildTools/3.5/mingw_64/bin/g++.exe" -std=c++1y
CXX14FLAGS = -O3 -Wno-unused-variable -Wno-unused-function

Note that I explicitly used -std=c++11 for CXX11, following the error message above.

  1. I have modified my Makevars.win file to:
CXX11 = "C:/RBuildTools/3.5/mingw_64/bin/g++.exe" -std=c++11
CXX14 = "C:/RBuildTools/3.5/mingw_64/bin/g++.exe" -std=c++1y
CXX14FLAGS=-O3 -march=native -mtune=native
CXX11FLAGS=-O3 -march=native -mtune=native
  1. pkgbuild::compile_dll() is executed successfully in the console.

  2. I restart R and then attempt Clean and Rebuild. Same errors apply, see attached screenshots:

[the very first error]

[snippet of errors appearing later]

Option 2
Leave Makevars as above, then modify Makevars.win file to:

CXX14=$(BINPREF)g++ -O2 -march=native -mtune=native
CXX14FLAGS=-O3 -march=native -mtune=native
CXX11FLAGS=-O3 -march=native -mtune=native

following this answer.

Same errors apply.

What happens when you try to compile a stan model locally? Can you try the rstan example model:

example(stan_model,package="rstan",run.dontrun=T)

Either with the Makevars files from Option 1 or Option 2 above, I receive the following:

stn_md> stancode <- 'data {real y_mean;} parameters {real y;} model {y ~ normal(y_mean,1);}'

stn_md> mod <- stan_model(model_code = stancode, verbose = TRUE)

TRANSLATING MODEL '73fc79f8b1915e8208c736914c86d1a1' FROM Stan CODE TO C++ CODE NOW.
successful in parsing the Stan model '73fc79f8b1915e8208c736914c86d1a1'.
COMPILING THE C++ CODE FOR MODEL '73fc79f8b1915e8208c736914c86d1a1' NOW.
OS: x86_64, mingw32; rstan: 2.21.2; Rcpp: 1.0.6; inline: 0.3.19 
Error in cleanup_makevar(old) : 
  argument "RMU" is missing, with no default
In addition: Warning message:
In system(paste(CXX, ARGS), ignore.stdout = TRUE, ignore.stderr = TRUE) :
  '-E' not found

@andrjohns

Based on your guidelines here, I used the following workaround, which successfully compiled the example stan model locally:

  1. Have an empty Makevars file.
  2. The Makevars.win file contains the following:
CXX14 = "C:/RBuildTools/3.5/mingw_64/bin/g++.exe" -std=c++1y
CXX14FLAGS += -mtune=native -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2
  1. Rename Makevars.win to Makevars.win.bak:
file.rename("~/.R/Makevars.win", "~/.R/Makevars.win.bak")
  1. Execute example(stan_model,package="rstan",run.dontrun=T). At the end of the execution, I receive the following warning message:
    2021-06-25_15h39_37

Testing this strategy on the compilation of the R package fails. devtools::check() yields:

i Updating Bernadette documentation
i Loading Bernadette
Error in Rcpp::loadModule(module = "stan_fit4dummy_lm_model_mod", what = TRUE,  : 
  Unable to load module "stan_fit4dummy_lm_model_mod": Failed to initialize module pointer: Error in FUN(X[[i]], ...): no such symbol _rcpp_module_boot_stan_fit4dummy_lm_model_mod in package Bernadette
Calls: <Anonymous> ... <Anonymous> -> run_ns_load_actions -> action -> <Anonymous>
Execution halted

Exited with status 1.

Hope this helps! :)

Best,
Jeffrey

Hi @jeffreypullin ,

Thank you for creating the pull request. Having merged with my repository, I attempted to compile the R package. Following guidelines from here, these are my steps:

  1. Moved one level up from the bernadette/ directory on my local machine.
  2. From the command line ran Rcmd.exe INSTALL --preclean bernadette.
  3. I am now receiving the following. Please note this is the case when my Makevars file contains the single line CXX14 = "C:/RBuildTools/3.5/mingw_64/bin/g++.exe" -std=c++1y. The Makevars.win.bak remains unchanged:
* installing to library 'C:/Program Files/R/R-3.6.0/library'
* installing *source* package 'Bernadette' ...
** using staged installation
** libs

*** arch - i386


"C:/RBuildTools/3.5/mingw_64/bin/g++.exe" -std=c++1y  -I"C:/PROGRA~1/R/R-36~1.0/include" -DNDEBUG -I"../inst/include" -I"C:/Program Files/R/R-3.6.0/library/StanHeaders/include/src" -DBOOST_DISABLE_ASSERTS -DEIGEN_NO_DEBUG -DRCPP_PARALLEL_USE_TBB=1 -I"C:/Program Files/R/R-3.6.0/library/BH/include" -I"C:/Program Files/R/R-3.6.0/library/Rcpp/include" -I"C:/Program Files/R/R-3.6.0/library/RcppEigen/include" -I"C:/Program Files/R/R-3.6.0/library/RcppParallel/include" -I"C:/Program Files/R/R-3.6.0/library/rstan/include" -I"C:/Program Files/R/R-3.6.0/library/StanHeaders/include"     -DRCPP_PARALLEL_USE_TBB=1 -I"C:/Program Files/R/R-3.6.0/library/RcppParallel/include" -D_REENTRANT -DSTAN_THREADS      -mtune=native -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -c RcppExports.cpp -o RcppExports.o


"C:/RBuildTools/3.5/mingw_64/bin/g++.exe" -std=c++1y  -I"C:/PROGRA~1/R/R-36~1.0/include" -DNDEBUG -I"../inst/include" -I"C:/Program Files/R/R-3.6.0/library/StanHeaders/include/src" -DBOOST_DISABLE_ASSERTS -DEIGEN_NO_DEBUG -DRCPP_PARALLEL_USE_TBB=1 -I"C:/Program Files/R/R-3.6.0/library/BH/include" -I"C:/Program Files/R/R-3.6.0/library/Rcpp/include" -I"C:/Program Files/R/R-3.6.0/library/RcppEigen/include" -I"C:/Program Files/R/R-3.6.0/library/RcppParallel/include" -I"C:/Program Files/R/R-3.6.0/library/rstan/include" -I"C:/Program Files/R/R-3.6.0/library/StanHeaders/include"     -DRCPP_PARALLEL_USE_TBB=1 -I"C:/Program Files/R/R-3.6.0/library/RcppParallel/include" -D_REENTRANT -DSTAN_THREADS      -mtune=native -O3 -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -c stanExports_dummy_lm_model.cc -o stanExports_dummy_lm_model.o
C:\RBuildTools\3.5\mingw_32\bin\nm.exe: RcppExports.o: File format not recognized
C:\RBuildTools\3.5\mingw_32\bin\nm.exe: stanExports_dummy_lm_model.o: File format not recognized
C:/RBuildTools/3.5/mingw_64/bin/g++.exe -std=c++1y -shared -s -static-libgcc -o Bernadette.dll tmp.def RcppExports.o stanExports_dummy_lm_model.o -LC:PROGRA~1RR-36~1.0libraryRCPPPA~1libi386 -ltbb -ltbbmalloc -LC:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386 -Wl,-rpath,C:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386 -ltbb -ltbbmalloc -LC:/PROGRA~1/R/R-36~1.0/bin/i386 -lR
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386/tbb.dll when searching for -ltbb
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386/tbb.dll when searching for -ltbb
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -ltbb
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386/tbbmalloc.dll when searching for -ltbbmalloc
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386/tbbmalloc.dll when searching for -ltbbmalloc
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -ltbbmalloc
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386/tbb.dll when searching for -ltbb
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386/tbb.dll when searching for -ltbb
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -ltbb
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386/tbbmalloc.dll when searching for -ltbbmalloc
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/Program Files/R/R-3.6.0/library/RcppParallel/lib/i386/tbbmalloc.dll when searching for -ltbbmalloc
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -ltbbmalloc
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/PROGRA~1/R/R-36~1.0/bin/i386/R.dll when searching for -lR
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: skipping incompatible C:/PROGRA~1/R/R-36~1.0/bin/i386/R.dll when searching for -lR
C:/RBuildTools/3.5/mingw_64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.3/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lR
collect2.exe: error: ld returned 1 exit status
no DLL was created
ERROR: compilation failed for package 'Bernadette'
* removing 'C:/Program Files/R/R-3.6.0/library/Bernadette'

Case when the Makevars file is empty:

$ Rcmd.exe INSTALL --preclean bernadette
 installing to library 'C:/Program Files/R/R-3.6.0/library'
* installing *source* package 'Bernadette' ...
** using staged installation
** libs

*** arch - i386
Error in .shlib_internal(args) :
  C++14 standard requested but CXX14 is not defined
* removing 'C:/Program Files/R/R-3.6.0/library/Bernadette'

Thanks,
Lampros

Hi there,

Any followup on the above?

Thanks.

Hi @bernadette-eu,

Sorry for the slow response.

After my changes the package builds on my machine, suggesting the problem you are having is windows specific. I don’t have a Windows box but maybe @andrjohns or @hsbadr could help?

Best,
Jeffrey

1 Like

It looks like you’ve an environment issue that causes linking problems to all DLLs, including R itself:

  ld.exe: skipping incompatible C:/PROGRA~1/R/R-36~1.0/bin/i386/R.dll when searching for -lR

This happens when the toolchain is incompatible with the one that was used to build R and all Windows libraries on CRAN or when you mix 32-bit and 64-bit binaries. Make sure that you run the 64-bit R with 64-bit Rtools compatible with R version.

Also, as suggested above, the Makevars issue could be fixed using:

          if (R.version$major < 4 && isTRUE(.Platform$OS.type == "windows")) {
            dotR <- file.path(Sys.getenv("HOME"), ".R")
            if (!file.exists(dotR)) dir.create(dotR)
            path_makevars <- ifelse(.Platform$OS.type == "windows", "Makevars.win", "Makevars")
            M <- file.path(dotR, path_makevars)
            if (!file.exists(M)) file.create(M)
            cat("\nCXX14FLAGS=-O3",
                "CXX14 = $(BINPREF)g++ -m$(WIN) -std=c++1y",
                "CXX11FLAGS=-O3",
                file = M, sep = "\n", append = TRUE)
          }
2 Likes

Eventually I upgraded to R version 4.1.0 and RTools 40 and re-configured the C toolchain. Having done that, the R library compliles with no errors.

Thank you all for your support on this.

Lampros

1 Like