I saw your post about map_rect working in rstan on Windows starting with R 3.6. That is good news and means that it should be possible to implement CRAN-approved R packages that make use of map_rect. However, this poses some interesting issues as I (and I assume other package authors, which is why I am posting this here) would probably not want to compile every Stan model using the compiler flags for threading as this lowers performance vis-a-vis non-threaded code. Instead, I think it would be optimal to be able to compile some models with threading support and some without depending to have both options for the user.
I imagine this is probably straightforward with editing the makefile to conditionally compile certain Stan models with certain flags, but could you provide some insight as to the best way to do that/whether rstantools can be modified to make this an option?
Also, we might be able to get threading to work (or at least not fail to compile) with rtools35. My guess is that the easiest thing to do is compile everything with threading and if the package or user does not want to utilize it for some model(s), then do Sys.setenv(STAN_NUM_THREADS = 1). If that makes things noticeably slower, then we will figure out something else.
I just tested this out in the last days, and threading seems to already ‘work’ (compile and run) on my win 7 64, R 3.5.2, rtools 3.5… though it seems that asking for extra cores just slows things down. In contrast, tried out r-testing (3.6 dev version) with rtools40 and both my win7 and 10 machines get stuck taking forever to compile…
edit: building ctsem from cran is working fine with r-testing, local version with map_rect spends hours compiling though. looking into it further…
Ok it builds fine if i construct a source package first and then install.packages . Some problem building from source in typical package development way using Rstudio. Also can’t install devtools with errors like:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lcurl-dualssl
collect2.exe: error: ld returned 1 exit status
make[1]: Leaving directory '/c/Users/driver/AppData/Local/Temp/RtmpaoAZPv/R.INSTALL2324438139e3/git2r/src'
no DLL was created
ERROR: compilation failed for package 'git2r'
* removing 'C:/Users/driver/Documents/R/win-library/testing/git2r'
ERROR: failed to lock directory 'C:/Users/driver/Documents/R/win-library/testing' for modifying
Try removing 'C:/Users/driver/Documents/R/win-library/testing/00LOCK-processx'
/usr/bin/cat: httr.out: No such file or directory
/usr/bin/cat: gh.out: No such file or directory
/usr/bin/cat: usethis.out: No such file or directory
Error in i.p(...) :
(converted from warning) installation of one or more packages failed,
probably ‘curl’, ‘git2r’, ‘processx’, ‘httr’, ‘gh’, ‘usethis’
Sorry if slightly off topic but seemed a relevant spot to discuss these issues…
It does build, it does run… but only with a single thread to my knowledge. At the moment when you start running multi-threaded programs then the program crashes.
BTW, right now threading will cost you some performance, but this is hopefully going away soon such that one will not notice performance penalties due to threading for almost all models… let’s see.
Maybe i have something oddly configured, but since asking for more cores slowed things down I inferred that it was at least responsive to such a request.
Ok maybe we can postpone this discussion to late April when R 3.6 is released and perhaps a new Stan version is as well that fixes the map_rect performance hit.