I need to run my codes with cmdstanr and I have made it successfully on my personal PC. However, the ultimate goal is to perform on server, which belongs to my university and is limited to Internet connection. So I wanna how to install cmastanr without Internet? I failed to find any guide about this.
Appreciated for any reply.
There are two components to working with cmdstanR:
cmdstanr R packagecmdstan
Both of these can be performed without internet if you can provide the files to the server.
For cmdstan, you can download the source from here. Then, create a folder in your R home directory called .cmdstan and extract the source there (e.g., you should have the folders .cmdstan/cmdstan-2.29.1)
For installing cmdstanr, you can download the R package source from here, and install like any other R package source archive.
Then, you can load cmdstanr and instruct it to build the cmdstan source directory that youāve downloaded:
cmdstanr::rebuild_cmdstan()
Then you should able to use cmdstanr as usual
Thanks for apply. When run cmdstanr::rebuild_cmdstan() , I got error message as followed.
Error in rethrow_call(c_processx_exec, command, c(command, args), pty, :
Command 'mingw32-make.exe' not found @win/processx.c:982 (processx_exec)
Type .Last.error.trace to see where the error occurred
Warning message:
Can't find CmdStan makefile to detect version number. Path may not point to valid installation.
Which necessary step did I miss?
The first error:
Command 'mingw32-make.exe' not found
Can be resolved by running :
cmdstanr::check_cmdstan_toolchain(fix = TRUE)
The second error:
Can't find CmdStan makefile to detect version number. Path may not point to valid installation.
Indicates that the cmdstan folder is not where cmdstanr expects it to be.
To check this, run:
file.path(Sys.getenv("HOME"), ".cmdstan")
Which will show where cmdstanr is looking. What do you get ask for the list of folders in that directory:
list.files(file.path(Sys.getenv("HOME"), ".cmdstan"))
I got following error when cmdstanr::check_cmdstan_toolchain(fix = TRUE).
Installing mingw32-make and writing RTools path to ~/.Renviron ...
Error in processx::run("pacman", args = c("-Syu", "mingw-w64-x86_64-make", :
System command 'pacman' failed, exit status: 1, stderr (last 10 lines):
E> error: failed to update mingw32 (invalid url for server)
E> error: failed retrieving file 'mingw64.db' from cloud.r-project.org : Could not resolve host: cloud.r-project.org
E> error: failed retrieving file 'mingw64.db' from cran.r-project.org : Could not resolve host: cran.r-project.org
E> error: failed retrieving file 'mingw64.db' from r-windows.github.io : Could not resolve host: r-windows.github.io
E> error: failed to update mingw64 (invalid url for server)
E> error: failed retrieving file 'ucrt64.db' from cloud.r-project.org : Could not resolve host: cloud.r-project.org
E> error: failed retrieving file 'ucrt64.db' from cran.r-project.org : Could not resolve host: cran.r-project.org
E> error: failed retrieving file 'ucrt64.db' from r-windows.github.io : Could not resolve host: r-windows.github.io
E> error: failed to update ucrt64 (invalid
[...]
Type .Last.error.trace to see where the error occurre
I have already install rtools40 and added C:\RTools\RTools40\usr\bin C:\RTools\RTools40\mingw64\bin to PATH environment variable.
Then, according to your advice for the second error, I get
> file.path(Sys.getenv("HOME"), ".cmdstan")
[1] "C:\\Users\\dell\\Documents/.cmdstan"
> list.files(file.path(Sys.getenv("HOME"), ".cmdstan"))
[1] "cmdstan-2.29.1"
Ah of course. You need an internet connection to install mingw32-make through RTools.
Can you try the steps I posted here for manually installing mingw32-make: Configure Rtools40 for CMDSTAN - #2 by andrjohns
Unfortunately, I failed to install mingw32-make manually with following messages.
č¦åļ¼database file for 'mingw32' does not exist (use '-Sy' to download)
č¦åļ¼database file for 'mingw64' does not exist (use '-Sy' to download)
č¦åļ¼database file for 'ucrt64' does not exist (use '-Sy' to download)
ę£åØå č½½č½Æ件å
...
ę£åØč§£ęä¾čµå
³ē³»...
č¦åļ¼ę ę³č§£å³ "mingw-w64-x86_64-gettext"ļ¼"mingw-w64-x86_64-make" ēäøäøŖä¾čµå
³ē³»
:: å äøŗę ę³č§£å³ä¾čµå
³ē³»ļ¼ä»„äøč½Æ件å
ę ę³čæč”ę“ę°ļ¼
mingw-w64-x86_64-make
My rtools is a Chinese version, so that the messages including Chinese. Hope itās easy to your translator.
It looks like itās having an issue with the gettext dependency, can you try installing that the same way? Link to the source here: https://cloud.r-project.org/bin/windows/Rtools/4.0/mingw64/mingw-w64-x86_64-gettext-0.19.8.1-9002-any.pkg.tar.xz
Itās helpful. Now I successfully installed mingw32-make in rtools. But the warnings still exist in rtools and Iām not sure if that matters.
warningļ¼database file for 'mingw32' does not exist (use '-Sy' to download)
warningļ¼database file for 'mingw64' does not exist (use '-Sy' to download)
warningļ¼database file for 'ucrt64' does not exist (use '-Sy' to download)
And then I run again cmdstanr::check_cmdstan_toolchain(fix = TRUE) in Rstudio, errors still
Installing mingw32-make and writing RTools path to ~/.Renviron ...
Error in processx::run("pacman", args = c("-Syu", "mingw-w64-x86_64-make", :
System command 'pacman' failed, exit status: 1, stderr:
E> error: failed to update mingw32 (unable to lock database)
E> error: failed to update mingw64 (unable to lock database)
E> error: failed to update ucrt64 (unable to lock database)
E> error: failed to synchronize all databases
Type .Last.error.trace to see where the error occurred
The cmdstanr::check_cmdstan_toolchain(fix = TRUE) call was for installing mingw32-make, itās not needed now that youāve done it manually. Can you try the cmdstanr::rebuild_cmdstan() call again?
Now I got the same messages as before.
> cmdstanr::rebuild_cmdstan()
Error in rethrow_call(c_processx_exec, command, c(command, args), pty, :
Command 'mingw32-make.exe' not found @win/processx.c:982 (processx_exec)
Type .Last.error.trace to see where the error occurred
Warning message:
Can't find CmdStan makefile to detect version number. Path may not point to valid installation.
Can you post the outputs from:
Sys.getenv("PATH")
list.files(file.path(Sys.getenv("RTOOLS40_HOME"), "mingw64", "bin"))
Sure.
> Sys.getenv("PATH")
[1] "C:\\Users\\dell\\Documents\\R\\win-library\\4.0\\rtools40\\usr\\bin;C:\\Program Files\\R\\R-4.1.2\\bin\\x64;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common;C:\\Program Files\\NVIDIA Corporation\\NVIDIA NvDLISR;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;D:\\Matlab\\runtime\\win64;D:\\Matlab\\bin;C:\\rtools40\\usr\\bin;C:\\rtools40\\mingw32\\bin;C:\\rtools40\\mingw64\\bin;C:\\Users\\dell\\AppData\\Local\\Microsoft\\WindowsApps;"
> list.files(file.path(Sys.getenv("RTOOLS40_HOME"), "mingw64", "bin"))
[1] "addr2line.exe" "ar.exe"
[3] "as.exe" "autopoint"
[5] "bunzip2.exe" "bzcat.exe"
[7] "bzcmp" "bzdiff"
[9] "bzegrep" "bzfgrep"
[11] "bzgrep" "bzip2.exe"
[13] "bzip2recover.exe" "bzless"
[15] "bzmore" "c++.exe"
[17] "c++filt.exe" "cc.exe"
[19] "cjpeg.exe" "cpp.exe"
[21] "djpeg.exe" "dlltool.exe"
[23] "dllwrap.exe" "dwp.exe"
[25] "elfedit.exe" "envsubst.exe"
[27] "fix-qdf" "g++.exe"
[29] "gcc-ar.exe" "gcc-nm.exe"
[31] "gcc-ranlib.exe" "gcc.exe"
[33] "gcov-tool.exe" "gcov.exe"
[35] "gendef.exe" "genidl.exe"
[37] "genlib.exe" "genpeimg.exe"
[39] "gettext.exe" "gettext.sh"
[41] "gettextize" "gfortran.exe"
[43] "gprof.exe" "jpegtran.exe"
[45] "ld.bfd.exe" "ld.exe"
[47] "ld.gold.exe" "mingw32-make.exe"
[49] "msgattrib.exe" "msgcat.exe"
[51] "msgcmp.exe" "msgcomm.exe"
[53] "msgconv.exe" "msgen.exe"
[55] "msgexec.exe" "msgfilter.exe"
[57] "msgfmt.exe" "msggrep.exe"
[59] "msginit.exe" "msgmerge.exe"
[61] "msgunfmt.exe" "msguniq.exe"
[63] "ngettext.exe" "nm.exe"
[65] "objcopy.exe" "objdump.exe"
[67] "pcre-config" "pcregrep.exe"
[69] "pcretest.exe" "pkg-config.exe"
[71] "qpdf.exe" "qpdf21.dll"
[73] "ranlib.exe" "rdjpgcom.exe"
[75] "readelf.exe" "recode-sr-latin.exe"
[77] "size.exe" "strings.exe"
[79] "strip.exe" "tjbench.exe"
[81] "widl.exe" "windmc.exe"
[83] "windres.exe" "wrjpgcom.exe"
[85] "x86_64-w64-mingw32-c++.exe" "x86_64-w64-mingw32-g++.exe"
[87] "x86_64-w64-mingw32-gcc-8.3.0.exe" "x86_64-w64-mingw32-gcc-ar.exe"
[89] "x86_64-w64-mingw32-gcc-nm.exe" "x86_64-w64-mingw32-gcc-ranlib.exe"
[91] "x86_64-w64-mingw32-gcc.exe" "x86_64-w64-mingw32-gfortran.exe"
[93] "x86_64-w64-mingw32-pkg-config.exe" "xgettext.exe"
[95] "xmlwf.exe" "zlib-flate.exe"
So mingw32-make.exe is in the rtools40/mingw64/bin folder, but your PATH is only including the rtools40/usr/bin folder.
Can you run the following command to append this to your PATH variable on startup:
write('PATH="${RTOOLS40_HOME}\\mingw64\\bin;${PATH}"', file = "~/.Renviron", append = TRUE)
Then restart R and try the rebuild_cmdstan() call again?
Here are the messages.
> set_cmdstan_path("C:\\Users\\dell\\Documents/.cmdstan")
CmdStan path set to: C:/Users/dell/Documents/.cmdstan
Warning message:
Can't find CmdStan makefile to detect version number. Path may not point to valid installation.
> cmdstanr::rebuild_cmdstan()
mingw32-make.exe: *** No rule to make target 'clean-all'. Stop.
mingw32-make.exe: *** No rule to make target 'build'. Stop.
Warning messages:
1: Can't find CmdStan makefile to detect version number. Path may not point to valid installation.
2: Can't find CmdStan makefile to detect version number. Path may not point to valid installation.
3: Can't find CmdStan makefile to detect version number. Path may not point to valid installation.
4: Can't find CmdStan makefile to detect version number. Path may not point to valid installation.
5: Can't find CmdStan makefile to detect version number. Path may not point to valid installation.
rebuild_cmdstan() canāt find the path at first, so set_cmdstan_path() is added. But Iām not sure if the path I specified in set_cmdstan_path() is correct.
Can you try instead:
set_cmdstan_path("C:\\Users\\dell\\Documents/.cmdstan/cmdstan-2.29.1")
Great! Now cmdstanr in the server is work! It will help me a lot. I canāt be more excited and grateful. Youāre my super hero today hah.
Iām also trying to install cmdstan on a secure server without internet access and this was very helpful, and it feels like it āalmostā got me all the way to the finish line - but not all the way: When I try to install_cmdstan() I get the error message "āGithub download of release list failedā
The details of what I did:
You donāt need to run install_cmdstan() if youāve already manually built cmdstan and set the path using set_cmdstan_path()
UPDATE: While I did not solve the problem of installing cmdstan (see below), I did solve my underlying problem: achieving within-chain parallellization on an offline server. I discovered that there is an rstan development build that uses stan 2.26.1, which supports the new reduce_sum function. I simply used the download.packages() function in R to download the two development versions recommended for windows users with Rtool42 and R versions 4.2, and installed these after uploading them to the server. The biggest hassle was identifying all the dependencies that had to be installed before rstan could be compiled.
Initial reply to original post:
You are right - re-reading the above thread I see that I got confused between rebuild_cmdstan() (which I am supposed to use) and install_cmdstan().
I now tried rebuild_cmdstan() - with my R home directory having a folder called .cmdstan, within which I had a folder with all the (extracted) files and subfolders from the cmdstan-2.31.0 file.
I get the following error message:
'cut' is not recognized as an internal or external command,
operable program or batch file.
'cut' is not recognized as an internal or external command,
operable program or batch file.
process_begin: CreateProcess(NULL, expr >= 8, ...) failed.
mingw32-make.exe: makefile:125: pipe: Bad file descriptor
INFO: Could not find files for the given pattern(s).
rm -f -r test
process_begin: CreateProcess(NULL, rm -f -r test, ...) failed.
make (e=2): The system cannot find the file specified.
mingw32-make.exe: *** [makefile:301: clean] Error 2
'cut' is not recognized as an internal or external command,
operable program or batch file.
'cut' is not recognized as an internal or external command,
operable program or batch file.
process_begin: CreateProcess(NULL, expr >= 8, ...) failed.
mingw32-make.exe: makefile:125: pipe: Bad file descriptor
INFO: Could not find files for the given pattern(s).
cp bin/windows-stanc bin/stanc.exe
process_begin: CreateProcess(NULL, cp bin/windows-stanc bin/stanc.exe, ...) failed.
make (e=2): The system cannot find the file specified.
mingw32-make.exe: *** [make/stanc:78: bin/stanc.exe] Error 2
This seems similar to an error message referenced here, where the recommendation is to
open up a git bash shell. Cd to the cmdstan directory ā eg:
cd /c/users/ajs/OneDrive/ajs/dev/cmdstan/
and run mingw32-make build
Using the windows file explorer I tried to open a command window in the cmdstan-2.31.0 folder, but I couldnāt figure out how to run the mingw32-make from the terminal window (the terminal did not recognize the filename as the mingw32-make file is in an entirely different location.)