How to install cmdstanr on server without Internet?

I need to run my codes with cmdstanr and I have made it successfully on my personal PC. However, the ultimate goal is to perform on server, which belongs to my university and is limited to Internet connection. So I wanna how to install cmastanr without Internet? I failed to find any guide about this.

Appreciated for any reply.

1 Like

There are two components to working with cmdstanR:

  • Installing the cmdstanr R package
  • Downloading and building cmdstan

Both of these can be performed without internet if you can provide the files to the server.

For cmdstan, you can download the source from here. Then, create a folder in your R home directory called .cmdstan and extract the source there (e.g., you should have the folders .cmdstan/cmdstan-2.29.1)

For installing cmdstanr, you can download the R package source from here, and install like any other R package source archive.

Then, you can load cmdstanr and instruct it to build the cmdstan source directory that you’ve downloaded:

cmdstanr::rebuild_cmdstan()

Then you should able to use cmdstanr as usual

3 Likes

Thanks for apply. When run cmdstanr::rebuild_cmdstan() , I got error message as followed.

Error in rethrow_call(c_processx_exec, command, c(command, args), pty,  : 
  Command 'mingw32-make.exe' not found @win/processx.c:982 (processx_exec)
Type .Last.error.trace to see where the error occurred
Warning message:
Can't find CmdStan makefile to detect version number. Path may not point to valid installation. 

Which necessary step did I miss?

The first error:

Command 'mingw32-make.exe' not found

Can be resolved by running :

cmdstanr::check_cmdstan_toolchain(fix = TRUE)

The second error:

Can't find CmdStan makefile to detect version number. Path may not point to valid installation. 

Indicates that the cmdstan folder is not where cmdstanr expects it to be.

To check this, run:

file.path(Sys.getenv("HOME"), ".cmdstan")

Which will show where cmdstanr is looking. What do you get ask for the list of folders in that directory:

list.files(file.path(Sys.getenv("HOME"), ".cmdstan"))
1 Like

I got following error when cmdstanr::check_cmdstan_toolchain(fix = TRUE).

Installing mingw32-make and writing RTools path to ~/.Renviron ...
Error in processx::run("pacman", args = c("-Syu", "mingw-w64-x86_64-make",  : 
  System command 'pacman' failed, exit status: 1, stderr (last 10 lines):
E> error: failed to update mingw32 (invalid url for server)
E> error: failed retrieving file 'mingw64.db' from cloud.r-project.org : Could not resolve host: cloud.r-project.org
E> error: failed retrieving file 'mingw64.db' from cran.r-project.org : Could not resolve host: cran.r-project.org
E> error: failed retrieving file 'mingw64.db' from r-windows.github.io : Could not resolve host: r-windows.github.io
E> error: failed to update mingw64 (invalid url for server)
E> error: failed retrieving file 'ucrt64.db' from cloud.r-project.org : Could not resolve host: cloud.r-project.org
E> error: failed retrieving file 'ucrt64.db' from cran.r-project.org : Could not resolve host: cran.r-project.org
E> error: failed retrieving file 'ucrt64.db' from r-windows.github.io : Could not resolve host: r-windows.github.io
E> error: failed to update ucrt64 (invalid 
[...]
Type .Last.error.trace to see where the error occurre

I have already install rtools40 and added C:\RTools\RTools40\usr\bin C:\RTools\RTools40\mingw64\bin to PATH environment variable.

Then, according to your advice for the second error, I get

> file.path(Sys.getenv("HOME"), ".cmdstan")
[1] "C:\\Users\\dell\\Documents/.cmdstan"
> list.files(file.path(Sys.getenv("HOME"), ".cmdstan"))
[1] "cmdstan-2.29.1"

Ah of course. You need an internet connection to install mingw32-make through RTools.

Can you try the steps I posted here for manually installing mingw32-make: Configure Rtools40 for CMDSTAN - #2 by andrjohns

1 Like

Unfortunately, I failed to install mingw32-make manually with following messages.

警告:database file for 'mingw32' does not exist (use '-Sy' to download)
警告:database file for 'mingw64' does not exist (use '-Sy' to download)
警告:database file for 'ucrt64' does not exist (use '-Sy' to download)
正在加载软件包...
正在解析依赖关系...
警告:无法解决 "mingw-w64-x86_64-gettext","mingw-w64-x86_64-make" 的一个依赖关系
:: 因为无法解决依赖关系,以下软件包无法进行更新:
      mingw-w64-x86_64-make

My rtools is a Chinese version, so that the messages including Chinese. Hope it’s easy to your translator.

It looks like it’s having an issue with the gettext dependency, can you try installing that the same way? Link to the source here: https://cloud.r-project.org/bin/windows/Rtools/4.0/mingw64/mingw-w64-x86_64-gettext-0.19.8.1-9002-any.pkg.tar.xz

1 Like

It’s helpful. Now I successfully installed mingw32-make in rtools. But the warnings still exist in rtools and I’m not sure if that matters.

warning:database file for 'mingw32' does not exist (use '-Sy' to download)
warning:database file for 'mingw64' does not exist (use '-Sy' to download)
warning:database file for 'ucrt64' does not exist (use '-Sy' to download)

And then I run again cmdstanr::check_cmdstan_toolchain(fix = TRUE) in Rstudio, errors still

Installing mingw32-make and writing RTools path to ~/.Renviron ...
Error in processx::run("pacman", args = c("-Syu", "mingw-w64-x86_64-make",  : 
  System command 'pacman' failed, exit status: 1, stderr:
E> error: failed to update mingw32 (unable to lock database)
E> error: failed to update mingw64 (unable to lock database)
E> error: failed to update ucrt64 (unable to lock database)
E> error: failed to synchronize all databases
Type .Last.error.trace to see where the error occurred

The cmdstanr::check_cmdstan_toolchain(fix = TRUE) call was for installing mingw32-make, it’s not needed now that you’ve done it manually. Can you try the cmdstanr::rebuild_cmdstan() call again?

1 Like

Now I got the same messages as before.

> cmdstanr::rebuild_cmdstan()
Error in rethrow_call(c_processx_exec, command, c(command, args), pty,  : 
  Command 'mingw32-make.exe' not found @win/processx.c:982 (processx_exec)
Type .Last.error.trace to see where the error occurred
Warning message:
Can't find CmdStan makefile to detect version number. Path may not point to valid installation. 

Can you post the outputs from:

Sys.getenv("PATH")
list.files(file.path(Sys.getenv("RTOOLS40_HOME"), "mingw64", "bin"))
1 Like

Sure.

> Sys.getenv("PATH")
[1] "C:\\Users\\dell\\Documents\\R\\win-library\\4.0\\rtools40\\usr\\bin;C:\\Program Files\\R\\R-4.1.2\\bin\\x64;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common;C:\\Program Files\\NVIDIA Corporation\\NVIDIA NvDLISR;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;D:\\Matlab\\runtime\\win64;D:\\Matlab\\bin;C:\\rtools40\\usr\\bin;C:\\rtools40\\mingw32\\bin;C:\\rtools40\\mingw64\\bin;C:\\Users\\dell\\AppData\\Local\\Microsoft\\WindowsApps;"
> list.files(file.path(Sys.getenv("RTOOLS40_HOME"), "mingw64", "bin"))
 [1] "addr2line.exe"                     "ar.exe"                           
 [3] "as.exe"                            "autopoint"                        
 [5] "bunzip2.exe"                       "bzcat.exe"                        
 [7] "bzcmp"                             "bzdiff"                           
 [9] "bzegrep"                           "bzfgrep"                          
[11] "bzgrep"                            "bzip2.exe"                        
[13] "bzip2recover.exe"                  "bzless"                           
[15] "bzmore"                            "c++.exe"                          
[17] "c++filt.exe"                       "cc.exe"                           
[19] "cjpeg.exe"                         "cpp.exe"                          
[21] "djpeg.exe"                         "dlltool.exe"                      
[23] "dllwrap.exe"                       "dwp.exe"                          
[25] "elfedit.exe"                       "envsubst.exe"                     
[27] "fix-qdf"                           "g++.exe"                          
[29] "gcc-ar.exe"                        "gcc-nm.exe"                       
[31] "gcc-ranlib.exe"                    "gcc.exe"                          
[33] "gcov-tool.exe"                     "gcov.exe"                         
[35] "gendef.exe"                        "genidl.exe"                       
[37] "genlib.exe"                        "genpeimg.exe"                     
[39] "gettext.exe"                       "gettext.sh"                       
[41] "gettextize"                        "gfortran.exe"                     
[43] "gprof.exe"                         "jpegtran.exe"                     
[45] "ld.bfd.exe"                        "ld.exe"                           
[47] "ld.gold.exe"                       "mingw32-make.exe"                 
[49] "msgattrib.exe"                     "msgcat.exe"                       
[51] "msgcmp.exe"                        "msgcomm.exe"                      
[53] "msgconv.exe"                       "msgen.exe"                        
[55] "msgexec.exe"                       "msgfilter.exe"                    
[57] "msgfmt.exe"                        "msggrep.exe"                      
[59] "msginit.exe"                       "msgmerge.exe"                     
[61] "msgunfmt.exe"                      "msguniq.exe"                      
[63] "ngettext.exe"                      "nm.exe"                           
[65] "objcopy.exe"                       "objdump.exe"                      
[67] "pcre-config"                       "pcregrep.exe"                     
[69] "pcretest.exe"                      "pkg-config.exe"                   
[71] "qpdf.exe"                          "qpdf21.dll"                       
[73] "ranlib.exe"                        "rdjpgcom.exe"                     
[75] "readelf.exe"                       "recode-sr-latin.exe"              
[77] "size.exe"                          "strings.exe"                      
[79] "strip.exe"                         "tjbench.exe"                      
[81] "widl.exe"                          "windmc.exe"                       
[83] "windres.exe"                       "wrjpgcom.exe"                     
[85] "x86_64-w64-mingw32-c++.exe"        "x86_64-w64-mingw32-g++.exe"       
[87] "x86_64-w64-mingw32-gcc-8.3.0.exe"  "x86_64-w64-mingw32-gcc-ar.exe"    
[89] "x86_64-w64-mingw32-gcc-nm.exe"     "x86_64-w64-mingw32-gcc-ranlib.exe"
[91] "x86_64-w64-mingw32-gcc.exe"        "x86_64-w64-mingw32-gfortran.exe"  
[93] "x86_64-w64-mingw32-pkg-config.exe" "xgettext.exe"                     
[95] "xmlwf.exe"                         "zlib-flate.exe"  

So mingw32-make.exe is in the rtools40/mingw64/bin folder, but your PATH is only including the rtools40/usr/bin folder.

Can you run the following command to append this to your PATH variable on startup:

write('PATH="${RTOOLS40_HOME}\\mingw64\\bin;${PATH}"', file = "~/.Renviron", append = TRUE)

Then restart R and try the rebuild_cmdstan() call again?

2 Likes

Here are the messages.

> set_cmdstan_path("C:\\Users\\dell\\Documents/.cmdstan")
CmdStan path set to: C:/Users/dell/Documents/.cmdstan
Warning message:
Can't find CmdStan makefile to detect version number. Path may not point to valid installation. 
> cmdstanr::rebuild_cmdstan()
mingw32-make.exe: *** No rule to make target 'clean-all'.  Stop.
mingw32-make.exe: *** No rule to make target 'build'.  Stop.
Warning messages:
1: Can't find CmdStan makefile to detect version number. Path may not point to valid installation. 
2: Can't find CmdStan makefile to detect version number. Path may not point to valid installation. 
3: Can't find CmdStan makefile to detect version number. Path may not point to valid installation. 
4: Can't find CmdStan makefile to detect version number. Path may not point to valid installation. 
5: Can't find CmdStan makefile to detect version number. Path may not point to valid installation. 

rebuild_cmdstan() can’t find the path at first, so set_cmdstan_path() is added. But I’m not sure if the path I specified in set_cmdstan_path() is correct.

Can you try instead:

set_cmdstan_path("C:\\Users\\dell\\Documents/.cmdstan/cmdstan-2.29.1")
2 Likes

Great! Now cmdstanr in the server is work! It will help me a lot. I can’t be more excited and grateful. You’re my super hero today hah.

7 Likes