How to install cmdstanr on server without Internet?

Mads_Lund_Pedersen · August 11, 2023, 8:49am

Hi. I’m also struggling to getting cmdstanr to work on a server. I have followed the steps of downloading source code for cmdstanr and cmdstan. I’m able to get cmdstanr to load and setting the path for cmdstan, but when running cmdstanr::rebuild_cmdstan() I get an error message saying it can’t find cmdstan/command.hpp (see screenshot). Any help greatly appreciated

Guido_Biele · August 11, 2023, 2:30pm

You are still trying to download something from the Internet (see the curl command).

It would also be good if you specify if
you are on a windwos or linux server (@Ole_Rogeberg seems to have been on windows).

I also remember that I used install_cmdstan when I installed cmdstan from scratch on a server without internet access. (Though i got stuck on some Rcpp problem)…

Mads_Lund_Pedersen · August 11, 2023, 7:06pm

Thanks, Guido.

I’m working on a linux server.

I don’t know how to prevent it trying to download (curl command), but that could be part of the issue.

Running install_cmdstan() gives an error that it can’t open URL, so that’s why tried I downloading source code and setting the path to the extracted folder.

Guido_Biele · August 13, 2023, 1:28pm

I guess you’ve already manually downloaded cmstan, copied it to the sever and pointed install_cmdatanr explicitly to the source file, correct?

If you’ve done all this, i unfortunately also don’t know what the problem is.

andrjohns · August 13, 2023, 2:41pm

The error indicates that some files are missing.

When you download the source code for cmdstan, make sure you either download from the “Releases” section or by cloning the repo recursively.

The repo readme has more info: GitHub - stan-dev/cmdstan: CmdStan, the command line interface to Stan

joels · August 13, 2023, 10:56pm

After reading through the latest entries in this thread, I decided to take another crack at installing cmdstanr/CmdStan on a Windows server with no internet connection, but I had some issues.

Here are the steps I performed:

Ran cmdstanr::check_cmdstan_toolchain() which returned “The C++ toolchain required for CmdStan is setup properly!.”
Went to Release v2.32.2 (15 May 2023) · stan-dev/cmdstan · GitHub and downloaded cmdstan-2.32.2.tar.gz. I unzipped this file to create the folder cmdstan-2.32.2. (I assumed this was what @andrjohns meant by the “Releases” section, but please let me know if I’m mistaken.)
Placed the folder in C:/Users/***/Documents/.cmdstan/ (username redacted).
Using set_cmdstan_path() I set the path to C:/Users/***/Documents/.cmdstan/cmdstan-2.32.2 and confirmed that this was successful using cmdstan_path().
Ran rebuild_cmdstan(). This produced a huge amount of console output, much of it in red “warning” text, but the function completed without throwing an error.

After these steps, I restarted R and ran the code below, which ultimately resulted in an error when I tried to fit a model using brms (see below for error messages). I’m not sure what to try next. Any ideas on what else I need to install?

> library(cmdstanr)
This is cmdstanr version 0.6.0
- CmdStanR documentation and vignettes: mc-stan.org/cmdstanr
- CmdStan path: C:/Users/***/Documents/.cmdstan/cmdstan-2.32.2
- CmdStan version: 2.32.2

> library(brms)
Loading required package: Rcpp
Loading 'brms' package (version 2.19.0). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar

> m1 = brm(mpg ~ hp, data=mtcars, backend="cmdstanr")
Compiling Stan program...
INFO: Could not find files for the given pattern(s).

In file included from C:/rtools43/ucrt64/include/c++/13.2.0/stdlib.h:36,
                 from C:/rtools43/ucrt64/lib/gcc/x86_64-w64-mingw32/13.2.0/include/mm_malloc.h:27,
                 from C:/rtools43/ucrt64/lib/gcc/x86_64-w64-mingw32/13.2.0/include/xmmintrin.h:34,
                 from C:/rtools43/ucrt64/lib/gcc/x86_64-w64-mingw32/13.2.0/include/immintrin.h:31,
                 from ../tbb_2020.3/include/tbb/machine/gcc_itsx.h:57,
                 from ../tbb_2020.3/include/tbb/machine/gcc_ia32_common.h:107,
                 from ../tbb_2020.3/include/tbb/machine/gcc_generic.h:232,
                 from ../tbb_2020.3/include/tbb/tbb_machine.h:197,
                 from ../tbb_2020.3/src/tbbmalloc/Synchronize.h:20,
                 from ../tbb_2020.3/src/tbbmalloc/Customize.h:29,
                 from ../tbb_2020.3/src/tbbmalloc/TypeDefinitions.h:54,
                 from ../tbb_2020.3/src/tbbmalloc/tbbmalloc_internal.h:21,
                 from ../tbb_2020.3/src/tbbmalloc/backend.cpp:19:
C:/rtools43/ucrt64/include/c++/13.2.0/cstdlib:141:11: error: 'at_quick_exit' has not been declared in '::'
  141 |   using ::at_quick_exit;
      |           ^~~~~~~~~~~~~

C:/rtools43/ucrt64/include/c++/13.2.0/cstdlib:164:11: error: 'quick_exit' has not been declared in '::'
  164 |   using ::quick_exit;
      |           ^~~~~~~~~~
C:/rtools43/ucrt64/include/c++/13.2.0/stdlib.h:43:14: error: 'at_quick_exit' has not been declared in 'std'
   43 |   using std::at_quick_exit;
      |              ^~~~~~~~~~~~~
C:/rtools43/ucrt64/include/c++/13.2.0/stdlib.h:46:14: error: 'quick_exit' has not been declared in 'std'
   46 |   using std::quick_exit;
      |              ^~~~~~~~~~

cc1plus.exe: note: unrecognized command-line option '-Wno-unknown-warning-option' may have been intended to silence earlier diagnostics

mingw32-make[1]: *** [C:/Users/***/Documents/.cmdstan/cmdstan-2.32.2/stan/lib/stan_math/lib/tbb_2020.3/build/Makefile.tbbmalloc:65: backend.o] Error 1

mingw32-make: *** [stan/lib/stan_math/make/libraries:173: stan/lib/stan_math/lib/tbb/tbbmalloc.def] Error 2

Error: An error occured during compilation! See the message above for more information.
>

andrjohns · August 14, 2023, 3:07am

The error here is because cmdstan is missing some compilation flags needed for compatibility, which are normally automatically set by cmdstanr during installation.

If you run:

    cmdstan_make_local(
      cpp_options = list(
        "CXXFLAGS += -Wno-nonnull -D_UCRT -Wno-deprecated-declarations",
        "TBB_CXXFLAGS= -D_UCRT"
      ),
      append = TRUE
    )

rebuild_cmdstan()

Your installation should be good to go

joels · August 14, 2023, 5:14am

rebuild_cmdstan() finished with the following messages:

mingw32-make[1]: Leaving directory 'C:/Users/***/Documents/.cmdstan/cmdstan-2.32.2/stan/lib/stan_math/lib/tbb'

NOTE: Please add C:/Users/***/Documents/.cmdstan/cmdstan-2.32.2/stan/lib/stan_math/lib/tbb to your PATH variable.
You may call

mingw32-make install-tbb

to automatically update your user configuration.
--- CmdStan v2.32.2 built ---

I wasn’t sure how to run mingw32-make install-tbb. That command isn’t recognized on the command line. If I run it from Powershell with .\mingw32-make install-tbb it pops up a “How do you want to open this file” dialogue. (I’m not that familiar with the inner workings of Windows.)

I tried fitting a model just to see what would happen. Error message is below.

> library(cmdstanr)
This is cmdstanr version 0.6.0
- CmdStanR documentation and vignettes: mc-stan.org/cmdstanr
- CmdStan path: C:/Users/***/Documents/.cmdstan/cmdstan-2.32.2
- CmdStan version: 2.32.2
> 
> library(brms)
Loading required package: Rcpp
Loading 'brms' package (version 2.19.0). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar

> m1 = brm(mpg ~ hp, data=mtcars, cores=4)
Compiling Stan program...
make cmd is
  make -f "C:/PROGRA~1/R/R-43~1.1/etc/x64/Makeconf" -f "C:/PROGRA~1/R/R-43~1.1/share/make/winshlib.mk" CXX='$(CXX14) $(CXX14STD)' CXXFLAGS='$(CXX14FLAGS)' CXXPICFLAGS='$(CXX14PICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX14LDFLAGS)' SHLIB_LD='$(SHLIB_CXX14LD)' SHLIB="file28fc6c2d632d.dll" WIN=64 TCLBIN= OBJECTS="file28fc6c2d632d.o"

make would use
if test "zfile28fc6c2d632d.o" != "z"; then \
  if test -e "file28fc6c2d632d-win.def"; then \
    echo g++  -shared -s -static-libgcc -o file28fc6c2d632d.dll file28fc6c2d632d-win.def file28fc6c2d632d.o  -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64" -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib"  -L"C:/PROGRA~1/R/R-43~1.1/bin/x64" -lR ; \
    g++  -shared -s -static-libgcc -o file28fc6c2d632d.dll file28fc6c2d632d-win.def file28fc6c2d632d.o  -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64" -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib"  -L"C:/PROGRA~1/R/R-43~1.1/bin/x64" -lR ; \
  else \
    echo EXPORTS > tmp.def; \
    nm file28fc6c2d632d.o | sed -n 's/^.* [BCDRT] / /p' | sed -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d' | sed 's/[^ ][^ ]*/"&"/g'  >> tmp.def; \
    echo g++  -shared -s -static-libgcc -o file28fc6c2d632d.dll tmp.def file28fc6c2d632d.o  -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64" -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib"  -L"C:/PROGRA~1/R/R-43~1.1/bin/x64" -lR ; \
    g++  -shared -s -static-libgcc -o file28fc6c2d632d.dll tmp.def file28fc6c2d632d.o  -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64" -L"C:/rtools43/x86_64-w64-mingw32.static.posix/lib"  -L"C:/PROGRA~1/R/R-43~1.1/bin/x64" -lR ; \
    rm -f tmp.def; \
  fi \
fi
Error in compileCode(f, code, language = language, verbose = verbose) : 
  C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: file28fc6c2d632d.o:file28fc6c2d632d.cpp:(.text$_ZN3tbb8internal26task_scheduler_observer_v3D0Ev[_ZN3tbb8internal26task_scheduler_observer_v3D0Ev]+0x1c): undefined reference to `tbb::internal::task_scheduler_observer_v3::observe(bool)'C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: file28fc6c2d632d.o:file28fc6c2d632d.cpp:(.text$_ZN3tbb10interface623task_scheduler_observerD1Ev[_ZN3tbb10interface623task_scheduler_observerD1Ev]+0x1c): undefined reference to `tbb::internal::task_scheduler_observer_v3::observe(bool)'C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: file28fc6c2d632d.o:file28fc6c2d632d.cpp:(.text$_ZN3tbb10interface623task_scheduler_observerD1Ev[_ZN3tbb10interface623task_scheduler_observerD1Ev]+0x37): undefined reference to `tbb::internal::task_scheduler_observer_v3::observe(bool)'C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: file28fc6c2d632d.o:file28fc6c2d632d.cpp:(.text$_ZN3tbb10interface
Error in sink(type = "output") : invalid connection

andrjohns · August 14, 2023, 5:21am

You don’t need to run mingw32-make install-tbb, since that’s all handled by cmdstanr.

Your current error is because brms is using the rstan backend, not cmdstanr. If you want to use the rstan backend then you need to install the 2.26 preview:

install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))

joels · August 14, 2023, 6:37am

Doh! I just forgot to add backend="cmdstanr". With that addition, the model fit completed as expected.

I did get a bunch of warnings, some of which I haven’t seen before when fitting models on my local Macbook Pro laptop. I’ve pasted them in below in case they’re something I need to address.

m1 = brm(mpg ~ hp, data=mtcars, cores=4, backend="cmdstanr",
         prior=prior(normal(0,1), class="b"))

Compiling Stan program...
In file included from stan/src/stan/model/model_header.hpp:11:
stan/src/stan/model/model_base_crtp.hpp:198: warning: 'void stan::model::model_base_crtp<M>::write_array(boost::random::ecuyer1988&, std::vector<double, std::allocator<double> >&, std::vector<int>&, std::vector<double, std::allocator<double> >&, bool, bool, std::ostream*) const [with M = model_0daeb6318b9223b37ecbc93c94ba9a4d_model_namespace::model_0daeb6318b9223b37ecbc93c94ba9a4d_model; boost::random::ecuyer1988 = boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014, 0, 2147483563>, boost::random::linear_congruential_engine<unsigned int, 40692, 0, 2147483399> >; std::ostream = std::basic_ostream<char>]' was hidden [-Woverloaded-virtual=]
  198 |   void write_array(boost::ecuyer1988& rng, std::vector<double>& theta,
      | 

C:/Users/***/AppData/Local/Temp/3/RtmpwjGnSt/model-28fc7827639e.hpp:518: note:   by 'model_0daeb6318b9223b37ecbc93c94ba9a4d_model_namespace::model_0daeb6318b9223b37ecbc93c94ba9a4d_model::write_array'
  518 |   write_array(RNG& base_rng, std::vector<double>& params_r, std::vector<int>&
      | 
stan/src/stan/model/model_base_crtp.hpp:136: warning: 'void stan::model::model_base_crtp<M>::write_array(boost::random::ecuyer1988&, Eigen::VectorXd&, Eigen::VectorXd&, bool, bool, std::ostream*) const [with M = model_0daeb6318b9223b37ecbc93c94ba9a4d_model_namespace::model_0daeb6318b9223b37ecbc93c94ba9a4d_model; boost::random::ecuyer1988 = boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014, 0, 2147483563>, boost::random::linear_congruential_engine<unsigned int, 40692, 0, 2147483399> >; Eigen::VectorXd = Eigen::Matrix<double, -1, 1>; std::ostream = std::basic_ostream<char>]' was hidden [-Woverloaded-virtual=]
  136 |   void write_array(boost::ecuyer1988& rng, Eigen::VectorXd& theta,
      | 
C:/Users/***/AppData/Local/Temp/3/RtmpwjGnSt/model-28fc7827639e.hpp:518: note:   by 'model_0daeb6318b9223b37ecbc93c94ba9a4d_model_namespace::model_0daeb6318b9223b37ecbc93c94ba9a4d_model::write_array'
  518 |   write_array(RNG& base_rng, std::vector<double>& params_r, std::vector<int>&
      |

Then during sampling I got about 30 or 40 warnings like this:

Start sampling
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in 'C:/Users/***/AppData/Local/Temp/3/RtmpwjGnSt/model-28fc7827639e.stan', line 36, column 4 to column 62)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.

Mads_Lund_Pedersen · August 14, 2023, 8:24am

Thanks, after downloading cmdstan-2.32.2-linux-arm64.tar.gz instead of cmdstan-2.32.2.tar.gz I was able to build cmdstan without errors. But when trying to run a model I got an error that it cannot execute binary file. Could this be related to limited rights on the cluster? If so, are there ways to solve this?

andrjohns · August 14, 2023, 8:33am

Is your cluster specifically an arm64 cluster? Otherwise you should download the regular cmdstan-2.32.2.tar.gz file

Mads_Lund_Pedersen · August 14, 2023, 9:43am

It looks like the cluster has AMD 64-core processors. When running the regular cmdstan-2.32.2.tar.gz file I got the error I reported previously, where it can’t find cmdstan/command.hpp (even though the file is there). I’m only able to build cmdstan using rebuild_cmdstan with arm64 and mips64el, but when using either of those I get the error that it cannot execute binary file.

andrjohns · August 14, 2023, 9:54am

If you navigate to the unzipped cmdstan-2.32 (regular) in the terminal and run make build, what’s the output?

Mads_Lund_Pedersen · August 14, 2023, 9:59am

andrjohns · August 14, 2023, 10:03am

Can you try redownloading the tar.gz file? Making sure it’s coming from the ‘Releases’ section?

The build shouldn’t be trying to download the nightly stanc if you’re building from a release tarball

Mads_Lund_Pedersen · August 14, 2023, 11:41am

Have done the following:

Downloaded cmdstan-2.32.2.tar.gz (highlighted in screenshot). Imported to the cluster
Extracted using tar -xvzf cmdstan-2.32.2.tar.gz.
Running make build

Leads to the same error - trying to download…

I’m able to run make build for the 64-core versions, but then I get the cannot execute binary file - error when trying to run the model.

Guido_Biele · August 14, 2023, 12:33pm

Assuming you are on colossus (a server in Norway), yes this is something one needs to take up with the local support. I hope you try ! :-)

andrjohns · August 14, 2023, 12:43pm

Can you post the output from the following terminal commands (again in the cmdstan folder):

ls bin
ls stan
ls stan/lib/stan_math

Mads_Lund_Pedersen · August 14, 2023, 1:26pm

the stan folder is empty for the regular file (cmdstan-2.32.2.tar.gz), but returns the following for the other tar.gz versions (arm64, but also armel).

Topic		Replies	Views
Installing rstan or cmdstanr from source Interfaces	3	34	February 7, 2025
Error in cmdstanr after installing cmdstanr CmdStan cmdstan , rstan , cmdstanr	7	759	August 4, 2023
Cmdstanr can't find makefile after installation...suspect network path problem General installation	3	1495	April 2, 2024
Cmdstanr installation error CmdStan installation , windows , cmdstanr	2	905	September 10, 2023
Problems installing cmdstan on Windows 11 General cmdstan , cmdstanr	2	743	April 25, 2023

How to install cmdstanr on server without Internet?

Related topics