Cmdstanr installation error: Command not found @win/processx.c:977

Guido_Biele · November 18, 2019, 5:08pm

Operating System: Windows 10
Interface Version: cmdstanr 0.0.0.9000
Compiler/Toolkit: Rtools

Hi,
I am trying to install cmdstan on windows via cmdstanr (using install_cmdstan()) and getting the following error:

Error in rethrow_call(c_processx_exec, command, c(command, args), stdin,  : 
  Command not found @win/processx.c:977

A quick internet search suggests that this could be due to the inability to find Rtools. I have Rtools installed and can compile model with Rstan.

Is this a problem I am causing, or is this a problem with the install_cmdstan() and associated files?

Thanks in advance - Guido

PS: cmdstanr, in particular the much shortened compilation time is awsome. I could install it without problems on my personal mac, but I needed on my windows machine at work.

rok_cesnovar · November 18, 2019, 4:02pm

Hi @Guido_Biele,

the problem here means that it cant find the command it is trying to run. In install_cmdstan() the only command we run is bash. We use bash to run the install script.

Unfortunately bash is not natively found on Windows, but it is part of Git for Windows for instance. This means that in order for install cmdstan to work on Windows you need to install Git for Windows (https://git-scm.com/download/win).

After installing please open the command line and try to run
bash --version
and
mingw32-make --version

If both run succesfully you are all set. The latter one should work for you already as that one comes with RTools. If not, then you will probably need to add C:\Program Files (x86)\Git\bin to the PATH variable. If you need helping doing that please let me know.

Thanks for discovering this! Everyone that tried cmdstanr on Windows, including myself, probably had either Git installed and in their PATH or some other version of bash from one of the other sources.

I made an issue to remove this dependency (https://github.com/stan-dev/cmdstanr/issues/88)

Thanks again for reporting!

Guido_Biele · November 18, 2019, 4:37pm

Thanks for your speedy response, @rok_cesnovar.

I had/have git installed, but there is still something amiss.

With

I think you mean to open the Windows command line, or do you mean to open the bash that comes with git?

Either way,

when I open the windows command line, neither the bash nor mingw32-make command is recognized (but the git command is, which tells me git is on the path)
when I open the bash that come line (from RStudio) the bash command is (naturally) recognized, as is the git command, but the mingw32-makeis not recognized.

Does this tell you anything?

Guido

rok_cesnovar · November 29, 2019, 2:03pm

Yes, I meant the Windows command line.

That seems like you need to set the PATH variable so that Windows can find both bash.exe and mingw32-make.exe.

You need to add both C:\Program Files (x86)\Git\bin and C:\RTools\mingw_64\bin. Maybe the later one is C:\Rtools\mingw_62\bin not sure right now.

If you have never done that on Windows follow these instructions (source):

In Search, search for and then select: System (Control Panel)
Click the Advanced system settings link.
Click Environment Variables . In the section System Variables , find the PATH environment variable and select it. Click Edit . If the PATH environment variable does not exist, click New .
In the Edit System Variable (or New System Variable ) window, specify the value of the PATH environment variable. Click OK . Close all remaining windows by clicking OK .
Reopen Command prompt window, and run the 2 commands.

Please note that you need to close any open Windows command lines and open new one that will have the freshly set PATH.

Guido_Biele · November 19, 2019, 8:52am

Thanks,
now I could install cmdstanr.

Unfortuantely, I am still having the problem that on the system I am working I am not not allowed to execute the executables that were generated (stanc.exe, diagnose.exe, stansummary.exe, print.exe). I have contacted my local IT support to solve this problem locally.

I know nothing about installation or rights management on windows. But, would it be possible to install cmdstanr in a way its executables are not automatically blocked by a restrictive rights management?

rok_cesnovar · November 19, 2019, 9:08am

Huh, I am recovering Windows user myself and I am afraid I also dont have much experience with permissions and all that.

One thing you could currently do is move the cmdstan installed files to a different location.

Its currently installed in C:/Users/username/.cmdstanr/cmdstan but you can move it to your Destkop or Documents folder and then call set_cmdstan_path() to point to the new location.

Hopefully that helps.

jonah · November 20, 2019, 5:40am

To follow up on this, we now have a branch of cmdstanr and PR that doesn’t require bash if anyone wants to try it out:

Guido_Biele · November 20, 2019, 9:51am

Copying the files to a different directory does not solve the problem.

rok_cesnovar · November 20, 2019, 10:00am

Then its probably an issue with permissions in the created folder. Please check the permission on the cmdstan/bin folder and try to add permission for execution.

I am not sure if cmdstanr can do anything to avoid this issue. Will investigate.

Guido_Biele · November 21, 2019, 8:11am

Yes, I knew this was a permission problem.
The thing is that at the place were I work users do not have rights to change permissions*.
In fact, I just got feedback from the IT-support they don’t want to give me permissions to use these .exe files, which means that I won’t be able to use cmdstanr if I can’t convince them otherwise.

I am not sure how common it is to have such a restrictive IT setup as at my working place. But If this is not so uncommon, it might worth it to see if there is a way to install cmdstanr more like a typical R package (i.e. without that .exe files are generated during installation.)

IT rights are generally very restrictive due to the type of data we are working with.

rok_cesnovar · November 21, 2019, 8:49am

Does that mean that you are not able to run any built exe files whatsoever? I would be a bit surprised that is the actual case.

You can test if we can get cmdstan working on your setup by:

downloading the latest release of cmdstan here https://github.com/stan-dev/cmdstan/releases/download/v2.21.0/cmdstan-2.21.0.tar.gz
unzip the file into your document folder or desktop folder or something like that and run mingw32-make build in the unzipped folder
run mingw32-make examples/bernoulli/bernoulli.exe at the same location
move to the examples/bernoulli folder and run bernoulli.exe with bernoulli.exe help

If you can get that to run then it was just a problem that cmdstan was initially installed to the C:/Users/username folder and the permissions set there were too restrictive. You can then use cmdstanr by simply pointing to this new location with set_cmdstan_path().

If you are also unable to run this bernoulli.exe then I am afraid that you will not be able to use cmdstanr on this system. Cmdstanr is envisioned as a lightweight wrapper for cmdstan which is essentially these exe files.

Guido_Biele · November 29, 2019, 2:03pm

My IT support finally informed me that I am allowed to run .exe files from C:/Rtools.

I assume that will be similar on other systems with very restrictive rights management. So maybe one could add the Rtools folder as an optional target folder for the installation of cmndstan through cmdstanr? (or just point to this possibility in the documentation)

alex_pike · July 30, 2020, 3:59pm

If anyone (after the new release of cmdstanr) is getting errors like the above, ` Error in rethrow_call(c_processx_exec, command, c(command, args), stdin, : Command not found @win/processx.c:977', and followed the information below here, I can recommend also adding Rtools\bin to your path, rather than just Rtools/mingw_64\bin otherwise you run into errors with ‘stanc’ (namely, make/stanc:61: recipe for target 'bin/stanc.exe' failed). Spent a while playing with this today and hoped it might help someone!

jonah · July 30, 2020, 4:32pm

Thanks @alex_pike, sorry you had to spend time figuring that out.

@rok_cesnovar Is there a way we can avoid people having to do this? Is this at all related to this issue?

rok_cesnovar · July 30, 2020, 4:48pm

Yes, with that we could help most users on Windows.

We would guess the paths based on default locations.

MatsuuraKentaro · July 30, 2020, 11:00pm

I’m getting a similar error on Windows 10, R-4.0.2, and rtools40:

> library(cmdstanr)
This is cmdstanr version 0.1.0
- Online documentation and vignettes at mc-stan.org/cmdstanr
- CmdStan path set to: C:/Users/kmatsuu/home/.cmdstanr/cmdstan-2.24.0
- Use set_cmdstan_path() to change the path
> cmdstan_path()
[1] "C:/Users/kmatsuu/home/.cmdstanr/cmdstan-2.24.0"
> file <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.stan")
> mod <- cmdstan_model(file)
Compiling Stan program...
 rethrow_call(c_processx_exec, command, c(command, args), stdin,  でエラー: 
  Command 'mingw32-make.exe' not found @win/processx.c:994 (processx_exec)
Type .Last.error.trace to see where the error occured
>

I have already added C:\rtools40\mingw64\bin to PATH. But I haven’t added C:\rtools40\bin to PATH because the folder doesn’t exist. Also, the file mingw32-make.exe does not exist in C:\rtools40\mingw64\bin and C:\rtools40\usr\bin (see attachment). In addition, the BINPREF in C:\Program Files\R\R-4.0.2\etc\x64\Makeconf seems to be set correctly as well. What should I do?

MatsuuraKentaro · July 31, 2020, 5:07am

I double-clicked on C:\rtools40\usr\bin\bash.exe, then I typed the following to install something:

pacman -Sy mingw-w64-x86_64-make

After that, I can successfully run cmdstan_model. I referred to:

CLRR · September 1, 2020, 3:05am

I faced the exactly the same situation to @MatsuuraKentaro (i.e. Command 'mingw32-make.exe' not found and mingw32-make.exe did not exist anywhere). Although I tried running pacman -Sy mingw-w64-x86_64-make on bash, bash only threw an error: bash: pacman: command not found…

When I executed the command on Rtools bash (msys2.exe), pacman always failed to install mingw32-make, too.

Is there any way to install mingw32-makewithout using pacman? Any kind of suggestions are welcomed.

rok_cesnovar · September 1, 2020, 6:17am

Hi,

sorry fot the troubles. Given that you have RTools 4.0 installed, run:

processx::run(
    "pacman",
    args = c("-Syu", "mingw-w64-x86_64-make","--noconfirm"),
    wd = file.path(Sys.getenv("RTOOLS40_HOME"), "usr", "bin"),
    echo = TRUE
)
write('PATH="${RTOOLS40_HOME}\\usr\\bin;${RTOOLS40_HOME}\\mingw64\\bin;${PATH}"', file = "~/.Renviron", append = TRUE)

and then restart the R session.

this will be handled automatically in the next version of the beta release of cmdstanr (after https://github.com/stan-dev/cmdstanr/pull/289 is merged).

CLRR · September 1, 2020, 8:39am

@rok_cesnovar

Thank you for your help but processx also failed to install mingw32-make…

The first command threw an error:

processx::run(
  "pacman", 
  args = c("-Syu", "mingw-w64-x86_64-make","--noconfirm"), 
  wd = file.path(Sys.getenv("RTOOLS40_HOME"),  "usr", "bin"), 
  echo = TRUE)
:: Synchronizing package databases...
Error in processx::run("pacman", args = c("-Syu", "mingw-w64-x86_64-make",  : 
  System command 'pacman' failed, exit status: -2147483647, stdout & stderr were printed
Type .Last.error.trace to see where the error occured

The trace said:

Stack trace:

 1. processx::run("pacman", args = c("-Syu", "mingw-w64-x86_64-make",  ...
 2. throw(new_process_error(res, call = sys.call(), echo = echo,  ...

 x System command 'pacman' failed, exit status: -2147483647, stdout & stderr were printed