Brms/cmdstanr looking for things in invalid directory

The location of cmdstanr should not matter.

We can turn on more debug info with:
options("cmdstanr_verbose" = TRUE)

and then run this again:

set_cmdstan_path("C:/cmdstan-2.26.1") 
file <- file.path(cmdstan_path(), " examples", "bernoulli", "bernoulli.stan")
mod <- cmdstan_model(file, quiet = FALSE, force_recompile = TRUE)

One other options is to add CMDSTAN location to a env. variable:

write('CMDSTAN="C:\\cmdstan-2.26.1"', file = "~/.Renviron", append = TRUE)

and then restart the R session.

I am a bit baffled why R insist on picking up the C:\Users\Folio.
If this still fails I would try installing with install_cmdstan() again.

> options("cmdstanr_verbose" = TRUE)
> set_cmdstan_path("C:/cmdstan-2.26.1") 
CmdStan path set to: C:/cmdstan-2.26.1
> file <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.stan")
> mod <- cmdstan_model(file, quiet = FALSE, force_recompile = TRUE)
Compiling Stan program...
Running mingw32-make.exe "C:/Rtemp/RtmpqgzyCr/model-1f88587c342.exe" \
  "STANCFLAGS += --name='bernoulli_model'"
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
Error: An error occured during compilation! See the message above for more information.

Editing .Renviron:

write('CMDSTAN="C:\\cmdstan-2.26.1"', file = "~/.Renviron", append = TRUE)

Restarting. Then:

> Sys.getenv("TMP")
[1] "C:/Rtemp"
> Sys.getenv("TEMP")
[1] "C:/Rtemp"
> require(brms)
Loading required package: brms
Loading required package: Rcpp
Loading 'brms' package (version 2.14.4). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar

> require(cmdstanr) #This has been previously installed as per the instructions at https://mc-stan.org/cmdstanr/articles/cmdstanr.html
Loading required package: cmdstanr
This is cmdstanr version 0.3.0
- Online documentation and vignettes at mc-stan.org/cmdstanr
- CmdStan path set to: C:/cmdstan-2.26.1
- Use set_cmdstan_path() to change the path

^ Note that it now reports the correct directory without me having had to call set_cmdstan_path(). Then:

> time0 <- Sys.time();testmod <- brm(dep.var ~ (1|trigger) + (1|CompLemma) + Word.Dist + time.period + polarity, prior = wdistpriors.both, family = categorical, data = d,
+                                   decomp = "QR", backend = "cmdstanr");Sys.time()-time0
Compiling Stan program...
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
Error: An error occured during compilation! See the message above for more information.
In addition: Warning messages:
1: Specifying global priors for regression coefficients in categorical models is deprecated. Please specify priors separately for each response category. 
2: Specifying global priors for regression coefficients in categorical models is deprecated. Please specify priors separately for each response category.

I guess the next thing to do is to uninstall both cmdstan and cmdstanr, then reinstall both of them from within RStudio. I’ll report back once I’ve done that.

Alright, here’s what happened when I did as instructed and tried installing cmdstan with the install_cmdstan() function of the cmdstanr library:

> install_cmdstan()
The CmdStan toolchain is setup properly!
* Latest CmdStan release is v2.26.1
* Installing CmdStan v2.26.1 in C:\Users\Folio 1040\Documents/.cmdstanr/cmdstan-2.26.1
* Downloading cmdstan-2.26.1.tar.gz from GitHub...
* Download complete
* Unpacking archive...
* Building CmdStan binaries...
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.

Warning message:
There was a problem during installation. See the error message(s) above. 

And that’s all she wrote.

It really doesn’t seem to like the space. Or perhaps it’s the mix of forward and backslashes in the path that’s throwing it off.

Has there been any solution to this ‘spaces in paths’ problem? I’ve discovered that the problematic directory name “Folio 1040” is actually the name of this laptop model, i.e. the directory name is a factory setting.

It will soon be time for me to fit (at a minimum) dozens or (preferably) hundreds of complex multilevel categorical models for my final report, so I need every possible speedup available. That means I must have recourse to all of that QR decomposition and within-chain parallelization stuff. But AFAIK neither of those options is available without a working install of cmdstan.

Try installing the latest version of cmdstanr, the issue should have been resolved.

1 Like

Many thanks for the prompt response! I did as suggested and installed what I assume to be the latest version, using the following command:

install.packages("cmdstanr", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))

This worked without errors. Then I loaded brms and cmdstanr:

Loading required package: brms
Loading required package: Rcpp
Loading 'brms' package (version 2.14.11). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar

> require(cmdstanr)
Loading required package: cmdstanr
This is cmdstanr version 0.4.0
- Online documentation and vignettes at mc-stan.org/cmdstanr
- CmdStan path set to: C:/Users/Folio 1040/Documents/.cmdstanr/cmdstan-2.26.1
- Use set_cmdstan_path() to change the path

And then I tried to fit my first model:

testmod <- brm(dep.var ~ (1|CompLemma) + ThatOmit + Word.Dist + time.period + polarity, prior = huoh, family = categorical, data = d, decomp = "QR", backend = "cmdstanr")

Compiling Stan program…
C:\Users\Folio:1: *** target pattern contains no ‘%’. Stop.
Error: An error occured during compilation! See the message above for more information.

Here’s the gist of sessionInfo()

R version 4.0.5 (2021-03-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

I can’t replicate your issue on windows 10, I made a Folio 1040 folder as a test and all works fine:

> set_cmdstan_path(".cmdstanr/Folio 1040/cmdstan-2.26.1/")
CmdStan path set to: C:/Users/Andrew Johnson/Documents/.cmdstanr/Folio 1040/cmdstan-2.26.1
> library(brms)
Loading required package: Rcpp
Loading 'brms' package (version 2.15.0). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar

> fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient),
+             data = epilepsy, family = poisson(), backend = "cmdstanr")
Compiling Stan program...
Start sampling
Running MCMC with 4 sequential chains...

I don’t think it will make a difference, but can you try updating your brms version to the latest CRAN version? Should be 2.15.

Thanks Andrew. I’ve now updated brms to version 2.15 and the error persists.

Btw, I presently have cmdstan installed in two directories. Mine and @rok_cesnovar’s earlier attempt to sidestep this problem was by installing cmdstan to ‘C:/cmdstan-2.26.1’. For some reason though, even if I call ````set_cmdstan_path(“C:/cmdstan-2.26.1”)``` before running brms in order to try and force R to look for cmdstan in a directory with no spaces in its name, the error message is still the same. In other words, R for some reason insists upon seeking cmdstan in the path with the space even if it’s explicitly told to seek it elsewhere.

Hmm, bit odd. Can you post the output from:

Sys.getenv("PATH")
1 Like

“C:\rtools40\usr\bin;C:\rtools40\mingw64\bin;C:\rtools40\usr\bin;C:\Program Files\R\R-4.0.5\bin\x64;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Rtools40\mingw64\bin\;C:\Rtools40\usr\bin\;C:\cmdstan-2.26.1\stan\lib\stan_math\lib\tbb\;C:rtools40mingw64bin;C:cmdstan-2.26.1stanlibstan_mathlibtbb;C:UsersFolio 1040AppDataLocalMicrosoftWindowsApps;;”

Ah ha! This looks like the culprit:

C:rtools40mingw64bin;C:cmdstan-2.26.1stanlibstan_mathlibtbb;C:UsersFolio 1040AppDataLocalMicrosoftWindowsApps;;"

Your PATH variable is all messed up, so when R/cmdstan tries to use it to find the compiler it throws an error. First I’ll get you to check whether any of this is getting set in R. Can you post the output from:

readLines("~/.Renviron")
1 Like
> readLines("~/.Renviron")
[1] "PATH=\"${RTOOLS40_HOME}\\usr\\bin;${PATH}\""                               
[2] "PATH=\"${RTOOLS40_HOME}\\usr\\bin;${RTOOLS40_HOME}\\mingw64\\bin;${PATH}\""
[3] "TMP=\"C:/Rtemp\""                                                          
[4] "TEMP=\"C:/Rtemp\""                                                         
[5] "TMP=\"C:/Rtemp\""                                                          
[6] "TEMP=\"C:/Rtemp\""        

Alright, so that means that the issue is (most likely) not coming from R. The next thing to check is your environment variables in Windows

If you go to the start menu and type ‘environment’, you should see the option:
image

Open that, and select the ‘Environment Variables’ button in the lower-right corner. You should see a window like the following:

There will be a ‘Path’ option under both the ‘User variables’ and ‘System variables’ headings. Can you open those up (double-click or hit ‘Edit’) and take a screenshot of each?

1 Like

Great! Can you edit that single User Environment variable, and delete everything except for: %USERPROFILE%\AppData\Local\Microsoft\WindowsApps.

Then restart R and see if you get a different error with cmdstanr/brms?

1 Like

Also, if it fails again can you post the result of Sys.getenv("PATH"), to see if that’s still a mess

1 Like
> require(brms)
Loading required package: brms
Loading required package: Rcpp
Loading 'brms' package (version 2.15.0). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar

> require(cmdstanr)
Loading required package: cmdstanr
This is cmdstanr version 0.4.0
- Online documentation and vignettes at mc-stan.org/cmdstanr
- CmdStan path set to: C:/Users/Folio 1040/Documents/.cmdstanr/cmdstan-2.26.1
- Use set_cmdstan_path() to change the path

> testmod <- brm(dep.var ~ (1|CompLemma) + ThatOmit + Word.Dist + time.period + polarity, prior = huoh, family = categorical, data = d, decomp = "QR", backend = "cmdstanr")

Compiling Stan program...
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
Error: An error occured during compilation! See the message above for more information.
> Sys.getenv("PATH")

[1] "C:/Users/Folio 1040/Documents/.cmdstanr/cmdstan-2.26.1/stan/lib/stan_math/lib/tbb;C:\\rtools40\\usr\\bin;C:\\rtools40\\mingw64\\bin;C:\\rtools40\\usr\\bin;C:\\Program Files\\R\\R-4.0.5\\bin\\x64;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\Program Files (x86)\\Intel\\iCLS Client\\;C:\\Program Files\\Intel\\iCLS Client\\;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Rtools40\\mingw64\\bin\\;C:\\Rtools40\\usr\\bin\\;C:\\cmdstan-2.26.1\\stan\\lib\\stan_math\\lib\\tbb\\;C:\\Users\\Folio 1040\\AppData\\Local\\Microsoft\\WindowsApps;"

Oof, more problems! I think I have another idea (sorry this is taking so long!).

I’m wondering whether the issue is that the path to the TBB is getting added to your PATH variable with forward-slashes instead of backslashes. I’ve made a fork of cmdstanr where this is patched. Can you try restarting R, then installing this patched version:

remotes::install_github("andrjohns/cmdstanr")

Then trying brms again? Also if you can post Sys.getenv("PATH") afterwards as well, to see if the fixes did anything, that would also be great

2 Likes

@andrjohns Hey, no problem. I really appreciate your patience with this. Here’s the latest, after reinstalling cmdstanr using your command:

> require(brms)
Loading required package: brms
Loading required package: Rcpp
Loading 'brms' package (version 2.15.0). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar

> require(cmdstanr)
Loading required package: cmdstanr
This is cmdstanr version 0.4.0.9000
- Online documentation and vignettes at mc-stan.org/cmdstanr
- CmdStan path set to: C:/Users/Folio 1040/Documents/.cmdstanr/cmdstan-2.26.1
- Use set_cmdstan_path() to change the path

> testmod <- brm(dep.var ~ (1|CompLemma) + ThatOmit + Word.Dist + time.period + polarity, prior = huoh, family = categorical, data = d, decomp = "QR", backend = "cmdstanr")
Compiling Stan program...
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
Error: An error occured during compilation! See the message above for more information.
> Sys.getenv("PATH")
[1] "C:\\Users\\Folio 1040\\Documents\\.cmdstanr\\cmdstan-2.26.1\\stan\\lib\\stan_math\\lib\\tbb;C:\\rtools40\\usr\\bin;C:\\rtools40\\mingw64\\bin;C:\\rtools40\\usr\\bin;C:\\Program Files\\R\\R-4.0.5\\bin\\x64;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\Program Files (x86)\\Intel\\iCLS Client\\;C:\\Program Files\\Intel\\iCLS Client\\;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Rtools40\\mingw64\\bin\\;C:\\Rtools40\\usr\\bin\\;C:\\cmdstan-2.26.1\\stan\\lib\\stan_math\\lib\\tbb\\;C:\\Users\\Folio 1040\\AppData\\Local\\Microsoft\\WindowsApps;"

Damn, I had high hopes for that one. Alright, the next step is track down exactly where in the cmdstanr process things are breaking. I’ve added some basic logging to my cmdstanr fork which will print out each of the compilation steps it’s about to complete. This way we can see which step is giving the error.

Can you reinstall my fork:

remotes::install_github("andrjohns/cmdstanr")

Then restart R and try your model again. Can you copy the output? It should have printed a series of steps like so (although yours will be interrupted by an error!):

> fit1 <- brm(count ~ zAge + zBase * Trt + (1|patient),
+             data = epilepsy, family = poisson(), backend = "cmdstanr")
[1] "Compile model is TRUE"
Compiling Stan program...
[1] "Copy Stan model to tempdir"
[1] "Add TBB to PATH"
[1] "Build Stanc flags"
[1] "Collate Stanc flags"
[1] "Collapse Stanc flags to single string"
[1] "Compile stan model, arguments are:"
[1] "command = mingw32-make.exe"
[1] "args = C:\\Users\\he184239\\AppData\\Local\\Temp\\RtmpAHZN1B\\model-1cdc4c594a4d.exe"
[2] "args = STANCFLAGS += --name='file1cdc4b662fc3_model'"                                
[1] "wd = C:/Users/he184239/Desktop/WD/.cmdstanr/cmdstan-2.26.1"
[1] "echo = FALSE"
[1] "echo_cmd = FALSE"
[1] "spinner = TRUE"
[1] "Copy executable from tempdir"
1 Like