Brms/cmdstanr looking for things in invalid directory

> testmod <- brm(dep.var ~ (1|CompLemma) + ThatOmit + Word.Dist + time.period + polarity, prior = huoh, family = categorical, data = d, decomp = "QR", backend = cmdstanr")
[1] "Compile model is TRUE"
Compiling Stan program...
[1] "Copy Stan model to tempdir"
[1] "Add TBB to PATH"
[1] "Build Stanc flags"
[1] "Collate Stanc flags"
[1] "Collapse Stanc flags to single string"
[1] "Compile stan model, arguments are:"
[1] "command = mingw32-make.exe"
[1] "args = C:\\Rtemp\\Rtmp8qsxUO\\model-d4c4a05238f.exe" "args = STANCFLAGS += --name='filed4c6523100f_model'"
[1] "wd = C:/Users/Folio 1040/Documents/.cmdstanr/cmdstan-2.26.1"
[1] "echo = FALSE"
[1] "echo_cmd = FALSE"
[1] "spinner = TRUE"
Running mingw32-make.exe "C:\Rtemp\Rtmp8qsxUO\model-d4c4a05238f.exe" \
  "STANCFLAGS += --name='filed4c6523100f_model'"
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
Error: An error occured during compilation! See the message above for more information.

Interesting, so it gets to the mingw32-make call and then errors. Let’s try compiling a model through cmdstan directly, to see if the same error pops up.

You’ll need to change your working directory to the cmdstan install:

# If it's installed in your home directory
setwd("~/.cmdstanr/cmdstan-2.26.1/")

Then delete the compiled code for the example model (gets automatically built when installing cmdstanr):

system("rm examples/bernoulli/bernoulli.hpp")
system("rm examples/bernoulli/bernoulli.exe")

Then try to compile the example model:

system("mingw32-make examples/bernoulli/bernoulli.exe",
       show.output.on.console = TRUE)

Thanks @andrjohns for debugging this! You are awesome!

I have a hunch it may be the wd in the run command (maybe processx does not handle spaces the way we thought it does).

wd <- cmdstan_path()
if(os_is_windows()) {
  wd <- utils::shortPathName(wd)
}

and then change this line to use the wd: cmdstanr/model.R at master · stan-dev/cmdstanr · GitHub

Can you add that to your debug branch? Thanks!

2 Likes

Oh good call. @blokeman I’ve updated my fork with Rok’s suggestion, can you try reinstalling from github and trying again?

1 Like
> require(brms)
Loading required package: brms
Loading required package: Rcpp
Loading 'brms' package (version 2.15.0). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar

> require(cmdstanr)
Loading required package: cmdstanr
This is cmdstanr version 0.4.0.9000
- Online documentation and vignettes at mc-stan.org/cmdstanr
- CmdStan path set to: C:/Users/Folio 1040/Documents/.cmdstanr/cmdstan-2.26.1
- Use set_cmdstan_path() to change the path
> setwd("C:/Users/Folio 1040/Documents/.cmdstanr/cmdstan-2.26.1")
> system("rm examples/bernoulli/bernoulli.hpp")
/usr/bin/rm: cannot remove 'examples/bernoulli/bernoulli.hpp': No such file or directory
[1] 1
> system("rm examples/bernoulli/bernoulli.exe")
/usr/bin/rm: cannot remove 'examples/bernoulli/bernoulli.exe': No such file or directory
[1] 1
> system("mingw32-make examples/bernoulli/bernoulli.exe",
+        show.output.on.console = TRUE)
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
[1] 2

As mentioned in an earlier post, I also have cmdstanr installed at C:\cmdstan-2.26.1. Setting that as the working directory and trying to run the same command produces the same error.

Another thing: when installing cmdstan (or cmdstanr, can’t remember which) 2 months ago, I do remember successfully running the exemplificatory Bernoulli model from Command Prompt. I’ve yet to successfully fit cmdstanr models from RStudio though.

Can you try running the brms model as normal? (After updating from my github)

1 Like
> remotes::install_github("andrjohns/cmdstanr")
Skipping install of 'cmdstanr' from a github remote, the SHA1 (d4c72fd8) has not changed since last install.
  Use `force = TRUE` to force installation
> require(brms)
Loading required package: brms
Loading required package: Rcpp
Loading 'brms' package (version 2.15.0). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar
	
> require(cmdstanr)
Loading required package: cmdstanr
This is cmdstanr version 0.4.0.9000
- Online documentation and vignettes at mc-stan.org/cmdstanr
- CmdStan path set to: C:/Users/Folio 1040/Documents/.cmdstanr/cmdstan-2.26.1
- Use set_cmdstan_path() to change the path

> testmod <- brm(dep.var ~ (1|CompLemma) + ThatOmit + Word.Dist + time.period + polarity, prior = huoh, family = categorical, data = d, decomp = "QR", backend = "cmdstanr")

[1] "Compile model is TRUE"
Compiling Stan program...
[1] "Copy Stan model to tempdir"
[1] "Add TBB to PATH"
[1] "Build Stanc flags"
[1] "Collate Stanc flags"
[1] "Collapse Stanc flags to single string"
[1] "Compile stan model, arguments are:"
[1] "command = mingw32-make.exe"
[1] "args = C:\\Rtemp\\RtmpE5PFNm\\model-2408110575bc.exe" "args = STANCFLAGS += --name='file240871b673af_model'"
[1] "wd = C:\\Users\\FOLIO1~1\\DOCUME~1\\CMDSTA~1\\CMDSTA~1.1"
[1] "echo = FALSE"
[1] "echo_cmd = FALSE"
[1] "spinner = TRUE"
Running mingw32-make.exe "C:\Rtemp\RtmpE5PFNm\model-2408110575bc.exe" \
  "STANCFLAGS += --name='file240871b673af_model'"
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.

Thanks for being so patient with us!

Can you run:

processx::run("Get-Command", args = "mingw32-make")
2 Likes
> processx::run("Get-Command", args = "mingw32-make")
Error in rethrow_call(c_processx_exec, command, c(command, args), pty,  : 
  Command 'Get-Command' not found @win/processx.c:982 (processx_exec)
Type .Last.error.trace to see where the error occured
> .Last.error.trace

 Stack trace:

 1. processx::run("Get-Command", args = "mingw32-make")
 2. process$new(command, args, echo_cmd = echo_cmd, wd = wd, windows_verbatim_args = windows_verbatim_args,  ...
 3. processx:::initialize(...)
 4. processx:::process_initialize(self, private, command, args, stdin,  ...
 5. rethrow_call(c_processx_exec, command, c(command, args), pty,  ...

 x Command 'Get-Command' not found @win/processx.c:982 (processx_exec) 

Hmm, that seems weird. Get-Command is part of Windows.

how about

processx::run("which", args = "mingw32-make")
1 Like
> processx::run("which", args = "mingw32-make")
$status
[1] 0

$stdout
[1] "/mingw64/bin/mingw32-make\n"

$stderr
[1] ""

$timeout
[1] FALSE

EDIT: There seems to be no file named ‘get-command’ anywhere on this machine:

What happens if you navigate to the cmdstan directory in command prompt and run:

mingw32-make examples/bernoulli/bernoulli.exe
1 Like

There were several screenfuls of output that I cannot make heads or tails of. It’s too much to paste without exceeding the character limit, so I’m attaching the paste as a textfile: output.txt (57.0 KB)

On another note, when brms fails, the following line (outputted by your debug version of cmdstanr) caught my attention:

"wd = C:\\Users\\FOLIO1~1\\DOCUME~1\\CMDSTA~1\\CMDSTA~1.1"

This is a truncated “DOS-style” path harking back to days of yore when no file or directory name could be more than six characters long. I wonder if it could be messing things up.

That output means that everything compiled and worked without issue. So the problem is definitely related to how R is calling the compiler.

The dos-style path was one of the changes added, to get rid of spaces in the path and have appropriate backslashes.

This is all incredibly strange and nothing I’ve remotely seen before.

Lets try some basic c++ compilation, what do you get from:

system("touch foo.cpp")
system("R CMD SHLIB foo.cpp")

Also, have you tried reinstalling RTools? Bit of a longshot though

1 Like
> system("touch foo.cpp")
[1] 0
> system("R CMD SHLIB foo.cpp")
"C:/rtools40/mingw64/bin/"g++ -std=gnu++11  -I"C:/PROGRA~1/R/R-40~1.5/include" -DNDEBUG          -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -c foo.cpp -o foo.o
C:/rtools40/mingw64/bin/g++ -std=gnu++11 -shared -s -static-libgcc -o foo.dll tmp.def foo.o -LC:/PROGRA~1/R/R-40~1.5/bin/x64 -lR
[1] 0

As is probably evident from this output, I have RTools version 4.0 installed. I seem to recall that brms required it.

EDIT: Oh sorry, I missed the question. No I haven’t tried reinstalling RTools. Worth a try. Let me do so now and report back.

I’ve now reinstalled RTools4.0 in the default directory, i.e. c:\rtools40. The resultant error message differs from the previous one:

[1] "Compile model is TRUE"
Compiling Stan program...
[1] "Copy Stan model to tempdir"
[1] "Add TBB to PATH"
[1] "Build Stanc flags"
[1] "Collate Stanc flags"
[1] "Collapse Stanc flags to single string"
[1] "Compile stan model, arguments are:"
[1] "command = mingw32-make.exe"
[1] "args = C:\\Rtemp\\Rtmp8sbxgs\\model-270470ab77c1.exe" "args = STANCFLAGS += --name='file270469ef2b65_model'"
[1] "wd = C:\\Users\\FOLIO1~1\\DOCUME~1\\CMDSTA~1\\CMDSTA~1.1"
[1] "echo = FALSE"
[1] "echo_cmd = FALSE"
[1] "spinner = TRUE"
Running mingw32-make.exe "C:\Rtemp\Rtmp8sbxgs\model-270470ab77c1.exe" \
  "STANCFLAGS += --name='file270469ef2b65_model'"
Error in rethrow_call(c_processx_exec, command, c(command, args), pty,  : 
  Command 'mingw32-make.exe' not found @win/processx.c:982 (processx_exec)
Type .Last.error.trace to see where the error occured

I remember something about having to manually download those mingw executables into a specific directory in order to make brms work the last time I installed rtools, but unfortunately the details are hazy.

Sorry I should have included that, you need get mingw32-make installed again and cmdstan rebuilt. You can do this using cmdstanr via:

library(cmdstanr)

check_cmdstan_toolchain(fix=TRUE)
rebuild_cmdstan()
1 Like

Thanks! Here’s what happens:

> check_cmdstan_toolchain(fix=TRUE)
Installing mingw32-make and writing RTools path to ~/.Renviron ...
The C++ toolchain required for CmdStan is setup properly!

> rebuild_cmdstan()
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.
C:\Users\Folio:1: *** target pattern contains no '%'.  Stop.

I’m at a bit of a loss here.

I don’t think this is due to spaces or naming, I created a User Account under the name ‘Folio 1040’, and had no issues with cmdstanr:

> library(cmdstanr)
This is cmdstanr version 0.4.0
- Online documentation and vignettes at mc-stan.org/cmdstanr
- CmdStan path set to: C:/Users/Folio 1040/Documents/.cmdstanr/cmdstan-2.26.1
- Use set_cmdstan_path() to change the path
> library(brms)
Loading required package: Rcpp
Loading 'brms' package (version 2.15.0). Useful instructions
can be found by typing help('brms'). A more detailed introduction
to the package is available through vignette('brms_overview').

Attaching package: ‘brms’

The following object is masked from ‘package:stats’:

    ar

> brm(count ~ zAge + zBase * Trt + (1|patient),
+     data = epilepsy, family = poisson(), backend = "cmdstanr",
+     refresh=0)
Compiling Stan program...
Start sampling
Running MCMC with 4 sequential chains...

Chain 1 finished in 11.7 seconds.
Chain 2 finished in 9.4 seconds.
Chain 3 finished in 8.7 seconds.
Chain 4 finished in 8.4 seconds.

Does it error even when just calling mingw32-make?

system("mingw32-make --version")
1 Like