Cmdstan and cmdstanr not handling include files the same way

Interfaces CmdStan
,
1

Hi,

I’m using cmdstanr 0.2.2 and CmdStan 2.25.0 on a Linux machine. I’m using two simple model files, one including the other.

test.stan (81 Bytes) test_functions.stan (25 Bytes)

I tried to compile the model in RStudio and got the below errors. I tried to use include_paths with no success: it just gives me a stanc error message about needing to “specify one model_file”.

> cmdstan_model("test.stan")
Compiling Stan program...
/
Syntax error in '/tmp/Rtmpcw36FD/model-34e0d7a3db5de.stan', line 2, column 0, include error:
   -------------------------------------------------
     1:  functions {
     2:  #include test_functions.stan
         ^
     3:    
     4:    real foo() {
   -------------------------------------------------

Could not find include file test_functions.stan in specified include paths.


make: *** [make/program:53: /tmp/Rtmpcw36FD/model-34e0d7a3db5de.hpp] Error 1
Error: An error occured during compilation! See the message above for more information.
> cmdstan_model("test.stan", include_paths = ".")
Compiling Stan program...
Please specify one model_file.
<snip>

If I use stanc on the commandline it is successful.

~/.cmdstanr/cmdstan-2.25.0/bin/stanc test.stan --include-paths .

Now I tried to use the below with no success. I’ve seen a bunch of other forum discussions and some suggested some such solutions. Apologies if I missed the answer that would make my cmdstanr consistent with stanc. It seems that the include_paths should just work like the stanc one (as it is documented). I’ve previously used rstan and these include paths seem to work well.

  • #include test_functions.stan
  • #include ./test_functions.stan
  • #include /test_functions.stan
  • #include “test_functions.stan”
  • #include “./test_functions.stan”
  • #include “./test_functions.stan”

Thanks!
Karim

2

Hey Karim, thanks for reporting this, that’s strange. I actually don’t get a stanc error when trying cmdstan_model("test.stan", include_paths = ".") with your Stan programs, so I’m not sure what’s happening here. I’m on Mac and not Linux but I don’t think this should be OS dependent (although I could be mistaken).

@rok_cesnovar or @bbbales2 Are you able to reproduce this on Linux?

3 
cmdstan_model("test2.stan", include_paths = ".")

works for me

cmdstan_model("test2.stan")

does not, so that’s probably the problem. I didn’t know you had to include the “.” path manually, so that’s good to know.

1 Like
4

It’s weird no one else is getting this error. Here’s the full output of the error message, there’s a code at the bottom. I don’t know if it helps.

> cmdstan_model("test.stan", include_paths = ".")
Compiling Stan program...
Please specify one model_file.

Usage: stanc [option] ... <model_file.stan>
  --debug-lex                      For debugging purposes: print the lexer actions
  --debug-parse                    For debugging purposes: print the parser actions
  --debug-ast                      For debugging purposes: print the undecorated AST, before semantic checking
  --debug-decorated-ast            For debugging purposes: print the decorated AST, after semantic checking
  --debug-generate-data            For debugging purposes: generate a mock dataset to run the model on
  --debug-mir                      For debugging purposes: print the MIR as an S-expression.
  --debug-mir-pretty               For debugging purposes: pretty-print the MIR.
  --debug-optimized-mir            For debugging purposes: print the MIR after it's been optimized. Only has an effect when optimizations are turned on.
  --debug-optimized-mir-pretty     For debugging purposes: pretty print the MIR after it's been optimized. Only has an effect when optimizations are turned on.
  --debug-transformed-mir          For debugging purposes: print the MIR after the backend has transformed it.
  --debug-transformed-mir-pretty   For debugging purposes: pretty print the MIR after the backend has transformed it.
  --dump-stan-math-signatures Dump out the list of supported type signatures for Stan Math backend.
  --warn-uninitialized             Emit warnings about uninitialized variables to stderr. Currently an experimental feature.
  --warn-pedantic                  Emit warnings about common mistakes in Stan programs.
  --auto-format                    Pretty prints the program to the console
  --print-canonical                Prints the canonicalized program to the console
  --version                        Display stanc version number
  --name                           Take a string to set the model name (default = "$model_filename_model")
  --O Allow                        the compiler to apply all optimizations to the Stan code.
  --o                              Take the path to an output file for generated C++ code (default = "$name.hpp")
  --print-cpp                      If set, output the generated C++ Stan model class to stdout.
  --allow-undefined                Do not fail if a function is declared but not defined
  --allow_undefined                Deprecated. Same as --allow-undefined.
  --include-paths                  Takes a comma-separated list of directories that may contain a file in an #include directive (default = "")
  --include_paths                  Deprecated. Same as --include-paths.
  --use-opencl                     If set, try to use matrix_cl signatures.
  --standalone-functions           If set, the generated C++ will be the standalone functions C++ code.
  -help                            Display this list of options
  --help                           Display this list of options
make: *** [make/program:53: /tmp/Rtmp5dvRrU/model-3ff0d7d430636.hpp] Error 127
Error: An error occured during compilation! See the message above for more information.
5

Ok, I also found that it works if I include the full path (in my home directory). Shouldn’t “.” work the same way if in that directory?

> cmdstan_model("test.stan", include_paths = "~/Code/takeup/stan_models")
Compiling Stan program...
functions {
#include "/test_functions.stan"
  
  real foo() {
    return 5.0;
  }
}
> getwd()
[1] "/media/karim/Code Drive/karimn-code/takeup/stan_models"
6

Even weirder, this does not work

> cmdstan_model("test.stan", include_paths = "/media/karim/Code Drive/karimn-code/takeup/stan_models")
Compiling Stan program...
Please specify one model_file.
<snip>

It works if I use a “~/Code/takeup/stan_models” path, but it fails if I use the path normalizePath() returns!

7

I think the problem here is that the full path to the included folder has spaces in them.

Works fine for me on linux as well.

1 Like
8

Sorry, does “.” work for you on Linux or the full path?

9

Both. The problem here is we automatically take the absolute path of “.” and supply that to cmdstan. And that absolute path has spaces in it in your case.

We need to fix that bug in cmdstanr. Will make an issue. In the meantime specify the path with ~/…

3 Likes
10

Ok, thanks for all your help, everyone.

2 Likes
11

Thanks @rok_cesnovar, I didn’t think to check for spaces in the path! And thanks @karimn for bringing this to our attention.

2 Likes
12

@rok_cesnovar FYI the spaces in paths issue is showing up in other functions too:

> xx <- read_cmdstan_csv(dist_fit_obj[[1]]$output_files())
grep: /media/karim/Code: No such file or directory
grep: Drive/karimn-code/takeup/data/stan_analysis_data/test_REDUCED_FORM_NO_RESTRICT-202012080958-1-8fb31c.csv: No such file or directory
Error: Supplied CSV file is corrupt!
13

Thanks!