CMDStanpy not compiling with uninformative error

General
1

Hello.

I’m running cmdstanpy with the below model.

data {
    int<lower=0> N; // number of records
    int<lower=0>  cl_pure_premium[N]; // pure premium
    vector[N] old_auto_c_score_n; //alert flag
}

parameters {
    real<lower=0> mu;
    real beta;

}

model {
       mu ~ normal(0,3);
       beta ~ normal(0,1);
       cl_pure_premium ~ poisson_log(mu + old_auto_c_score_n*beta);
}

generated quantities {
      vector[N] eta = mu + old_auto_c_score * beta;
  int y_rep[N];
  if (max(eta) > 20) {
    // avoid overflow in poisson_log_rng
    print("max eta too big: ", max(eta));
    for (n in 1:N)
      y_rep[n] = -1;
  } else {
      for (n in 1:N)
        y_rep[n] = poisson_log_rng(eta[n]);
  }
}

There is an uninformative error given below and I’m not sure what to do. I’m running this on Windows through the pycharm IDE.

C:\Users\JORDAN.HOWELL.GITDIR\AppData\Local\Continuum\anaconda3\envs\cmdstan\python.exe C:/Users/JORDAN.HOWELL.GITDIR/PycharmProjects/motorcyclebayes/models/mc_coll_bayes_model_script.py
INFO:cmdstanpy:compiling stan file C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.stan to exe file C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.exe
Traceback (most recent call last):
  File "C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes_model_script.py", line 11, in <module>
    proc_parallel_model = CmdStanModel(
  File "c:\users\jordan.howell.gitdir\src\cmdstanpy\cmdstanpy\model.py", line 215, in __init__
    raise ValueError(
ValueError: Unable to compile Stan model file: C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.stan.
INFO:cmdstanpy:deleting tmpfiles dir: C:\Users\JORDAN~1.GIT\AppData\Local\Temp\tmp49w2f7m3
INFO:cmdstanpy:done

Process finished with exit code 1

I’ve re-installed cmdstanpy already. Has anyone seen this before and know what to do?

2

Hi, after the compilation error, can you check the stdoutput in the temp dir?

Check the following folder

cmdstanpy._TMPDIR

and look for .txt file.

3

There is no file there. At the end of the error above, there is a message saying that tmpfiles are being deleted.

4

Ok, do you run your code with the script?

Can you try hopping into the interactive session

python -i script.py

and try again

5

So I ran this from the command terminal:

python -i mc_coll_bayes_model_script.py

I got this but still no file in the temp dir:

INFO:cmdstanpy:compiling stan file C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.stan to exe file C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.exe
Traceback (most recent call last):
  File "C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes_model_script.py", line 11, in <module>
    proc_parallel_model = CmdStanModel(
  File "c:\users\jordan.howell.gitdir\src\cmdstanpy\cmdstanpy\model.py", line 215, in __init__
    raise ValueError(
ValueError: Unable to compile Stan model file: C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.stan.
6

@WardBrian @mitzimorris did we have some keyword to pipe all output to shell?

7

the compile method uses the logging everywhere. I recommend setting the logging level to logging.DEBUG - cf cmdstanpy/model.py at 0b50ae9a8bfdbe011cc9eb258a6f2477d7aefc90 · stan-dev/cmdstanpy · GitHub

add these two stmts to your script:

import logging
logging.getLogger("cmdstanpy").setLevel(logging.DEBUG)
8

Thanks. Using that, a lot came out.

DEBUG:cmdstanpy:cmd: .\bin\stanc.exe --info C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.stan
cwd: C:\Users\JORDAN.HOWELL.GITDIR\.cmdstan\cmdstan-2.28.2
DEBUG:cmdstanpy:Command ['.\\bin\\stanc.exe', '--info', 'C:\\Users\\JORDAN.HOWELL.GITDIR\\PycharmProjects\\motorcyclebayes\\models\\mc_coll_bayes.stan']
	error during processing Operation not permitted
DEBUG:cmdstanpy:cmd: where.exe tbb.dll
cwd: None
DEBUG:cmdstanpy:TBB already found in load path
INFO:cmdstanpy:compiling stan file C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.stan to exe file C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.exe
DEBUG:cmdstanpy:cmd: mingw32-make STAN_THREADS=True STAN_OPENCL=True C:/Users/JORDAN.HOWELL.GITDIR/PycharmProjects/motorcyclebayes/models/mc_coll_bayes.exe
cwd: C:\Users\JORDAN.HOWELL.GITDIR\.cmdstan\cmdstan-2.28.2
DEBUG:cmdstanpy:Console output:

At global scope:
cc1plus.exe: warning: unrecognized command line option '-Wno-ignored-attributes'
cc1plus.exe: warning: unrecognized command line option '-Wno-ignored-attributes'
cc1plus.exe: warning: unrecognized command line option '-Wno-int-in-bool-context'
make/program:11: recipe for target 'src/cmdstan/main_threads_opencl.o' failed
mingw32-make: *** [src/cmdstan/main_threads_opencl.o] Error 1

Command ['mingw32-make', 'STAN_THREADS=True', 'STAN_OPENCL=True', 'C:/Users/JORDAN.HOWELL.GITDIR/PycharmProjects/motorcyclebayes/models/mc_coll_bayes.exe']
	error during processing No such file or directory

Traceback (most recent call last):
  File "C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes_model_script.py", line 13, in <module>
    proc_parallel_model = CmdStanModel(
  File "c:\users\jordan.howell.gitdir\src\cmdstanpy\cmdstanpy\model.py", line 215, in __init__
    raise ValueError(
ValueError: Unable to compile Stan model file: C:\Users\JORDAN.HOWELL.GITDIR\PycharmProjects\motorcyclebayes\models\mc_coll_bayes.stan.
INFO:cmdstanpy:deleting tmpfiles dir: C:\Users\JORDAN~1.GIT\AppData\Local\Temp\tmpp3b0onid
INFO:cmdstanpy:done
9

What compiler do you use? Have you set GPU settings correctly / added needed drivers?

10

I assume whatever compiler was installed when I installed cmdstanpy. My GPU works for neural nets, etc but I wasn’t aware cmdstanpy was using it

11

You are trying to use OpenCL? Or do you use cpu stuff from it?

12

I guess just CPU. I was using proc_parallel because I figured it was putting one chain on each cpu core. That was a guess though.

13

I ended up using this…

mc_model = os.path.join(cmdstan_path(),'examples', 'poisson', 'mc_coll.stan')
my_model = CmdStanModel(stan_file=mc_model)
my_model.name
my_model.stan_file
my_model.exe_file
my_model.code()

It compiled with no errors this time so maybe a problem with proc_parallel and my computer setup?

14

I got the exact same error with CMDStanpy. Now on Linux.

ValueError: Unable to compile Stan model file: /home/stan_stuff/Unpool_model/unpooled_treated_model.stan.

But what’s really weird (and the reason why I resurrect this thread) is that I get this error only for some stan files. So the following line works fine:

test1 = cmdstanpy.CmdStanModel(stan_file = '/home/stan_stuff/test_stan/bernoulli.stan',
                     cpp_options={'STAN_THREADS': 'TRUE'},
                     compile='force')

This is using the standard bernoulli.stan from “Hello, World!” — CmdStanPy 1.0.4 documentation

While the following line gives the aforementioned error:

test2 = cmdstanpy.CmdStanModel(stan_file = '/home/stan_stuff/Unpool_model/unpooled_treated_model.stan',
                     cpp_options={'STAN_THREADS': 'TRUE'},
                     compile='force')

Note that yesterday the lines described above (creating test2) did work. I was trying to get OpenCL to work and may have changed something.

But what’s really really weird is that if I make a copy of unpooled_treated_model.stan with a different name, that it then compiles fine. So:

test3 = cmdstanpy.CmdStanModel(stan_file = '/home/stan_stuff/Unpool_model/unpooled_treated_model_copy.stan',
                     cpp_options={'STAN_THREADS': 'TRUE'},
                     compile='force')

works fine.

So for now I have a workaround (creating a copy of a Stan file). Hope it helps someone who has the same problem.

I have also tried out the extra two lines that @mitzimorris wrote above (in CMDStanpy not compiling with uninformative error - #7 by mitzimorris) but I don’t see any extra lines in my output.

Here is the output of cmdstanpy.show_versions():
output of cmstandpy.show_versions.txt (460 Bytes)

15

if the logger level is at DEBUG, and the model is sometimes failing to compile,
this sure sounds like the C++ compiler and/or loader/linker is failing to find the right libraries, or any other myriad C++ compiler issues incurred by the combination of TBB and GPU libs.

could you try to compile the model from the command line? you need to cd to the CmdStan install directory and run make from there.

make STAN_THREADS=TRUE home/stan_stuff/Unpool_model/unpooled_treated_model
1 Like
16

Thanks for your quick reply!

Running this make gives directly many errors. If I redirect them to a file I get the enclosed file.
result_of_make.txt (629.9 KB)