Python 3.7 / pystan 19.0.0 error "DistutilsExecError: command 'gcc' failed with exit status 1"

Interfaces
1

Operating System: macOS Sierra Version 10.12.6
Interface Version: pystan 19.0.0
Compiler/Toolkit: gcc (GCC) 4.8.5
I am getting a error "DistutilsExecError: command ‘gcc’ failed with exit status 1” while compiling any STAN program.
I read and tried some solutions from the previous posts, but none of them works.
What is the ultimate solution?

Please, find the STAN script and error messages below.

import pickle
import pystan

bernoulli model

model_code = “”"
data {
int<lower=0> N;
int<lower=0,upper=1> y[N];
}
parameters {
real<lower=0,upper=1> theta;
}
model {
for (n in 1:N)
y[n] ~ bernoulli(theta);
}
“”"
data = dict(N=10, y=[0, 1, 0, 0, 0, 0, 0, 0, 0, 1])
model = pystan.StanModel(model_code=model_code,verbose=True)
fit = model.sampling(data=data)
with open(‘model.pkl’, ‘wb’) as f:
pickle.dump(model, f, protocol=pickle.HIGHEST_PROTOCOL)

load it at some future point

with open(‘model.pkl’, ‘rb’) as f:
model = pickle.load(f)

run with different data

fit = model.sampling(data=dict(N=5, y=[1, 1, 0, 1, 0]))

INFO:INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_6a7878734d32aa2c4ce7e5fa08a21901 NOW.
INFO:pystan:OS: darwin, Python: 3.7.3 (default, Mar 27 2019, 16:54:48)
[Clang 4.0.1 (tags/RELEASE_401/final)], Cython 0.29.7
Compiling /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/stanfit4anon_model_6a7878734d32aa2c4ce7e5fa08a21901_1770875115268925820.pyx because it changed.
[1/1] Cythonizing /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/stanfit4anon_model_6a7878734d32aa2c4ce7e5fa08a21901_1770875115268925820.pyx
building ‘stanfit4anon_model_6a7878734d32aa2c4ce7e5fa08a21901_1770875115268925820’ extension
creating /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/var
creating /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/var/folders
creating /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/var/folders/ky
creating /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn
creating /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T
creating /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/anaconda3/envs/stan37/include -arch x86_64 -I/anaconda3/envs/stan37/include -arch x86_64 -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -I/var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep -I/anaconda3/envs/stan37/lib/python3.7/site-packages/pystan -I/anaconda3/envs/stan37/lib/python3.7/site-packages/pystan/stan/src -I/anaconda3/envs/stan37/lib/python3.7/site-packages/pystan/stan/lib/stan_math -I/anaconda3/envs/stan37/lib/python3.7/site-packages/pystan/stan/lib/stan_math/lib/eigen_3.3.3 -I/anaconda3/envs/stan37/lib/python3.7/site-packages/pystan/stan/lib/stan_math/lib/boost_1.69.0 -I/anaconda3/envs/stan37/lib/python3.7/site-packages/pystan/stan/lib/stan_math/lib/sundials_4.1.0/include -I/anaconda3/envs/stan37/lib/python3.7/site-packages/numpy/core/include -I/anaconda3/envs/stan37/include/python3.7m -c /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/stanfit4anon_model_6a7878734d32aa2c4ce7e5fa08a21901_1770875115268925820.cpp -o /var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/var/folders/ky/6v6kh1kn4k74zbzmym2fdjxh0000gn/T/tmp9dfpyhep/stanfit4anon_model_6a7878734d32aa2c4ce7e5fa08a21901_1770875115268925820.o -O2 -ftemplate-depth-256 -Wno-unused-function -Wno-uninitialized -std=c++1y

DistutilsExecError Traceback (most recent call last)
/anaconda3/envs/stan37/lib/python3.7/distutils/unixccompiler.py in _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts)
117 self.spawn(compiler_so + cc_args + [src, ‘-o’, obj] +
–> 118 extra_postargs)
119 except DistutilsExecError as msg:

/anaconda3/envs/stan37/lib/python3.7/distutils/ccompiler.py in spawn(self, cmd)
908 def spawn(self, cmd):
–> 909 spawn(cmd, dry_run=self.dry_run)
910

/anaconda3/envs/stan37/lib/python3.7/distutils/spawn.py in spawn(cmd, search_path, verbose, dry_run)
35 if os.name == ‘posix’:
—> 36 _spawn_posix(cmd, search_path, dry_run=dry_run)
37 elif os.name == ‘nt’:

/anaconda3/envs/stan37/lib/python3.7/distutils/spawn.py in _spawn_posix(cmd, search_path, verbose, dry_run)
158 “command %r failed with exit status %d”
–> 159 % (cmd, exit_status))
160 elif os.WIFSTOPPED(status):

DistutilsExecError: command ‘gcc’ failed with exit status 1

During handling of the above exception, another exception occurred:

CompileError Traceback (most recent call last)
in
16 “”"
17 data = dict(N=10, y=[0, 1, 0, 0, 0, 0, 0, 0, 0, 1])
—> 18 model = pystan.StanModel(model_code=model_code,verbose=True)
19 fit = model.sampling(data=data)
20 with open(‘model.pkl’, ‘wb’) as f:

/anaconda3/envs/stan37/lib/python3.7/site-packages/pystan/model.py in init(self, file, charset, model_name, model_code, stanc_ret, include_paths, boost_lib, eigen_lib, verbose, obfuscate_model_name, extra_compile_args)
347
348 try:
–> 349 build_extension.run()
350 finally:
351 if redirect_stderr:

/anaconda3/envs/stan37/lib/python3.7/distutils/command/build_ext.py in run(self)
338
339 # Now actually compile and link everything.
–> 340 self.build_extensions()
341
342 def check_extensions_list(self, extensions):

/anaconda3/envs/stan37/lib/python3.7/distutils/command/build_ext.py in build_extensions(self)
447 self._build_extensions_parallel()
448 else:
–> 449 self._build_extensions_serial()
450
451 def _build_extensions_parallel(self):

/anaconda3/envs/stan37/lib/python3.7/distutils/command/build_ext.py in _build_extensions_serial(self)
472 for ext in self.extensions:
473 with self._filter_build_errors(ext):
–> 474 self.build_extension(ext)
475
476 @contextlib.contextmanager

/anaconda3/envs/stan37/lib/python3.7/distutils/command/build_ext.py in build_extension(self, ext)
532 debug=self.debug,
533 extra_postargs=extra_args,
–> 534 depends=ext.depends)
535
536 # XXX outdated variable, kept here in case third-part code

/anaconda3/envs/stan37/lib/python3.7/distutils/ccompiler.py in compile(self, sources, output_dir, macros, include_dirs, debug, extra_preargs, extra_postargs, depends)
572 except KeyError:
573 continue
–> 574 self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
575
576 # Return all object filenames, not just the ones we just built.

/anaconda3/envs/stan37/lib/python3.7/distutils/unixccompiler.py in _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts)
118 extra_postargs)
119 except DistutilsExecError as msg:
–> 120 raise CompileError(msg)
121
122 def create_static_lib(self, objects, output_libname,

CompileError: command ‘gcc’ failed with exit status 1

2

You need to update your compiler.

3

Do you mean re-install gcc?

4

I mean upgrade. You need C++14 compatible compiler.

5

I installed gcc and pystan in my virtual environment:
conda install gcc
conda install pystan
And now the compiler works. But it uses -std=c++11.
gcc version 4.8.5 (GCC)
Why do I need c++14?

6

2.19 uses --c++1y.

You can try to find gcc 5 or higher from conda, if you have problems in future.

Also gcc from brew should work.

7

Thank you for info.

8

I’m dealing with the same basic problem, and I can’t figure out how to fix it. I have completely uninstalled and reinstalled Anaconda, command line tools (making sure to run the command to deal with headers), and I’m still getting CompileError: command 'gcc' failed with exit status 1.

I can’t figure out how to get a gcc version above 4.8.5 from conda. I installed gcc with homebrew, and it seems to be a much more recent version, but I can’t get pystan to use it (/usr/local/bin is already in my path, and it didn’t use it even with command line tools uninstalled).

I’m on macOS Mojave 10.14.5. I just reinstalled Anaconda 3, and I used pip to install pystan 2.19.0.0.

When I type gcc --version into the terminal, I get:

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include- 
dir=/usr/include/c++/4.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

This is driving me crazy, and I would very much like to use Stan and pystan for an analysis I’m working on, so any advice on how to get this working would be greatly appreciated. Thanks.

9

Three questions:

  • In what folder or directory is your Anaconda installation?
  • What is the output of “which gcc” (without the quotes)?
  • What is the output of “echo $PATH” (without the quotes)?
10
  1. ~/anaconda3
  2. /usr/bin/gcc
  3. /Users/nsilbert/anaconda3/bin:/Users/nsilbert/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/MacVim.app/Contents/bin
11

Okay, so what happens if you run the following command?:

ls /usr/local/bin | grep gcc
12

gcc-9
gcc-ar-9
gcc-nm-9
gcc-ranlib-9
x86_64-apple-darwin18-gcc-9
x86_64-apple-darwin18-gcc-9.1.0
x86_64-apple-darwin18-gcc-ar-9
x86_64-apple-darwin18-gcc-nm-9
x86_64-apple-darwin18-gcc-ranlib-9

13

Two more questions:

  • Do you have a g++-9 in /usr/local/bin?
  • If the answer to the previous question is “yes”, what happens if you run export CXX=g++9 and then start a Python session in which you run PyStan?
14

If I run ls /usr/local/bin | grep g++, I get:

g++-9
x86_64-apple-darwin18-g++-9

I ran export CXX=g++-9 (and export CXX=g++9), and I got the same error/failure.

15

I’m not sure what else to try. You might want to look here for ideas:

16

Hi, try

CC=gcc-9

I’m not sure if distutils even uses CXX.

import pystan
import os
os.environ["CC"] = "gcc-9"
# just to be sure
os.environ["CXX"] = "g++-9"
1 Like
17

That did the trick! Thanks (to both ahartikainen and jjramsey)!

1 Like
18

Brief follow-up: I wrote a new script in which I only included os.environ['CC'] = 'gcc-9', and I got an ImportError error that I hadn’t seen before. I added os.environ['CXX'] = 'g++-9' and the script works fine (at least with respect to compiling the model). I can copy and paste the error message if it would be of use.

1 Like
19

That makes sense. Behind the scenes, your Stan file is translated into a C++ file that is then compiled into an executable. Without os.environ['CXX'] = 'g++-9', either no C++ compiler or the wrong C++ compiler will be found, so you then should get an error of some sort.