Bin/stanc segfault on cluster

Interfaces CmdStan
1

So I’m trying to get CmdStan going on a cluster and I get:

--- Translating Stan model to C++ code ---
bin/stanc  examples/bernoulli/bernoulli.stan --o=examples/bernoulli/bernoulli.hpp
make: *** [examples/bernoulli/bernoulli.hpp] Segmentation fault

I requested plenty of memory, I cloned develop and pulled in the submodules, it has
sane modules loaded:

[ks79a@ghpcc06 cmdstan]$ module list
Currently Loaded Modulefiles:
  1) gcc/5.1.0     2) glibc/2.23    3) cmake/3.2.2   4) git/2.1.3     5) R/3.3.1

uname -a gives:

Linux c35b15 2.6.32-642.4.2.el6.x86_64 #1 SMP Mon Aug 15 02:06:41 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux

I’m confused. Ring any bells for you guys?

Full output below but…

Job <make examples/bernoulli/bernoulli> was submitted from host <ghpcc06> by user <ks79a> in cluster <umghpcc>.
Job was executed on host(s) <c35b15>, in queue <condo_uma_nicholas_reich>, as user <ks79a> in cluster <umghpcc>.
</home/ks79a> was used as the home directory.
</home/ks79a/cmdstan/.> was used as the working directory.
Started at Wed Apr 12 10:52:46 2017
Results reported on Wed Apr 12 10:57:43 2017

Your job looked like:

------------------------------------------------------------
# LSBATCH: User input
make examples/bernoulli/bernoulli
------------------------------------------------------------

Exited with exit code 2.

Resource usage summary:

    CPU time :                                   251.83 sec.
    Max Memory :                                 1823 MB
    Average Memory :                             689.25 MB
    Total Requested Memory :                     4096.00 MB
    Delta Memory :                               2273.00 MB
    Max Processes :                              6
    Max Threads :                                7
    Run time :                                   297 sec.
    Turnaround time :                            302 sec.

The output (if any) follows:

g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O3 -o bin/cmdstan/stanc.o src/cmdstan/stanc.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/expression07_grammar_inst.o stan/src/stan/lang/grammars/expression07_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/statement_grammar_inst.o stan/src/stan/lang/grammars/statement_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/indexes_grammar_inst.o stan/src/stan/lang/grammars/indexes_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/functions_grammar_inst.o stan/src/stan/lang/grammars/functions_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/whitespace_grammar_inst.o stan/src/stan/lang/grammars/whitespace_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/expression_grammar_inst.o stan/src/stan/lang/grammars/expression_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/bare_type_grammar_inst.o stan/src/stan/lang/grammars/bare_type_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/term_grammar_inst.o stan/src/stan/lang/grammars/term_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/var_decls_grammar_inst.o stan/src/stan/lang/grammars/var_decls_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/statement_2_grammar_inst.o stan/src/stan/lang/grammars/statement_2_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/program_grammar_inst.o stan/src/stan/lang/grammars/program_grammar_inst.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/grammars/semantic_actions_def.o stan/src/stan/lang/grammars/semantic_actions_def.cpp
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe    -c -O0 -o bin/stan/lang/ast_def.o stan/src/stan/lang/ast_def.cpp
ar -rs bin/libstanc.a bin/stan/lang/grammars/expression07_grammar_inst.o bin/stan/lang/grammars/statement_grammar_inst.o bin/stan/lang/grammars/indexes_grammar_inst.o bin/stan/lang/grammars/functions_grammar_inst.o bin/stan/lang/grammars/whitespace_grammar_inst.o bin/stan/lang/grammars/expression_grammar_inst.o bin/stan/lang/grammars/bare_type_grammar_inst.o bin/stan/lang/grammars/term_grammar_inst.o bin/stan/lang/grammars/var_decls_grammar_inst.o bin/stan/lang/grammars/statement_2_grammar_inst.o bin/stan/lang/grammars/program_grammar_inst.o bin/stan/lang/grammars/semantic_actions_def.o bin/stan/lang/ast_def.o
ar: creating bin/libstanc.a
g++ -I src -I stan/src -isystem stan/lib/stan_math/ -isystem stan/lib/stan_math/lib/eigen_3.2.9 -isystem stan/lib/stan_math/lib/boost_1.62.0 -isystem stan/lib/stan_math/lib/cvodes_2.9.0/include -Wall -DEIGEN_NO_DEBUG  -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DNO_FPRINTF_OUTPUT -pipe   -lpthread  -O0 -o bin/stanc bin/cmdstan/stanc.o  -Lbin -lstanc

--- Translating Stan model to C++ code ---
bin/stanc  examples/bernoulli/bernoulli.stan --o=examples/bernoulli/bernoulli.hpp
make: *** [examples/bernoulli/bernoulli.hpp] Segmentation fault
2

Huh, this segfaults when I just run stanc --version and it segfaults before the first line in main… ok.

3

This can happen if you compiled on one platform and are trying to run on another. The binaries have to match.

4

Actually I just learned that everything I compile on this cluster segfaults, so… yeah… fixing that first.