CmdStan on Linux server - Chain finished unexpectedly with schools example (but not with bernoulli example)

Interfaces CmdStan
1

Hello,

I have been trying to fit models using cmdstan and cmdstanr on a Linux server.
cmdstanr version 0.7.1
cmdstan version 2.30.1
Operating System: Linux Centos7
I can get the Bernoulli example (see attached code with reproducible example) to compile and run successfully. However, when I try to implement the schools example (see attached code with reproducible example) the model compiles and starts sampling but ends unexpectedly after a few iterations. I can’t understand why the Bernoulli model would work while the schools model wouldn’t. Does anybody have any suggestions as to why that may be or what I should check to try and diagnose this? I am copying the output that I get when I compile and run the failing model (warnings occurring during compilation seem to be the same with both the model that works and the model that does not work). I have seen a few posts with what appear to be very similar issues on the forum but none of them seems to have found a solution.

CmdStan path set to: /modeling/spack/opt/spack/linux-centos7-haswell/gcc-12.2.0/cmdstan-2.30.1-6flytnlf npq7xf6xldbnkxstwkljxmy7
In file included from stan/lib/stan_math/lib/boost_1.78.0/boost/multi_array/multi_array_ref.hpp:32,
from stan/lib/stan_math/lib/boost_1.78.0/boost/multi_array.hpp:34,
from stan/lib/stan_math/lib/boost_1.78.0/boost/numeric/odeint/algebra/multi_array_alge bra.hpp:22,
from stan/lib/stan_math/lib/boost_1.78.0/boost/numeric/odeint.hpp:63,
from stan/lib/stan_math/stan/math/prim/functor/ode_rk45.hpp:9,
from stan/lib/stan_math/stan/math/prim/functor/integrate_ode_rk45.hpp:6,
from stan/lib/stan_math/stan/math/prim/functor.hpp:14,
from stan/lib/stan_math/stan/math/rev/fun.hpp:196,
from stan/lib/stan_math/stan/math/rev.hpp:10,
from stan/lib/stan_math/stan/math.hpp:19,
from stan/src/stan/model/model_header.hpp:4,
from /tmp/RtmplSlamc/model-a6ed4d92e1b1.hpp:3:
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:180:45: warning: ‘template<class _Arg, class _ Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
180 | : public boost::functional::detail::unary_function<typename unary_traits::ar gument_type,bool>
| ^~~~~~~~~~~~~~

In file included from /modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g 6x5lksmvapqw3dnkce3p/lib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/string:48,
from /modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g 6x5lksmvapqw3dnkce3p/lib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/locale_clas ses.h:40,
from /modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g 6x5lksmvapqw3dnkce3p/lib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/ios_base.h: 41,
from /modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g 6x5lksmvapqw3dnkce3p/lib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/ios:42,
from /modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g 6x5lksmvapqw3dnkce3p/lib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/istream:38,
from /modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g 6x5lksmvapqw3dnkce3p/lib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/sstream:38,
from /modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g 6x5lksmvapqw3dnkce3p/lib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/complex:45,
from stan/lib/stan_math/lib/eigen_3.3.9/Eigen/Core:96,
from stan/lib/stan_math/lib/eigen_3.3.9/Eigen/Dense:1,
from stan/lib/stan_math/stan/math/prim/fun/Eigen.hpp:22,
from stan/lib/stan_math/stan/math/rev.hpp:4:
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:117:12: note: decl ared here
117 | struct unary_function
| ^~~~~~~~~~~~~~
stan/
lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:214:45: warning: ‘template<class _Arg1, class _Arg2 , class _Result> struct std::binary_function’ is deprecated [-Wdeprecated-declarations]
214 | : public boost::functional::detail::binary_function<
| ^~~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:131:12: note: decl ared here
131 | struct binary_function
| ^~~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:252:45: warning: ‘template<class _Arg, class _ Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
252 | : public boost::functional::detail::unary_function<
| ^~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:117:12: note: decl ared here
117 | struct unary_function
| ^~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:299:45: warning: ‘template<class _Arg, class _ Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
299 | : public boost::functional::detail::unary_function<
| ^~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:117:12: note: decl ared here
117 | struct unary_function
| ^~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:345:57: warning: ‘template<class _Arg, class _ Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
345
| class mem_fun_t : public boost::functional::detail::unary_function<T*, S>
| ^~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:117:12: note: decl ared here
117 | struct unary_function
| ^~~~~~~~~~~~~~

stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:361:58: warning: ‘template<class _Arg1, class _Arg2, class _Result> struct std::binary_function’ is deprecated [-Wdeprecated-declarations]
361 | class mem_fun1_t : public boost::functional::detail::binary_function<T*, A, S>
| ^~~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:131:12: note: decl ared here
131 | struct binary_function
| ^~~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:377:63: warning: ‘template<class _Arg, class _ Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
377 | class const_mem_fun_t : public boost::functional::detail::unary_function<const T*, S>
| ^~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:117:12: note: decl ared here
117 | struct unary_function
| ^~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:393:64: warning: ‘template<class _Arg1, class _Arg2, class _Result> struct std::binary_function’ is deprecated [-Wdeprecated-declarations]
393 | class const_mem_fun1_t : public boost::functional::detail::binary_function<const T*, A, S>
| ^~~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:131:12: note: decl ared here
131 | struct binary_function
| ^~~~~~~~~~~~~~~
stan/lib/stan_math/lib/boo
st_1.78.0/boost/functional.hpp:438:61: warning: ‘template<class _Arg, class Result> struct std::unary function’ is deprecated [-Wdeprecated-declarations]
438 | class mem_fun_ref_t : public boost::functional::detail::unary_function<T&, S>
| ^~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:117:12: note: decl ared here
117 | struct unary_function
| ^~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:454:62: warning: ‘template<class _Arg1, class _Arg2, class _Result> struct std::binary_function’ is deprecated [-Wdeprecated-declarations]
454 | class mem_fun1_ref_t : public boost::functional::detail::binary_function<T&, A, S>
| ^~~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:131:12: note: decl ared here
131 | struct binary_function
| ^~~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:470:67: warning: ‘template<class _Arg, class _ Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
470 | class const_mem_fun_ref_t : public boost::functional::detail::unary_function<const T&, S>
| ^~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:117:12: note: decl ared here
117 | struct unary_function
| ^~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:487:68: warni
ng: ‘template<class _Arg1, class _Arg2, class _Result> struct std::binary_function’ is deprecated [-Wde precated-declarations]
487 | class const_mem_fun1_ref_t : public boost::functional::detail::binary_function<const T&, A, S>
| ^~~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:131:12: note: decl ared here
131 | struct binary_function
| ^~~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:533:73: warning: ‘template<class _Arg, class _ Result> struct std::unary_function’ is deprecated [-Wdeprecated-declarations]
533 | class pointer_to_unary_function : public boost::functional::detail::unary_function<Arg,Resu lt>
| ^~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:117:12: note: decl ared here
117 | struct unary_function
| ^~~~~~~~~~~~~~
stan/lib/stan_math/lib/boost_1.78.0/boost/functional.hpp:557:74: warning: ‘template<class _Arg1, class _Arg2, class _Result> struct std::binary_function’ is deprecated [-Wdeprecated-declarations]
557 | class pointer_to_binary_function : public boost::functional::detail::binary_function<Arg1,A rg2,Result>
| ^~~~~~~~~~~~~~~
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/l ib/gcc/x86_64-pc-linux-gnu/12.2.0/…/…/…/…/include/c++/12.2.0/bits/stl_function.h:131:12: note: decl ared here
131 | struct binary_function
| ^~~~~~~~~~~~~~~

Running MCMC with 1 chain…

Chain 1 Iteration: 1 / 1000 [ 0%] (Warmup)
Warning: Chain 1 finished unexpectedly!

Warning message:
No chains finished successfully. Unable to retrieve the fit.
Error: No chains finished successfully. Unable to retrieve the draws.
Execution halted

schools.R (360 Bytes)
bernoulli.R (295 Bytes)
bernoulli.stan (244 Bytes)
schools.stan (366 Bytes)

2

Do you get any additional output if you run the model outside of R on the command line?

3

Hi,
thanks for your answer! This is the output I get when running on the command line:

[XXXXXXX]$ ./schools sample data file=schools_data.R
method = sample (Default)
sample
num_samples = 1000 (Default)
num_warmup = 1000 (Default)
save_warmup = 0 (Default)
thin = 1 (Default)
adapt
engaged = 1 (Default)
gamma = 0.050000000000000003 (Default)
delta = 0.80000000000000004 (Default)
kappa = 0.75 (Default)
t0 = 10 (Default)
init_buffer = 75 (Default)
term_buffer = 50 (Default)
window = 25 (Default)
algorithm = hmc (Default)
hmc
engine = nuts (Default)
nuts
max_depth = 10 (Default)
metric = diag_e (Default)
metric_file = (Default)
stepsize = 1 (Default)
stepsize_jitter = 0 (Default)
num_chains = 1 (Default)
id = 1 (Default)
data
file = schools_data.R
init = 2 (Default)
random
seed = 3569292808 (Default)
output
file = output.csv (Default)
diagnostic_file = (Default)
refresh = 100 (Default)
sig_figs = -1 (Default)
profile_file = profile.csv (Default)
num_threads = 1 (Default)

Gradient evaluation took 3.3e-05 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.33 seconds.
Adjust your expectations accordingly!

Iteration: 1 / 2000 [ 0%] (Warmup)
Segmentation fault (core dumped)

4

Could you re-compile with CXXFLAGS=-ggdb3 and run the model under gdb? Trying to find where the segfault would be happening.

5

I tried compiling my model from the command line using the following command:

make CXXFLAGS=‘-ggdb3’ /PATH/TO/MODEL

But I got the following error:

— Translating Stan model to C++ code —
bin/stanc --warn-pedantic --o=/modeling/ib700/CalCAST_code/schools.hpp /modeling/ib700/CalCAST_code/schools.stan

— Compiling, linking C++ code —
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/bin/g++ -ggdb3 -DBOOST_DISABLE_ASSERTS -c -Wno-ignored-attributes -x c++ -o /path/to/model/schools.o /path/to/model/schools.hpp
/path/to/model/schools.hpp:3:10: fatal error: stan/model/model_header.hpp: No such file or directory
3 | #include <stan/model/model_header.hpp>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

However, looking around the forum I found this post and decided to try the following:

make STAN_THREADS=TRUE STANCFLAGS=–O1 /PATH/TO/MODEL

This makes my model complete succesfully! Any idea why?

6

I think for a first attempt you can not recompile Stan, just the model. It’s possible that will lead to a dead end pointing us somewhere deeper which would make us want to recompile everything, but for now I wouldn’t say it is necessary. make/local or export CXXFLAGS=-ggdb3 should work

7

Re-posting my answer here since I edited my previus post while you were replying:

I tried compiling my model from the command line using the following command:

make CXXFLAGS=‘-ggdb3’ /PATH/TO/MODEL

But I got the following error:

— Translating Stan model to C++ code —
bin/stanc --warn-pedantic --o=/modeling/ib700/CalCAST_code/schools.hpp /modeling/ib700/CalCAST_code/schools.stan

— Compiling, linking C++ code —
/modeling/spack/opt/spack/linux-centos7-haswell/gcc-4.8.5/gcc-12.2.0-jvrrfd27hm6g6x5lksmvapqw3dnkce3p/bin/g++ -ggdb3 -DBOOST_DISABLE_ASSERTS -c -Wno-ignored-attributes -x c++ -o /path/to/model/schools.o /path/to/model/schools.hpp
/path/to/model/schools.hpp:3:10: fatal error: stan/model/model_header.hpp: No such file or directory
3 | #include <stan/model/model_header.hpp>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

However, looking around the forum I found this post and decided to try the following:

make STAN_THREADS=TRUE STANCFLAGS=–O1 /PATH/TO/MODEL

This makes my model complete succesfully! Any idea why?

8

There is an issue where make CXXFLAGS=... ends up re-setting all of the C++ flags stan itself tries to set. You can use CXXFLAGS=... make, though, and it should work

9

Thanks. You may have missed what I wrote at the end of my reply (past the error message), but looking around the forum I found this post and decided to try the following:

make STAN_THREADS=TRUE STANCFLAGS=–O1 /PATH/TO/MODEL

This makes my model complete succesfully! Does this give you any idea as to what may be going on?

10

I did indeed miss that, my bad!

STAN_THREADS changes quite a few of the internals of Stan to be thread safe and allow multiple threads to sample at once. STANCFLAGS=-O1 tells the Stan-to-C++ compiler it can perform some well-tested optimizations.

As a result, a lot of the compiled code will probably be different with those two combined. If you’d like to keep looking for an issue without them you can, but if you’re happy to continue with the now-working model that is also understandable :)

11

I would definitely want to figure out what the issue is, because even though I can now run this simple test model, my actual models (which run with no problem on my Windows setup) are still giving me the same error, even after adding those flags.

I tried to re-compile the test model with CXXFLAGS=-ggdb3 and run the model under gdb, but I get yet another error:

Dwarf Error: wrong version in compilation unit header (is 5, should be 2, 3, or 4)

I very much appreciate your help so far, but I may just have to stick with Windows unfortunately.

12

That error sounds like what would happen if you have a gdb version much older than your compiler. If your server is using modules maybe you need to load a newer version?

13

Yes you are right, I’ll look into that. Thanks again for being so responsive, I really appreciate that!

14

Hi,
I may have managed to get gdb to work. When I run the model under gdb I get the following error, any idea what this may mean? Thanks!

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7350b7c in free () from /lib64/libc.so.6

15

That’s definitely an interesting place to segfault. If you enter bt after that it should give us a bigger picture idea of where it may be happening

16

Here’s the output of bt:

#0 0x00007ffff7350b7c in free () from /lib64/libc.so.6
#1 0x000000000054ee8d in stan::mcmc::base_nuts<stan::model::model_base, stan::mcmc::diag_e_metric, stan::mcmc::expl_leapfrog, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >::transition(stan::mcmc::sample&, stan::callbacks::logger&) ()
#2 0x000000000054f82f in stan::mcmc::adapt_diag_e_nuts<stan::model::model_base, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >::transition(stan::mcmc::sample&, stan::callbacks::logger&) ()
#3 0x000000000045b858 in void stan::services::util::generate_transitions<stan::model::model_base, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >(stan::mcmc::base_mcmc&, int, int, int, int, int, bool, bool, stan::services::util::mcmc_writer&, stan::mcmc::sample&, stan::model::model_base&, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> >&, stan::callbacks::interrupt&, stan::callbacks::logger&, unsigned long, unsigned long) [clone .constprop.1] ()
#4 0x0000000000512d85 in int stan::services::sample::hmc_nuts_diag_e_adaptstan::model::model_base(stan::model::model_base&, stan::io::var_context const&, stan::io::var_context const&, unsigned int, unsigned int, double, int, int, int, bool, int, double, double, int, double, double, double, double, unsigned int, unsigned int, unsigned int, stan::callbacks::interrupt&, stan::callbacks::logger&, stan::callbacks::writer&, stan::callbacks::writer&, stan::callbacks::writer&) ()
#5 0x0000000000515077 in int stan::services::sample::hmc_nuts_diag_e_adapt<stan::model::model_base, std::shared_ptrstan::io::var_context, stan::callbacks::writer, stan::callbacks::unique_stream_writer<std::basic_ostream<char, std::char_traits > >, stan::callbacks::unique_stream_writer<std::basic_ostream<char, std::char_traits > > >(stan::model::model_base&, unsigned long, std::vector<std::shared_ptrstan::io::var_context, std::allocator<std::shared_ptrstan::io::var_context > > const&, unsigned int, unsigned int, double, int, int, int, bool, int, double, double, int, double, double, double, double, unsigned int, unsigned int, unsigned int, stan::callbacks::interrupt&, stan::callbacks::logger&, std::vector<stan::callbacks::writer, std::allocatorstan::callbacks::writer >&, std::vector<stan::callbacks::unique_stream_writer<std::basic_ostream<char, std::char_traits > >, std::allocator<stan::callbacks::unique_stream_writer<std::basic_ostream<char, std::char_traits > > > >&, std::vector<stan::callbacks::unique_stream_writer<std::basic_ostream<char, std::char_traits > >, std::allocator<stan::callbacks::unique_stream_writer<std::basic_ostream<char, std::char_traits > > > >&) ()
#6 0x000000000048e484 in cmdstan::command(int, char const**) ()
#7 0x0000000000421de6 in main ()

17

Hm, it’s still not giving line numbers even with -ggdb3, which makes it a bit difficult. I’ve not seen a seg fault in that part of the code before.

Let’s see if @stevebronder has any ideas on how to dig further

18

The only time I’ve seen issues like this is if Stan or dependencies of Stan were built with one compiler and then the model is built with another compiler. Did you recently update to gcc-12? If so can you either

  1. do make clean-all in the cmdstan repo and then make build
  2. delete the cmdstan repo from the linux server, redownload it, and then do a clean build