Eigen library failure with MPI on cluster

Hey it’s me, that guy who always finds weird bugs with MPI…

This time I have a compiled Stan model that fails with the following error (running on SLURM on a linux cluster):

Loading module 'openmpi/1.10.2'
Loading module 'gcc/4.9.3'
idealstan_stan_code: stan/lib/stan_math/lib/eigen_3.3.7/Eigen/src/Core/DenseCoeffsBase.h:408: Eigen::DenseCoeffsBase<Derived, 1>::Scalar& Eigen::DenseCoeffsBase<Derived, 1>::operator[](Eigen::Index) [with Derived = Eigen::Matrix<double, -1, 1>; Eigen::DenseCoeffsBase<Derived, 1>::Scalar = double; Eigen::Index = long int]: Assertion `index >= 0 && index < size()' failed.
[compute-12-17:12676] *** Process received signal ***
[compute-12-17:12676] Signal: Aborted (6)
[compute-12-17:12676] Signal code:  (-6)
[compute-12-17:12676] [ 0] /lib64/libpthread.so.0(+0xf100)[0x2ae2a4a22100]
[compute-12-17:12676] [ 1] /lib64/libc.so.6(gsignal+0x37)[0x2ae2a4c645f7]
[compute-12-17:12676] [ 2] /lib64/libc.so.6(abort+0x148)[0x2ae2a4c65ce8]
[compute-12-17:12676] [ 3] /lib64/libc.so.6(+0x2e566)[0x2ae2a4c5d566]
[compute-12-17:12676] [ 4] /lib64/libc.so.6(+0x2e612)[0x2ae2a4c5d612]
[compute-12-17:12676] [ 5] /scratch/rmk7/wb/idealstan_stan_code[0x424a4d]
[compute-12-17:12676] [ 6] /scratch/rmk7/wb/idealstan_stan_code[0x54f95b]
[compute-12-17:12676] [ 7] /scratch/rmk7/wb/idealstan_stan_code[0x6ab952]
[compute-12-17:12676] [ 8] /scratch/rmk7/wb/idealstan_stan_code[0x6c6c33]
[compute-12-17:12676] [ 9] /scratch/rmk7/wb/idealstan_stan_code[0x678192]
[compute-12-17:12676] [10] /scratch/rmk7/wb/idealstan_stan_code[0x423c06]
[compute-12-17:12676] [11] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2ae2a4c50b15]
[compute-12-17:12676] [12] /scratch/rmk7/wb/idealstan_stan_code[0x4248b9]
[compute-12-17:12676] *** End of error message ***

Any idea what issue I could be having with Eigen? This is using cmdstan-2.23 with the STANC2 compiler.

Does the model run without mpi?

(Probably unrelated, but that compiler is really old…it would not hurt to upgrade if you can)

Is mpi needed or can a threaded version also do?

No ideas for now unfortunately, but a fix for the stanc3 bug will be live tomorrow. Might be worth a try if stanc3 has fixed this since.

You will only need to run
rn bin/linux-stanc
make build

And you should be able to use it. Will post when it goes up on Github.

what bug are you referring to?

I have run the model with threading with rstan 2.19.

I just tried to compile with threads (after cleaning and rebuilding cmdstan) and got this unusual error:

g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare     -DSTAN_THREADS -I stan/lib/stan_math/lib/tbb_2019_U8/include -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.7 -I stan/lib/stan_math/lib/boost_1.72.0 -I stan/lib/stan_math/lib/sundials_5.2.0/include    -DBOOST_DISABLE_ASSERTS      -c  -x c++ -o /scratch/rmk7/wb/idealstan_stan_code.o /scratch/rmk7/wb/idealstan_stan_code.hpp
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
     s__ << "[{\"name\":\"sigma_abs_free\",\"type\":{\"name\":\"vector\",\"length\":" << num_bills << "},\"block\":\"parameters\"},{\"name\":\"L_full\",\"type\":{\"name\":\"vector\",\"length\":" << num_legis << "},\"block\":\"parameters\"},{\"name\":\"m_sd_free\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N_fix << "},\"block\":\"parameters\"},{\"name\":\"gp_sd_free\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N_fix << "},\"block\":\"parameters\"},{\"name\":\"L_tp2\",\"type\":{\"name\":\"array\",\"length\":" << gp_nT << ",\"element_type\":{\"name\":\"vector\",\"length\":" << num_legis << "}},\"block\":\"parameters\"},{\"name\":\"ls_int\",\"type\":{\"name\":\"vector\",\"length\":" << num_ls << "},\"block\":\"parameters\"},{\"name\":\"L_tp1_var\",\"type\":{\"name\":\"array\",\"length\":" << (T - 1) << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (logical_gt(T, 1) ? num_legis : 0) << "}},\"block\":\"parameters\"},{\"name\":\"L_AR1\",\"type\":{\"name\":\"vector\",\"length\":" << ((primitive_value(logical_gt(T, 1)) && primitive_value( logical_eq(time_proc, 3))) ? num_legis : 0) << "},\"block\":\"parameters\"},{\"name\":\"sigma_reg_free\",\"type\":{\"name\":\"vector\",\"length\":" << num_bills << "},\"block\":\"parameters\"},{\"name\":\"legis_x\",\"type\":{\"name\":\"vector\",\"length\":" << LX << "},\"block\":\"parameters\"},{\"name\":\"sigma_reg_x\",\"type\":{\"name\":\"vector\",\"length\":" << SRX << "},\"block\":\"parameters\"},{\"name\":\"sigma_abs_x\",\"type\":{\"name\":\"vector\",\"length\":" << SAX << "},\"block\":\"parameters\"},{\"name\":\"B_int_free\",\"type\":{\"name\":\"vector\",\"length\":" << num_bills << "},\"block\":\"parameters\"},{\"name\":\"A_int_free\",\"type\":{\"name\":\"vector\",\"length\":" << num_bills << "},\"block\":\"parameters\"},{\"name\":\"steps_votes3\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(1), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes4\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(2), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes5\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(3), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes6\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(4), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes7\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(5), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes8\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(6), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes9\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(7), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes10\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(8), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm3\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(1), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm4\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(2), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm5\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(3), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm6\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(4), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm7\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(5), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm8\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(6), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm9\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(7), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm10\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(8), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"extra_sd\",\"type\":{\"name\":\"real\"},\"block\":\"parameters\"},{\"name\":\"time_var_gp_free\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N_fix << "},\"block\":\"parameters\"},{\"name\":\"time_var_free\",\"type\":{\"name\":\"vector\",\"length\":" << ((primitive_value(logical_gt(T, 1)) && primitive_value( logical_neq(time_proc, 4))) ? (num_legis - 1) : 0) << "},\"block\":\"parameters\"},{\"name\":\"L_tp1\",\"type\":{\"name\":\"array\",\"length\":" << T << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (logical_gt(T, 1) ? num_legis : 0) << "}},\"block\":\"transformed_parameters\"},{\"name\":\"time_var_full\",\"type\":{\"name\":\"vector\",\"length\":" << (logical_gt(T, 1) ? num_legis : 0) << "},\"block\":\"transformed_parameters\"},{\"name\":\"time_var_gp_full\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N << "},\"block\":\"transformed_parameters\"},{\"name\":\"m_sd_full\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N << "},\"block\":\"transformed_parameters\"},{\"name\":\"gp_sd_full\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N << "},\"block\":\"transformed_parameters\"},{\"name\":\"varparams\",\"type\":{\"name\":\"array\",\"length\":" << S << ",\"element_type\":{\"name\":\"vector\",\"length\":" << vP << "}},\"block\":\"transformed_parameters\"},{\"name\":\"dparams\",\"type\":{\"name\":\"vector\",\"length\":" << dP << "},\"block\":\"transformed_parameters\"}]";
     ^
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37772:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
     s__ << "[{\"name\":\"sigma_abs_free\",\"type\":{\"name\":\"vector\",\"length\":" << num_bills << "},\"block\":\"parameters\"},{\"name\":\"L_full\",\"type\":{\"name\":\"vector\",\"length\":" << num_legis << "},\"block\":\"parameters\"},{\"name\":\"m_sd_free\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N_fix << "},\"block\":\"parameters\"},{\"name\":\"gp_sd_free\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N_fix << "},\"block\":\"parameters\"},{\"name\":\"L_tp2\",\"type\":{\"name\":\"array\",\"length\":" << gp_nT << ",\"element_type\":{\"name\":\"vector\",\"length\":" << num_legis << "}},\"block\":\"parameters\"},{\"name\":\"ls_int\",\"type\":{\"name\":\"vector\",\"length\":" << num_ls << "},\"block\":\"parameters\"},{\"name\":\"L_tp1_var\",\"type\":{\"name\":\"array\",\"length\":" << (T - 1) << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (logical_gt(T, 1) ? num_legis : 0) << "}},\"block\":\"parameters\"},{\"name\":\"L_AR1\",\"type\":{\"name\":\"vector\",\"length\":" << ((primitive_value(logical_gt(T, 1)) && primitive_value( logical_eq(time_proc, 3))) ? num_legis : 0) << "},\"block\":\"parameters\"},{\"name\":\"sigma_reg_free\",\"type\":{\"name\":\"vector\",\"length\":" << num_bills << "},\"block\":\"parameters\"},{\"name\":\"legis_x\",\"type\":{\"name\":\"vector\",\"length\":" << LX << "},\"block\":\"parameters\"},{\"name\":\"sigma_reg_x\",\"type\":{\"name\":\"vector\",\"length\":" << SRX << "},\"block\":\"parameters\"},{\"name\":\"sigma_abs_x\",\"type\":{\"name\":\"vector\",\"length\":" << SAX << "},\"block\":\"parameters\"},{\"name\":\"B_int_free\",\"type\":{\"name\":\"vector\",\"length\":" << num_bills << "},\"block\":\"parameters\"},{\"name\":\"A_int_free\",\"type\":{\"name\":\"vector\",\"length\":" << num_bills << "},\"block\":\"parameters\"},{\"name\":\"steps_votes3\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(1), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes4\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(2), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes5\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(3), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes6\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(4), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes7\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(5), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes8\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(6), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes9\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(7), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes10\",\"type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_rat, cons_list(index_uni(8), nil_index_list()), \\"n_cats_rat\\") - 1) << "},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm3\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(1), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm4\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(2), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm5\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(3), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm6\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(4), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm7\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(5), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm8\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(6), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm9\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(7), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"steps_votes_grm10\",\"type\":{\"name\":\"array\",\"length\":" << num_bills_grm << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (rvalue(n_cats_grm, cons_list(index_uni(8), nil_index_list()), \\"n_cats_grm\\") - 1) << "}},\"block\":\"parameters\"},{\"name\":\"extra_sd\",\"type\":{\"name\":\"real\"},\"block\":\"parameters\"},{\"name\":\"time_var_gp_free\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N_fix << "},\"block\":\"parameters\"},{\"name\":\"time_var_free\",\"type\":{\"name\":\"vector\",\"length\":" << ((primitive_value(logical_gt(T, 1)) && primitive_value( logical_neq(time_proc, 4))) ? (num_legis - 1) : 0) << "},\"block\":\"parameters\"},{\"name\":\"L_tp1\",\"type\":{\"name\":\"array\",\"length\":" << T << ",\"element_type\":{\"name\":\"vector\",\"length\":" << (logical_gt(T, 1) ? num_legis : 0) << "}},\"block\":\"transformed_parameters\"},{\"name\":\"time_var_full\",\"type\":{\"name\":\"vector\",\"length\":" << (logical_gt(T, 1) ? num_legis : 0) << "},\"block\":\"transformed_parameters\"},{\"name\":\"time_var_gp_full\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N << "},\"block\":\"transformed_parameters\"},{\"name\":\"m_sd_full\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N << "},\"block\":\"transformed_parameters\"},{\"name\":\"gp_sd_full\",\"type\":{\"name\":\"vector\",\"length\":" << gp_N << "},\"block\":\"transformed_parameters\"},{\"name\":\"varparams\",\"type\":{\"name\":\"array\",\"length\":" << S << ",\"element_type\":{\"name\":\"vector\",\"length\":" << vP << "}},\"block\":\"transformed_parameters\"},{\"name\":\"dparams\",\"type\":{\"name\":\"vector\",\"length\":" << dP << "},\"block\":\"transformed_parameters\"}]";
     ^
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
/scratch/rmk7/wb/idealstan_stan_code.hpp:37778:5: error: stray ‘\’ in program
make: *** [/scratch/rmk7/wb/idealstan_stan_code] Error 1

What version would you recommend? Is this mainly gcc or also openmpi?

I wouldn’t worry about gcc being 4.9.3, but a 1.x openmpi is simply ancient.

This looks like a problem of stanc3 transpiler. You can test it on stanc2 by adding “STANC2=true” in make/local.

Sorry, should have been clearer. It was as bug that forced most MPI users to roll with STANC2. That is why I referred to it as “the bug” as Robert was using stanc2.

The bug causes MPI issues with map_rect:


Its fixed now and if you run
rm bin/linux-stanc bin/windows-stanc bin/mac-stanc
And
Make build

It should download the fixed version.

@saudiwin please try it out if you have the chance.

2 Likes

Sure I’ll give it a go soon! Am currently trying to run the above model with threads (this particular model is small enough to run on a single node). Could the bug perhaps be that running MPI on a single node produces errors? Because I previously ran MPI across nodes without a problem (same cmdstan, etc).

This would be unlikely.

I’m still getting the same error with stanc3 in the latest version (2.23). The model compiles fine with stanc2.

Do you mean the 2.23 release? That release had this bug.

You can do two things:

Just for reference, what OS version is the SLURM cluster running? Presumably RHEL/Centos.