Error in rstan_2.16.2 after upgrading to R 3.4.1 with Clang 4.0.0

Interfaces RStan
1

I can compile and run the 8schools example fine, but for example with bugs_examples/vol2/dugongs/dugongs.stan I get the following error message:

SAMPLING FOR MODEL ‘dugongs’ NOW (CHAIN 1).
empty_nested() must be true before calling recover_memory()
[1] "Error in sampler$call_sampler(args_list[[i]]) : " " c++ exception (unknown reason)"
error occurred during calling the sampler; sampling not done

Operating System: macOS 10.12.6

Interface Version: RStan 2.16.2

Output of writeLines(readLines(file.path(Sys.getenv(“HOME”), “.R/Makevars”))):
CC=/usr/local/clang4/bin/clang
CXX=/usr/local/clang4/bin/clang++ -arch x86_64 -ftemplate-depth-256
LDFLAGS=-L/usr/local/clang4/lib -fopenmp

created by rstan at Tue Oct 14 13:50:01 2014

CXXFLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -Wc++11-extensions -Wc++11-long-long -fopenmp
R_XTRA_CPPFLAGS = -I$(R_INCLUDE_DIR) #set_by_rstan

Output of devtools::session_info("rstan”):
Session info ------------------------------------------------------------------------------------------------------------------------------------------
setting value
version R version 3.4.1 (2017-06-30)
system x86_64, darwin15.6.0
ui RStudio (1.0.153)
language (EN)
collate en_GB.UTF-8
tz Europe/London
date 2017-09-14

Packages ----------------------------------------------------------------------------------------------------------------------------------------------
package * version date source
BH 1.65.0-1 2017-08-24 CRAN (R 3.4.1)
colorspace 1.3-2 2016-12-14 CRAN (R 3.4.0)
dichromat 2.0-0 2013-01-24 CRAN (R 3.4.0)
digest 0.6.12 2017-01-27 CRAN (R 3.4.0)
ggplot2 2.2.1 2016-12-30 CRAN (R 3.4.0)
graphics * 3.4.1 2017-07-07 local
grDevices * 3.4.1 2017-07-07 local
grid 3.4.1 2017-07-07 local
gridExtra 2.2.1 2016-02-29 CRAN (R 3.4.0)
gtable 0.2.0 2016-02-26 CRAN (R 3.4.0)
inline 0.3.14 2015-04-13 CRAN (R 3.4.0)
labeling 0.3 2014-08-23 CRAN (R 3.4.0)
lattice 0.20-35 2017-03-25 CRAN (R 3.4.1)
lazyeval 0.2.0 2016-06-12 CRAN (R 3.4.0)
magrittr 1.5 2014-11-22 CRAN (R 3.4.0)
MASS 7.3-47 2017-02-26 CRAN (R 3.4.1)
Matrix 1.2-10 2017-05-03 CRAN (R 3.4.1)
methods * 3.4.1 2017-07-07 local
munsell 0.4.3 2016-02-13 CRAN (R 3.4.0)
plyr 1.8.4 2016-06-08 CRAN (R 3.4.0)
RColorBrewer 1.1-2 2014-12-07 CRAN (R 3.4.0)
Rcpp 0.12.12 2017-07-15 CRAN (R 3.4.1)
RcppEigen 0.3.3.3.0 2017-05-01 CRAN (R 3.4.0)
reshape2 1.4.2 2016-10-22 CRAN (R 3.4.0)
rlang 0.1.1 2017-05-18 CRAN (R 3.4.0)
rstan 2.16.2 2017-07-03 CRAN (R 3.4.1)
scales 0.4.1 2016-11-09 CRAN (R 3.4.0)
StanHeaders 2.16.0-1 2017-07-03 CRAN (R 3.4.1)
stats * 3.4.1 2017-07-07 local
stats4 3.4.1 2017-07-07 local
stringi 1.1.5 2017-04-07 CRAN (R 3.4.0)
stringr 1.2.0 2017-02-18 CRAN (R 3.4.0)
tibble 1.3.3 2017-05-28 CRAN (R 3.4.0)
tools 3.4.1 2017-07-07 local
utils * 3.4.1 2017-07-07 local

2

Can you try without the -fopenmp? I think that shouldn’t cause problems unless OpenMP is used, but maybe it does.

3

This is my new Makevars, without any CXXFLAGS or R_XTRA_CPPFLAGS:

CC=/usr/local/clang4/bin/clang
CXX=/usr/local/clang4/bin/clang++ -arch x86_64 -ftemplate-depth-256
LDFLAGS=-L/usr/local/clang4/lib

I still get the same error with ‘dugongs’ (302) and ‘birats’ (300) but not with ‘8schools’
Does Stan cache a copy of compiled models anywhere, that I need to clear?

4

If you have gdb installed, can you add CXXFLAGS = -O0 -g to ~/.R/Makevars and then start R with

R -d gdb

Then type

run

at the gdb prompt to start R. Do stan_demo("dugongs", chains = 1) or whatever and wait for the unknown C++ exception, which will throw you back into gdb. Then type

bt

and paste that here. Then you can just do

quit

to exit.

5

The stack was 94 levels deep, so there is a lot of output below. This is the result that I got from lldb:

(lldb) breakpoint set -E c++
Breakpoint 1: 4 locations.
(lldb) process continue
Process 82965 resuming

stan_demo(“birats”, chains = 1)

DIAGNOSTIC(S) FROM PARSER:
Warning (non-fatal): Comments beginning with # are deprecated. Please use // in place of # for line comments.
Warning (non-fatal): Comments beginning with # are deprecated. Please use // in place of # for line comments.
Warning (non-fatal): Comments beginning with # are deprecated. Please use // in place of # for line comments.
Warning (non-fatal): Comments beginning with # are deprecated. Please use // in place of # for line comments.
Warning (non-fatal): Comments beginning with # are deprecated. Please use // in place of # for line comments.
Warning (non-fatal): Comments beginning with # are deprecated. Please use // in place of # for line comments.
Warning (non-fatal): assignment operator ← deprecated in the Stan language; use = instead.
Warning (non-fatal): Comments beginning with # are deprecated. Please use // in place of # for line comments.
Warning (non-fatal): Comments beginning with # are deprecated. Please use // in place of # for line comments.

recompiling to avoid crashing R session
In file included from file144151e196a9a.cpp:8:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:12:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/gevv_vvv_vari.hpp:5:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/var.hpp:7:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/math/tools/config.hpp:13:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config.hpp:39:
/Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config/compiler/clang.hpp:200:11: warning: ‘BOOST_NO_CXX11_RVALUE_REFERENCES’ macro redefined [-Wmacro-redefined]
# define BOOST_NO_CXX11_RVALUE_REFERENCES
^
:6:9: note: previous definition is here
#define BOOST_NO_CXX11_RVALUE_REFERENCES 1
^
1 warning generated.

SAMPLING FOR MODEL ‘birats’ NOW (CHAIN 1).

Gradient evaluation took 0.001639 seconds
1000 transitions using 10 leapfrog steps per transition would take 16.39 seconds.
Adjust your expectations accordingly!

Process 82965 stopped

  • thread #1, queue = ‘com.apple.main-thread’, stop reason = breakpoint 1.3
    frame #0: 0x000000010850db70 libc++abi.dylib__cxa_throw libc++abi.dylib__cxa_throw:
    → 0x10850db70 <+0>: pushq %rbp
    0x10850db71 <+1>: movq %rsp, %rbp
    0x10850db74 <+4>: pushq %r15
    0x10850db76 <+6>: pushq %r14
    (lldb) bt
  • thread #1, queue = ‘com.apple.main-thread’, stop reason = breakpoint 1.3

    • frame #0: 0x000000010850db70 libc++abi.dylib`__cxa_throw

      frame #1: 0x000000011d38335b file144151e196a9a.so`void stan::math::domain_errorstan::math::var(function=“inv_wishart_lpdf”, name=“LDLT_Factor of random variable”, y=0x00007fff5fbec200, msg1=“is not positive definite. last conditional variance is “, msg2=”.”) at domain_error.hpp:36

      frame #2: 0x000000011d387da3 file144151e196a9a.so`void stan::math::check_ldlt_factor<stan::math::var, -1, -1>(function=“inv_wishart_lpdf”, name=“LDLT_Factor of random variable”, A=0x00007fff5fbec628) at check_ldlt_factor.hpp:42

      frame #3: 0x000000011d387399 file144151e196a9a.so`boost::math::tools::promote_args<stan::math::var, int, double, float, float, float>::type stan::math::inv_wishart_lpdf<true, stan::math::var, int, double>(W=0x00007fff5fbecc68, nu=0x00007fff5fbec984, S=0x0000000104191dd8) at inv_wishart_lpdf.hpp:71

      frame #4: 0x000000011d37bd95 file144151e196a9a.so`boost::math::tools::promote_args<stan::math::var, int, double, float, float, float>::type stan::math::inv_wishart_log<true, stan::math::var, int, double>(W=0x00007fff5fbecc68, nu=0x00007fff5fbec984, S=0x0000000104191dd8) at inv_wishart_log.hpp:38

      frame #5: 0x000000011d377db7 file144151e196a9a.so`stan::math::var model144157b4f772d_birats_namespace::model144157b4f772d_birats::log_prob<true, true, stan::math::var>(this=0x0000000104191d70, params_r__=size=66, params_i__=size=0, pstream__=0x00007fff5fbed658) const at file144151e196a9a.cpp:336

      frame #6: 0x000000011d3da1b3 file144151e196a9a.sostan::math::var model144157b4f772d_birats_namespace::model144157b4f772d_birats::log_prob<true, true, stan::math::var>(this=0x0000000104191d70, params_r=0x00007fff5fbed510, pstream=0x00007fff5fbed658) const at file144151e196a9a.cpp:369 frame #7: 0x000000011d3d8d87 file144151e196a9a.sostan::math::var stan::model::model_functional<model144157b4f772d_birats_namespace::model144157b4f772d_birats>::operator(this=0x00007fff5fbed648, x=0x00007fff5fbed510)stan::math::var(Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1> const&) const at model_functional.hpp:23
      frame #8: 0x000000011d3d8717 file144151e196a9a.sovoid stan::math::gradient<stan::model::model_functional<model144157b4f772d_birats_namespace::model144157b4f772d_birats> >(f=0x00007fff5fbed648, x=0x00007fff5fbee408, fx=0x00007fff5fbee428, grad_fx=0x00007fff5fbee430) at gradient.hpp:51 frame #9: 0x000000011d3d6b13 file144151e196a9a.sovoid stan::model::gradient<model144157b4f772d_birats_namespace::model144157b4f772d_birats>(model=0x0000000104191d70, x=0x00007fff5fbee408, f=0x00007fff5fbee428, grad_f=0x00007fff5fbee430, logger=0x00007fff5fbf0740) at gradient.hpp:31
      frame #10: 0x000000011d3fc5ed file144151e196a9a.sostan::mcmc::base_hamiltonian<model144157b4f772d_birats_namespace::model144157b4f772d_birats, stan::mcmc::diag_e_point, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >::update_potential_gradient(this=0x00007fff5fbee458, z=0x00007fff5fbee400, logger=0x00007fff5fbf0740) at base_hamiltonian.hpp:69 frame #11: 0x000000011d3fc413 file144151e196a9a.sostan::mcmc::expl_leapfrog<stan::mcmc::diag_e_metric<model144157b4f772d_birats_namespace::model144157b4f772d_birats, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > > >::update_q(this=0x00007fff5fbee450, z=0x00007fff5fbee400, hamiltonian=0x00007fff5fbee458, epsilon=1, logger=0x00007fff5fbf0740) at expl_leapfrog.hpp:27
      frame #12: 0x000000011d3fc218 file144151e196a9a.sostan::mcmc::base_leapfrog<stan::mcmc::diag_e_metric<model144157b4f772d_birats_namespace::model144157b4f772d_birats, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > > >::evolve(this=0x00007fff5fbee450, z=0x00007fff5fbee400, hamiltonian=0x00007fff5fbee458, epsilon=1, logger=0x00007fff5fbf0740) at base_leapfrog.hpp:24 frame #13: 0x000000011d403209 file144151e196a9a.sostan::mcmc::base_hmc<model144157b4f772d_birats_namespace::model144157b4f772d_birats, stan::mcmc::diag_e_metric, stan::mcmc::expl_leapfrog, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >::init_stepsize(this=0x00007fff5fbee3f8, logger=0x00007fff5fbf0740) at base_hmc.hpp:96
      frame #14: 0x000000011d4015e5 file144151e196a9a.sovoid stan::services::util::run_adaptive_sampler<stan::mcmc::adapt_diag_e_nuts<model144157b4f772d_birats_namespace::model144157b4f772d_birats, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >, model144157b4f772d_birats_namespace::model144157b4f772d_birats, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >(sampler=0x00007fff5fbee3f8, model=0x0000000104191d70, cont_vector=size=66, num_warmup=1000, num_samples=1000, num_thin=1, refresh=200, save_warmup=true, rng=0x00007fff5fbee5e8, interrupt=0x00007fff5fbf0738, logger=0x00007fff5fbf0740, sample_writer=0x0000000104194d90, diagnostic_writer=0x00007fff5fbf05f0) at run_adaptive_sampler.hpp:53 frame #15: 0x000000011d4012a9 file144151e196a9a.soint stan::services::sample::hmc_nuts_diag_e_adapt<model144157b4f772d_birats_namespace::model144157b4f772d_birats>(model=0x0000000104191d70, init=0x00000001041133e0, init_inv_metric=0x00007fff5fbee8d0, random_seed=1020848125, chain=1, init_radius=2, num_warmup=1000, num_samples=1000, num_thin=1, save_warmup=true, refresh=200, stepsize=1, stepsize_jitter=0, max_depth=10, delta=0.80000000000000004, gamma=0.050000000000000003, kappa=0.75, t0=10, init_buffer=75, term_buffer=50, window=25, interrupt=0x00007fff5fbf0738, logger=0x00007fff5fbf0740, init_writer=0x00007fff5fbf0598, sample_writer=0x0000000104194d90, diagnostic_writer=0x00007fff5fbf05f0) at hmc_nuts_diag_e_adapt.hpp:106
      frame #16: 0x000000011d2f3b86 file144151e196a9a.soint stan::services::sample::hmc_nuts_diag_e_adapt<model144157b4f772d_birats_namespace::model144157b4f772d_birats>(model=0x0000000104191d70, init=0x00000001041133e0, random_seed=1020848125, chain=1, init_radius=2, num_warmup=1000, num_samples=1000, num_thin=1, save_warmup=true, refresh=200, stepsize=1, stepsize_jitter=0, max_depth=10, delta=0.80000000000000004, gamma=0.050000000000000003, kappa=0.75, t0=10, init_buffer=75, term_buffer=50, window=25, interrupt=0x00007fff5fbf0738, logger=0x00007fff5fbf0740, init_writer=0x00007fff5fbf0598, sample_writer=0x0000000104194d90, diagnostic_writer=0x00007fff5fbf05f0) at hmc_nuts_diag_e_adapt.hpp:164 frame #17: 0x000000011d2c0c87 file144151e196a9a.soint rstan::(anonymous namespace)::command<model144157b4f772d_birats_namespace::model144157b4f772d_birats, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >(args=0x00007fff5fbf5470, model=0x0000000104191d70, holder=0x00007fff5fbf5460, qoi_idx=size=69, fnames_oi=size=69, base_rng=0x0000000104191df0) at stan_fit.hpp:648
      frame #18: 0x000000011d1a9363 file144151e196a9a.sorstan::stan_fit<model144157b4f772d_birats_namespace::model144157b4f772d_birats, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >::call_sampler(this=0x0000000104191ce0, args_=0x000000011095a670) at stan_fit.hpp:1196 frame #19: 0x000000011d44d67a file144151e196a9a.soRcpp::CppMethod1<rstan::stan_fit<model144157b4f772d_birats_namespace::model144157b4f772d_birats, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >, SEXPREC*, SEXPREC*>::operator(this=0x000000010f902150, object=0x0000000104191ce0, args=0x00007fff5fbf5dd0)(rstan::stan_fit<model144157b4f772d_birats_namespace::model144157b4f772d_birats, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > >, SEXPREC**) at Module_generated_CppMethod.h:111
      frame #20: 0x000000011d261276 file144151e196a9a.soRcpp::class_<rstan::stan_fit<model144157b4f772d_birats_namespace::model144157b4f772d_birats, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> > > >::invoke_notvoid(this=0x000000010f909120, method_xp=0x0000000110bdd3b8, object=0x00000001095ad830, args=0x00007fff5fbf5dd0, nargs=1) at class.h:234 frame #21: 0x000000010839322f Rcpp.soCppMethod__invoke_notvoid(SEXPREC      ) + 255
      frame #22: 0x000000010010a75d libR.dylibdo_External(call=<unavailable>, op=0x000000010183e278, args=0x0000000109575868, env=0x0000000109575830) at dotcode.c:548 [opt] frame #23: 0x000000010013bd09 libR.dylibRf_eval(e=, rho=0x0000000109575830) at eval.c:728 [opt]
      frame #24: 0x000000010014dee0 libR.dylibdo_begin(call=<unavailable>, op=0x0000000101828828, args=0x000000011126dc68, rho=0x0000000109575830) at eval.c:2192 [opt] frame #25: 0x000000010013ba38 libR.dylibRf_eval(e=, rho=0x0000000109575830) at eval.c:700 [opt]
      frame #26: 0x000000010014bd82 libR.dylibR_execClosure(call=0x000000010e39ef90, newrho=0x0000000109575830, sysparent=<unavailable>, rho=<unavailable>, arglist=<unavailable>, op=0x0000000109577f88) at eval.c:1607 [opt] frame #27: 0x000000010013fe44 libR.dylibbcEval(body=, rho=0x0000000116e29870, useCache=) at eval.c:6405 [opt]
      frame #28: 0x000000010013b8d1 libR.dylibRf_eval(e=<unavailable>, rho=<unavailable>) at eval.c:624 [opt] frame #29: 0x0000000100149de9 libR.dylibforcePromise(e=0x0000000109580718) at eval.c:520 [opt]
      frame #30: 0x00000001001551ee libR.dylibgetvar [inlined] FORCE_PROMISE(value=<unavailable>, symbol=<unavailable>, rho=<unavailable>, keepmiss=FALSE) at eval.c:4718 [opt] frame #31: 0x00000001001551e6 libR.dylibgetvar(symbol=0x000000010187b578, rho=0x0000000109580830, dd=, keepmiss=FALSE, vcache=, sidx=) at eval.c:4760 [opt]
      frame #32: 0x000000010013ece0 libR.dylibbcEval(body=<unavailable>, rho=0x0000000109580830, useCache=<unavailable>) at eval.c:6194 [opt] frame #33: 0x000000010013b8d1 libR.dylibRf_eval(e=, rho=) at eval.c:624 [opt]
      frame #34: 0x0000000100149de9 libR.dylibforcePromise(e=0x00000001095808d8) at eval.c:520 [opt] frame #35: 0x00000001001551ee libR.dylibgetvar [inlined] FORCE_PROMISE(value=, symbol=, rho=, keepmiss=FALSE) at eval.c:4718 [opt]
      frame #36: 0x00000001001551e6 libR.dylibgetvar(symbol=0x000000010187b578, rho=0x0000000109580a98, dd=<unavailable>, keepmiss=FALSE, vcache=<unavailable>, sidx=<unavailable>) at eval.c:4760 [opt] frame #37: 0x000000010013ece0 libR.dylibbcEval(body=, rho=0x0000000109580a98, useCache=) at eval.c:6194 [opt]
      frame #38: 0x000000010013b8d1 libR.dylibRf_eval(e=<unavailable>, rho=<unavailable>) at eval.c:624 [opt] frame #39: 0x0000000100149de9 libR.dylibforcePromise(e=0x000000010957ff18) at eval.c:520 [opt]
      frame #40: 0x00000001001551ee libR.dylibgetvar [inlined] FORCE_PROMISE(value=<unavailable>, symbol=<unavailable>, rho=<unavailable>, keepmiss=FALSE) at eval.c:4718 [opt] frame #41: 0x00000001001551e6 libR.dylibgetvar(symbol=0x000000010187b578, rho=0x00000001095801b8, dd=, keepmiss=FALSE, vcache=, sidx=) at eval.c:4760 [opt]
      frame #42: 0x000000010013ece0 libR.dylibbcEval(body=<unavailable>, rho=0x00000001095801b8, useCache=<unavailable>) at eval.c:6194 [opt] frame #43: 0x000000010013b8d1 libR.dylibRf_eval(e=, rho=) at eval.c:624 [opt]
      frame #44: 0x0000000100149de9 libR.dylibforcePromise(e=0x0000000109580260) at eval.c:520 [opt] frame #45: 0x00000001001551ee libR.dylibgetvar [inlined] FORCE_PROMISE(value=, symbol=, rho=, keepmiss=FALSE) at eval.c:4718 [opt]
      frame #46: 0x00000001001551e6 libR.dylibgetvar(symbol=0x000000010187b578, rho=0x000000010957f558, dd=<unavailable>, keepmiss=FALSE, vcache=<unavailable>, sidx=<unavailable>) at eval.c:4760 [opt] frame #47: 0x000000010013ece0 libR.dylibbcEval(body=, rho=0x000000010957f558, useCache=) at eval.c:6194 [opt]
      frame #48: 0x000000010013b8d1 libR.dylibRf_eval(e=<unavailable>, rho=<unavailable>) at eval.c:624 [opt] frame #49: 0x0000000100149de9 libR.dylibforcePromise(e=0x000000010957f600) at eval.c:520 [opt]
      frame #50: 0x00000001001551ee libR.dylibgetvar [inlined] FORCE_PROMISE(value=<unavailable>, symbol=<unavailable>, rho=<unavailable>, keepmiss=FALSE) at eval.c:4718 [opt] frame #51: 0x00000001001551e6 libR.dylibgetvar(symbol=0x000000010187b578, rho=0x000000010957f8a0, dd=, keepmiss=FALSE, vcache=, sidx=) at eval.c:4760 [opt]
      frame #52: 0x000000010013ece0 libR.dylibbcEval(body=<unavailable>, rho=0x000000010957f8a0, useCache=<unavailable>) at eval.c:6194 [opt] frame #53: 0x000000010013b8d1 libR.dylibRf_eval(e=, rho=) at eval.c:624 [opt]
      frame #54: 0x000000010014bd82 libR.dylibR_execClosure(call=0x0000000101b5fa78, newrho=0x000000010957f8a0, sysparent=<unavailable>, rho=<unavailable>, arglist=<unavailable>, op=0x000000010957f590) at eval.c:1607 [opt] frame #55: 0x000000010013fe44 libR.dylibbcEval(body=, rho=0x000000010957f558, useCache=) at eval.c:6405 [opt]
      frame #56: 0x000000010013b8d1 libR.dylibRf_eval(e=<unavailable>, rho=<unavailable>) at eval.c:624 [opt] frame #57: 0x000000010014bd82 libR.dylibR_execClosure(call=0x0000000101b5da78, newrho=0x000000010957f558, sysparent=, rho=, arglist=, op=0x0000000109580b40) at eval.c:1607 [opt]
      frame #58: 0x000000010013fe44 libR.dylibbcEval(body=<unavailable>, rho=0x00000001095801b8, useCache=<unavailable>) at eval.c:6405 [opt] frame #59: 0x000000010013b8d1 libR.dylibRf_eval(e=, rho=) at eval.c:624 [opt]
      frame #60: 0x000000010014bd82 libR.dylibR_execClosure(call=0x0000000101b5d748, newrho=0x00000001095801b8, sysparent=<unavailable>, rho=<unavailable>, arglist=<unavailable>, op=0x0000000109580ad0) at eval.c:1607 [opt] frame #61: 0x000000010013fe44 libR.dylibbcEval(body=, rho=0x0000000109580a98, useCache=) at eval.c:6405 [opt]
      frame #62: 0x000000010013b8d1 libR.dylibRf_eval(e=<unavailable>, rho=<unavailable>) at eval.c:624 [opt] frame #63: 0x000000010014bd82 libR.dylibR_execClosure(call=0x0000000101e22228, newrho=0x0000000109580a98, sysparent=, rho=, arglist=, op=0x0000000101b517c8) at eval.c:1607 [opt]
      frame #64: 0x000000010013fe44 libR.dylibbcEval(body=<unavailable>, rho=0x0000000109580830, useCache=<unavailable>) at eval.c:6405 [opt] frame #65: 0x000000010013b8d1 libR.dylibRf_eval(e=, rho=) at eval.c:624 [opt]
      frame #66: 0x000000010014bd82 libR.dylibR_execClosure(call=0x000000010e39ef20, newrho=0x0000000109580830, sysparent=<unavailable>, rho=<unavailable>, arglist=<unavailable>, op=0x0000000101e18dc8) at eval.c:1607 [opt] frame #67: 0x000000010013fe44 libR.dylibbcEval(body=, rho=0x0000000116e29870, useCache=) at eval.c:6405 [opt]
      frame #68: 0x000000010013b8d1 libR.dylibRf_eval(e=<unavailable>, rho=<unavailable>) at eval.c:624 [opt] frame #69: 0x000000010014bd82 libR.dylibR_execClosure(call=0x0000000101590ce8, newrho=0x0000000116e29870, sysparent=, rho=, arglist=, op=0x000000010106fa38) at eval.c:1607 [opt]
      frame #70: 0x000000010013bbb6 libR.dylibRf_eval(e=0x0000000101590ce8, rho=0x0000000116e2ac38) at eval.c:747 [opt] frame #71: 0x000000010014dee0 libR.dylibdo_begin(call=, op=0x0000000101828828, args=0x0000000101590d20, rho=0x0000000116e2ac38) at eval.c:2192 [opt]
      frame #72: 0x000000010013ba38 libR.dylibRf_eval(e=<unavailable>, rho=0x0000000116e2ac38) at eval.c:700 [opt] frame #73: 0x000000010014bd82 libR.dylibR_execClosure(call=0x0000000116e2b470, newrho=0x0000000116e2ac38, sysparent=, rho=, arglist=, op=0x0000000101071d40) at eval.c:1607 [opt]
      frame #74: 0x000000010014c533 libR.dylibR_execMethod(op=<unavailable>, rho=0x0000000116e2b710) at eval.c:1776 [opt] frame #75: 0x0000000100fd4258 methods.soR_dispatchGeneric + 2360
      frame #76: 0x000000010018e5bf libR.dylibdo_standardGeneric(call=<unavailable>, op=<unavailable>, args=<unavailable>, env=0x0000000116e2b710) at objects.c:1169 [opt] frame #77: 0x000000010013bd51 libR.dylibRf_eval(e=, rho=0x0000000116e2b710) at eval.c:732 [opt]
      frame #78: 0x000000010014dee0 libR.dylibdo_begin(call=<unavailable>, op=0x0000000101828828, args=0x000000010108f2d0, rho=0x0000000116e2b710) at eval.c:2192 [opt] frame #79: 0x000000010013ba38 libR.dylibRf_eval(e=, rho=0x0000000116e2b710) at eval.c:700 [opt]
      frame #80: 0x000000010014bd82 libR.dylibR_execClosure(call=0x0000000116e2b470, newrho=0x0000000116e2b710, sysparent=<unavailable>, rho=<unavailable>, arglist=<unavailable>, op=0x000000010138f8d0) at eval.c:1607 [opt] frame #81: 0x000000010013bbb6 libR.dylibRf_eval(e=0x0000000116e2b470, rho=0x00000001107d0908) at eval.c:747 [opt]
      frame #82: 0x00000001000dd2eb libR.dylibdo_docall(call=<unavailable>, op=<unavailable>, args=0x0000000113a0ee88, rho=<unavailable>) at coerce.c:2506 [opt] frame #83: 0x000000010014026c libR.dylibbcEval(body=, rho=0x0000000116e2c098, useCache=) at eval.c:6434 [opt]
      frame #84: 0x000000010013b8d1 libR.dylibRf_eval(e=<unavailable>, rho=<unavailable>) at eval.c:624 [opt] frame #85: 0x000000010014bd82 libR.dylibR_execClosure(call=0x0000000108c16990, newrho=0x0000000116e2c098, sysparent=, rho=, arglist=, op=0x0000000104f516a0) at eval.c:1607 [opt]
      frame #86: 0x000000010013fe44 libR.dylibbcEval(body=<unavailable>, rho=0x00000001107d0908, useCache=<unavailable>) at eval.c:6405 [opt] frame #87: 0x000000010013b8d1 libR.dylibRf_eval(e=, rho=) at eval.c:624 [opt]
      frame #88: 0x000000010014bd82 libR.dylibR_execClosure(call=0x0000000110f1ba38, newrho=0x00000001107d0908, sysparent=<unavailable>, rho=<unavailable>, arglist=<unavailable>, op=0x00000001057bc520) at eval.c:1607 [opt] frame #89: 0x000000010013bbb6 libR.dylibRf_eval(e=0x0000000110f1ba38, rho=0x0000000101854070) at eval.c:747 [opt]
      frame #90: 0x000000010017be38 libR.dylibRf_ReplIteration(rho=0x0000000101854070, savestack=<unavailable>, browselevel=<unavailable>, state=0x00007fff5fbfe970) at main.c:258 [opt] frame #91: 0x000000010017d32f libR.dylibrun_Rmainloop [inlined] R_ReplConsole(rho=, savestack=0, browselevel=0) at main.c:308 [opt]
      frame #92: 0x000000010017d2c6 libR.dylibrun_Rmainloop at main.c:1059 [opt] frame #93: 0x0000000100000f5b Rmain + 27
      frame #94: 0x00007fffc8479235 libdyld.dylib`start + 1

      (lldb) process continue
      Process 82965 resuming
      [1] “Error in sampler$call_sampler(args_list[[i]]) : "
      [2] " c++ exception (unknown reason)”
      error occurred during calling the sampler; sampling not done

6

Thanks. It looks as if there is a typical C++ exception being thrown due to a matrix not being numerically positive definite and that takes down the whole process, which it shouldn’t. @Bob_Carpenter do you see anything fishy?

7

Well, the problem’s definitely what Ben says it is. This is happening when we do an LDLT factorization and then check that everything solved OK so that the original input was positive definite.

I would assume it’s just the usual instability in inverse Wisharts. We haven’t put a lot of effort into them.

But it still shouldn’t crash in this way—the domain errors should be caught by the sampler. Is the error in the generated quantities block by any chance?

8

This also shows that it went through the initialization iteration OK, so it’s not a problem there, but in some place it gets to later when sampling.

9

Is it possible that it’s the exception handling itself that’s the problem? Or cleaning up after the exception.

The first time I run it, I just get “Error in sampler$call_sampler(args_list[[i]]) : " " c++ exception (unknown reason)”

It’s only on the second and subsequent runs of the Stan model that the error message changes to “empty_nested() must be true before calling recover_memory()”

It doesn’t happen for every model (8schools runs fine, but dugongs and birats has the error)

I don’t get the error in R 3.3.3 with Apple LLVM, only with R 3.4.1 and the Clang 4.0.0 compiler.

10

The exception handling is definitely a problem. That exception should be caught.

Does it happen if you only run a single chain? It may also be some kind of issue with parallelization.

11

Yes, when I ran it in the debugger I used

stan_demo(“birats”, chains = 1)

This describes my setup:

Important: since R 3.4.0 release we are now providing binaries for OS X 10.11 (El Capitan) and higher using non-Apple toolkit to provide support for OpenMP and C++17 standard features. Please read the corresponding note below.

I’m using clang-4.0.0-darwin15.6-Release.tar.gz from here:
http://r.research.att.com/libs/

12

I have a workaround for now, which is to keep using the old (Apple LLVM) toolchain instead of the unofficial Clang 4.0.0 binaries from CRAN. For example, the following local Makevars doesn’t exhibit the error:

CC=clang
CXX=clang++ -arch x86_64 -ftemplate-depth-256

This corresponds to the Xcode version of Clang:

Apple LLVM version 8.1.0 (clang-802.0.42)
13

Can one of the Stan developers with a Mac download R’s version of clang and figure out what the problem is? Coatless has a Bash installation script here:

curl -O http://r.research.att.com/libs/clang-4.0.0-darwin15.6-Release.tar.gz
# Extract binary onto root directory
tar fvxz clang-4.0.0-darwin15.6-Release.tar.gz -C /

# Overwrite the ~/.R/Makevars
cat <<- EOF > ~/.R/Makevars
# The following statements are required to use the clang4 binary
CC=/usr/local/clang4/bin/clang
CXX=/usr/local/clang4/bin/clang++
CXX11=$CXX
CXX14=$CXX
CXX17=$CXX
CXX1X=$CXX
LDFLAGS=-L/usr/local/clang4/lib
# End clang4 inclusion statements
EOF
14

Looks like I cannot put that clang into the root folder without errors. So I just put it at a folder say ~/Downloads/tmp.

But I cannot use this compiler at all.

install.packages(‘rstan’,type=‘source’)
— Please select a CRAN mirror for use in this session —
trying URL ‘https://cloud.r-project.org/src/contrib/rstan_2.16.2.tar.gz’
Content type ‘application/x-gzip’ length 733666 bytes (716 KB)
==================================================
downloaded 716 KB

  • installing source package ‘rstan’ …
    ** package ‘rstan’ successfully unpacked and MD5 sums checked
    ** libs
    /Users/jq/Downloads/tmp/usr/local/clang4/bin/clang++ -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I"…/inst/include" -I"…/inst/include/boost_not_in_BH" -I""/Library/Frameworks/R.framework/Resources/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"" -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -I"/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/3.4/Resources/library/RcppEigen/include" -I"/Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include" -I"/Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include" -I/usr/local/include -fPIC -Wall -g -O2 -c chains.cpp -o chains.o
    In file included from chains.cpp:19:
    In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/mat/fun/mean.hpp:4:
    In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/arr/err/check_nonzero_size.hpp:4:
    In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/prim/scal/err/invalid_argument.hpp:4:
    In file included from /Users/jq/Downloads/tmp/usr/local/clang4/bin/…/include/c++/v1/typeinfo:61:
    In file included from /Users/jq/Downloads/tmp/usr/local/clang4/bin/…/include/c++/v1/exception:82:
    In file included from /Users/jq/Downloads/tmp/usr/local/clang4/bin/…/include/c++/v1/cstdlib:86:
    /Users/jq/Downloads/tmp/usr/local/clang4/bin/…/include/c++/v1/stdlib.h:94:15: fatal error:
    ‘stdlib.h’ file not found
    #include_next <stdlib.h>
    ^~~~~~~~~~
    1 error generated.
    make: *** [chains.o] Error 1
    ERROR: compilation failed for package ‘rstan’
  • removing ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rstan’
  • restoring previous ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/rstan’

The downloaded source packages are in
‘/private/var/folders/60/t4btzd511qd7bpvdj2y1x9d40000gn/T/Rtmpc1BLu0/downloaded_packages’
Warning message:
In install.packages(“rstan”, type = “source”) :
installation of package ‘rstan’ had non-zero exit status

15

Looks like it has to be in the root folder in order to find …/include/c++/v1/stdlib.h. What was the error when you unpacked it?

16

You should untar in /usr/local/ not in the root directory.

$ cd /usr/local/
$ sudo tar xvzf ~/Downloads/clang+llvm-4.0.0-x86_64-apple-darwin.tar.xz

I had another go with the official LLVM release of Clang 4.0.0 on macOS 10.12.6 (Sierra). My Makevars was as follows:

CC=/usr/local/clang+llvm-4.0.0-x86_64-apple-darwin/bin/clang
CXX=/usr/local/clang+llvm-4.0.0-x86_64-apple-darwin/bin/clang++ -arch x86_64 -ftemplate-depth-256
LDFLAGS=-L/usr/local/clang+llvm-4.0.0-x86_64-apple-darwin/lib
CXXFLAGS = -O0 -g

This worked fine for both ‘birats’ and ‘dugongs’, which leads me to believe that the problem is with the unofficial CRAN build of Clang 4.0.0 and not a Stan issue per se.

17

I suspect you are correct but I would like to get something more concrete before raising it with CRAN.

1 Like
18

Can you make a Stan file with

functions {
  int fib(int n);
  int fib(int n) {
    if (n <= 0) reject("n must be positive");
    return n <= 2 ? 1 : fib(n - 1) + fib(n - 2);
  }
}

and then do

rstan::expose_stan_functions("filename")
fib(-1L)

to see if R segfaults on the exception thrown?

19

Or even just call Rcpp::sourceCpp() on a C++ file like this

#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
int throw_exception() {
  std::stringstream errmsg;
  errmsg << "this is an exception";
  throw std::domain_error(errmsg.str());
  return 0;
}
20

The output of the C++ file is as follows:

Rcpp::sourceCpp(‘Dropbox/blog/RcppException.cpp’)
throw_exception()
Error in throw_exception() : c++ exception (unknown reason)

The output of the Stan file was:

fib(-1L)
Error in fib(-1L) : c++ exception (unknown reason)