Error in stanc - c++ exception (unknown reason)


#1

[Please include Stan program and accompanying data if possible]

Hello,

I’m fairly new to Stan and I’m trying to estimate a factor stochastic volatility model , but I get the following error message

> fit <- stan(file = "2DimFactor.stan",data = dat,chains = 1)
Error in stanc(file = file, model_code = model_code, model_name = model_name,  : 
  c++ exception (unknown reason)

I don’t know how to interpret this error. Is there something wrong with my code, is it a problem with the compiler?

I have seen this error before in other questions but I have not managed to find the solution to my problem.

Stan code for my model:

//2 dim factor model

data {
  int<lower=0> n; // Sample size
  int<lower=0> p; // Number of time series
  matrix[p,n] y;  // data
}

parameters {
  vector[n] h_std; //standardized factors
  matrix[p,n] x_std; //standardized idiosyncratic processes
  vector[p] beta; //loading matrix (vector in this case)
  int<lower=-1,upper=1> phi_h; 
  int<lower=0> sigma_h; 
  vector<lower=-1,upper=1>[p] phi_x; 
  vector[p] mu_x; 
  vector<lower = 0>[p] sigma_x; 
}

transformed parameters {
  matrix[p,n] x; //indiosyncratic processes
  vector[n] h;   // factor volatility


  //Scale with standard deviation
  h = h_std * sigma_h;
  h[1] = h[1]/sqrt(1 - square(phi_h));
  for (t in 2:n)
    h[t] = h[t] + phi_h*h[t-1];
}

  
  //Scale with standard deviation
  for(t in 1:p){
    for(i in 1:n)
    x[t,] = x[t,]*sigma_x[t];
    x[t,] = x[t,] + mu_x[t];
    x[t,1] = x[t,1]/sqrt(1 - square(phi_x[t]));
  }
  for(t in 2:n){
    for(i in 1:p){
      x[i,t] = x[i,t] + phi_x(i)*(x[i,t-1] - mu_x(i));
    }
  }

}

model {
  //Prior parameters 
  int B_beta = 1; 
  int b_mu = 0; 
  int B_mu = 100; 
  int a_0 = 20; 
  int b_0 = 1.5; 
  int B_sigma = 1;
  
  //Covariance matrix for observations as function of parameters
  matrix[p,p] Sigma;

  //Prior sigma_h
  target += log(2*sigma_h) + gamma_lpdf(square(sigma_h),0.5,1/(2*B_sigma)); //density of sigma if sigma^2 is gamma(0.5,0.5)

  //Prior phi_h
  target += log(2) + beta_lpdf(2*phi_h - 1,a0,b0); // density of phi if (1 + phi)/2 is beta(a0,b0)

  for(i in 1:p){
  target += log(2*sigma_x[i]) + gamma_lpdf(square(sigma_x[i]),0.5,1/(2*B_sigma)); //prior sigma_x if sigma^2 is gamma(0.5,0.5)
  target += log(2) + beta_lpdf(2*phi_x[i]-1,a0,b0); //prior phi_h
  mu_x[i] ~ normal(b_mu,B_mu); //prior mu
  }
  
 beta ~ normal(B_beta,1); 
  
  
  h_std ~ normal(0,1);
  x_std[1,] ~ normal(0,1); 
  x_std[2,] ~ normal(0,1); 
  
  //Constribution from observations
  for(i in 1:n){
    Sigma[1,1] = square(beta[1])*exp(h[i]) + exp(x[1,i]); 
    Sigma[1,2] = beta[1]*beta[2]*exp(h[i]); 
    Sigma[2,2] = square(beta[2])*exp(h[i]) + exp(x[2,i]); 
    Sigma[2,1] = Sigma[1,2]; 
    y[,i] ~ multi_normal(0,Sigma);
  }
}

If it helps, this is what my Makevars file looks like:

# The following statements are required to use the clang4 binary
#CC=clang
#CXX=clang++
#CXX=g++-mp-4.8 -arch x86_64
#CC=gcc-mp-4.8
CC=clang
CXX=clang++
# End clan4 inclusion statements

Any tips are much appreciated!

Thanks,

Jens


#2

You cannot declare integer parameters in Stan.


#3

Thank you, that was a blunder, but it did not fix the problem with the error message. Does this error message come from an error in the program or from somewhere else? In any case, it does not give any information about where the error occurs.


#4

I too have suddenly got this major problem of uninformative error messages. My version information is as follows:
Rstan: 2.16.2
Rstudio : Version 1.0.153
R itself.

platform       x86_64-apple-darwin15.6.0   
arch           x86_64                      
os             darwin15.6.0                
system         x86_64, darwin15.6.0        
status                                     
major          3                           
minor          4.1                         
year           2017                        
month          06                          
day            30                          
svn rev        72865                       
language       R                           
version.string R version 3.4.1 (2017-06-30)
nickname       Single Candle     

#5

Update: have just moved to R 3.4.2: still get this problem. Can anyone explain what is going on?


#6

Yes: The R binary that comes from CRAN is compiled with a version of clang that does not handle exceptions properly.

It should work if either you use the clang from Xcode to compile your models or you install RStan from the develop branch on GitHub or you compile R yourself. But it has been known to not work when despite doing things that should work.


#7

Thank you for responding so quickly. It is reassuring to know that it is a real and known problem and not just me.
As a mere applied statistician, however, I shall have to try to work out what your suggested approaches mean and then apply them, but I am sorry to say that I have never heard of “clang”, “Xcode” or “the develop branch on Github”. I have no idea how to use the clang to compile my models or how to compile R myself.
If you or somebody to point me to a source of guidance on these things I should be everlastingly grateful.


#8

Thank you for all the tips!

I have now tried to install RStan from the develop branch from,

https://github.com/stan-dev/rstan/blob/develop/rstan/install.R

but for some reason this did not work. I got the error message:

Warning: unable to access index for repository http://rstan.org/repo/src/contrib:
  cannot open URL 'http://rstan.org/repo/src/contrib/PACKAGES'

I have also tried to follow the instructions on

https://github.com/stan-dev/rstan/wiki/Installing-RStan-on-Mac-or-Linux

but without luck.

Maybe someone can show me what their Makevars file looks like?
This is what mine looks like:

# The following statements are required to use the clang4 binary

CXX=g++ -arch x86_64 -ftemplate-depth-256 -stdlib=libstdc++
CXXFLAGS="-mtune=native  -O3 -Wall -pedantic -Wconversion
#CC=clang
#CXX=clang++
#CXX=g++-mp-4.8 -arch x86_64
#CC=gcc-mp-4.8
#SHLIB_OPENMP_CXXFLAGS= -fopenmp
#CC = /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
#CCX = /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
# End clan4 inclusion statements

The one I’m trying now is the one from,

https://groups.google.com/forum/#!topic/stan-dev/4Z4gMExWwbQ

but I thought

#CC=clang
#CXX=clang++

was the correct one.


#9

clang is the C++ compiler on a Mac. First, make sure if you have the devtools R package installed with

library(devtools)

If that fails, then do

install.packages("devtools")

Then do

devtools::install_github("stan-dev/rstan", ref = "develop",
                         subdir = "rstan/rstan")

Then try to do your Stan model again.


#10

I got things working now. I don’t know why it’s working now, but all I did was making my makevars file look like this:

CC=clang
CXX=clang++

and reinstall the packages by running

install.packages(c("Rcpp", "rstan"), type = "source")

from

https://github.com/stan-dev/rstan/wiki/Installing-RStan-on-Mac-or-Linux

Thank you very much @bgoodri for taking you time answering these basic questions!


#11

Thanks, Ben
That seems to have worked: I now get informative error messages. I am most
grateful for your help.
Jeremy


#12

Yes, thanks Ben!

We’ve been losing a lot of time on the project to vagaries of compiler compatibilities and configurations often beyond our control. We’re trying to stay abreast, but it’s a battle. Ben somehow knows a ridiculous amount about the differences between compilers often before they’re even officially released.


#13

Oh dear! I have had a few weeks of not using Stan and with my first attempt today the problem of non-informative error messages has returned, exactly as described before. So if I run:

parameters{
  vector[10] alpha;
}
model{
  alpha ~ normal(0, 1)
}

I get:

Error in stanc(model_code = paste(program, collapse = "\n"), model_name = model_cppname,  : 
  c++ exception (unknown reason)

And if I try to do what worked last time, namely

devtools::install_github("stan-dev/rstan", ref = "develop",
                         subdir = "rstan/rstan")

I get:

* installing *source* package ‘rstan’ ...
** libs
Error: package ‘StanHeaders’ 2.17.1 was found, but >= 2.17.2 is required by ‘rstan’
* removing ‘/Users/Jeremy1/Rlibs/rstan’
* restoring previous ‘/Users/Jeremy1/Rlibs/rstan’
Installation failed: Command failed (1)
Warning message:
GitHub repo contains submodules, may not function as expected! 

And when I try to find StanHeaders 2.17.2 CRAN says 2.17.1 is the current version.

Can anyone help, please?


#14

You have to get the right StanHeaders too, which is a bit difficult. Try

source("https://raw.githubusercontent.com/stan-dev/rstan/develop/install_StanHeaders.R",
       echo = TRUE)

#15

Thanks for such a quick response. I have tried that, apparently with success but when I then try to download rstan, R studio enters a very long loop giving out many verbose messages, ending with:

1 warning generated.
clang++ -arch x86_64 -ftemplate-depth-256 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I"../inst/include" -I"../inst/include/boost_not_in_BH" -I"`"/Library/Frameworks/R.framework/Resources/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/Users/Jeremy1/Rlibs/Rcpp/include" -I"/Users/Jeremy1/Rlibs/RcppEigen/include" -I"/Users/Jeremy1/Rlibs/BH/include" -I"/Users/Jeremy1/Rlibs/StanHeaders/include" -I/usr/local/include   -fPIC  -O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -Wno-ignored-attributes -Wno-deprecated-declarations -c misc.cpp -o misc.o
clang++ -arch x86_64 -ftemplate-depth-256 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I"../inst/include" -I"../inst/include/boost_not_in_BH" -I"`"/Library/Frameworks/R.framework/Resources/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/Users/Jeremy1/Rlibs/Rcpp/include" -I"/Users/Jeremy1/Rlibs/RcppEigen/include" -I"/Users/Jeremy1/Rlibs/BH/include" -I"/Users/Jeremy1/Rlibs/StanHeaders/include" -I/usr/local/include   -fPIC  -O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -Wno-ignored-attributes -Wno-deprecated-declarations -c sparse_extractors.cpp -o sparse_extractors.o
clang++ -arch x86_64 -ftemplate-depth-256 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I"../inst/include" -I"../inst/include/boost_not_in_BH" -I"`"/Library/Frameworks/R.framework/Resources/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -DFUSION_MAX_VECTOR_SIZE=12 -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/Users/Jeremy1/Rlibs/Rcpp/include" -I"/Users/Jeremy1/Rlibs/RcppEigen/include" -I"/Users/Jeremy1/Rlibs/BH/include" -I"/Users/Jeremy1/Rlibs/StanHeaders/include" -I/usr/local/include   -fPIC  -O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -Wno-ignored-attributes -Wno-deprecated-declarations -c stanc.cpp -o stanc.o
In file included from stanc.cpp:22:
In file included from /Users/Jeremy1/Rlibs/StanHeaders/include/src/stan/lang/compiler.hpp:7:
In file included from /Users/Jeremy1/Rlibs/StanHeaders/include/src/stan/lang/parser.hpp:5:
In file included from /Users/Jeremy1/Rlibs/StanHeaders/include/src/stan/lang/grammars/program_grammar.hpp:6:
In file included from /Users/Jeremy1/Rlibs/StanHeaders/include/src/stan/lang/grammars/whitespace_grammar.hpp:4:
In file included from /Users/Jeremy1/Rlibs/BH/include/boost/spirit/include/qi.hpp:16:
In file included from /Users/Jeremy1/Rlibs/BH/include/boost/spirit/home/qi.hpp:14:
In file included from /Users/Jeremy1/Rlibs/BH/include/boost/spirit/home/qi/action.hpp:14:
In file included from /Users/Jeremy1/Rlibs/BH/include/boost/spirit/home/qi/action/action.hpp:14:
In file included from /Users/Jeremy1/Rlibs/BH/include/boost/spirit/home/qi/meta_compiler.hpp:14:
In file included from /Users/Jeremy1/Rlibs/BH/include/boost/spirit/home/support/meta_compiler.hpp:16:
In file included from /Users/Jeremy1/Rlibs/BH/include/boost/spirit/include/phoenix_limits.hpp:11:
/Users/Jeremy1/Rlibs/BH/include/boost/phoenix/core/limits.hpp:44:10: warning: 'BOOST_PHOENIX_NO_VARIADIC_EXPRESSION' macro redefined [-Wmacro-redefined]
# define BOOST_PHOENIX_NO_VARIADIC_EXPRESSION
         ^
<command line>:6:9: note: previous definition is here
#define BOOST_PHOENIX_NO_VARIADIC_EXPRESSION 1
        ^
In file included from stanc.cpp:23:
In file included from /Users/Jeremy1/Rlibs/StanHeaders/include/src/stan/lang/compile_functions.hpp:4:
In file included from /Users/Jeremy1/Rlibs/StanHeaders/include/src/stan/lang/generator/generate_standalone_functions.hpp:7:
In file included from /Users/Jeremy1/Rlibs/StanHeaders/include/src/stan/lang/generator/generate_function_instantiations.hpp:5:
In file included from /Users/Jeremy1/Rlibs/StanHeaders/include/src/stan/lang/generator/generate_function_instantiation.hpp:6:
/Users/Jeremy1/Rlibs/StanHeaders/include/src/stan/lang/generator/generate_function_instantiation_body.hpp:44:43: warning: range-based for loop is a C++11 extension [-Wc++11-extensions]
      for (const std::string& namespace_i : namespaces) {
                                          ^
2 warnings generated.
clang++ -arch x86_64 -ftemplate-depth-256 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o rstan.so chains.o init.o lang__ast_def.o lang__grammars__bare_type_grammar_inst.o lang__grammars__expression07_grammar_inst.o lang__grammars__expression_grammar_inst.o lang__grammars__functions_grammar_inst.o lang__grammars__indexes_grammar_inst.o lang__grammars__program_grammar_inst.o lang__grammars__semantic_actions.o lang__grammars__statement_2_grammar_inst.o lang__grammars__statement_grammar_inst.o lang__grammars__term_grammar_inst.o lang__grammars__var_deccls_grammar_inst.o lang__grammars__whitespace_grammar_inst.o misc.o sparse_extractors.o stanc.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Users/Jeremy1/Rlibs/rstan/libs
** R
** inst
** tests
** preparing package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (rstan)
Warning message:
GitHub repo contains submodules, may not function as expected! 

and when I try the minimal reproducible example I am back with the non-informative error message again.
I do now appear to have StanHeaders 2.17.2, however.

If there is any more information that anyone would like about my setup, please ask.

It would be very helpful to have some advice on what I should now do.


#16

Problem solved although I am not sure how. The last step I took was to run:

install.packages(c("Rcpp", "rstan"), type = "source")

That took several minutes and caused a great flow of to me incomprehensible messages, but the main point is I now know what was wrong with my latest Stan program.


#17

Reinstalling rstan from source is the only way to currently fix the issue of missing error messages during initialization. The rstan devs are working on a fix.