Compilation error on linux but not on windows

I’m getting an error when I try to compile some stan code on linux. The same linux machine can compile other stan code without a problem, and the problematic code compiles on windows without any problem.

This is the error I get:

Error in compileCode(f, code, language = language, verbose = verbose) : Compilation ERROR, function(s)/method(s) not created! In file included from /usr/local/lib/R/site-library/RcppEigen/include/Eigen/Core:388:0, from /usr/local/lib/R/site-library/RcppEigen/include/Eigen/Dense:1, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/mat/fun/Eigen_NumTraits.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/core/matrix_vari.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/core.hpp:14, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/mat.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/model/model_header.hpp:4, from filec71d4d889a

I saved all the output to this gist

This is the session info for the linux machine

> sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default
BLAS: /usr/lib/openblas-base/libblas.so.3
LAPACK: /usr/lib/libopenblasp-r0.2.19.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=C              LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] bayesplot_1.5.0    tictoc_1.0         rstan_2.17.3       StanHeaders_2.17.2 forcats_0.3.0      stringr_1.3.1     
 [7] dplyr_0.7.5        purrr_0.2.5        readr_1.1.1        tidyr_0.8.1        tibble_1.4.2       ggplot2_2.2.1     
[13] tidyverse_1.2.1    MASS_7.3-50       

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.17               lubridate_1.7.4            lattice_0.20-35            listenv_0.7.0             
 [5] assertthat_0.2.0           digest_0.6.15              psych_1.8.4                R6_2.2.2                  
 [9] cellranger_1.1.0           plyr_1.8.4                 ggridges_0.5.0             stats4_3.5.0              
[13] httr_1.3.1                 pillar_1.2.3               rlang_0.2.1                lazyeval_0.2.1            
[17] curl_3.2                   readxl_1.1.0               rstudioapi_0.7             googleAuthR_0.6.3         
[21] drat_0.1.4                 foreign_0.8-70             munsell_0.5.0              broom_0.4.4               
[25] compiler_3.5.0             googleComputeEngineR_0.2.0 modelr_0.1.2               pkgconfig_2.0.1           
[29] mnormt_1.5-5               globals_0.12.1             openssl_1.0.1              tidyselect_0.2.4          
[33] gridExtra_2.3              codetools_0.2-15           future_1.8.1               crayon_1.3.4              
[37] grid_3.5.0                 nlme_3.1-137               jsonlite_1.5               gtable_0.2.0              
[41] magrittr_1.5               scales_0.5.0               zip_1.0.0                  cli_1.0.0                 
[45] stringi_1.2.3              reshape2_1.4.3             bindrcpp_0.2.2             xml2_1.2.0                
[49] tools_3.5.0                glue_1.2.0                 hms_0.4.2                  parallel_3.5.0            
[53] yaml_2.1.19                inline_0.3.15              colorspace_1.3-2           rvest_0.3.2               
[57] memoise_1.1.0              knitr_1.20                 bindr_0.1.1                haven_1.1.2   

and this is the session info for the windows machine that can compile the same code:

> sessionInfo()
R version 3.4.3 (2017-11-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server >= 2012 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] bindrcpp_0.2       bayesplot_1.4.0    tictoc_1.0         rstan_2.17.2       StanHeaders_2.17.1 forcats_0.3.0      stringr_1.2.0     
 [8] dplyr_0.7.4        purrr_0.2.5        readr_1.1.1        tidyr_0.8.1        tibble_1.4.1       ggplot2_2.2.1      tidyverse_1.2.1   
[15] MASS_7.3-47       

loaded via a namespace (and not attached):
 [1] tidyselect_0.2.4 reshape2_1.4.3   haven_1.1.1      lattice_0.20-35  colorspace_1.3-2 stats4_3.4.3     htmltools_0.3.6  yaml_2.1.16     
 [9] rlang_0.2.1      pillar_1.0.1     foreign_0.8-69   glue_1.2.0       modelr_0.1.2     readxl_1.1.0     bindr_0.1        plyr_1.8.4      
[17] munsell_0.4.3    gtable_0.2.0     cellranger_1.1.0 rvest_0.3.2      codetools_0.2-15 psych_1.8.4      evaluate_0.10.1  labeling_0.3    
[25] inline_0.3.14    knitr_1.18       parallel_3.4.3   broom_0.4.4      Rcpp_0.12.14     backports_1.1.2  scales_0.5.0     jsonlite_1.5    
[33] gridExtra_2.3    mnormt_1.5-5     hms_0.4.2        digest_0.6.13    stringi_1.1.6    grid_3.4.3       rprojroot_1.3-2  cli_1.0.0       
[41] tools_3.4.3      magrittr_1.5     lazyeval_0.2.1   crayon_1.3.4     pkgconfig_2.0.1  rsconnect_0.8.5  xml2_1.2.0       lubridate_1.7.4 
[49] assertthat_0.2.0 rmarkdown_1.10   httr_1.3.1       rstudioapi_0.7   R6_2.2.2         nlme_3.1-131     compiler_3.4.3  

In a previous post I had a similar problem. But this time I don’t see an integer that could be generating that problem. In case it is helful, this is my stan code:

data {
  int<lower = 0, upper = 1> run_estimation;       // a switch to evaluate the likelihood
  int<lower=0> N ;                                // number of data items
  int<lower=0> K ;                                // number of predictors
  matrix[N, K] x ;                                // predictor matrix
  vector[N]    y ;                                // outcome vector
}

transformed data {
  matrix[N, K] Q_ast ;
  matrix[K, K] R_ast ;
  matrix[K, K] R_ast_inverse ;
  real mean_y = mean(y) ;
  real sd_y = sd(y) ;
  vector[N] y_std;
  for (n in 1:N) y_std[n] = (y[n] - mean_y)/sd_y ;
  // thin and scale the QR decomposition
  Q_ast = qr_Q(x)[, 1:K] * sqrt(N - 1) ;
  R_ast = qr_R(x)[1:K, ] / sqrt(N - 1) ;
  R_ast_inverse = inverse(R_ast) ;
}

parameters {
  real alpha_ast ;          // intercept
  vector[K] theta ;     // coefficients on Q_ast
  real<lower=0> sigma_ast ; // error scale
}

model {
  if(run_estimation==1) y_std ~ normal(alpha_ast + Q_ast * theta , sigma_ast) ; // likelihood
  theta ~ normal(0,1) ;  
  alpha_ast ~ normal(0,1) ;
  sigma_ast ~ normal(0,1) ;
}

generated quantities {
  vector[K] beta ;
  vector[N] y_sim ;
  real alpha ;
  real<lower=0> sigma ;
  alpha = alpha_ast*sd_y + mean_y ;
  beta = R_ast_inverse * theta *sd_y ; // coefficients on x
  sigma = sigma_ast*sd_y ;
  for(n in 1:N) y_sim[n] = normal_rng(alpha_ast +  Q_ast[n,] * theta , sigma_ast)  * sd_y + mean_y ;
}

Am I doing something wrong?

Thanks for the help!

The gist does not indicate what the error is so my guess is that you ran out of RAM doing the QR decomposition.

The compilation crashes before i try to fit the model. The command that I’m running is

compiled_model2 <- stan_model("modelQR.stan")

Just to check, i did it while keeping a eye on my ram with htop and it never got close to using all the ram.

What are the values of N and K?

N is 10000, and K 20.

I’m just generating some fake data and trying to recover the parameters with and without the QR decomposition as a learning exercise.

It realized it was going to need way more RAM that was available and gave up.

Both the window machine and the linux machine that i’m using at the moment have 16 GB. But because i’m doing this on a VM is easy to up the ram to 32 GB to test your hypothesis. I did that and I still get the same error. Moreover, at the point I get the error stan does not know what my N and K are. This is all the code that I need to run to get that error:

library(rstan)
compiled_model2 <- stan_model("modelQR.stan")

So no mention of N and K at all. Am I missing something? Thanks for the help!

Ah, try with / sqrt(N - 1.0); in transformed data.

2 Likes

Thanks! I guess this is the same error that I encountered before. Do you know if in the near future stan will be able to give a more informative error message when this happens?

It won’t happen in the near future.

1 Like