Compilation error on linux but not on windows

rstan

#1

I’m getting an error when I try to compile some stan code on linux. The same linux machine can compile other stan code without a problem, and the problematic code compiles on windows without any problem.

This is the error I get:

Error in compileCode(f, code, language = language, verbose = verbose) : Compilation ERROR, function(s)/method(s) not created! In file included from /usr/local/lib/R/site-library/RcppEigen/include/Eigen/Core:388:0, from /usr/local/lib/R/site-library/RcppEigen/include/Eigen/Dense:1, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/mat/fun/Eigen_NumTraits.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/core/matrix_vari.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/core.hpp:14, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/mat.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/stan/math.hpp:4, from /usr/local/lib/R/site-library/StanHeaders/include/src/stan/model/model_header.hpp:4, from filec71d4d889a

I saved all the output to this gist

This is the session info for the linux machine

> sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default
BLAS: /usr/lib/openblas-base/libblas.so.3
LAPACK: /usr/lib/libopenblasp-r0.2.19.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=C              LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] bayesplot_1.5.0    tictoc_1.0         rstan_2.17.3       StanHeaders_2.17.2 forcats_0.3.0      stringr_1.3.1     
 [7] dplyr_0.7.5        purrr_0.2.5        readr_1.1.1        tidyr_0.8.1        tibble_1.4.2       ggplot2_2.2.1     
[13] tidyverse_1.2.1    MASS_7.3-50       

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.17               lubridate_1.7.4            lattice_0.20-35            listenv_0.7.0             
 [5] assertthat_0.2.0           digest_0.6.15              psych_1.8.4                R6_2.2.2                  
 [9] cellranger_1.1.0           plyr_1.8.4                 ggridges_0.5.0             stats4_3.5.0              
[13] httr_1.3.1                 pillar_1.2.3               rlang_0.2.1                lazyeval_0.2.1            
[17] curl_3.2                   readxl_1.1.0               rstudioapi_0.7             googleAuthR_0.6.3         
[21] drat_0.1.4                 foreign_0.8-70             munsell_0.5.0              broom_0.4.4               
[25] compiler_3.5.0             googleComputeEngineR_0.2.0 modelr_0.1.2               pkgconfig_2.0.1           
[29] mnormt_1.5-5               globals_0.12.1             openssl_1.0.1              tidyselect_0.2.4          
[33] gridExtra_2.3              codetools_0.2-15           future_1.8.1               crayon_1.3.4              
[37] grid_3.5.0                 nlme_3.1-137               jsonlite_1.5               gtable_0.2.0              
[41] magrittr_1.5               scales_0.5.0               zip_1.0.0                  cli_1.0.0                 
[45] stringi_1.2.3              reshape2_1.4.3             bindrcpp_0.2.2             xml2_1.2.0                
[49] tools_3.5.0                glue_1.2.0                 hms_0.4.2                  parallel_3.5.0            
[53] yaml_2.1.19                inline_0.3.15              colorspace_1.3-2           rvest_0.3.2               
[57] memoise_1.1.0              knitr_1.20                 bindr_0.1.1                haven_1.1.2   

and this is the session info for the windows machine that can compile the same code:

> sessionInfo()
R version 3.4.3 (2017-11-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server >= 2012 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] bindrcpp_0.2       bayesplot_1.4.0    tictoc_1.0         rstan_2.17.2       StanHeaders_2.17.1 forcats_0.3.0      stringr_1.2.0     
 [8] dplyr_0.7.4        purrr_0.2.5        readr_1.1.1        tidyr_0.8.1        tibble_1.4.1       ggplot2_2.2.1      tidyverse_1.2.1   
[15] MASS_7.3-47       

loaded via a namespace (and not attached):
 [1] tidyselect_0.2.4 reshape2_1.4.3   haven_1.1.1      lattice_0.20-35  colorspace_1.3-2 stats4_3.4.3     htmltools_0.3.6  yaml_2.1.16     
 [9] rlang_0.2.1      pillar_1.0.1     foreign_0.8-69   glue_1.2.0       modelr_0.1.2     readxl_1.1.0     bindr_0.1        plyr_1.8.4      
[17] munsell_0.4.3    gtable_0.2.0     cellranger_1.1.0 rvest_0.3.2      codetools_0.2-15 psych_1.8.4      evaluate_0.10.1  labeling_0.3    
[25] inline_0.3.14    knitr_1.18       parallel_3.4.3   broom_0.4.4      Rcpp_0.12.14     backports_1.1.2  scales_0.5.0     jsonlite_1.5    
[33] gridExtra_2.3    mnormt_1.5-5     hms_0.4.2        digest_0.6.13    stringi_1.1.6    grid_3.4.3       rprojroot_1.3-2  cli_1.0.0       
[41] tools_3.4.3      magrittr_1.5     lazyeval_0.2.1   crayon_1.3.4     pkgconfig_2.0.1  rsconnect_0.8.5  xml2_1.2.0       lubridate_1.7.4 
[49] assertthat_0.2.0 rmarkdown_1.10   httr_1.3.1       rstudioapi_0.7   R6_2.2.2         nlme_3.1-131     compiler_3.4.3  

In a previous post I had a similar problem. But this time I don’t see an integer that could be generating that problem. In case it is helful, this is my stan code:

data {
  int<lower = 0, upper = 1> run_estimation;       // a switch to evaluate the likelihood
  int<lower=0> N ;                                // number of data items
  int<lower=0> K ;                                // number of predictors
  matrix[N, K] x ;                                // predictor matrix
  vector[N]    y ;                                // outcome vector
}

transformed data {
  matrix[N, K] Q_ast ;
  matrix[K, K] R_ast ;
  matrix[K, K] R_ast_inverse ;
  real mean_y = mean(y) ;
  real sd_y = sd(y) ;
  vector[N] y_std;
  for (n in 1:N) y_std[n] = (y[n] - mean_y)/sd_y ;
  // thin and scale the QR decomposition
  Q_ast = qr_Q(x)[, 1:K] * sqrt(N - 1) ;
  R_ast = qr_R(x)[1:K, ] / sqrt(N - 1) ;
  R_ast_inverse = inverse(R_ast) ;
}

parameters {
  real alpha_ast ;          // intercept
  vector[K] theta ;     // coefficients on Q_ast
  real<lower=0> sigma_ast ; // error scale
}

model {
  if(run_estimation==1) y_std ~ normal(alpha_ast + Q_ast * theta , sigma_ast) ; // likelihood
  theta ~ normal(0,1) ;  
  alpha_ast ~ normal(0,1) ;
  sigma_ast ~ normal(0,1) ;
}

generated quantities {
  vector[K] beta ;
  vector[N] y_sim ;
  real alpha ;
  real<lower=0> sigma ;
  alpha = alpha_ast*sd_y + mean_y ;
  beta = R_ast_inverse * theta *sd_y ; // coefficients on x
  sigma = sigma_ast*sd_y ;
  for(n in 1:N) y_sim[n] = normal_rng(alpha_ast +  Q_ast[n,] * theta , sigma_ast)  * sd_y + mean_y ;
}

Am I doing something wrong?

Thanks for the help!


#2

The gist does not indicate what the error is so my guess is that you ran out of RAM doing the QR decomposition.


#3

The compilation crashes before i try to fit the model. The command that I’m running is

compiled_model2 <- stan_model("modelQR.stan")

Just to check, i did it while keeping a eye on my ram with htop and it never got close to using all the ram.


#4

What are the values of N and K?


#5

N is 10000, and K 20.

I’m just generating some fake data and trying to recover the parameters with and without the QR decomposition as a learning exercise.


#6

It realized it was going to need way more RAM that was available and gave up.


#7

Both the window machine and the linux machine that i’m using at the moment have 16 GB. But because i’m doing this on a VM is easy to up the ram to 32 GB to test your hypothesis. I did that and I still get the same error. Moreover, at the point I get the error stan does not know what my N and K are. This is all the code that I need to run to get that error:

library(rstan)
compiled_model2 <- stan_model("modelQR.stan")

So no mention of N and K at all. Am I missing something? Thanks for the help!


#8

Ah, try with / sqrt(N - 1.0); in transformed data.


#9

Thanks! I guess this is the same error that I encountered before. Do you know if in the near future stan will be able to give a more informative error message when this happens?


#10

It won’t happen in the near future.