MacBook M1 problems

Modeling
1

I am having a problem running a model on my MacBook Pro (13-inch, M1, 2020) Big Sur 11.6.
The model runs perfectly on my old Mid 2014 MacBook Pro which is also using Big Sur 11.6.
When I try to run the program on the newer machine it stalls executing the stan command and gives this message:

[1] "Error in sampler$call_sampler(args_list[[i]]) : "               
[2] "  VECTOR_ELT() can only be applied to a 'list', not a 'integer'"
error occurred during calling the sampler; sampling not done

I have tried running the 8 schools example and that is fine. So the problem is some incompatibility between my code and the new machine.

Here is my stan code:

functions {

  real gpoispoint_lpdf(vector y, real mu, real k, real sigma, real u) {
    // generalised Pareto log pdf 
    int N = rows(y);
    real inv_k = inv(k);
    if (k<0 && max(y-mu)/sigma > -inv_k)
      reject("k<0 and max(y-mu)/sigma > -1/k; found k, sigma =", k, sigma);
    if (sigma<=0)
      reject("sigma<=0; found sigma =", sigma);
    if (fabs(k) > 1e-15)
      return -(1+inv_k)*sum(log1p((y-mu) * (k/sigma))) -N*log(sigma) -
      54 * (1 + k * (u - mu) / sigma) ^ -inv_k;
    else
      return -sum(y-mu)/sigma -N*log(sigma) - 
      (54 * exp(u - mu) / sigma); // limit k->0
  }
  
    real gpoispoint_cdf(vector y, real mu, real k, real sigma) {
    //   cdf
    real inv_k = inv(k);
    if (k<0 && max(y-mu)/sigma > -inv_k)
      reject("k<0 and max(y-mu)/sigma > -1/k; found k, sigma =", k, sigma);
    if (sigma<=0)
      reject("sigma<=0; found sigma =", sigma);
    if (fabs(k) > 1e-15)
      return exp(sum(log1m_exp((-inv_k)*(log1p((y-mu) * (k/sigma))))));
    else
      return exp(sum(log1m_exp(-(y-mu)/sigma))); // limit k->0
  }
    real gpoispoint_lcdf(vector y, real mu, real k, real sigma) {
    //  log cdf
    real inv_k = inv(k);
    if (k<0 && max(y-mu)/sigma > -inv_k)
      reject("k<0 and max(y-mu)/sigma > -1/k; found k, sigma =", k, sigma);
    if (sigma<=0)
      reject("sigma<=0; found sigma =", sigma);
    if (fabs(k) > 1e-15)
      return sum(log1m_exp((-inv_k)*(log1p((y-mu) * (k/sigma)))));
    else
      return sum(log1m_exp(-(y-mu)/sigma)); // limit k->0
  }
  
    real gpoispoint_lccdf(vector y, real mu, real k, real sigma) {
    //   log ccdf
    real inv_k = inv(k);
    if (k<0 && max(y-mu)/sigma > -inv_k)
      reject("k<0 and max(y-mu)/sigma > -1/k; found k, sigma =", k, sigma);
    if (sigma<=0)
      reject("sigma<=0; found sigma =", sigma);
    if (fabs(k) > 1e-15)
      return (-inv_k)*sum(log1p((y-mu) * (k/sigma)));
    else
      return -sum(y-mu)/sigma; // limit k->0
  }
    real gpoispoint_rng(real mu, real k, real sigma) {
    //  rng
    if (sigma<=0)
      reject("sigma<=0; found sigma =", sigma);
    if (fabs(k) > 1e-15)
      return mu + (uniform_rng(0,1)^-k -1) * sigma / k;
    else
      return mu - sigma*log(uniform_rng(0,1)); // limit k->0
  }
 
  
}

data {
  int<lower=0> N;
  vector[N] y;
  vector[3] par1;
  vector[3] par2;
  int<lower = 0, upper = 1> type;
 
  real u;
}
parameters {
  real<lower = 0> qtilde1;
  real<lower = 0>  qtilde2;
  real<lower = 0>  qtilde3; 
}
transformed parameters{
  real mu;
  real<lower = 0> nu = qtilde3 / qtilde2;
  real xi = log10(nu); 
  real<lower = 0> sigma;
  
  
  
  sigma  = qtilde2 * xi / (nu * (nu - 1));
  mu = qtilde1 - qtilde2 / nu;
  }
model {
   // priors
   if(type == 0){ 
   qtilde1 ~ gamma(par1[1], par2[1]); 
   qtilde2 ~ gamma(par1[2], par2[2]);
   qtilde3 ~ gamma(par1[3], par2[3]);
   } else {
   qtilde1 ~ lognormal(par1[1], par2[1]); 
   qtilde2 ~ lognormal(par1[2], par2[2]);
   qtilde3 ~ lognormal(par1[3], par2[3]);
   }
 
  target += gpoispoint_lpdf(y | mu, xi, sigma,  u) ;
 
}

And here is my session info:

+       file = stanfile,
+       data = ds,
+       refresh = -1,
+       verbose = FALSE,
+       chains = n_chains,
+       seed = 2,
+       iter = niter,
+       init  = init_ll,
+       control = list(adapt_delta = 0.9 ,
+                      stepsize = 0.1)
+     )
[1] "Error in sampler$call_sampler(args_list[[i]]) : "               
[2] "  VECTOR_ELT() can only be applied to a 'list', not a 'integer'"
error occurred during calling the sampler; sampling not done
> sessionInfo()
R version 4.1.1 (2021-08-10)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Big Sur 11.6

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets 
[6] methods   base     

other attached packages:
 [1] RcppArmadillo_0.10.7.0.0 ggraph_2.0.5            
 [3] shinystan_2.5.0          shiny_1.6.0             
 [5] lubridate_1.7.10         rstan_2.21.2            
 [7] StanHeaders_2.21.0-7     reshape2_1.4.4          
 [9] dplyr_1.0.7              assertive_0.3-6         
[11] ggplot2_3.3.5            evdbayes_1.1-1          
[13] knitr_1.33               euqr1_0.0.1.0           

loaded via a namespace (and not attached):
  [1] matrixStats_0.60.1         xts_0.12.1                
  [3] assertive.models_0.0-2     threejs_0.3.3             
  [5] assertive.datetimes_0.0-3  tools_4.1.1               
  [7] utf8_1.2.2                 R6_2.5.1                  
  [9] DT_0.18                    DBI_1.1.1                 
 [11] colorspace_2.0-2           assertive.data_0.0-3      
 [13] withr_2.4.2                assertive.reflection_0.0-5
 [15] tidyselect_1.1.1           gridExtra_2.3             
 [17] prettyunits_1.1.1          processx_3.5.2            
 [19] curl_4.3.2                 compiler_4.1.1            
 [21] cli_3.0.1                  assertive.properties_0.0-4
 [23] shinyjs_2.0.0              assertive.files_0.0-2     
 [25] colourpicker_1.1.0         scales_1.1.1              
 [27] dygraphs_1.1.1.6           ggridges_0.5.3            
 [29] callr_3.7.0                stringr_1.4.0             
 [31] digest_0.6.27              base64enc_0.1-3           
 [33] assertive.numbers_0.0-2    pkgconfig_2.0.3           
 [35] htmltools_0.5.1.1          fastmap_1.1.0             
 [37] htmlwidgets_1.5.3          rlang_0.4.11              
 [39] rstudioapi_0.13            farver_2.1.0              
 [41] generics_0.1.0             zoo_1.8-9                 
 [43] jsonlite_1.7.2             gtools_3.9.2              
 [45] crosstalk_1.1.1            inline_0.3.19             
 [47] magrittr_2.0.1             loo_2.4.1                 
 [49] bayesplot_1.8.1            Rcpp_1.0.7                
 [51] munsell_0.5.0              fansi_0.5.0               
 [53] viridis_0.6.2              lifecycle_1.0.0           
 [55] stringi_1.7.3              assertive.base_0.0-9      
 [57] MASS_7.3-54                pkgbuild_1.2.0            
 [59] plyr_1.8.6                 grid_4.1.1                
 [61] ggrepel_0.9.1              parallel_4.1.1            
 [63] promises_1.2.0.1           crayon_1.4.1              
 [65] miniUI_0.1.1.1             lattice_0.20-44           
 [67] graphlayouts_0.7.1         assertive.code_0.0-3      
 [69] ps_1.6.0                   pillar_1.6.2              
 [71] igraph_1.2.6               markdown_1.1              
 [73] assertive.sets_0.0-3       codetools_0.2-18          
 [75] stats4_4.1.1               glue_1.4.2                
 [77] evaluate_0.14              V8_3.4.2                  
 [79] RcppParallel_5.1.4         tweenr_1.0.2              
 [81] vctrs_0.3.8                httpuv_1.6.2              
 [83] polyclip_1.10-0            tidyr_1.1.3               
 [85] gtable_0.3.0               purrr_0.3.4               
 [87] assertive.strings_0.0-3    assertthat_0.2.1          
 [89] ggforce_0.3.3              xfun_0.25                 
 [91] mime_0.11                  tidygraph_1.2.0           
 [93] xtable_1.8-4               assertive.types_0.0-3     
 [95] later_1.3.0                assertive.data.uk_0.0-2   
 [97] viridisLite_0.4.0          rsconnect_0.8.24          
 [99] tibble_3.1.3               shinythemes_1.2.0         
[101] assertive.matrices_0.0-2   ellipsis_0.3.2            
[103] assertive.data.us_0.0-2   
> print(sessionInfo())
R version 4.1.1 (2021-08-10)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Big Sur 11.6

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets 
[6] methods   base     

other attached packages:
 [1] RcppArmadillo_0.10.7.0.0 ggraph_2.0.5            
 [3] shinystan_2.5.0          shiny_1.6.0             
 [5] lubridate_1.7.10         rstan_2.21.2            
 [7] StanHeaders_2.21.0-7     reshape2_1.4.4          
 [9] dplyr_1.0.7              assertive_0.3-6         
[11] ggplot2_3.3.5            evdbayes_1.1-1          
[13] knitr_1.33               euqr1_0.0.1.0           

loaded via a namespace (and not attached):
  [1] matrixStats_0.60.1         xts_0.12.1                
  [3] assertive.models_0.0-2     threejs_0.3.3             
  [5] assertive.datetimes_0.0-3  tools_4.1.1               
  [7] utf8_1.2.2                 R6_2.5.1                  
  [9] DT_0.18                    DBI_1.1.1                 
 [11] colorspace_2.0-2           assertive.data_0.0-3      
 [13] withr_2.4.2                assertive.reflection_0.0-5
 [15] tidyselect_1.1.1           gridExtra_2.3             
 [17] prettyunits_1.1.1          processx_3.5.2            
 [19] curl_4.3.2                 compiler_4.1.1            
 [21] cli_3.0.1                  assertive.properties_0.0-4
 [23] shinyjs_2.0.0              assertive.files_0.0-2     
 [25] colourpicker_1.1.0         scales_1.1.1              
 [27] dygraphs_1.1.1.6           ggridges_0.5.3            
 [29] callr_3.7.0                stringr_1.4.0             
 [31] digest_0.6.27              base64enc_0.1-3           
 [33] assertive.numbers_0.0-2    pkgconfig_2.0.3           
 [35] htmltools_0.5.1.1          fastmap_1.1.0             
 [37] htmlwidgets_1.5.3          rlang_0.4.11              
 [39] rstudioapi_0.13            farver_2.1.0              
 [41] generics_0.1.0             zoo_1.8-9                 
 [43] jsonlite_1.7.2             gtools_3.9.2              
 [45] crosstalk_1.1.1            inline_0.3.19             
 [47] magrittr_2.0.1             loo_2.4.1                 
 [49] bayesplot_1.8.1            Rcpp_1.0.7                
 [51] munsell_0.5.0              fansi_0.5.0               
 [53] viridis_0.6.2              lifecycle_1.0.0           
 [55] stringi_1.7.3              assertive.base_0.0-9      
 [57] MASS_7.3-54                pkgbuild_1.2.0            
 [59] plyr_1.8.6                 grid_4.1.1                
 [61] ggrepel_0.9.1              parallel_4.1.1            
 [63] promises_1.2.0.1           crayon_1.4.1              
 [65] miniUI_0.1.1.1             lattice_0.20-44           
 [67] graphlayouts_0.7.1         assertive.code_0.0-3      
 [69] ps_1.6.0                   pillar_1.6.2              
 [71] igraph_1.2.6               markdown_1.1              
 [73] assertive.sets_0.0-3       codetools_0.2-18          
 [75] stats4_4.1.1               glue_1.4.2                
 [77] evaluate_0.14              V8_3.4.2                  
 [79] RcppParallel_5.1.4         tweenr_1.0.2              
 [81] vctrs_0.3.8                httpuv_1.6.2              
 [83] polyclip_1.10-0            tidyr_1.1.3               
 [85] gtable_0.3.0               purrr_0.3.4               
 [87] assertive.strings_0.0-3    assertthat_0.2.1          
 [89] ggforce_0.3.3              xfun_0.25                 
 [91] mime_0.11                  tidygraph_1.2.0           
 [93] xtable_1.8-4               assertive.types_0.0-3     
 [95] later_1.3.0                assertive.data.uk_0.0-2   
 [97] viridisLite_0.4.0          rsconnect_0.8.24          
 [99] tibble_3.1.3               shinythemes_1.2.0         
[101] assertive.matrices_0.0-2   ellipsis_0.3.2            
[103] assertive.data.us_0.0-2

Any help would be appreciated.

1 Like
2

Have just realised that I included some extra code on top of the session info. Sorry!

3

Usually with this error that means your function is being fed an integer instead of a list.

Can you check the data types being fed into your function?

4

Thanks. The function isn’t mine at all. VECTOR_ELT() is buried somewhere in Stan ( I think) so I do not know how to check what data types it is getting.
But the main thing is the identical code works perfectly on my old MacBook.

5

That error pops up when the data being passed to an Rcpp-exposed program (like rstan's implementation) doesn’t match what has been specified. I would guess that this is due to some edge case with Rcpp on M1.

Do you get the same error with the rstan example model or only your own?

example(stan_model,package="rstan",run.dontrun=T)
1 Like
6

That example case appears to run normally, as did 8 schools. I do not quite understand what you mean by " this is due to some edge case with Rcpp on M1." Could you put it another way, please?

7

I meant that Rcpp's implementation of compiling and calling c++ code from R may have some M1-specific issues that are causing this.

The next step is to try to identify which specific part of your model is resulting in this error. To do this, you need to start with the simplest possible version of your model, and then iteratively re-add model code until the error occurs.

For example, you would start with just the data block of your model to see whether the passing of data arguments was causing the error:

data {
  int<lower=0> N;
  vector[N] y;
  vector[3] par1;
  vector[3] par2;
  int<lower = 0, upper = 1> type;
  real u;
}
parameters {
  real y; 
}
model {
  y ~ normal(0, 1);
}

And then iteratively add back in the rest of the model until the error occurs. Also make sure that you’re using the exact same R code to call the model as in the original

1 Like
8

Many thanks. I suspected it would come to something like this. I’ll try now and see what happens.

2 Likes
9

Hi @jgcolman ,
I am having the same error on M1 Macbook - when I delete the control list part, model is running but once I added the control list with adapt_delta it is giving the same error you got. Have you come up with possible solution?

10

Just wanted to say that I’m not sure how to solve these issues with M1 and Rcpp, but an alternative would be to avoid Rcpp altogether and use the new CmdStanR interface instead of RStan to run Stan from R:

1 Like
11

Thanks @jonah , both packages were installed but I did not have any problem with cmdstanr on M1 Macbook just rstan was giving this error. I think I will just stick with cmdstanr.

1 Like
12

Thanks from me too, @Jonah. CmdStanR looks like a more attractive way forward than the line by line investigation proposed earlier - a task which I have kept postponing.

1 Like
13

I have now written and run a test program using CmdStanR as proposed. It operates smoothly and very fast. So, first, thank you.
But I do have a supplementary question. I will wish to incorporate my work in an R package and I have, naturally, read the two vignettes:

It is not clear to me how much of this material remains valid if one is working through CmdStanR rather than rstan.
Can you advise?

14

I should have noticed before now that your experience explains why I could run some Stan programs and not others: the ones that would not run were my own with a control list. As for a solution, I am following @Jonah and using the CmdStanR interface.