“Alrighty” sounds like progress, this is good.
readLines("~/.Renviron")
Error in file(con, “r”) : cannot open the connection
In addition: Warning message:
In file(con, “r”) :
cannot open file ‘/Users/jdsal/.Renviron’: No such file or directory
Thanks much, hugely appreciate the help.
Unfortunately that’s the extent of my knowledge with macs sorry! I believe @jonah is on Mac, so he might be able to get you over the line here.
Sorry that you can’t get started with Stan just yet!
So if I understand correctly:
and running the schools example model just hangs for ~6 hrs?
That’s pretty strange! I don’t think I’ve seen that before, although I’ve seen a few people report issues with Mojave. One thing to try is reinstalling Rcpp from source and then, if necessary, also reinstalling RStan from source again after installing Rcpp from source:
install.packages("Rcpp", type = "source")
# try running model again, if same issue then
install.packages("rstan", type = "source")
And if that gets it working then I recommend creating a Makevars file:
# create the file
file.create(file.path(Sys.getenv("HOME"), ".R", "Makevars"))
# open the file for editing
file.edit(file.path(Sys.getenv("HOME"), ".R", "Makevars"))
add these lines
CXX14FLAGS=-O3 -march=native -mtune=native
CXX14FLAGS += -arch x86_64 -ftemplate-depth-256
That should speed up your models (once you get them working).
Jonah, hi there. Thanks for coming on board the thread.
The fresh install of Rcpp from source, then trying the model, yielded the same result as before- the model appears stuck. And, the fresh install of Rcpp then rstan from source, yielded the same as well.
A couple points maybe relevant. First, I’m experiencing these identical problems on two machines that I’m troubleshooting in parallel, both with new R 4.0.2 installed, one running OSX 10.13.6 High Sierra and the other 10.14.6 Mojave.
Second, perhaps relevant, even after quitting out of the R console, the processes continue to run in the background, or at least separate R processes continue to suck up CPU space shown on the system activity monitor.
Yeah, we used to have a lot of trouble with the newer OS X Catalina, but now it’s the opposite and Catalina seems to be fine and the older Mac OSes seem to result in more issues. Often installing Rcpp and RStan from source fixes the issue, but if that didn’t fix it for you then I’m not sure what else to try. If we can’t figure this out soon but you want to run Stan models from R then another option is to try the new CmdStanR interface.
But @bgoodri might have more suggestions. Ben have you seen this before? Any ideas? Here’s a summary:
And installing Rcpp and then RStan from source didn’t fix the problem.
Maybe
remotes::install_github("bgoodri/inline")
and passing verbose = TRUE to stan or sampling will reveal something about how it was (mis)compiled. Look for the part that says “make would call …”.
Hey, thanks for chiming in! The below model has been ‘running’ and appears stuck, as before. In the interest of being verbose, I’m including the whole process.
> remotes::install_github("bgoodri/inline")
Downloading GitHub repo bgoodri/inline@HEAD
─ preparing ‘inline’:/private/var/folders/h1/rq6w0dz12pbc3lpb129lhlhh0000gn/T/Rtmp6yGZDy/remotes16ec822742797/bgoodri-inline-f80c7f9/DESCRIPTION’ ...
─ installing the package to process help pages
es
* installing *source* package ‘inline’ ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (inline)
> library(rstan)
Loading required package: StanHeaders
Loading required package: ggplot2
rstan (Version 2.21.2, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores()).
To avoid recompilation of unchanged Stan programs, we recommend calling
rstan_options(auto_write = TRUE)
>
> model <-
+ "data {
+ int<lower=0> J;
+ real y[J];
+ real<lower=0> sigma[J];
+ }
+ parameters {
+ real mu;
+ real<lower=0> tau;
+ vector[J] eta;
+ }
+ transformed parameters {
+ vector[J] theta = mu + tau * eta;
+ }
+ model {
+ target += normal_lpdf(eta | 0, 1);
+ target += normal_lpdf(y | theta, sigma);
+ }"
>
> schools_dat <- list(J = 8,
+ y = c(28, 8, -3, 7, -1, 1, 18, 12),
+ sigma = c(15, 10, 16, 11, 9, 11, 10, 18))
> fit <- stan(model_code=model, data = schools_dat, verbose=T)
TRANSLATING MODEL 'c6e5b88b131a8ef63e73c683a2589167' FROM Stan CODE TO C++ CODE NOW.
successful in parsing the Stan model 'c6e5b88b131a8ef63e73c683a2589167'.
Try first doing
rstan_options(javascript = FALSE)
Interesting. First running
rstan_options(javascript = FALSE)
Then the model
> fit <- stan(model_code=model, data = schools_dat, verbose=T)
causes the same stuck process, but this time I get an Apple spinning colorwheel and cannot Esc out of the process, which I could do before.
And
example(cxxfunction, package = "inline", run.dontrun = TRUE)
works?
No, still the same.
> rstan_options(javascript = FALSE)
>
> example(cxxfunction, package = "inline", run.dontrun = TRUE)
cxxfnc> # default plugin
cxxfnc> fx <- cxxfunction( signature(x = "integer", y = "numeric" ) , '
cxxfnc+ return ScalarReal( INTEGER(x)[0] * REAL(y)[0] ) ;
cxxfnc+ ' )
And even after quitting out of the console, activity monitor still shows an R process running in the background.
OK. This is probably an issue with your toolchain, rather than something specific to Stan. @coatless might have previously encountered someone with this hanging under
R 4.0.2
Mac OS X Mojave or High Sierra
No Makevars file
No Renviron file
Fresh install of the latest RStanOdd, not sure why RStan is hanging in this instance.
Few quick questions:
Rcpp::evalCpp("1 + 1", verbose = TRUE)
sessionInfo()
cat(readLines(file.path(Sys.getenv("HOME"), ".R", "Makevars")), sep = "\n")
update.packages(ask = FALSE, checkBuilt = TRUE)
Another consideration, but maybe upgrade from macOS Mojave 10.14.6 to macOS Catalina 10.15?
Thanks much. I’m reporting outputs here from a machine running Mohave, but I get the same behavior on a different machine I just upgraded to Catalina (and did a fresh R and RStan install on).
> Rcpp::evalCpp("1 + 1", verbose = TRUE)
Generated code for function definition:
--------------------------------------------------------
#include <Rcpp.h>
using namespace Rcpp;
// [[Rcpp::export]]
SEXP get_value(){ return wrap( 1 + 1 ) ; }
Generated extern "C" functions
--------------------------------------------------------
#include <Rcpp.h>
// get_value
SEXP get_value();
RcppExport SEXP sourceCpp_1_get_value() {
BEGIN_RCPP
Rcpp::RObject rcpp_result_gen;
Rcpp::RNGScope rcpp_rngScope_gen;
rcpp_result_gen = Rcpp::wrap(get_value());
return rcpp_result_gen;
END_RCPP
}
Generated R functions
-------------------------------------------------------
`.sourceCpp_1_DLLInfo` <- dyn.load('/private/var/folders/_t/y8rbmtzj4jz0qxpl_bt_wqlw0000gp/T/Rtmpf0xuzk/sourceCpp-x86_64-apple-darwin17.0-1.0.5/sourcecpp_131e1414200d5/sourceCpp_2.so')
get_value <- Rcpp:::sourceCppFunction(function() {}, FALSE, `.sourceCpp_1_DLLInfo`, 'sourceCpp_1_get_value')
rm(`.sourceCpp_1_DLLInfo`)
Building shared library
--------------------------------------------------------
DIR: /private/var/folders/_t/y8rbmtzj4jz0qxpl_bt_wqlw0000gp/T/Rtmpf0xuzk/sourceCpp-x86_64-apple-darwin17.0-1.0.5/sourcecpp_131e1414200d5
/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB -o 'sourceCpp_2.so' 'file131e138e8e1d3.cpp'
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include" -I"/private/var/folders/_t/y8rbmtzj4jz0qxpl_bt_wqlw0000gp/T/Rtmpf0xuzk/sourceCpp-x86_64-apple-darwin17.0-1.0.5" -I/usr/local/include -fPIC -Wall -g -O2 -c file131e138e8e1d3.cpp -o file131e138e8e1d3.o
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o sourceCpp_2.so file131e138e8e1d3.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
[1] 2
>
And,
> sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Mojave 10.14.6
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.0.2 tools_4.0.2 Rcpp_1.0.5
>
And,
> cat(readLines(file.path(Sys.getenv("HOME"), ".R", "Makevars")), sep = "\n")
Error in file(con, "r") : cannot open the connection
In addition: Warning message:
In file(con, "r") :
cannot open file '/Users/jdsal/.R/Makevars': No such file or directory
>
Then following
update.packages(ask = FALSE, checkBuilt = TRUE)
I get the same hanging behavior as described above if I run the schools example.
I’m not sure what’s going on exactly.
My suggestion is double check whether Xcode CLI is up-to-date. I think a new update was released recently.
Try to update with Software Update in System Preferences or open up Terminal and type:
softwareupdate --all --install --force
If that doesn’t provide an option to upgrade Xcode CLI, please remove the old version and update to a newer one:
sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install
I really appreciate the help. I updated to Catalina (10.15.6). I followed this Reinstall compiler tools for MacOS. hellowworld.cpp worked, but the schools example caused the same hanging.
Following https://discourse.mc-stan.org/t/rstan-compilation-hangs/15150 I did from-scratch install of XCode CLI, R, and RStan. Still no fix.
Hi folks. I gave up on this thread back in September, without solving the hanging problem. Ended up reverting to R v3.6x to get work done.
I recently upgraded to R 4.1.0 for a few reasons, and with fresh updates and installs of Rccp, RStan, and CLI. Hopefully revisiting this again someone can help. Here’s the session info:
R version 4.1.0 (2021-05-18)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.6
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRblas.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base
other attached packages:
[1] dplyr_1.0.6 plyr_1.8.6 rethinking_2.13 rstan_2.21.2 ggplot2_3.3.3 StanHeaders_2.21.0-7
loaded via a namespace (and not attached):
[1] Rcpp_1.0.6 pillar_1.6.1 compiler_4.1.0 prettyunits_1.1.1 tools_4.1.0 pkgbuild_1.2.0 lattice_0.20-44 jsonlite_1.7.2 lifecycle_1.0.0
[10] tibble_3.1.2 gtable_0.3.0 pkgconfig_2.0.3 rlang_0.4.11 cli_2.5.0 curl_4.3.1 mvtnorm_1.1-1 loo_2.4.1 coda_0.19-4
[19] gridExtra_2.3 withr_2.4.2 generics_0.1.0 vctrs_0.3.8 stats4_4.1.0 grid_4.1.0 tidyselect_1.1.1 glue_1.4.2 inline_0.3.19
[28] R6_2.5.0 processx_3.5.2 fansi_0.5.0 purrr_0.3.4 callr_3.7.0 magrittr_2.0.1 MASS_7.3-54 codetools_0.2-18 matrixStats_0.59.0
[37] scales_1.1.1 ps_1.6.0 ellipsis_0.3.2 shape_1.4.6 colorspace_2.0-1 V8_3.4.2 utf8_1.2.1 RcppParallel_5.1.4 munsell_0.5.0
[46] crayon_1.4.1
>
Can you try the preview of the next version of rstan to see if the problem is still there?
Restart R,making sure that rstan doesn’t get loaded and run:
remove.packages(c("StanHeaders", "rstan"))
install.packages("StanHeaders", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
This worked! Andrew, you just fixed a 10-month problem, you are amazing.
thanks!!