Suggestions for OS X install wiki

[mike-lawrence]

The wiki currently suggests installing the command line tools by installing Xcode, a 5GB download & 9GB install size. I suggest it instead suggest simply running xcode-select --install in a terminal, which installs simply the command line tools alone.

Further, I think some of the packages on which RStan depends require things like openssl, curl, etc, in which case I think we should be recommending folks use the mature/popular homebrew package management system.

Indeed, a simple shell script to install everything from scratch would be:

#!/bin/sh

#install homebrew (automatically installs command-line tools)
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

#install R
brew cask install r-app

#alternative for longer install but possibly-faster-r performance:
#brew install openblas
#brew install r --with-openblas

#install RStudio
brew cask install rstudio

#install rstan 
echo '
dotR <- file.path(Sys.getenv("HOME"), ".R") 
if (!file.exists(dotR)) dir.create(dotR) 
M <- file.path(dotR, "Makevars") 
if (!file.exists(M)) file.create(M) 
cat( 
  "\nCXXFLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function  -Wno-macro-redefined", 
  "\nCC=clang", 
  "CXX=clang++ -arch x86_64 -ftemplate-depth-256", 
    file = M, sep = "\n", append = TRUE) 
Sys.setenv(MAKEFLAGS = paste0("-j",parallel::detectCores()/2)) 
install.packages("rstan", repos = "https://cloud.r-project.org/", dependencies=TRUE) 
' > /tmp/temp.r
Rscript /tmp/temp.r

Also, the bit after the #install rstan comment could even be made into a homebrew formula so that users could simply do brew install rstan.

[Bob_Carpenter]

This one’s up to @bgoodri. Xcode isn’t getting any smaller or easier to install.

Could this perhaps go in as an alternative? I don’t trust long shell scripts to keep working over time without someone committed to maintaining them.

[bgoodri]

I don’t have a strong opinion about what people who have Macs should use. But now that the bug with exceptions has been worked around for Stan models compiled with clang4, maybe we should just tell people o install that?

[mitzimorris]

I tried installing clang4 and on my mac (High Sierra) and couldn’t get it to work, FWIW - there’s not enough information on what needs to be in the Makevars to get them to work with clang4 - I tried a bunch of stuff (like a monkey, sure, but it’s the done thing) and none of it worked.

[mitzimorris]

similar less that great experiences with homebrew - it works, but it always needs updating. for the non-programmer mac user, xcode is most definitely the path of least resistance. (also for this programmer).

[mike-lawrence]

Brew by default checks for and pulls updates to its formulas, yes, but that hardly seems a downside. I’ve never had any issues with it as a package manager. Just as good as those on any Linux system I’ve used. And I have a hard time believing that the multi-step process of installing xcode is any easier for any kind of user than firing up terminal and pasting a single command.

[mitzimorris]

@mike-lawrence

Just as good as those on any Linux system I’ve used.

absolutely. sounds like I’ve confused brew with macports. I’ve filed them all in the same jumbled “linux for max” drawer.

but if you’re not a linux admin user, it’s just not that easy to get it installed and set up. many mac users don’t know how to fire up a terminal window!

[mike-lawrence]

Ah, yes, macports was a mess. Homebrew is far superior.

[mike-lawrence]

If we’re concerned about mac users that can’t figure out how to launch the terminal.app, it should be relatively straight forward to make an apple script app that runs the pertinent terminal commands.

[mitzimorris]

@mike-lawrence

suggest it instead suggest simply running xcode-select --install in a terminal, which installs simply the command line tools alone.

this is an excellent suggestion and I will add it to the wiki page.

[Bob_Carpenter]

@mike-lawrence

macports was a mess. Homebrew is far superior.

One of the compilers we needed to test was only available on Macports. The whole thing’s a mess.

I’m not a huge fan of Linux package management. It never worked very well for me because of all the dependencies. And we’ve just repeated the same mess with RStan because of CRAN policy on small packages (so we have to depend on BH’s version of Boost, RcppEigen’s verison of Eigen, etc.

[mike-lawrence]

@mitzimorris

many mac users don’t know how to fire up a terminal window

Ok, how about this: https://github.com/mike-lawrence/install_rstan_mac/archive/master.zip

Inside the zip will be a file called “install_rstan.applescript” that, when double-clicked, will open applescript editor where we can tell users to click the “play” button.

Note that in the long run, it’d probably be better for the Stan team to get “identified developer” status from apple and bundle this script into an app (in applescript editor, export-> application), which would be more familiar for users (and a couple fewer mouse clicks).

[Bob_Carpenter]

Installation and compile time are two big issues we’re going to try to tackle head on this year. We’re hiring right now (at Columbia) for an operations/build/deployment person to help out.

[mike-lawrence]

Fyi, there’s a nice gui installer for the Mac OS tool chain now: https://github.com/coatless/r-macos-rtools

[Bob_Carpenter]

Cool! Has anyone verified that it worked?

[coatless]

Greetings and Salutations folks,

Yes, it does indeed work as the toolchain tries to mirror exactly how CRAN builds the binary.

For details, see: https://gist.github.com/coatless/ef89c622858b18164195befee2f06d55

Though, it’s noisy in the default state because it does not include the CXXFLAGS recommended on the Installing RStan on Mac or Linux wiki page.

Disclaimer: I developed the r-macos-rtools installer linked by @mike-lawrence. Great to see its already spreading!

Sincerely,

JJB

[bgoodri]

Thanks @coatless for putting all this together.

[Bob_Carpenter]

@coatless

the toolchain tries to mirror exactly how CRAN builds the binary.

Neat, and thanks for explaining (and sharing the build).

@coatless

it’s noisy in the default state because it does not include the CXXFLAGS recommended on the Installing RStan on Mac or Linux wiki page.

What is the default?

Both -O3 (in CXXFLAGS) and -mtune=native (in CXX) are necessary for optimal compiled code. The former is what’s going to unfold everything as we expect and the latter’s going to allow loop vectorization, etc.

[coatless]

@Bob_Carpenter

coatless:

it’s noisy in the default state because it does not include the CXXFLAGS recommended on the Installing RStan on Mac or Linux wiki page.

What is the default?

Both -O3 (in CXXFLAGS) and -mtune=native (in CXX) are necessary for optimal compiled code. The former is what’s going to unfold everything as we expect and the latter’s going to allow loop vectorization, etc.

The default flags for macOS are:

-fPIC  -Wall -g -O2

rstan from Installing RStan on Mac or Linux wiki page recommends:

CXXFLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function  -Wno-macro-redefined

“Noise” is generated due to the lack of -Wno-unused-variable -Wno-unused-function -Wno-macro-redefined in the default flags. There is also less optimization -O2 vs. -O3.

[Bob_Carpenter]

It’d be interesting to see what the speed of -O2 vs. -O3 is and what the difference is with and without the native specifications (which mainluy enable lower-level CPU optimizations, which are important for a lot of loop unfolding and thus matrix operations).