Rstan install - need to compile from source in order to see parser error messages

didn’t read the docs, or see this message in Ben’s announcement:

The main change to rstan is that it should work with a Mac that uses the upstream version of clang (which CRAN now uses) rather than Xcode’s clang. It is necessary that the upstream clang be installed in /usr/local/clang4/bin/clang++. After that, there should be no more “unknown exception” messages.

because I don’t have clang4 installed at the prescribed path, I triggered this error. Will install clang4 and retest.

installed clang4, using this nifty installer:

still not working for me.
cheers,
Mitzi

Can you tell which part of


is not working?

grepl("^darwin", R.version$os)
grepl("clang4", rstan:::get_CXX())
Sys.which("install_name_tool")
file.exists("/usr/local/clang4/lib/libc++.1.dylib")
file.exists("/usr/lib/libc++.1.dylib")

all of them seem to be working -

grepl(“^darwin”, R.version$os)
[1] TRUE
grepl(“clang4”, rstan:::get_CXX())
[1] TRUE
Sys.which(“install_name_tool”)
install_name_tool
“/usr/bin/install_name_tool”
file.exists(“/usr/local/clang4/lib/libc++.1.dylib”)
[1] TRUE
file.exists(“/usr/lib/libc++.1.dylib”)
[1] TRUE

and can run clang4 from command line, FWIW:

/usr/local/clang4/bin/clang++ -v
clang version 4.0.0 (tags/RELEASE_400/final)
Target: x86_64-apple-darwin17.3.0
Thread model: posix
InstalledDir: /usr/local/clang4/bin

Is it just parser errors that do this or do syntactically correct models that occasionally throw numerical exceptions (like funnel) do this as well?

I’ve seen this just for parser errors. With the same version, I’ve seen rejections with appropriate messages.

What if you do (in a clean R session)

  cmd <- paste(
    "install_name_tool",
    "-change",
    "/usr/local/clang4/lib/libc++.1.dylib",
    "/usr/lib/libc++.1.dylib",
    system.file("libs", "rstan.so", package = "rstan")
  )
  system(cmd)
  library(rstan)
  stanc("program_with_syntax_error.stan")

in a fresh R session, trying to run a syntactically correct model - the funnel w/ centered parameterization and no data - stan command fails.

niter=100;
nthin=1;
nchains=4;
fit_cp_0 = stan(file=“funnel_cp.stan”, data=d_0, iter=niter, thin=nthin, chains=nchains, save_warmup=FALSE);

data exists, model compiles in cmdstan 2.17 - doesn’t run in R:

In file included from file16b41811ba3.cpp:8:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hp p:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/autodiffstackstorage.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/memory/stack_alloc.hpp:8:
In file included from /usr/local/clang4/bin/…/include/c++/v1/cstdlib:86:
/usr/local/clang4/bin/…/include/c++/v1/stdlib.h:94:15: fatal error: ‘stdlib.h’ file not found
#include_next <stdlib.h>

tried this - it didn’t work.
corrected syntactically incorrect file and tried again w/ command stan_model - this fails to link the generated .cpp file - same set of error messages as below for the funnel model.

checking what the installer did - it overwrites any existing ~/.R/Makevars file -

cat .R/Makevars

CC=/usr/local/clang4/bin/clang
CXX=/usr/local/clang4/bin/clang++
CXX1X=/usr/local/clang4/bin/clang++
CXX98=/usr/local/clang4/bin/clang++
CXX11=/usr/local/clang4/bin/clang++
CXX14=/usr/local/clang4/bin/clang++
CXX17=/usr/local/clang4/bin/clang++
LDFLAGS=-L/usr/local/clang4/lib

is trying to #include_next <stdlib.h>? WTF?

still having fun - trying to use xcode clang++ compiler.

blew away my ~/.R/Makevars file, which was overwritten by the clang4 installer, and created the following Makevars file, per instructions on the RStan wiki for installs on Mac - current file is:

cat .R/Makevars

CXXFLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function
CC=clang
CXX=clang++ -arch x86_64 -ftemplate-depth-256

in a fresh RStudio session, try stan_model command for syntactially correct Stan program:

> stan_model("hello.stan")
In file included from file196e5bbee5d6.cpp:8:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:12:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/gevv_vvv_vari.hpp:5:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/var.hpp:7:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/math/tools/config.hpp:13:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config.hpp:39:
/Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config/compiler/clang.hpp:200:11: warning: 'BOOST_NO_CXX11_RVALUE_REFERENCES' macro redefined [-Wmacro-redefined]
#  define BOOST_NO_CXX11_RVALUE_REFERENCES
          ^
<command line>:6:9: note: previous definition is here
#define BOOST_NO_CXX11_RVALUE_REFERENCES 1
        ^
1 warning generated.

this works:

> fx <- inline::cxxfunction( signature(x = "integer", y = "numeric" ) , '
+ 	return ScalarReal( INTEGER(x)[0] * REAL(y)[0] ) ;
+                            ' )
> fx( 2L, 5 ) 
[1] 10

OK, this works:

file ~/.R/Makevars

CXXFLAGS=-O3 -mtune=native -march=native
CXXFLAGS= -Wno-macro-redefined -Wno-unused-variable -Wno-unused-function
CC=clang
CXX=clang++ -arch x86_64 -ftemplate-depth-256

as before, installed RStan from source:

install.packages(“rstan”, type = “source”)

now getting parser error messages for syntax errors,
and syntactically correct files compile and run, with warning messages.

One final fix required in RStudio session from command line:

Sys.setenv(TZ=“America/New_York”)

to get rid of stupid error messages:

Warning messages:
1: 1: In as.POSIXlt.POSIXct(x, tz) :In as.POSIXlt.POSIXct(x, tz) :
unknown timezone ‘default/America/New_York’

The permafix for the TZ problem is to edit either your personal or site installation’s Renviron file to add whatever you want your default TZ to be.

giving the clang4 install another go, but still seeing problems trying to run a simple valid model through stan_model - linker fails with message:

/usr/local/clang4/bin/…/include/c++/v1/stdlib.h:94:15: fatal error: ‘stdlib.h’ file not found

similar to what was discussed in this thread:

I suspect the Makevars file created by that installer is missing stuff needed by rstan et al, but reading the R docs and that other thread I can’t figure out what those should be.

For what it may be worth, last week I was also having problems with RStan 2.17 swallowing parser warnings but I was able to get everything working using the vanilla Xcode toolkit (OS X 10.12.6, Xcode 9.2, Apple LLVM version 9.0.0 (clang-900.0.39.2)) by installing StanHeaders from source and then RStanArm 2.17.2 from source.

interesting. so don’t have to install RStan from source, just the other two?
do you need RStanArm from source if you’re only using RStan?

the problem remains that RStan users won’t see parser error messages without doing some kind of install from source and previously this wasn’t necessary, so existing users who upgrade will hit this error as well as any new users.

I’ve just done a pass as editing the RStan Wiki page: “Installing RStan on Mac or Linux”, but I didn’t just flat out say “you’re going to have to install from source” because maybe that’s too strong?

I did, however, update the section on crafting your Makevars file to the set of flags that seem to be necessary for make RStan compile without throwing lots of warnings.

Eyeballs would be most welcome! https://github.com/stan-dev/rstan/wiki/Installing-RStan-on-Mac-or-Linux

1 Like

Sorry, autocomplete typo – I meant StanHeaders and then RStan. No RStanArm. One issue I noticed is that if you install RStan from source then there’s a clash with StanHeaders which is precompiled presumably using a different version of clang. Compiling both from source ensures that everything is compiled with the same toolkit.

Hi,

I stumbled into this the other day (maybe my OS X Mojave installation did something?) After Mojave installation I downloaded newest XCode, and ran Coatless’ installer. I changed my Makevars from the Coatless’ installer default to this:

So I guess the above uses OS X’s clang only?

Then I ran:

install.packages("StanHeaders", repos = "https://cloud.r-project.org/", dependencies=TRUE, type="source")
install.packages("rstan", repos = "https://cloud.r-project.org/", dependencies=TRUE, type="source")

and it all seems to work :)

so glad it works!