Rstan install - need to compile from source in order to see parser error messages

mitzimorris · January 5, 2018, 4:25pm

Operating System: MacOS High Sierra
Interface Version: rstan 2.17.2
Compiler/Toolkit: clang/xcode

the downloaded rstan package can’t get compiler error messages from stanc.

I just upgraded to High Sierra (what’s in a name? celebrating legal pot in CA?).
In RStudio, I upgraded all installed packages - this gave me RStan 2.17.2.
Next I tried to compile a model with a simple syntax error:

Error in stanc(file = file, model_code = model_code, model_name = model_name,  : 
  c++ exception (unknown reason)

the fix that worked for me is:

library(devtools)

this required a restart of the R session, then:

install.packages("rstan", type = "source")

Posting this for future reference - next I’ll check the RStan install page (because I don’t read the docs, I just write them).

cheers,
Mitzi

mitzimorris · January 5, 2018, 4:56pm

didn’t read the docs, or see this message in Ben’s announcement:

The main change to rstan is that it should work with a Mac that uses the upstream version of clang (which CRAN now uses) rather than Xcode’s clang. It is necessary that the upstream clang be installed in /usr/local/clang4/bin/clang++. After that, there should be no more “unknown exception” messages.

because I don’t have clang4 installed at the prescribed path, I triggered this error. Will install clang4 and retest.

mitzimorris · January 5, 2018, 7:07pm

installed clang4, using this nifty installer:

still not working for me.
cheers,
Mitzi

bgoodri · January 5, 2018, 9:05pm

Can you tell which part of

github.com

stan-dev/rstan/blob/develop/rstan/rstan/R/cxxfunplus.R#L145


} 


cxxfunctionplus <- function(sig = character(), body = character(),
                          plugin = "default", includes = "",
                          settings = getPlugin(plugin), 
                          save_dso = FALSE, module_name = "MODULE", 
                          ..., verbose = FALSE) {
fx <- cxxfunction(sig = sig, body = body, plugin = plugin, includes = includes, 
                  settings = settings, ..., verbose = verbose)
dso_last_path <- dso_path(fx)
if (grepl("^darwin", R.version$os) && grepl("clang4", get_CXX())) {
  cmd <- paste(
    "install_name_tool",
    "-change",
    "/usr/local/clang4/lib/libc++.1.dylib",
    "/usr/lib/libc++.1.dylib",
    dso_last_path
  )
  system(cmd)
  dyn.unload(dso_last_path)
  dyn.load(dso_last_path)

is not working?

grepl("^darwin", R.version$os)
grepl("clang4", rstan:::get_CXX())
Sys.which("install_name_tool")
file.exists("/usr/local/clang4/lib/libc++.1.dylib")
file.exists("/usr/lib/libc++.1.dylib")

mitzimorris · January 5, 2018, 10:25pm

all of them seem to be working -

grepl(“^darwin”, R.version$os)
[1] TRUE
grepl(“clang4”, rstan:::get_CXX())
[1] TRUE
Sys.which(“install_name_tool”)
install_name_tool
“/usr/bin/install_name_tool”
file.exists(“/usr/local/clang4/lib/libc++.1.dylib”)
[1] TRUE
file.exists(“/usr/lib/libc++.1.dylib”)
[1] TRUE

and can run clang4 from command line, FWIW:

/usr/local/clang4/bin/clang++ -v
clang version 4.0.0 (tags/RELEASE_400/final)
Target: x86_64-apple-darwin17.3.0
Thread model: posix
InstalledDir: /usr/local/clang4/bin

bgoodri · January 5, 2018, 11:11pm

Is it just parser errors that do this or do syntactically correct models that occasionally throw numerical exceptions (like funnel) do this as well?

syclik · January 5, 2018, 11:13pm

I’ve seen this just for parser errors. With the same version, I’ve seen rejections with appropriate messages.

bgoodri · January 5, 2018, 11:22pm

What if you do (in a clean R session)

  cmd <- paste(
    "install_name_tool",
    "-change",
    "/usr/local/clang4/lib/libc++.1.dylib",
    "/usr/lib/libc++.1.dylib",
    system.file("libs", "rstan.so", package = "rstan")
  )
  system(cmd)
  library(rstan)
  stanc("program_with_syntax_error.stan")

mitzimorris · January 5, 2018, 11:28pm

in a fresh R session, trying to run a syntactically correct model - the funnel w/ centered parameterization and no data - stan command fails.

niter=100;
nthin=1;
nchains=4;
fit_cp_0 = stan(file=“funnel_cp.stan”, data=d_0, iter=niter, thin=nthin, chains=nchains, save_warmup=FALSE);

data exists, model compiles in cmdstan 2.17 - doesn’t run in R:

In file included from file16b41811ba3.cpp:8:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hp p:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/autodiffstackstorage.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/memory/stack_alloc.hpp:8:
In file included from /usr/local/clang4/bin/…/include/c++/v1/cstdlib:86:
/usr/local/clang4/bin/…/include/c++/v1/stdlib.h:94:15: fatal error: ‘stdlib.h’ file not found
#include_next <stdlib.h>

mitzimorris · January 5, 2018, 11:38pm

tried this - it didn’t work.
corrected syntactically incorrect file and tried again w/ command stan_model - this fails to link the generated .cpp file - same set of error messages as below for the funnel model.

mitzimorris · January 5, 2018, 11:45pm

checking what the installer did - it overwrites any existing ~/.R/Makevars file -

cat .R/Makevars

CC=/usr/local/clang4/bin/clang
CXX=/usr/local/clang4/bin/clang++
CXX1X=/usr/local/clang4/bin/clang++
CXX98=/usr/local/clang4/bin/clang++
CXX11=/usr/local/clang4/bin/clang++
CXX14=/usr/local/clang4/bin/clang++
CXX17=/usr/local/clang4/bin/clang++
LDFLAGS=-L/usr/local/clang4/lib

bgoodri · January 5, 2018, 11:50pm

is trying to #include_next <stdlib.h>? WTF?

mitzimorris · January 6, 2018, 12:18am

still having fun - trying to use xcode clang++ compiler.

blew away my ~/.R/Makevars file, which was overwritten by the clang4 installer, and created the following Makevars file, per instructions on the RStan wiki for installs on Mac - current file is:

cat .R/Makevars

CXXFLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function
CC=clang
CXX=clang++ -arch x86_64 -ftemplate-depth-256

in a fresh RStudio session, try stan_model command for syntactially correct Stan program:

> stan_model("hello.stan")
In file included from file196e5bbee5d6.cpp:8:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/src/stan/model/model_header.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/mat.hpp:4:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core.hpp:12:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/gevv_vvv_vari.hpp:5:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/StanHeaders/include/stan/math/rev/core/var.hpp:7:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/math/tools/config.hpp:13:
In file included from /Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config.hpp:39:
/Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config/compiler/clang.hpp:200:11: warning: 'BOOST_NO_CXX11_RVALUE_REFERENCES' macro redefined [-Wmacro-redefined]
#  define BOOST_NO_CXX11_RVALUE_REFERENCES
          ^
<command line>:6:9: note: previous definition is here
#define BOOST_NO_CXX11_RVALUE_REFERENCES 1
        ^
1 warning generated.

this works:

> fx <- inline::cxxfunction( signature(x = "integer", y = "numeric" ) , '
+ 	return ScalarReal( INTEGER(x)[0] * REAL(y)[0] ) ;
+                            ' )
> fx( 2L, 5 ) 
[1] 10

mitzimorris · January 6, 2018, 12:51am

OK, this works:

file ~/.R/Makevars

CXXFLAGS=-O3 -mtune=native -march=native
CXXFLAGS= -Wno-macro-redefined -Wno-unused-variable -Wno-unused-function
CC=clang
CXX=clang++ -arch x86_64 -ftemplate-depth-256

as before, installed RStan from source:

install.packages(“rstan”, type = “source”)

now getting parser error messages for syntax errors,
and syntactically correct files compile and run, with warning messages.

One final fix required in RStudio session from command line:

Sys.setenv(TZ=“America/New_York”)

to get rid of stupid error messages:

Warning messages:
1: 1: In as.POSIXlt.POSIXct(x, tz) :In as.POSIXlt.POSIXct(x, tz) :
unknown timezone ‘default/America/New_York’

The permafix for the TZ problem is to edit either your personal or site installation’s Renviron file to add whatever you want your default TZ to be.

mitzimorris · January 9, 2018, 2:28am

giving the clang4 install another go, but still seeing problems trying to run a simple valid model through stan_model - linker fails with message:

/usr/local/clang4/bin/…/include/c++/v1/stdlib.h:94:15: fatal error: ‘stdlib.h’ file not found

similar to what was discussed in this thread:

I suspect the Makevars file created by that installer is missing stuff needed by rstan et al, but reading the R docs and that other thread I can’t figure out what those should be.

betanalpha · January 9, 2018, 3:33am

For what it may be worth, last week I was also having problems with RStan 2.17 swallowing parser warnings but I was able to get everything working using the vanilla Xcode toolkit (OS X 10.12.6, Xcode 9.2, Apple LLVM version 9.0.0 (clang-900.0.39.2)) by installing StanHeaders from source and then RStanArm 2.17.2 from source.

mitzimorris · January 9, 2018, 4:06am

interesting. so don’t have to install RStan from source, just the other two?
do you need RStanArm from source if you’re only using RStan?

mitzimorris · January 9, 2018, 4:18am

the problem remains that RStan users won’t see parser error messages without doing some kind of install from source and previously this wasn’t necessary, so existing users who upgrade will hit this error as well as any new users.

I’ve just done a pass as editing the RStan Wiki page: “Installing RStan on Mac or Linux”, but I didn’t just flat out say “you’re going to have to install from source” because maybe that’s too strong?

I did, however, update the section on crafting your Makevars file to the set of flags that seem to be necessary for make RStan compile without throwing lots of warnings.

Eyeballs would be most welcome! https://github.com/stan-dev/rstan/wiki/Installing-RStan-on-Mac-or-Linux

betanalpha · January 9, 2018, 4:25am

Sorry, autocomplete typo – I meant StanHeaders and then RStan. No RStanArm. One issue I noticed is that if you install RStan from source then there’s a clash with StanHeaders which is precompiled presumably using a different version of clang. Compiling both from source ensures that everything is compiled with the same toolkit.

torkar · October 2, 2018, 8:05am

Hi,

I stumbled into this the other day (maybe my OS X Mojave installation did something?) After Mojave installation I downloaded newest XCode, and ran Coatless’ installer. I changed my Makevars from the Coatless’ installer default to this:

So I guess the above uses OS X’s clang only?

Then I ran:

install.packages("StanHeaders", repos = "https://cloud.r-project.org/", dependencies=TRUE, type="source")
install.packages("rstan", repos = "https://cloud.r-project.org/", dependencies=TRUE, type="source")

and it all seems to work :)

Topic		Replies	Views
Mac compileCode() error Interfaces rstan	11	702	August 1, 2020
Can't compile code using RStan RStan compiler	8	1165	November 30, 2023
Can't compile a simple C file with Rstan on MacOS Sonoma 14.5 Developers rstan , bug	3	442	July 15, 2024
Can't compile simple stan code on M1 MacOS Ventura RStan	18	1405	July 30, 2023
Trouble with initial use of rstan Interfaces rstan	2	628	November 11, 2019

Rstan install - need to compile from source in order to see parser error messages

Related topics