RStan unable to find g++ when compiling model

Interfaces RStan
1

Hello,

We are trying to run a computation, with RStan, but when the job is submitted, it fails when compiling the model, with the below error message:

Error in compileCode(f, code, language = language, verbose = verbose) :
  Compilation ERROR, function(s)/method(s) not created! /bin/sh: /bin/g++: No such file or directory
make: *** [file3843752b0cf6.o] Error 127
Calls: brm ... <Anonymous> -> cxxfunctionplus -> cxxfunction -> compileCode
In addition: Warning message:
running command '/usr/local/R/3.4.0/lib64/R/bin/R CMD SHLIB file3843752b0cf6.cpp 2> file3843752b0cf6.cpp.err.txt' had status 1
Execution halted

I’ve found suggested solutions, but they only apply to Rtools, and this issue is happening on RHEL 7.3.

Am I missing something obvious? Please see below for additional details and please let me know what additional information I can provide.

Thanks for any insight or direction you can offer!

R 3.4.0 (compiled, not installed via package manager)
RStan 2.15.1 (installed via installed.packages(“rstan”))

Inside of R, the output from “system(‘which g++’)” is:

/usr/bin/g++

The output from “system(‘g++ -v’)” is:

Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: …/configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)

Contents of ~/.R/Makevars:

CXX = /bin/g++
CC = /bin/gcc
FC = /bin/gfortran

Output from env, from the terminal, is:

MODULE_VERSION_STACK=3.2.10
MANPATH=/usr/local/gdal/2.1.3/share/gdal:/usr/local/proj/4.9.3/share/man:/usr/local/curl/7.53.1/share/man:/usr/local/openmpi/1.10.6/gnu/4.8.5-rhel/ib/man:/usr/local/Modules/3.2.10/share/man:/opt/sge/man:/usr/share/man/overrides:/usr/share/man:/usr/local/share/man
XDG_SESSION_ID=30771
HOSTNAME=du-hpc
TERM=xterm
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT= 22
PERL5LIB=/usr/lib64/perl5
SGE_CELL=default
SGE_ARCH=lx-amd64
OPTS=-O3
QTDIR=/usr/lib64/qt-3.3
OLDPWD=/home/bfotovic/.R
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/5
QT_GRAPHICSSYSTEM_CHECKED=1
USER=bfotovic
LD_LIBRARY_PATH=/usr/local/R/3.4.0/lib64:/usr/local/gdal/2.1.3/lib:/usr/local/proj/4.9.3/lib:/usr/local/curl/7.53.1/lib:/usr/local/lapack/3.7.0/lib:/usr/local/blas/3.7.0/lib:/usr/local/geos/3.6.1/lib:/usr/local/openmpi/1.10.6/gnu/4.8.5-rhel/ib/lib
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.jpg=01;35:.jpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.axv=01;35:.anx=01;35:.ogv=01;35:.ogx=01;35:.aac=01;36:.au=01;36:.flac=01;36:.mid=01;36:.midi=01;36:.mka=01;36:.mp3=01;36:.mpc=01;36:.ogg=01;36:.ra=01;36:.wav=01;36:.axa=01;36:.oga=01;36:.spx=01;36:*.xspf=01;36:
MODULE_VERSION=3.2.10
MAIL=/var/spool/mail/bfotovic
PATH=/usr/local/R/3.4.0/bin:/usr/local/gdal/2.1.3/bin:/usr/local/proj/4.9.3/bin:/usr/local/curl/7.53.1/bin:/usr/local/geos/3.6.1/bin:/usr/local/openmpi/1.10.6/gnu/4.8.5-rhel/ib/bin:/usr/local/Modules/3.2.10/bin:/opt/sge/bin:/opt/sge/bin/lx-amd64:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/bfotovic/bin
PWD=/home/bfotovic
LMFILES=/usr/local/Modules/modulefiles/null:/usr/local/Modules/modulefiles/modules:/usr/local/Modules/modulefiles/mpi/openmpi-1.10.6-gcc4.8.5:/usr/local/Modules/modulefiles/tools/geos-3.6.1:/usr/local/Modules/modulefiles/libs/blas-3.7.0:/usr/local/Modules/modulefiles/libs/lapack-3.7.0:/usr/local/Modules/modulefiles/tools/curl-7.53.1:/usr/local/Modules/modulefiles/tools/proj-4.9.3:/usr/local/Modules/modulefiles/tools/gdal-2.1.3:/usr/local/Modules/modulefiles/apps/R-3.4.0
SGE_EXECD_PORT=6445
LANG=en_US.UTF-8
MODULEPATH=/usr/local/Modules/versions:/usr/local/Modules/$MODULE_VERSION/modulefiles:/usr/local/Modules/modulefiles
SGE_QMASTER_PORT=6444
LOADEDMODULES=null:modules:mpi/openmpi-1.10.6-gcc4.8.5:tools/geos-3.6.1:libs/blas-3.7.0:libs/lapack-3.7.0:tools/curl-7.53.1:tools/proj-4.9.3:tools/gdal-2.1.3:apps/R-3.4.0
SGE_ROOT=/opt/sge
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/bfotovic
MATHEMATICA_HOME=/usr/local/mathematica/11.1
PYTHONPATH=/usr/lib64/python2.7
LOGNAME=bfotovic
QTLIB=/usr/lib64/qt-3.3/lib
SSH_CONNECTION=130.253.15.56 37074 130.253.21.117 22
GXX_ROOT=/usr/lib/gcc/x86_64-redhat-linux/4.8.5
MODULESHOME=/usr/local/Modules/3.2.10
LESSOPEN=||/usr/bin/lesspipe.sh %s
SGE_CLUSTER_NAME=p6444
XDG_RUNTIME_DIR=/run/user/565
DISPLAY=localhost:13.0
INCLUDE=/usr/local/gdal/2.1.3/include:/usr/local/proj/4.9.3/include:/usr/local/curl/7.53.1/include:/usr/local/geos/3.6.1/include
BASH_FUNC_module()=() { eval /usr/local/Modules/$MODULE_VERSION/bin/modulecmd bash $*
}
_=/usr/bin/env

2

This

does not match with

You should probably just have

CXX=g++
CC=gcc
FC=gfortran
3

Hey bgoodri,

Thanks for the suggestion!

I made the change to ~/.R/Makevars:

CXX=g++
CC=gcc
FC=gfortran

And we’re still getting the same error message.

What additional steps could/should I attempt?

Thanks!

4

You mean it now says
Compilation ERROR, function(s)/method(s) not created! /bin/sh: /usr/bin/g++: No such file or directory

5

Sorry for my slow response.

Here is the error message now:

Error in compileCode(f, code, language = language, verbose = verbose) :
Compilation ERROR, function(s)/method(s) not created! /bin/sh: g++: command not found
make: *** [file56907e8d301d.o] Error 127
Calls: brm … → cxxfunctionplus → cxxfunction → compileCode
In addition: Warning message:
running command ‘/usr/local/R/3.4.0/lib64/R/bin/R CMD SHLIB file56907e8d301d.cpp 2> file56907e8d301d.cpp.err.txt’ had status 1
Execution halted

Thanks!

6

My guess is that the PATH of the worker that compiles the Stan model is not the same as the PATH in the output of env. Do you get more information in the log if you call stan_model with verbose = TRUE?

7

I haven’t tried your suggestion (though I can compile R and packages, using it is something I’m still learning, along with researchers I assist), but I want to share some behavior:

The error I encountered appears when trying to run the R analysis through a job queue system; submission script calls Rscript .

When I run R and enter the commands directly, everything works, including the compilation of the stan model. So, from what I can tell, there seems to be a pathing challenge when calling Rscript.

If you have any advice for this, please let me know, but I think the issue may exist outside of your package.

Thank you for your patience and assistance!

8

Indeed, there was a path issue, on some of the nodes we tried to run a stan model with. After it was corrected, the stan model compiles consistently every time!

9

Ahh, now you tell me. RStan has been known to not work with Rscript for reasons that are both mysterious and beyond our control, although I can’t remember it manifesting itself in failing to find the compiler. Usually the problem is that you have to call library(Rcpp) explicitly in the script or specify --default-packages=Rcpp. But with either of those, I can compile Stan programs on my machine. Are you sure the Rscript that is being called is on the same filesystem as the one where g++ is installed?

10

To answer your question: as g++ didn’t exist, on the compute nodes, g++ and Rscript weren’t on the same file system.

On the compute nodes, R wasn’t able to find g++, but on the head node, it was. The “path issue” was that a g++ package wasn’t explicitly installed, on the compute nodes. After installing the g++ package, everything worked correctly.

Thank you!