Rstan won't Load in Rstudio Server (linux)

Interfaces RStan
1

To be honest this is likely more of an rstudio question than an rstan question, but I am hoping there may be someone here who can help with my issue.

I am trying to get rstan setup on a linux cluster (RHEL 7.6). The only available gcc version was 4.8.5, so the administrators installed gcc version 7.2.0 for me. In ~/.profile I load the newer gcc version. Launching R from a bash terminal then correctly uses the new version of gcc and I can install and load rstan (2.18.2) successfully.

The issue I am having is that when trying to load rstan through rstudio server I get the following error:

library(rstan)
Error: package or namespace load failed for ‘rstan’ in dyn.load(file, DLLpath = DLLpath, …):
unable to load shared object ‘/home/cquirk/R/x86_64-redhat-linux-gnu-library/3.5/rstan/libs/rstan.so’:
/lib64/libstdc++.so.6: version `CXXABI_1.3.8’ not found (required by /home/cquirk/R/x86_64-redhat-linux-gnu-library/3.5/rstan/libs/rstan.so)

This occurs despite being able to successfully reinstall rstan through rstudio.

Here is some potentially relevant output:

$ cat ~/.R/Makevars

CXX14 = g++
CXX14FLAGS = -fPIC

$ cat ~/.Renviron

TMPDIR = /home/cquirk/TMPDIR
PATH = /usr/local/gcc/7.2.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
MANPATH = /usr/local/gcc/7.2.0/share/man
LD_LIBRARY_PATH = /usr/local/gcc/7.2.0/lib64:/usr/local/gcc/7.2.0/lib:/usr/lib64/R/lib:/lib:/usr/lib/jvm/jre/lib/amd64/server:/usr/lib/jvm/jre/lib/amd64:/usr/lib/jvm/java/lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib
LIBRARY_PATH = /usr/local/gcc/7.2.0/lib64:/usr/local/gcc/7.2.0/lib
CPATH = /usr/local/gcc/7.2.0/include
CMAKE_PREFIX_PATH = /usr/local/gcc/7.2.0
CC = /usr/local/gcc/7.2.0/bin/gcc
CXX = /usr/local/gcc/7.2.0/bin/g++
FC = /usr/local/gcc/7.2.0/bin/gfortran
F77 = /usr/local/gcc/7.2.0/bin/gfortran
F90 = /usr/local/gcc/7.2.0/bin/gfortran

> Sys.getenv('LD_LIBRARY_PATH')

[1] “/usr/local/gcc/7.2.0/lib64:/usr/local/gcc/7.2.0/lib:/usr/lib64/R/lib:/lib:/usr/lib/jvm/jre/lib/amd64/server:/usr/lib/jvm/jre/lib/amd64:/usr/lib/jvm/java/lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib”

> devtools::session_info("rstan")

─ Session info ─────────────────────────────────────────────────────────────────────────────────────────────────────
setting value
version R version 3.5.1 (2018-07-02)
os Red Hat Enterprise Linux
system x86_64, linux-gnu
ui RStudio
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz America/Chicago
date 2019-02-13

─ Packages ─────────────────────────────────────────────────────────────────────────────────────────────────────────
package * version date lib source
assertthat 0.2.0 2017-04-11 [2] CRAN (R 3.5.0)
backports 1.1.3 2018-12-14 [2] CRAN (R 3.5.1)
BH 1.69.0-1 2019-01-07 [1] CRAN (R 3.5.1)
callr 3.1.1 2018-12-21 [1] CRAN (R 3.5.1)
cli 1.0.1 2018-09-25 [2] CRAN (R 3.5.1)
colorspace 1.4-0 2019-01-13 [1] CRAN (R 3.5.1)
crayon 1.3.4 2017-09-16 [2] CRAN (R 3.5.0)
desc 1.2.0 2018-05-01 [2] CRAN (R 3.5.1)
digest 0.6.18 2018-10-10 [2] CRAN (R 3.5.1)
fansi 0.4.0 2018-10-05 [2] CRAN (R 3.5.1)
ggplot2 * 3.1.0 2018-10-25 [2] CRAN (R 3.5.1)
glue 1.3.0 2018-07-17 [2] CRAN (R 3.5.1)
gridExtra 2.3 2017-09-09 [1] CRAN (R 3.5.1)
gtable 0.2.0 2016-02-26 [2] CRAN (R 3.5.0)
inline 0.3.15 2018-05-18 [1] CRAN (R 3.5.1)
labeling 0.3 2014-08-23 [2] CRAN (R 3.5.0)
lattice 0.20-38 2018-11-04 [2] CRAN (R 3.5.1)
lazyeval 0.2.1 2017-10-29 [2] CRAN (R 3.5.0)
loo 2.0.0 2018-04-11 [1] CRAN (R 3.5.1)
magrittr 1.5 2014-11-22 [2] CRAN (R 3.5.0)
MASS 7.3-51.1 2018-11-01 [2] CRAN (R 3.5.1)
Matrix 1.2-15 2018-11-01 [2] CRAN (R 3.5.1)
matrixStats 0.54.0 2018-07-23 [1] CRAN (R 3.5.1)
mgcv 1.8-26 2018-11-21 [2] CRAN (R 3.5.1)
munsell 0.5.0 2018-06-12 [2] CRAN (R 3.5.0)
nlme 3.1-137 2018-04-07 [2] CRAN (R 3.5.1)
pillar 1.3.1 2018-12-15 [2] CRAN (R 3.5.1)
pkgbuild 1.0.2 2018-10-16 [1] CRAN (R 3.5.1)
pkgconfig 2.0.2 2018-08-16 [2] CRAN (R 3.5.1)
plyr 1.8.4 2016-06-08 [2] CRAN (R 3.5.0)
prettyunits 1.0.2 2015-07-13 [2] CRAN (R 3.5.1)
processx 3.2.1 2018-12-05 [1] CRAN (R 3.5.1)
ps 1.3.0 2018-12-21 [1] CRAN (R 3.5.1)
R6 2.3.0 2018-10-04 [2] CRAN (R 3.5.1)
RColorBrewer 1.1-2 2014-12-07 [2] CRAN (R 3.5.0)
Rcpp 1.0.0 2018-11-07 [2] CRAN (R 3.5.1)
RcppEigen 0.3.3.5.0 2018-11-24 [2] CRAN (R 3.5.1)
reshape2 1.4.3 2017-12-11 [2] CRAN (R 3.5.0)
rlang 0.3.1 2019-01-08 [1] CRAN (R 3.5.1)
rprojroot 1.3-2 2018-01-03 [2] CRAN (R 3.5.0)
rstan 2.18.2 2018-11-07 [1] CRAN (R 3.5.1)
scales 1.0.0 2018-08-09 [2] CRAN (R 3.5.1)
StanHeaders * 2.18.1 2019-01-28 [1] CRAN (R 3.5.1)
stringi 1.2.4 2018-07-20 [2] CRAN (R 3.5.1)
stringr 1.3.1 2018-05-10 [2] CRAN (R 3.5.0)
tibble 2.0.1 2019-01-12 [1] CRAN (R 3.5.1)
utf8 1.1.4 2018-05-24 [2] CRAN (R 3.5.0)
viridisLite 0.3.0 2018-02-01 [2] CRAN (R 3.5.0)
withr 2.1.2 2018-03-15 [2] CRAN (R 3.5.0)

[1] /home/cquirk/R/x86_64-redhat-linux-gnu-library/3.5
[2] /usr/lib64/R/library
[3] /usr/share/R/library

It seems as if rstudio is not respecting my LD_LIBRARY_PATH. The administrator confirmed that if LD_LIBRARY_PATH is set in in the global RStudio Server configuration file (in a test environment) everything seems to work correctly. I am apparently not the first person to have this issue, but it seems like all of the solutions require a global change to the RStudio Server configuration and making a system wide change for this issue is probably not going to be possible. So I am somewhat stuck (at least in terms of using rstudio server, which I have a strong preference for).

Like I said, I realize this isn’t strictly an rstan issue but I’d appreciate any advice anyone can offer.

2

Can you try changing the first line of ~/.R/Makevars to CXX14 = g++-7 or CXX14 = /full/path/to/g++-7?

3

Hi, thanks for the response Ben!

I tried again with CXX14 = /usr/local/gcc/7.2.0/bin/g++ (g++-7 results in command not found) but I have the same behavior. Rstan installs fine, but then won’t load after the library call (in rstudio, after launching R from the command line rstan is still loading in).

1 Like
4

I also had to abandon (for the moment) the installation of Rstan + Rstudio server on the cluster.

My IT admin told similar thing (his suggestion did not work at the end and he told me that there are no current solutions). It seems more an issue with Rstudio, being not as powerful as R

You are forcing Rstan to compile with a version of gcc that rstudio has no idea where it is. For your Rstan to work, Rstduio has to be able to see the shared libraries that come with gcc 7.3.0.

If you see your error trace, the glibxx library your Rstan is looking at is not the one from gcc 7.3.0:

Error: package or namespace load failed for ‘rstan’ in dyn.load(file, DLLpath = DLLpath, …):
unable to load shared object ‘/stornext/HPCScratch/home/mangiola.s/R/x86_64-pc-linux-gnu- library/3.5/rstan/libs/rstan.so’:
/usr/lib64/libstdc++.so.6 : version `GLIBCXX_3.4.20’ not found (required by …

It is instead looking at the system’s gcc library, which is version 4.8.5 and located at /usr/lib64/. The GLIBXX library for the gcc version you want is at /stornext/System/data/apps/gcc/gcc-7.3.0/lib64/libstdc++.so. 6

In the past, the Rstan that has been installed works because it was compiled it with a gcc version which matches the R version Rstudio is using. Only in this case Rstudio will be able to know where the gcc libraries are. Otherwise, Rstudio is extremely limiting, in does not see you library path or anything else that is not added at the config file on startup. Adding the libraries you need is not viable in the config file as non admin users have no write permission and also it modifies all users’ sessions.

I only have two recommendations:

  1. Forget about Rstudio and use the command line. The solution in this case is:

module load gcc/7.3.0
module load R

  1. If you really need Rstudio, use this overly complicated work around:
    dirs<-c(“/stornext/System/data/apps/gcc/gcc-7.3.0 /lib64/”,“/stornext/System/data/apps/hdf5/hdf5-1.8. 20/lib”)
    for ( dir in dirs ) {
    libs<-list.files(dir,pattern=“*.so”)
    for ( lib in libs ) {
    dyn.load(paste(dir,lib,sep=“/”))
    }
    }

These lines of code will dynamically load all the libraries for gcc 7.3.0 and hdf5 1.8.20 in every Rstudio session you start as long as those lines are in your R profile. This outs overheads in the amount of memory your sessions will use.

Similarly, this library issue in Rstudio will occur for every other module that depends on shared libraries to work.

5

Thanks for your message. It looks like the solution going forward for us is for a single compute node to be setup to work with rstan.

6

Just to follow up, this is what ended up working for my version of g++:

CXX14FLAGS=-O3 -march=native -mtune=native -fPIC -std=c++1y
CXX14=g++

I’m guessing this is probably not very stable, but it is working for now at least

2 Likes