R Package interfacing with Stan - models much slower than outside package

Interfaces RStan
1

Hi all,
I’m building an R package that uses some Stan functions I wrote. I’ve been using rstantools and the guidelines here (Guidelines for developers of R packages interfacing with Stan • rstantools) and here (R Package Maintainers Using Stan - Please Read).

I’ve gotten the package compiled, but sampling is ~10 times slower using the compiled versions of the same functions as running them outside my R package on the same data. I’m using rstan and I’m guessing this has something to do with running the chains in parallel. Below are excerpts from my NAMESPACE and DESCRIPTION files for reference.

Any thoughts are appreciated.
MG

Excerpt from NAMESPACE File
import(MASS)
import(Rcpp)
import(ape)
import(bridgesampling)
import(devtools)
import(ggplot2)
import(ggsci)
import(loo)
import(methods)
import(rstantools)
import(treeplyr)
importFrom(RcppParallel,RcppParallelLibs)
importFrom(rstan,sampling)

Excerpt from DESCRIPTION FILE
Depends: 
    R (>= 3.4.0)
Imports: 
    methods,
    Rcpp (>= 0.12.0),
    RcppParallel (>= 5.1.7),
    rstan (>= 2.18.1),
    rstantools (>= 2.3.1),
    devtools,
    ape,
    treeplyr,
    ggplot2,
    ggsci,
    MASS,
    loo,
    bridgesampling
LinkingTo: 
    BH (>= 1.66.0),
    Rcpp (>= 0.12.0),
    RcppEigen (>= 0.3.3.3.0),
    RcppParallel (>= 5.0.1),
    rstan (>= 2.18.1),
    StanHeaders (>= 2.18.0)
2

That’s very strange, I haven’t seen that before. @andrjohns Any ideas what might cause that?

Is the package source code available on GitHub or another public repository?

What makes you think that? I don’t think that should make things slower unless the models are super fast. If your models are really fast to run then it can take longer to run in parallel than sequentially because of some extra overhead, but for models that take a non-trivial amount of time it should be faster to run in parallel.

3

Thanks @jonah. I’m still doing testing of the package so the code is not available yet.

I actually meant that my package is missing something that allows for the chains to run in parallel. I use the code below normally to “enable some compiler optimizations to improve the estimation speed of the model” from Configuring C Toolchain for Mac · stan-dev/rstan Wiki · GitHub

However, checking now, when I run the non-R package versions of the same Stan program without running that code first, it does not seem to have an effect - the non-package version of the functions take the same amount of time to run (i.e. 10 times faster than the package versions).

Thanks,
MG

dotR <- file.path(Sys.getenv("HOME"), ".R")
if (!file.exists(dotR)) dir.create(dotR)
M <- file.path(dotR, "Makevars")
if (!file.exists(M)) file.create(M)
arch <- ifelse(R.version$arch == "aarch64", "arm64", "x86_64")
cat(paste("\nCXX14FLAGS += -O3 -mtune=native -arch", arch, "-ftemplate-depth-256"),
    file = M, sep = "\n", append = FALSE)
4

I had a problem like this, and it was because a flag was set to create a debug build:

2 Likes
5

Interesting, I didn’t actually know pkgload did that. Good to know!

6

me neither, this caused me a bunch of headaches in the past, thanks :) :)

7

Thanks for the ideas - I tried adding PKG_BUILD_EXTRA_FLAGS=“false” to m .Renviron file as that link suggests but Stan models are still 10 times slower within the package than our of it. Other thoughts?
Thanks!

8

I would try to build/install the package in different ways to see whether the problem persists. There is the pkgload thing from that github link, there is the traditional R CMD build followed by install.packages(), and there might be some other way to install packages from Rstudio. And, esp if you are on linux, the commands can be run as root (sudo) or as a regular user. These can influence where R looks for packages and installation flags. So you would hope to find one type of installation with fast models, then figure out why that one was fast while others were slow.

1 Like
9

Thanks Ed - that solved it. I tried different methods and the Rstudio Clean and Install option was successful. Why - not sure yet.

2 Likes