Comments and questions about building packages with rstantools

[dbarneche]

Hi @martinmodrak, @jonah, and @bgoodri,

Thank you all very much for the explanations, this was extremely helpful!

If you don’t mind, I have one more larger question that builds on the above, but perhaps should be moved to a separate topic (happy to do it if you prefer).

I am currently co-developing another package with my colleague @beckyfisher using rstantools. The point of the package is to provide a specific set of non-linear models (about 10 different equations), in a similar way to how rstanarm makes the SSfunctions.

There are three things that we would like to allow the user to specify on top of the model type:
a) specifying a family for the response variable;
b) priors;
c) addition of an offset parameter on which they can add hierarchical effects, e.g. if \mu = a x^b, it would implement a variant like \mu = o + a x^b, with o being fixed and random, whereas a and b would be fixed only.

We started using rstantools (please see initial attempt here), but the combinations of families * non-linear equations became quite large (~50 stan models) in inst/stan, so pre-compilation during installation via install_github became prohibitive (about 1 hour to download and install).

We decided (at least for now) to implement (a–c) using brms (please see current version here) instead because it is more flexible, however it requires compilation on the go. We still would like to be able to implement (a–c) using rstantools, but the main questions then would be:

1 - Would the (eventual) download a binary version be too heavy considering a set of 50+ pre-compiled models?

2 - Is there a way to modularise the stan code by placing small bits and pieces of variants (e.g. changes to \mu formulas on link scales, changes to prior distributions) under inst/include such that the final pre-compiled product is much lighter?

Thanks again!
Best,
D

[bgoodri]

There is no way a package with 50 Stan programs would be able to go on CRAN and even without CRAN would be difficult to install. You need to use a bunch of if, else if, \dots else statements to express more models in fewer Stan files like is done in rstanarm.

[dbarneche]

Thanks for clarifying @bgoodri, we suspected it was going to be too heavy.

[jeffreypullin]

I’m also having trouble getting my RStan dependent package to build with GitHub Actions.

I get the same error as @dbarneche - a failure to link to tbb i.e.

clang++ -mmacosx-version-min=10.13 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o rater.so RcppExports.o stanExports_class_conditional_dawid_skene.o stanExports_dawid_skene.o stanExports_grouped_data.o stanExports_hierarchical_dawid_skene.o -L'/Users/runner/runners/2.263.0/work/_temp/Library/RcppParallel/lib/' -Wl,-rpath,/Users/runner/runners/2.263.0/work/_temp/Library/RcppParallel/lib/ -ltbb -ltbbmalloc -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: directory not found for option '-L'/Users/runner/runners/2.263.0/work/_temp/Library/RcppParallel/lib/''
ld: library not found for -ltbb
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [rater.so] Error 1
ERROR: compilation failed for package ‘rater’

you can see the full logs in the check results for this PR. It fails on both Mac and Linux but passes on Windows

This is pretty odd, because the package builds fine on my Mac and on Travis, which I believe uses Linux.

I don’t really know if this is fixable and there is a known workaround, but I thought that I’d just record that this is a general problem and hope that someone would have some insight.

Thanks!

[jonah]

@jeffreypullin thanks for sharing and sorry for the hassle. I think the fix is for us to get new versions of rstan and rstantools on CRAN that depend on RcppParallel, which includes TBB and should avoid errors like this

@jeffreypullin

ld: library not found for -ltbb

I know @bgoodri and @wds15 are working on getting rstan submitted. I think the version of rstantools on GitHub is ready to go but is waiting on rstan.

[jeffreypullin]

@jonah I was writing a long comment about this when I realized what the problem is: although rstantools has been rolled back to 2.0.0

install.packages("rstantools")

on a Mac will still install 2.1.0 because that is what the current CRAN binaries are. 2.1.0 is however incompatible with the latest StanHeaders causing the failure.

I don’t see any easy way to fix this except to wait for the CRAN binaries to update.

cc fyi @dbarneche

[wds15]

just install it from source on Mac. That should give you the right version… still inconvenient, of course…

[mcol]

This should do what @wds15 suggested:

install.packages("rstantools", type="source")

[jeffreypullin]

Hi @mcol, @wds15, thanks for your responses.

That’s what I’m doing currently - the problem is GitHub Actions where I can’t customize the installation type easily (or at least I don’t think I can)

Best!

[dbarneche]

@jeffreypullin, this is our temporary fix:

nschiett/fishflux/blob/master/.github/workflows/R-CMD-check.yml

[mcol]

I’m not familiar with Github actions, but perhaps it’s posible to specify the rstantools github repository as the preferred repository ahead of CRAN? This page shows an example of how to setup actions for R, and it seems you can specify a repository with “uses”: https://github.com/r-lib/actions/tree/master/setup-r (I’m just guessing though)!

[rok_cesnovar]

I think just adding

install.packages("rstantools", type="source")
before
remotes::install_deps(dependencies = TRUE)

in the Install dependencies part of the github action script should also work. It would build from source on all OS though.

[jeffreypullin]

Hi all, thanks for all your help - it’s now building successfully! I ended up using @rok_cesnovar’s suggestion to just install rstantools from GitHub. For anyone trying to emulate this, I put the line:

remotes::install_github("stan-dev/rstantools@v2.0.0")

at the start of the install dependencies block.

[bgoodri]

rstantools 2.1.x is now on CRAN again, so the GitHub business should be unnecessary

[wwguo]

Will install.packages("myownpackage.tar.gz") triggers pre-compailation? It installs my package, but it recompiles.

I packed my R and stan codes and want to share with my colleagues. I have 90 stan files in my package so it took hours to compile. I don’t want my colleagues all spending so long time to install my package.

[mcol]

Yes, install.packages recompiles the package. R packages cannot contain executables or libraries, so will always (as far as I know) require compilation on installation.

[wwguo]

Thank you! Your answer saves me lots of time from keeping digging in and makes me so sad.

[Kbus007]

Hi everyone,

I am building a R package (not yet on CRAN) which depends on rstan and was built with rstantools. Following this thread, I was wondering if there is currently an easy way (i.e. one or two lines of code) for people to install the package without the need to compile all the stan models?

From the previous posts, it seems that install_github will automatically compile the stan models again even though the models were already pre-compiled.

Thanks for your help!

[bgoodri]

install_github requires compilation. I gather you are trying to commit the compiled version of the models to the GitHub repository, but the compiled code is depends on the OS, the compiler, and the version of R. So, it will not generally work on a user’s computer, which is why install_github builds it locally from source. Once you get your package onto CRAN, it builds binary versions for all recent versions of Mac and Windows.

[Kbus007]

Thanks a lot for your quick answer!

I will then ask the users to download a C++ compiler as explained on the rstan website to install the package from Github. Once the package is on CRAN, a C++ compiler should not be necessary anymore.