Rstan 2.26 crashes with no error when (accidentally) using # as comment

hsbadr · December 15, 2021, 3:19am

Install RcppParallel first from CRAN, preferably from source code, and check its flags:

install.packages("RcppParallel")

Then, install StanHeaders v2.26.6 followed by rstan v2.26.6 (will be ready soon).

remove.packages(c("rstan", "StanHeaders"))
install.packages("StanHeaders", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
install.packages("rstan", \repos = c("https://mc-stan.org/r-packages/", getOption("repos")))

stemangiola · December 15, 2021, 4:13am

Please let me know when x.x.6 is available.

I have installed RcppParallel

> install.packages("RcppParallel")
Installing package into ‘/stornext/Home/data/allstaff/m/mangiola.s/R/library_for_cell_sig_with_new_rstan/rstudio’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/RcppParallel_5.1.4.tar.gz'
Content type 'application/x-gzip' length 1967752 bytes (1.9 MB)

hsbadr · December 15, 2021, 4:26am

Sure thing.

Have you checked the flags? Also, it’s worth updating your R packages, which might cause a problem if you’ve upgraded major R version.

update.packages(checkBuilt = TRUE, ask = FALSE)

stemangiola · December 15, 2021, 4:49am

Sorry, I am unfamiliar with this vocabulary :) Could you be more explicit?

hsbadr · December 15, 2021, 4:54am

No problem :)
I mean:

RcppParallel::LdFlags()
RcppParallel::CxxFlags()

BTW. rstan/StanHeaders packages v2.26.6 are ready for testing. Good luck!

stemangiola · December 15, 2021, 6:25am

for version 5.1.4 I get empty (before LdFlags returned a value before I deleted rstan 2.26.5)

> RcppParallel::LdFlags()
> RcppParallel::CxxFlags()

And if I try to compile my model it “badly” fails.

(I appreciate the time you are spending)

hsbadr · December 15, 2021, 6:54am

You need to fix RcppParallel before installing or testing RStan.
Unset TBB environment variables:

Sys.unsetenv("TBB_LIB")
Sys.unsetenv("TBB_INC")

Then, install RcppParallel and check its compiler flags:

install.packages("RcppParallel", type = "source")
RcppParallel::LdFlags()
RcppParallel::CxxFlags()

Make sure that you have the latest Rtools (UCRT), if on Windows.

Then, proceed with RStan.

stemangiola · December 15, 2021, 7:15am

I am not sure why my RcppParallel is broken, and I can’t get the flags back

> unloadNamespace( asNamespace("rstan"))
> unloadNamespace( asNamespace("StanHeaders"))
> unloadNamespace( asNamespace("RcppParallel"))
> # Parallel
> .libPaths("~/R/library_for_cell_sig_with_new_rstan/rstudio")
> Sys.unsetenv("TBB_LIB")
> Sys.unsetenv("TBB_INC")
> install.packages("RcppParallel", type = "source")
Installing package into ‘/stornext/Home/data/allstaff/m/mangiola.s/R/library_for_cell_sig_with_new_rstan/rstudio’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/RcppParallel_5.1.4.tar.gz'
Content type 'application/x-gzip' length 1967752 bytes (1.9 MB)
==================================================
downloaded 1.9 MB

* installing *source* package ‘RcppParallel’ ...
** package ‘RcppParallel’ successfully unpacked and MD5 sums checked
** using staged installation
** preparing to configure package 'RcppParallel' ...
*** configured file: 'src/Makevars.in' => 'src/Makevars'
** finished configure for package 'RcppParallel'
** libs
(tbb) Building TBB using bundled sources ...
make[1]: Entering directory `/stornext/HPCScratch/mangiola.s_HPC_scratch/.Rtemp/RtmptF7fgH/R.INSTALL2a8e47fc33bb/RcppParallel/src/tbb/src'
make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
OS: linux
arch=intel64
compiler=gcc
runtime=cc11.1.0_libc2.17_kernel3.10.0
tbb_build_prefix=linux_intel64_gcc_cc11.1.0_libc2.17_kernel3.10.0
work_dir=/stornext/HPCScratch/mangiola.s_HPC_scratch/.Rtemp/RtmptF7fgH/R.INSTALL2a8e47fc33bb/RcppParallel/src/build/linux_intel64_gcc_cc11.1.0_libc2.17_kernel3.10.0_release
make[1]: Leaving directory `/stornext/HPCScratch/mangiola.s_HPC_scratch/.Rtemp/RtmptF7fgH/R.INSTALL2a8e47fc33bb/RcppParallel/src/tbb/src'
(tbb) TBB compilation finished successfully.
/stornext/System/data/apps/gcc/gcc-11.1.0/bin/g++ -std=gnu++14 -I"/stornext/System/data/apps/R/R-4.1.2/lib64/R/include" -DNDEBUG -I../inst/include  -I/usr/local/include  -std=gnu++11 -DRCPP_PARALLEL_USE_TBB=1 -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -fpic  -g -O2  -c init.cpp -o init.o
/stornext/System/data/apps/gcc/gcc-11.1.0/bin/g++ -std=gnu++14 -I"/stornext/System/data/apps/R/R-4.1.2/lib64/R/include" -DNDEBUG -I../inst/include  -I/usr/local/include  -std=gnu++11 -DRCPP_PARALLEL_USE_TBB=1 -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -fpic  -g -O2  -c options.cpp -o options.o
/stornext/System/data/apps/gcc/gcc-11.1.0/bin/g++ -std=gnu++14 -shared -L/stornext/System/data/apps/R/R-4.1.2/lib64/R/lib -L/usr/local/lib64 -o RcppParallel.so init.o options.o -L/stornext/System/data/apps/R/R-4.1.2/lib64/R/lib -lR
installing via 'install.libs.R' to /stornext/Home/data/allstaff/m/mangiola.s/R/library_for_cell_sig_with_new_rstan/rstudio/00LOCK-RcppParallel/00new/RcppParallel
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (RcppParallel)

The downloaded source packages are in
	‘/stornext/HPCScratch/mangiola.s_HPC_scratch/.Rtemp/RtmpCevbQD/downloaded_packages’
> RcppParallel::LdFlags()
> RcppParallel::CxxFlags()

hsbadr · December 15, 2021, 12:42pm

What’s the output of:

RcppParallel::tbbLibraryPath()

We could switch to this in RStan too.

hsbadr · December 15, 2021, 1:02pm

@stemangiola You reported that one of your models worked with v2.26.5. Does it still work with v2.26.6? If so, we can skip RcppParallel flags, since one of the models compiles successfully, and troubleshoot the failed model (which could be a different problem). Are you able to compile 8school model now?

stemangiola · December 16, 2021, 12:48am

I found the problem,

if the program has a comment starting with # it fails badly, not providing any error message and preventing the compilation of working models until the session is restarted.

hsbadr · December 16, 2021, 12:59am

Great! That fixes your model, but kind of different from the OP. You originally reported that the example fails to compile with v2.26.4, which is fixed by both v2.26.5 & v2.26.6; right? I’m trying to evaluate the changes in those versions.

As for the parser error, we handle some of those comment-related errors for stanc3, but it seems that we need to extend the preprocessor to handle more cases.

stemangiola · December 16, 2021, 1:12am

No.

I am 90% sure that the comment # crashes all three versions in the same way. I have eliminated the comment and the model works, but this does not depend on the versions.

hsbadr · December 16, 2021, 1:18am

Ok. Thanks!

stemangiola · December 16, 2021, 1:21am

Thanks for your help!

alex_pike · June 20, 2022, 2:45pm

Just to say - I came across this issue today. I think I have up to date versions of the packages mentioned: R version 4.1.0, RStan 2.26.3, Rcpp 1.0.8.3, StanHeaders 2.26.3. It would be great if there was an informative error message, rather than just failing with the unhelpful ‘parser failed badly’.

rok_cesnovar · June 20, 2022, 2:50pm

Can you try upgrading to the latest 2.26.X version and see if the issue persists? The current rstan experimental version is 2.26.11.

Run the following:

remove.packages(c("rstan", "StanHeaders"))
install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))

alex_pike · June 20, 2022, 3:03pm

Same issue - it may be worth mentioning I’m using hBayesDM, which is a wrapper around RStan.

packageVersion(‘rstan’)
[1] ‘2.26.11’
packageVersion(‘StanHeaders’)
[1] ‘2.26.11’

hsbadr · June 24, 2022, 10:14pm

That’s a known issue that we need to fix when parsing the headers from the code. The haspound conditional needs to be updated to distinguish between #include & comments.

Topic		Replies	Views
Future rstan is broken with clang++ Developers	66	3476	November 11, 2019
Stan not running after updating to r version 4.0.2 and stan updated from 2.19.3 to 2.21 RStan	72	6487	September 20, 2020
Rstan crashes R at end of model compilation RStan rstan , compiler , brms	8	852	April 12, 2021
HELP！Can someone help me find out why this error is reported? RStan rstan	19	1102	November 3, 2022
Error in rstan_2.16.2 after upgrading to R 3.4.1 with Clang 4.0.0 RStan	60	6462	December 5, 2017

Rstan 2.26 crashes with no error when (accidentally) using # as comment

Related topics