Compilation flags to solve "too many sections" in package that depends on rstan

General
1

Hi! I have an R package that depends on rstan (repo here). On some (not all) machines, I get a compilation error when running devtools::install() or devtools::document():

as: stanExports_joint_continuous.o: too many sections (56364)
   C:\Users\abiga\AppData\Local\Temp\ccTyskic.s: Assembler messages:
   C:\Users\abiga\AppData\Local\Temp\ccTyskic.s: Fatal error: can't write 150 bytes to section .text of stanExports_joint_continuous.o: 'file too big'
    as: stanExports_joint_continuous.o: too many sections (56364)
    C:\Users\abiga\AppData\Local\Temp\ccTyskic.s: Fatal error: stanExports_joint_continuous.o: file too big
   make: *** [C:/PROGRA~1/R/R-43~1.3/etc/x64/Makeconf:270: stanExports_joint_continuous.o] Error 1
   ERROR: compilation failed for package 'eDNAjoint'

I’ve noticed similar posts:

  1. post 1 that mentions having: -O3 in CXX14FLAGS of the ~/.R/Makevars file
  2. post 2 that mentions adding PKG_CXXFLAGS += -Wa,-mbig-obj to src/Makevars.win.

I have tried the following edits to src/Makevars:

  1. Added an -O3 compilation flag: PKG_CXXFLAGS = $(shell "$(R_HOME)/bin$(R_ARCH_BIN)/Rscript" -e "RcppParallel::CxxFlags()") $(shell "$(R_HOME)/bin$(R_ARCH_BIN)/Rscript" -e "StanHeaders:::CxxFlags()") -O3
  2. Added PKG_CXXFLAGS += -Wa,-mbig-obj as a new line below CXX_STD = CXX17.

Neither of these solutions worked for me. I’m wondering if anyone here has any tips? Also, I’m curious if I am editing the correct Makevars file? I’ve noticed that when I edit src/Makevars.win rather than src/Makevars, edits get removed when I try to compile.

Please let me know if there is more information I can share.

Thanks so much!
Best,
Abby Keller

2

Hi Abby, this has generally only been an issue for local testing when using RStudio, the error doesn’t occur during normal installation of the package.

For adding the compiler flags you mentioned, you need to edit the configure script in your package which calls rstantools. I’ve got examples of this here: Pkgbuild::compile_dll() on previously working rstantools-based package fails with "file too big" error - #17 by andrjohns

If that doesn’t work for you, feel free to leave a link to your package source and I’ll debug locally

1 Like
3

@andrjohns This worked! Thank you so much!

4

@andrjohns

Thanks for your help with this! I actually have one follow up question.

I’m hoping that users with different operating systems will be able to install the package from source. When I add the configuration to the configure and configure.win files, I can install with Windows but I get the following error in Linux:

as: unrecognized option '-mbig-obj' make: *** [/usr/lib/R/etc/Makeconf:204: RcppExports.o] Error 1

When I try to conditionally add the -mbig-obj flag based on the operating system:

echo -e "ifeq (\$(OS_TYPE), windows)\n\tPKG_CXXFLAGS += -Wa,-mbig-obj\nendif" >> ./src/Makevars

I get the following error:

Makevars:11: *** missing separator. Stop. Warning in system(paste(MAKE, p1(paste("-f", shQuote(makefiles))), "compilers"), : running command 'make -f 'Makevars' -f '/usr/lib/R/etc/Makeconf' -f '/usr/share/R/share/make/shlib.mk' compilers' had status 2
Error in if (nzchar(cxx)) { : argument is of length zero

Do you have any tips? I can also just alert Windows users to add these flags themselves, but it would be nice to do this conditionally.

Thanks so much for your help!
Best,
Abby

5

You can just remove it from the configure file, since only configure.win is run on windows

1 Like
6

Oh right - thanks so much!