Not reproducible compilation error with rstan calling Rcpp date_datetime during cluster execution of a snakemake file


I got the following not reproducible error for one of my brms models while executing a snakemake pipeline on a DRMAA cluster. When I rerun the job, it worked. Any ideas why?
Thank you in advance.

Compiling the C++ model
Error in compileCode(f, code, language = language, verbose = verbose) : 
  Compilation ERROR, function(s)/method(s) not created! In file included from /home/[username]/miniconda3/envs/[env_name]/lib/R/library/Rcpp/include/Rcpp.h:62:0,
             from /home/[username]/miniconda3/envs/[env_name]/lib/R/library/rstan/include/rstan/io/rlist_ref_var_context.hpp:16,
             from /home/[username]/miniconda3/envs/[env_name]/lib/R/library/rstan/include/rstan/stan_fit.hpp:18,
             from /home/[username]/miniconda3/envs/[env_name]/lib/R/library/rstan/include/rstan/rstaninc.hpp:3,
             from file781663cd529b.cpp:1060:
/home/[username]/miniconda3/envs/[env_name]/lib/R/library/Rcpp/include/Rcpp/date_datetime/date_datetime.h:25:10: fatal error: /home/[username]/miniconda3/envs/[env_name]/lib/R/library/Rcpp/include/Rcpp/date_datetime/Date.h: Remote I/O error
compilation terminated.
make: *** [/home/[username]/miniconda3/envs/[env_name]/lib/R/etc/Makeconf:167: file781663cd529b.o] Error 1
Calls: build_models ... cxxfunctionplus -> <Anonymous> -> cxxfunction -> compileCode
In addition: Warning message:
In system(cmd, intern = !verbose) :
  running command '/home/[username]/miniconda3/envs/[env_name]/lib/R/bin/R CMD SHLIB file781663cd529b.cpp 2> file781663cd529b.cpp.err.txt' had status 1
Error in sink(type = "output") : invalid connection
Calls: build_model
  • Operating System:
    Platform: x86_64-conda_cos6-linux-gnu (64-bit)
    Running under: Debian GNU/Linux 8 (jessie)
  • brms Version: r-brms=2.9.0
  • r-rcpp=1.0.1
  • r-rcppeigen=
  • r-base=3.5.1

I am sorry, I don’t know what this error means. If someone has an idea that probably @bgoodri.

Not really. The various Conda setups rarely work with RStan but the Remote I/O error suggests it might have been something random.

Thanks anyway. I think it might be due to file system latency or sth like that.

@paul.buerkner What happens in the sink command in cxxfunctionplus. Is it possible that a temporary file is written? Where is the temporary directory? I wonder if this is the actual problem of my errors.

I got a lot of these compilation errors, all different libraries, all having the Error in sink(type=“output”)" invalid connection error in the end. The compilation errors usually disappear if I rerun the jobs or if I run the script not on the cluster. It still bothers me a lot.

This does not happen on the brms side but rather in rstan or even deeper so I really don’t know what is happening.

Thank you anyway!

Perhaps you could try changing the default setting of the temporary directory and see if it gets you any further: