R package compile problems on certain platforms

Charles_Driver · July 10, 2020, 9:15am

This is pretty specific and @bgoodri may be the only one who can help.
R package ctsem has caught the Ripley virus and I’ve been given very limited time to resolve every CRAN check error shown here: https://cran.r-project.org/web/checks/check_results_ctsem.html

Basically I’m having trouble building on:
r-devel-linux-x86_64-debian-clang
r-patched-linux-x86_64
r-patched-solaris-x86

I’m at a bit of a loss and pretty ready to support CRAN alternatives or therapy sessions, but maybe somebody can help…

wds15 · July 10, 2020, 9:29am

I had similar trouble a while ago with OncoBayes2. Look at the change logs from the R package

https://cran.r-project.org/web/packages/OncoBayes2/index.html

there you see that I had to do multiple attempts to fix the sanitiser issues. You should compare the stan model between the releases where it was fixed - it’s some problematic indexing stuff. There is an issue in stan somewhere, but we have not yet gotten to the bottom of it.

So basically slicing with indices into data-structures is the root cause which you have to avoid somehow.

Charles_Driver · July 10, 2020, 9:37am

Thanks… are you saying that any references of e.g. matrices like:
mymatrix[intarray, intarray]
is going to be problematic? If so then I guess I should give up now…

wds15 · July 10, 2020, 9:48am

avoiding these resolved my problem, yes. You should look up the issue for this for more details (sorry, I don’t find it right now).

bgoodri · July 10, 2020, 1:43pm

I’ve got to do a presentation in an hour but I will help figure it out this afternoon.

Charles_Driver · July 10, 2020, 5:29pm

‘Maybe’ I found a problem – instead of the following in makevars:
-DBOOST_MATH_OVERFLOW_ERROR_POLICY=errno_on_error
I had the windows argument:
-DRCPP_PARALLEL_USE_TBB=1

The original form compiled on my ubuntu with gcc and clang, but it’s my only guess / hope so far.

bgoodri · July 10, 2020, 6:42pm

You can have both and probably should. Are we talking about

https://www.r-project.org/nosvn/R.check/r-patched-solaris-x86/ctsem-00install.html

or something else?

Charles_Driver · July 10, 2020, 6:44pm

There is also:
https://www.r-project.org/nosvn/R.check/r-devel-linux-x86_64-debian-clang/ctsem-00install.html
https://www.r-project.org/nosvn/R.check/r-patched-linux-x86_64/ctsem-00install.html

bgoodri · July 10, 2020, 6:49pm

Those look like it either timed out or ran out of RAM. One thing that I think all of the R packages can move toward is having a separate generated quantities block that is drawn from by calling rstan::gqs. Have you worked with standalone generated quantities before?

Charles_Driver · July 10, 2020, 6:53pm

No, but I already compile a separate object for the complex generated quantities… I notice when compiling with clang 10 on Ubuntu that I am getting a hang when anything greater than -O1 is set, though this doesn’t seem to be the case on CRAN…

Charles_Driver · July 10, 2020, 7:07pm

Yes actually it seems to hang at the same point where it is killed on the debian clang box, when I use -O2 or greater with clang on ubuntu:

* installing *source* package ‘ctsem’ ...
** using staged installation
** libs
"/usr/lib/R/bin/Rscript" -e "source(file.path('..', 'tools', 'make_cc.R')); make_cc(commandArgs(TRUE))" stan_files/ctsm.stan
Wrote C++ file "stan_files/ctsm.cc"


clang++ -std=gnu++14 -I"/usr/share/R/include" -DNDEBUG -I"../inst/include" -I"/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src" -DBOOST_DISABLE_ASSERTS -DEIGEN_NO_DEBUG -DBOOST_MATH_OVERFLOW_ERROR_POLICY=errno_on_error -I'/home/driver/R/x86_64-pc-linux-gnu-library/4.0/BH/include' -I'/home/driver/R/x86_64-pc-linux-gnu-library/4.0/Rcpp/include' -I'/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include' -I'/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppParallel/include' -I'/home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include' -I'/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include'    -I'/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppParallel/include' -D_REENTRANT -DSTAN_THREADS   -fpic  -mtune=native -march=native -fPIC -fpic  -O2 -Wall  -c stan_files/ctsm.cc -o stan_files/ctsm.o
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Dense:1:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Core:535:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Dense:2:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/LU:47:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Dense:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Cholesky:12:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Jacobi:29:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Dense:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Cholesky:43:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Dense:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/QR:17:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Householder:27:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Dense:5:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/SVD:48:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Dense:6:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Geometry:58:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:13:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Dense:7:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Eigenvalues:58:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:14:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Sparse:26:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/SparseCore:66:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:14:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Sparse:27:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/OrderingMethods:71:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:14:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Sparse:29:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/SparseCholesky:43:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:14:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Sparse:32:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/SparseQR:35:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:3:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp:14:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/Sparse:33:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/IterativeLinearSolvers:46:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:22:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/RcppEigen.h:25:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/RcppEigenForward.h:32:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/CholmodSupport:45:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:22:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/RcppEigen.h:25:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/RcppEigenForward.h:35:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/unsupported/Eigen/KroneckerProduct:34:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/unsupported/Eigen/../../Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:22:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/RcppEigen.h:25:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/RcppEigenForward.h:39:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/unsupported/Eigen/Polynomials:135:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/unsupported/Eigen/../../Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:22:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/RcppEigen.h:25:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/RcppEigenForward.h:40:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/unsupported/Eigen/SparseExtra:51:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/unsupported/Eigen/../../Eigen/src/Core/util/ReenableStupidWarnings.h:10:30: warning: pragma diagnostic pop could not pop, no matching push [-Wunknown-pragmas]
    #pragma clang diagnostic pop
                             ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:35:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:12:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat.hpp:283:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/prob/gaussian_dlm_obs_rng.hpp:138:7: warning: unused variable 'n' [-Wunused-variable]
  int n = G.rows();  // number of states
      ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:35:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:17:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat/fun/columns_dot_self.hpp:8:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat/fun/dot_self.hpp:38:41: warning: all paths through this function will call itself [-Winfinite-recursion]
  inline static double square(double x) { return square(x); }
                                        ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:35:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:70:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat/functor/integrate_ode_adams.hpp:5:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat/functor/cvodes_integrator.hpp:126:18: warning: unused variable 'coupled_size' [-Wunused-variable]
    const size_t coupled_size = cvodes_data.coupled_ode_.size();
                 ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:35:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:6:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/core.hpp:46:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints.hpp:14:13: warning: unused function 'set_zero_all_adjoints' [-Wunused-function]
static void set_zero_all_adjoints() {
            ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:35:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:6:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/core.hpp:47:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/core/set_zero_all_adjoints_nested.hpp:17:13: warning: 'static' function 'set_zero_all_adjoints_nested' declared in header file should be declared 'static inline' [-Wunneeded-internal-declaration]
static void set_zero_all_adjoints_nested() {
            ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:35:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:12:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat.hpp:278:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/prob/dirichlet_log.hpp:5:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/prob/dirichlet_lpmf.hpp:60:9: warning: unused type alias 'T_partials_vec' [-Wunused-local-typedef]
  using T_partials_vec = typename Eigen::Matrix<T_partials_return, -1, 1>;
        ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:35:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:12:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat.hpp:328:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/prob/poisson_log_glm_log.hpp:5:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/prim/mat/prob/poisson_log_glm_lpmf.hpp:52:9: warning: unused type alias 'T_alpha_val' [-Wunused-local-typedef]
  using T_alpha_val = typename std::conditional_t<
        ^
In file included from stan_files/ctsm.cc:3:
In file included from stan_files/ctsm.hpp:18:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/rstaninc.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/rstan/include/rstan/stan_fit.hpp:35:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/services/diagnose/diagnose.hpp:10:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/test_gradients.hpp:7:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/src/stan/model/log_prob_grad.hpp:4:
In file included from /home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat.hpp:51:
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat/fun/squared_distance.hpp:27:11: warning: unused type alias 'idx_t' [-Wunused-local-typedef]
    using idx_t = typename index_type<matrix_v>::type;
          ^
/home/driver/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat/fun/squared_distance.hpp:64:11: warning: unused type alias 'idx_t' [-Wunused-local-typedef]
    using idx_t = typename index_type<matrix_d>::type;
          ^
In file included from stan_files/ctsm.cc:3:
stan_files/ctsm.hpp:1712:24: warning: unused typedef 'local_scalar_t__' [-Wunused-local-typedef]
        typedef double local_scalar_t__;
                       ^

bgoodri · July 10, 2020, 7:14pm

clang++ is usually faster to compile but it can sometimes be slower on giant files. For me, clang++-10 eventually installs ctsem under -O3 but ctsem.o is 11MB. The largest one in rstanarm is about 7.5 MB.

What object were you referring to when you said you had a separate generated quantities?

Charles_Driver · July 10, 2020, 7:16pm

ok. ctsmgen.stan is used for prior / posterior checks. ctsm.stan (the main file) contains about 60 lines of generated quantities.

bgoodri · July 10, 2020, 7:18pm

OK. Let me see if I can put together a PR.

Charles_Driver · July 10, 2020, 7:20pm

gosh. good luck :/ and thanks!

bgoodri · July 10, 2020, 8:05pm

I started to do this on the gqs branch that I just invited you to on GitHub. At this point, you could get the generated quantities that were formerly in the ctsm.stan file by calling

gqs(stanmodels$ctsm_gqs, data_list, draws = as.matrix(post))

where post is produced by sampling(stanmodels$ctsm, data_list, ...).

The next step I am afraid I will mess up without understanding the details of the model, but basically everything that is in the transformed parameters block of ctsm.stan that isn’t needed to evaluate the posterior kernel needs to get moved into the generated quantities block of ctsm_gqs.stan (and then deleted from its parameters block). This way, ctsm.stan will get smaller and ctsm_gqs.stan will get bigger but the compilation should be easier.

Charles_Driver · July 10, 2020, 8:11pm

Ok. I am working on condensing some things. I can’t really see the more major step you propose working though – yes there is a bunch of stuff that can be taken out of transformed pars and put in gen quantities, but gen quantities then depends on nearly everything else in the model anyway so would have to get very large. I’ll think on it and try my luck I guess. Thanks.

bgoodri · July 10, 2020, 8:18pm

That is the idea. But the things that are in the transformed parameters block of ctsm.stan don’t have to be recalculated for ctsm_gqs.stan. You can just pass them back their realizations as part of that big matrix called draws in rstan::gqs. So, it is mostly just a manner of redistributing the code across ctsm.stan and ctsm_gqs.stan so that each is bite-sized.

Charles_Driver · July 10, 2020, 8:23pm

Sure. But I’m not storing all the process / measurement matrices that may change at each time point, because that generates headaches of it’s own, I think – and they all have to be regenerated to compute the interesting quantities. No doubt some improvement is possible, I just don’t think it’s so straightforward unfortunately :/

bgoodri · July 10, 2020, 8:24pm

It isn’t so straightforward, which is why I am scared to do it. But try it and then I’ll run over it again.

Topic		Replies	Views
R pkg using rstantools cran pre-check failed Developers	2	404	June 7, 2018
Stan & R 4.2 on Windows General	16	2446	July 28, 2022
Cran issues with Stan packages on r-devel-linux-x86_64-debian-clang + r-devel-linux-x86_64-debian-gcc Developers	5	617	December 4, 2019
R 4.3, rstan 2.26.22, windows, github actions -- errors when compiling models from package Developers	2	504	May 4, 2023
R4.0, rtools 4.0 and rstan 2.19 on windows Interfaces	2	602	April 24, 2020

R package compile problems on certain platforms

Related topics