Issues including stan math autodiff functions in R package

[eweine]

Hey all,

I’m attempting to use the autograd function from stan math in an R package that I’m building. However, I’m having trouble getting even a simple example to compile.

I’m attempting to compile the following code which I got from the Stan Math Vignette

// [[Rcpp::depends(BH)]]
// [[Rcpp::depends(RcppEigen)]]
// [[Rcpp::depends(RcppParallel)]]
// [[Rcpp::depends(StanHeaders)]]
#include <stan/math/mix.hpp> // stuff from mix/ must come first
#include <stan/math.hpp>     // finally pull in everything from rev/ and prim/
#include <Rcpp.h>
#include <RcppEigen.h>       // do this AFTER including stuff from stan/math

// [[Rcpp::plugins(cpp17)]]

/* Objective function */

// [[Rcpp::export]]
auto f(Eigen::VectorXd x, Eigen::VectorXd a) { // objective function in doubles
  using stan::math::dot_self;                  // dot_self() is a dot product with self
  return dot_self( (x - a) );
}

/* Gradient */

// [[Rcpp::export]]
auto g(Eigen::VectorXd x, Eigen::VectorXd a) {  // gradient by AD using Stan
  double fx;
  Eigen::VectorXd grad_fx;
  using stan::math::dot_self;
  stan::math::gradient([&a](auto x) { return dot_self( (x - a) ); },
                       x, fx, grad_fx);
  return grad_fx;
}

However, when I attempt to compile this, I get the following output:

   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"../inst/include" -I"/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/StanHeaders/include/src" -DBOOST_DISABLE_ASSERTS -DEIGEN_NO_DEBUG -DBOOST_MATH_OVERFLOW_ERROR_POLICY=errno_on_error  -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/BH/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppArmadillo/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppParallel/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/rstan/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/StanHeaders/include' -I/opt/R/arm64/include    -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppParallel/include' -D_REENTRANT -DSTAN_THREADS   -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c RcppExports.cpp -o RcppExports.o
   In file included from RcppExports.cpp:5:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/RcppEigen.h:25:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/RcppEigenForward.h:29:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/Eigen/Sparse:26:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/Eigen/SparseCore:61:
   /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/Eigen/src/SparseCore/TriangularSolver.h:273:13: warning: variable 'count' set but not used [-Wunused-but-set-variable]
         Index count = 0;
               ^
   In file included from RcppExports.cpp:5:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/RcppEigen.h:25:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/RcppEigenForward.h:29:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/Eigen/Sparse:29:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/Eigen/SparseLU:35:
   /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/Eigen/src/SparseLU/SparseLU_heap_relax_snode.h:78:9: warning: variable 'nsuper_et_post' set but not used [-Wunused-but-set-variable]
     Index nsuper_et_post = 0; // Number of relaxed snodes in postordered etree 
           ^
   /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/Eigen/src/SparseLU/SparseLU_heap_relax_snode.h:79:9: warning: variable 'nsuper_et' set but not used [-Wunused-but-set-variable]
     Index nsuper_et = 0; // Number of relaxed snodes in the original etree 
           ^
   In file included from RcppExports.cpp:5:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/RcppEigen.h:25:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/RcppEigenForward.h:31:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/unsupported/Eigen/IterativeSolvers:46:
   /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/unsupported/Eigen/src/IterativeSolvers/IDRS.h:72:10: warning: variable 'replacements' set but not used [-Wunused-but-set-variable]
                           Index replacements = 0;
                                 ^
   In file included from RcppExports.cpp:5:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/RcppEigen.h:25:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/RcppEigenForward.h:37:
   In file included from /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/unsupported/Eigen/SparseExtra:45:
   /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppEigen/include/unsupported/Eigen/src/SparseExtra/MarketIO.h:246:7: warning: variable 'count' set but not used [-Wunused-but-set-variable]
     int count = 0;
         ^
   RcppExports.cpp:23:34: error: function 'f' with deduced return type cannot be used before it is defined
       rcpp_result_gen = Rcpp::wrap(f(x, a));
                                    ^
   RcppExports.cpp:16:6: note: 'f' declared here
   auto f(Eigen::VectorXd x, Eigen::VectorXd a);
        ^
   RcppExports.cpp:35:34: error: function 'g' with deduced return type cannot be used before it is defined
       rcpp_result_gen = Rcpp::wrap(g(x, a));
                                    ^
   RcppExports.cpp:28:6: note: 'g' declared here
   auto g(Eigen::VectorXd x, Eigen::VectorXd a);
        ^
   5 warnings and 2 errors generated.
   make: *** [RcppExports.o] Error 1
   ERROR: compilation failed for package ‘vbGLMMR’
─  removing ‘/private/var/folders/vp/3nx4m_j97gx29nvg87trnw940000gn/T/RtmpoX4D9g/devtools_install_9b5b4afaaf2c/vbGLMMR’
Error in `(function (command = NULL, args = character(), error_on_status = TRUE, …`:
! System command 'R' failed
---
Exit status: 1
stdout & stderr: <printed>
---

Below is my Makevars file:

STANHEADERS_SRC = $(shell "$(R_HOME)/bin$(R_ARCH_BIN)/Rscript" -e "message()" -e "cat(system.file('include', 'src', package = 'StanHeaders', mustWork = TRUE))" -e "message()" | grep "StanHeaders")

STANC_FLAGS = $(shell "$(R_HOME)/bin$(R_ARCH_BIN)/Rscript" -e "cat(ifelse(utils::packageVersion('rstan') >= 2.26, '-DUSE_STANC3',''))")
PKG_CPPFLAGS = -I"../inst/include" -I"$(STANHEADERS_SRC)" -DBOOST_DISABLE_ASSERTS -DEIGEN_NO_DEBUG -DBOOST_MATH_OVERFLOW_ERROR_POLICY=errno_on_error $(STANC_FLAGS)
PKG_CXXFLAGS = $(shell "$(R_HOME)/bin$(R_ARCH_BIN)/Rscript" -e "RcppParallel::CxxFlags()") $(shell "$(R_HOME)/bin$(R_ARCH_BIN)/Rscript" -e "StanHeaders:::CxxFlags()")
PKG_LIBS = $(shell "$(R_HOME)/bin$(R_ARCH_BIN)/Rscript" -e "RcppParallel::RcppParallelLibs()") $(shell "$(R_HOME)/bin$(R_ARCH_BIN)/Rscript" -e "StanHeaders:::LdFlags()")

CXX_STD = CXX17

In my description, I have

Imports:
    methods,
    Rcpp (>= 1.0.8),
    RcppParallel (>= 5.0.1),
    reticulate,
    rstan (>= 2.18.1),
    rstantools (>= 2.4.0)
LinkingTo: 
    BH (>= 1.66.0),
    Rcpp (>= 0.12.0),
    RcppArmadillo,
    RcppEigen (>= 0.3.3.3.0),
    RcppParallel (>= 5.0.1),
    rstan (>= 2.18.1),
    StanHeaders (>= 2.18.0)

I’m on Mac OS if that makes a difference. I also have rstan version 2.32.6 (Stan version 2.32.2) installed.

Any help with the above would be greatly appreciated! I searched various forum posts and github repos to assemble this makevars / description, but perhaps I am not up to date on the latest advice and am doing something wrong.

Thanks for your help!

Eric.

[Bob_Carpenter]

This discussion has moved here:

Issue getting Hessian with stan math

Hey all, I’m having trouble getting a hang of the autodiff capabilities of stan math. I’m attempting to derive the gradient and the Hessian of the function f(L) = Tr(LL^{T}A), where A is a known matrix and L is a lower triangular matrix. I’m using the log-cholesky parameterization here for convenience because ultimately I would like to solve an optimization problem (that is much more complex than this) with a unique solution. Below is my code to compute the gradient: // [[Rcpp::export]] Eige…