Compilation error when manually specifying gradients

Interfaces RStan
1

Hello,

I have a model that ran too slowly, without divergences or any other issues in sampling, so I manually specified gradients for the functions in the likelihood. The functions take vector arguments and return a vector. When compiling the model on RStan and Windows 10 x64, I get the following error.

Error in compileCode(f, code, language = language, verbose = verbose) :
Compilation ERROR, function(s)/method(s) not created! In file included from E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/functor/integrate_ode_adams.hpp:4:0,
from E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat.hpp:68,
from E:\Downloads\Software\Stanmath\math-develop/stan/math.hpp:4,
from E:/R/library/StanHeaders/include/src/stan/model/model_header.hpp:4,
from file2441e9f602.cpp:8:
E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/functor/cvodes_integrator.hpp: In member function ‘std::vector<std::vector<typename stan::return_type<T_initial, T_param>::type> > stan::math::cvodes_integrator::integrate(const F&, const std::vector<T_initial>&, double, const std::vector&, const std::vector<T_param>&, const std::vector&, const std::vector&, std::ostream*, double, double, long int)’:
E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/fun
In addition: Warning message:
In system(cmd, intern = !verbose) :
running command ‘E:/R/bin/x64/R CMD SHLIB file2441e9f602.cpp 2> file2441e9f602.cpp.err.txt’ had status 1
Error in sink(type = “output”) : invalid connection

On compiling the model with verbose=TRUE, I found this at the location of the first error.

Compilation argument:
E:/R/bin/x64/R CMD SHLIB file244154a2103.cpp 2> file244154a2103.cpp.err.txt
In file included from E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/functor/integrate_ode_adams.hpp:4:0,
from E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat.hpp:68,
from E:\Downloads\Software\Stanmath\math-develop/stan/math.hpp:4,
from E:/R/library/StanHeaders/include/src/stan/model/model_header.hpp:4,
from file244154a2103.cpp:8:
E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/functor/cvodes_integrator.hpp: In member function ‘std::vector<std::vector<typename stan::return_type<T_initial, T_param>::type> > stan::math::cvodes_integrator::integrate(const F&, const std::vector<T_initial>&, double, const std::vector&, const std::vector<T_param>&, const std::vector&, const std::vector&, std::ostream*, double, double, long int)’:
E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/functor/cvodes_integrator.hpp:111:39: error: too few arguments to function ‘void* CVodeCreate(int, int)’
void* cvodes_mem = CVodeCreate(Lmm);
^
In file included from E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/functor/cvodes_utils.hpp:4:0,
from E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat.hpp:66,
from E:\Downloads\Software\Stanmath\math-develop/stan/math.hpp:4,
from E:/R/library/StanHeaders/include/src/stan/model/model_header.hpp:4,
from file244154a2103.cpp:8:
E:\R\library\StanHeaders\include/cvodes/cvodes.h:546:23: note: declared here
SUNDIALS_EXPORT void *CVodeCreate(int lmm, int iter);
^
file244154a2103.cpp: At global scope:
file244154a2103.cpp:91:48: error: ‘model24468e1abe_deconv_db_trapz_grad_namespace::stan::io’ has not been declared
model24468e1abe_deconv_db_trapz_grad(stan::io::var_context& context__,
^
file244154a2103.cpp:91:63: error: expected ‘)’ before ‘&’ token
model24468e1abe_deconv_db_trapz_grad(stan::io::var_context& context__,
^
file244154a2103.cpp:97:48: error: ‘model24468e1abe_deconv_db_trapz_grad_namespace::stan::io’ has not been declared
model24468e1abe_deconv_db_trapz_grad(stan::io::var_context& context__,
^
file244154a2103.cpp:97:63: error: expected ‘)’ before ‘&’ token
model24468e1abe_deconv_db_trapz_grad(stan::io::var_context& context__,
^
file244154a2103.cpp:104:26: error: ‘model24468e1abe_deconv_db_trapz_grad_namespace::stan::io’ has not been declared
void ctor_body(stan::io::var_context& context__,
^
file244154a2103.cpp:104:41: error: expected ‘,’ or ‘…’ before ‘&’ token
void ctor_body(stan::io::var_context& context__,
^
file244154a2103.cpp:400:38: error: ‘io’ in namespace ‘model24468e1abe_deconv_db_trapz_grad_namespace::stan’ does not name a type
void transform_inits(const stan::io::var_context& context__,
^
file244154a2103.cpp:400:53: error: expected unqualified-id before ‘&’ token
void transform_inits(const stan::io::var_context& context__,

I also attempted to compile the model on Ubuntu 18.04, and got the following error under verbose=TRUE.

file13e84e54d7a6.cpp:91:50: error: ‘model13e82a6ed0e9_deconv_db_trapz_grad_namespace::stan::io’ has not been declared
model13e82a6ed0e9_deconv_db_trapz_grad(stan::io::var_context& context__,
^~
file13e84e54d7a6.cpp:91:65: error: expected ‘)’ before ‘&’ token
model13e82a6ed0e9_deconv_db_trapz_grad(stan::io::var_context& context__,
^
file13e84e54d7a6.cpp:97:50: error: ‘model13e82a6ed0e9_deconv_db_trapz_grad_namespace::stan::io’ has not been declared
model13e82a6ed0e9_deconv_db_trapz_grad(stan::io::var_context& context__,
^~
file13e84e54d7a6.cpp:97:65: error: expected ‘)’ before ‘&’ token
model13e82a6ed0e9_deconv_db_trapz_grad(stan::io::var_context& context__,
^
file13e84e54d7a6.cpp:104:26: error: ‘model13e82a6ed0e9_deconv_db_trapz_grad_namespace::stan::io’ has not been declared
void ctor_body(stan::io::var_context& context__,
^~
file13e84e54d7a6.cpp:104:41: error: expected ‘,’ or ‘…’ before ‘&’ token
void ctor_body(stan::io::var_context& context__,
^
file13e84e54d7a6.cpp:400:38: error: ‘io’ in namespace ‘model13e82a6ed0e9_deconv_db_trapz_grad_namespace::stan’ does not name a type
void transform_inits(const stan::io::var_context& context__,
^~
file13e84e54d7a6.cpp:400:53: error: expected unqualified-id before ‘&’ token
void transform_inits(const stan::io::var_context& context__,
^

2

It is unlikely anyone is going to be able to fix this without seeing the code.

3

The relevant .hpp and .stan files are in this repository.

4

OK, I will look at it after the R packages for 2.19 get released. At first glance, I think the problem is that Stan is passing the arguments as var but your C++ signatures are defined for double. So, you need to take in the var versions, demote them to double versions with stan::math::value_of, compute your stuff, and then call precomputed_gradients() at the end.

5

The first four lines of each Stan interface function is supposed to take care of that. For example, in Mgradstan.hpp, lines 68-72, I take the const Eigen::Matrix<var, -1, 1> alphabeta argument and change it to a vector using

Eigen::VectorXd alphabeta_d (alphabeta.size());
for (int i=0; i<alphabeta_d.size(); i++)
{
alphabeta_d(i) = alphabeta(i).val();
}

Would this not do the same job as stan::math::value_of?

6

Yeah, in theory that should be fine. I haven’t looked closely at it yet.

7

I just ran a few tests, and I get that error even when I run the code in the vignette here: Interfacing with External C++ Code.

I ran the sinc.hpp example given there. The compiled code section provided there matches up with my output (up to where it is truncated, of course), but I get the same error when I attempt to compile the model.

Error in compileCode(f, code, language = language, verbose = verbose) :
Compilation ERROR, function(s)/method(s) not created! In file included from E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/functor/integrate_ode_adams.hpp:4:0,
from E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat.hpp:68,
from E:\Downloads\Software\Stanmath\math-develop/stan/math.hpp:4,
from E:/R/library/StanHeaders/include/src/stan/model/model_header.hpp:4,
from file1fbc144d1ea7.cpp:8:
E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/functor/cvodes_integrator.hpp: In member function ‘std::vector<std::vector<typename stan::return_type<T_initial, T_param>::type> > stan::math::cvodes_integrator::integrate(const F&, const std::vector<T_initial>&, double, const std::vector&, const std::vector<T_param>&, const std::vector&, const std::vector&, std::ostream*, double, double, long int)’:
E:\Downloads\Software\Stanmath\math-develop/stan/math/rev/mat/f
In addition: Warning message:
In system(cmd, intern = !verbose) :
running command ‘E:/R/bin/x64/R CMD SHLIB file1fbc144d1ea7.cpp 2> file1fbc144d1ea7.cpp.err.txt’ had status 1
Error in sink(type = “output”) : invalid connection

When verbose=TRUE is set, these are the first errors I get.

E:\R\library\StanHeaders\include/cvodes/cvodes.h:546:23: note: declared here
SUNDIALS_EXPORT void *CVodeCreate(int lmm, int iter);
^
file1fbc3e3430ba.cpp: At global scope:
file1fbc3e3430ba.cpp:40:38: error: ‘model1fbc770a482d_external_namespace::stan::io’ has not been declared
model1fbc770a482d_external(stan::io::var_context& context__,
^
file1fbc3e3430ba.cpp:40:53: error: expected ‘)’ before ‘&’ token
model1fbc770a482d_external(stan::io::var_context& context__,
^
file1fbc3e3430ba.cpp:46:38: error: ‘model1fbc770a482d_external_namespace::stan::io’ has not been declared
model1fbc770a482d_external(stan::io::var_context& context__,
^
file1fbc3e3430ba.cpp:46:53: error: expected ‘)’ before ‘&’ token
model1fbc770a482d_external(stan::io::var_context& context__,
^
file1fbc3e3430ba.cpp:53:26: error: ‘model1fbc770a482d_external_namespace::stan::io’ has not been declared
void ctor_body(stan::io::var_context& context__,
^

8

This problem has been solved. I switched to cmdstan to compile using Stan 2.19, and the problem went away.