External C++ Code with Precomputed Gradients

Interfaces CmdStan
1

Hi everybody,

I’m trying to compile a model using an external C++ code with precomputed gradients in CmdStan following the examples of the PyStan and the RStan interfaces docs.

For that purpose, I’m using the following toy example:

  • external_grad.stan file:
functions {
    real my_func(real A, real B);
}
data {
    real B;
}
parameters {
    real A;
}
transformed parameters {
    real C = my_func(A, B);
}
model {
    C ~ std_normal();
}
  • external_manual.hpp file:
#include <cmath>
#include <ostream>
#include <stan/math.hpp>

namespace external_grad_model_namespace {

	using namespace stan::math;

	inline double my_func (double A, double B, std::ostream* pstream) {
	  double C = A*B*B+A;

	  return C;
	}

	inline var my_func (const var& A_var, const var& B_var, std::ostream* pstream) {
	  // Compute the value
	  double A = A_var.val(),
		 B = B_var.val(),
		 C = my_func(A, B, pstream);

	  // Compute the partial derivatives:
	  double dC_dA = B*B+1.0,
		 dC_dB = 2.0*A*B;

	  // Autodiff wrapper:
	  return var(new precomp_vv_vari(
	    C,             // The _value_ of the output
	    A_var.vi_,     // The input gradient wrt A
	    B_var.vi_,     // The input gradient wrt B
	    dC_dA,         // The partial introduced by this function wrt A
	    dC_dB          // The partial introduced by this function wrt B
	  ));
	}

	inline var my_func (double A, const var& B_var, std::ostream* pstream) {
	  double B = B_var.val(),
		 C = my_func(A, B, pstream),
		 dC_dB = 2.0*A*B;
	  return var(new precomp_v_vari(C, B_var.vi_, dC_dB));
	}

	inline var my_func (const var& A_var, double B, std::ostream* pstream) {
	  double A = A_var.val(),
		 C = my_func(A, B, pstream),
		 dC_dA = B*B+1.0;
	  return var(new precomp_v_vari(C, A_var.vi_, dC_dA));
	}
}

Unfortunately, the compilation throws the following error:

(stan-env) miguel@miguel-ASUS-Laptop-X407UBR:~/anaconda3/envs/stan-env/bin/cmdstan$ make STANCFLAGS=--allow-undefined USER_HEADER=examples/external_grad/external_manual.hpp examples/external_grad/external_grad

--- Translating Stan model to C++ code ---
bin/stanc --allow-undefined --o=examples/external_grad/external_grad.hpp examples/external_grad/external_grad.stan

--- Compiling, linking C++ code ---
/home/miguel/anaconda3/envs/stan-env/bin/x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/miguel/anaconda3/envs/stan-env/include -std=c++1y -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes      -I /home/miguel/anaconda3/envs/stan-env/include/    -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.9 -I stan/lib/stan_math/lib/boost_1.75.0 -I stan/lib/stan_math/lib/sundials_5.7.0/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/miguel/anaconda3/envs/stan-env/include    -DBOOST_DISABLE_ASSERTS   -DTBB_INTERFACE_NEW  -DTBB_INTERFACE_NEW     -c -include examples/external_grad/external_manual.hpp -x c++ -o examples/external_grad/external_grad.o examples/external_grad/external_grad.hpp
In file included from <command-line>:
./examples/external_grad/external_manual.hpp: In function 'stan::math::var external_grad_model_namespace::my_func(double, const var&, std::ostream*)':
./examples/external_grad/external_manual.hpp:39:14: error: expected primary-expression before '(' token
   39 |    return var(new precomp_v_vari(C, B_var.vi_, dC_dB));
      |              ^
./examples/external_grad/external_manual.hpp:39:19: error: expected type-specifier before 'precomp_v_vari'
   39 |    return var(new precomp_v_vari(C, B_var.vi_, dC_dB));
      |                   ^~~~~~~~~~~~~~
./examples/external_grad/external_manual.hpp: In function 'stan::math::var external_grad_model_namespace::my_func(const var&, double, std::ostream*)':
./examples/external_grad/external_manual.hpp:46:14: error: expected primary-expression before '(' token
   46 |    return var(new precomp_v_vari(C, A_var.vi_, dC_dA));
      |              ^
./examples/external_grad/external_manual.hpp:46:19: error: expected type-specifier before 'precomp_v_vari'
   46 |    return var(new precomp_v_vari(C, A_var.vi_, dC_dA));
      |                   ^~~~~~~~~~~~~~
make: *** [make/program:58: examples/external_grad/external_grad] Error 1

It seems that the precomp_v_vari class is not founded. By checking the Stan Math Library docs, I’m not able to found that class either, I only found the precomp_vv_vari class.

So my question is, what is the correct way to compile my model in CmdStan?

Additional information:

  • Operating System: Ubuntu 18
  • CmdStan Version: 2.28.2
  • Compiler/Toolkit: g++ 7.5.0
1 Like
2

Hi Miguel, Stan now uses the make_callback_var function to specify pre-computed gradients, I put together an example of using this for pre-computed gradients over in this post: Gamma distribution quantile and inverse quantile function - #42 by andrjohns

2 Likes
3

So, according to your example, I just need to remove the functions:

inline var my_func (double A, const var& B_var, std::ostream* pstream);
inline var my_func (const var& A_var, double B, std::ostream* pstream);

keeping only the function that has all the parameters as var inputs:

inline var my_func (const var& A_var, const var& B_var, std::ostream* pstream);

Is that correct?

4

No, you still keep the same number of function definitions, you just use make_callback_var instead of precomp_v_vari

5

My model now compiles with no errors, thank you!

I’m attaching the corrected code of my toy example based on @andrjohns’s suggestions for anyone stuck on the same error.

  • external_manual.hpp file (corrected):
#include <ostream>
#include <stan/math/rev/core.hpp>

namespace external_grad_model_namespace {

	using namespace stan::math;

	inline double my_func(double A, double B, std::ostream* pstream) {
		return A * B * B + A;
	}

	inline var my_func (const var& A_var, const var& B_var, std::ostream* pstream) {
		// Extract values from inputs
		double A = A_var.val();
		double B = B_var.val();
		double C = my_func(A, B, pstream);
		// Compute the partial derivatives
		double dC_dA = B * B + 1.0;
		double dC_dB = 2.0 * A * B;
		// Return the new parameter with values of the function and gradients
		return make_callback_var(C, [A_var, B_var, dC_dA, dC_dB](auto& vi) {
			A_var.adj() += vi.adj() * dC_dA;
			B_var.adj() += vi.adj() * dC_dB;
		});
	}

	inline var my_func(double A, const var& B_var, std::ostream* pstream) {
		// Extract values from inputs
	  	double B = B_var.val();
		double C = my_func(A, B, pstream);
		// Compute the partial derivatives
		double dC_dB = 2.0 * A * B;
		// Return the new parameter with values of the function and gradients
		return make_callback_var(C, [B_var, dC_dB](auto& vi) {
			B_var.adj() += vi.adj() * dC_dB;
		});
	}

	inline var my_func(const var& A_var, double B, std::ostream* pstream) {
		// Extract values from inputs
	  	double A = A_var.val();
		double C = my_func(A, B, pstream);
		// Compute the partial derivatives
		double dC_dA = B * B + 1.0;
		// Return the new parameter with values of the function and gradients
		return make_callback_var(C, [A_var, dC_dA](auto& vi) {
			A_var.adj() += vi.adj() * dC_dA;
		});
	}
}
4 Likes