External C++ function error compilation

Interfaces CmdStan
1

Hi everyone!
I am trying to understand how to develop an external function for the CmdStan interface using a C++ code.
The version I have is 2.25, I added in the make/local CXXFLAGS_PROGRAM += -include $(USER_HEADER) but I did not solve the problem yet.
I followed the tutorial in the manual and I reproduced the same example.

functions{
	real make_odds (real theta);
}

data{
	int<lower=0> N;
	int<lower=0,upper=1> y[N];
}

parameters{
	real<lower=0,upper=1> theta;
}

model{
	theta~beta(1,1);
	y~bernoulli(theta);
}

generated quantities{
	real odds;
	odds=make_odds(theta);
}

namespace bernoulli_model_namespace{

template <typename T0__> inline typename boost::math::tools::promote_args<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
	return theta/(1-theta);
}
}

The error I get when I run

STANCFLAGS=--allow-undefined USER_HEADER=examples/external-function/make_odds.hpp make examples/external-function/bernoulli

is the following:

    --- Compiling, linking C++ code ---
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes      -I stan/lib/stan_math/lib/tbb_2019_U8/include   -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.9 -I stan/lib/stan_math/lib/boost_1.72.0 -I stan/lib/stan_math/lib/sundials_5.6.1/include -I/home/fbrarda/cmdstan-petsc/stan/lib/stan_math/lib/boost_1.72.0    -DBOOST_DISABLE_ASSERTS         -c -include examples/external-function/make_odds.hpp -Wno-ignored-attributes   -include examples/external-function/make_odds.hpp -x c++ -o examples/external-function/bernoulli.o examples/external-function/bernoulli.hpp
In file included from <command-line>:0:0:
./examples/external-function/make_odds.hpp:3:42: error: ‘boost’ has not been declared
 template <typename T0__> inline typename boost::math::tools::promote_args<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
                                          ^
./examples/external-function/make_odds.hpp:3:74: error: expected unqualified-id before ‘<’ token
 template <typename T0__> inline typename boost::math::tools::promote_args<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
                                                                          ^
In file included from <command-line>:0:0:
./examples/external-function/make_odds.hpp:3:42: error: ‘boost’ has not been declared
 template <typename T0__> inline typename boost::math::tools::promote_args<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
                                          ^
./examples/external-function/make_odds.hpp:3:74: error: expected unqualified-id before ‘<’ token
 template <typename T0__> inline typename boost::math::tools::promote_args<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
                                                                          ^

I do not understand why I get this error and neither why the boost library is included twice (also from another directory - cmdstan-petsc which uses the 2.24 version of CmdStan).
I very much appreciate any help to understand where I am wrong.

Francesco

1 Like
2

Hi,
sorry for not getting to you earlier. Just to be clear: you are able to compile models without custom functions without any issue, right?

That sounds a bit suspicious - the user header currently needs to be included in a specific place in the file, not on top, so this might be problematic… What happens if you remove this?

3

Hi Martin,
Yes, I am able to compile models without functions defined in external codes. For instance, the Bernoulli example already present in the examples directory works.
I removed the user header from the make/local, cancelled all binaries and built everything again, nothing has changed.

1 Like
4

If you are running CmdStan 2.25 can you try the old (deprecated) flag --allow_undefined. There was a bug with the non-deprecated flag (see release notes for 2.26.0: Release v2.26.0 (26 January 2021) · stan-dev/cmdstan · GitHub).

1 Like
5

The docs also mention, you might need to pass --name=bernoulli_model to stanc.
Also, could you share the .hpp file generated by stanc? Should be something like examples/external-function/bernoulli.hpp

6

I tried also with --allow_undefined when I had the version 2.25, but the error did not change. Now I have 2.26.1 since I rebuild everything.
I added also stan::math:: before boost, but I guess it is not useful.
Here you can find the .hpp file generated.
bernoulli.hpp (14.8 KB)

7

In that case its an issue with including of Boost. In this case it isnt even needed as you can just write:

template <typename T0__> inline promote_args_t<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
	return theta/(1-theta);
}

Maybe also include stan/math.hpp if that doesnt work.

1 Like
8

I tried with and without include stan/math.hpp. None of them worked. The error I get is the same and is the following

In file included from **<command-line>:0:0** :

**./examples/external-function/make_odds.hpp:5:33:** **error:** ‘ **promote_args_t** ’ does not name a type

template <typename T0__> inline promote_args_t<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){

**^**

**cc1plus:** **warning:** unrecognized command line option ‘ **-Wno-ignored-attributes** ’

**cc1plus:** **warning:** unrecognized command line option ‘ **-Wno-ignored-attributes** ’

How can I fix the problem with boost?

Thank you very much.

9

Based off the error I think it just needs to be

template <typename T0__> 
inline stan::promote_args_t<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
	return theta/(1-theta);
}
2 Likes
10

Thank you for your suggestion. This was the error I got

/cmdstan$ STANCFLAGS=--allow-undefined USER_HEADER=examples/external-function/make_odds.hpp make examples/external-function/bernoulli

--- Compiling, linking C++ code ---
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes      -I stan/lib/stan_math/lib/tbb_2020.3/include   -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.9 -I stan/lib/stan_math/lib/boost_1.75.0 -I stan/lib/stan_math/lib/sundials_5.7.0/include -I/home/fbrarda/cmdstan-petsc/stan/lib/stan_math/lib/boost_1.72.0    -DBOOST_DISABLE_ASSERTS         -c -Wno-ignored-attributes   -include examples/external-function/make_odds.hpp -x c++ -o examples/external-function/bernoulli.o examples/external-function/bernoulli.hpp
In file included from <command-line>:0:0:
./examples/external-function/make_odds.hpp:5:33: error: need ‘typename’ before ‘stan::promote_args_t<T>::type’ because ‘stan::promote_args_t<T>’ is a dependent scope
 template <typename T0__> inline stan::promote_args_t<T0__>::type make_odds(const T0__& theta, std::ost
                             ^
cc1plus: warning: unrecognized command line option ‘-Wno-ignored-attributes’
cc1plus: warning: unrecognized command line option ‘-Wno-ignored-attributes’
make: *** [examples/external-function/bernoulli] Error 1

By adding typename as it requires and #include <stan/math.hpp> I have a new error, which is the following

--- Compiling, linking C++ code ---
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes      -I stan/lib/stan_math/lib/tbb_2020.3/include   -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.9 -I stan/lib/stan_math/lib/boost_1.75.0 -I stan/lib/stan_math/lib/sundials_5.7.0/include -I/home/fbrarda/cmdstan-petsc/stan/lib/stan_math/lib/boost_1.72.0    -DBOOST_DISABLE_ASSERTS         -c -Wno-ignored-attributes   -include examples/external-function/make_odds.hpp -x c++ -o examples/external-function/bernoulli.o examples/external-function/bernoulli.hpp
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes      -I stan/lib/stan_math/lib/tbb_2020.3/include   -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.9 -I stan/lib/stan_math/lib/boost_1.75.0 -I stan/lib/stan_math/lib/sundials_5.7.0/include -I/home/fbrarda/cmdstan-petsc/stan/lib/stan_math/lib/boost_1.72.0    -DBOOST_DISABLE_ASSERTS               -Wl,-L,"/home/fbrarda/cmdstan/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/home/fbrarda/cmdstan/stan/lib/stan_math/lib/tbb"      examples/external-function/bernoulli.o src/cmdstan/main.o        -Wl,-L,"/home/fbrarda/cmdstan/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/home/fbrarda/cmdstan/stan/lib/stan_math/lib/tbb"   stan/lib/stan_math/lib/sundials_5.7.0/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_5.7.0/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_5.7.0/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_5.7.0/lib/libsundials_kinsol.a  stan/lib/stan_math/lib/tbb/libtbb.so.2 -o examples/external-function/bernoulli
examples/external-function/bernoulli.o: In function `stan::model::model_base_crtp<bernoulli_model_namespace::bernoulli_model>::write_array(boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> >&, std::vector<double, std::allocator<double> >&, std::vector<int, std::allocator<int> >&, std::vector<double, std::allocator<double> >&, bool, bool, std::ostream*) const':
bernoulli.hpp:(.text._ZNK4stan5model15model_base_crtpIN25bernoulli_model_namespace15bernoulli_modelEE11write_arrayERN5boost6random23additive_combine_engineINS6_26linear_congruential_engineIjLj40014ELj0ELj2147483563EEENS8_IjLj40692ELj0ELj2147483399EEEEERSt6vectorIdSaIdEERSD_IiSaIiEESG_bbPSo[_ZNK4stan5model15model_base_crtpIN25bernoulli_model_namespace15bernoulli_modelEE11write_arrayERN5boost6random23additive_combine_engineINS6_26linear_congruential_engineIjLj40014ELj0ELj2147483563EEENS8_IjLj40692ELj0ELj2147483399EEEEERSt6vectorIdSaIdEERSD_IiSaIiEESG_bbPSo]+0x107): undefined reference to `boost::math::tools::promote_args<double, float, float, float, float, float>::type bernoulli_model_namespace::make_odds<double>(double const&, std::ostream*)'
examples/external-function/bernoulli.o: In function `stan::model::model_base_crtp<bernoulli_model_namespace::bernoulli_model>::write_array(boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014u, 0u, 2147483563u>, boost::random::linear_congruential_engine<unsigned int, 40692u, 0u, 2147483399u> >&, Eigen::Matrix<double, -1, 1, 0, -1, 1>&, Eigen::Matrix<double, -1, 1, 0, -1, 1>&, bool, bool, std::ostream*) const':
bernoulli.hpp:(.text._ZNK4stan5model15model_base_crtpIN25bernoulli_model_namespace15bernoulli_modelEE11write_arrayERN5boost6random23additive_combine_engineINS6_26linear_congruential_engineIjLj40014ELj0ELj2147483563EEENS8_IjLj40692ELj0ELj2147483399EEEEERN5Eigen6MatrixIdLin1ELi1ELi0ELin1ELi1EEESG_bbPSo[_ZNK4stan5model15model_base_crtpIN25bernoulli_model_namespace15bernoulli_modelEE11write_arrayERN5boost6random23additive_combine_engineINS6_26linear_congruential_engineIjLj40014ELj0ELj2147483563EEENS8_IjLj40692ELj0ELj2147483399EEEEERN5Eigen6MatrixIdLin1ELi1ELi0ELin1ELi1EEESG_bbPSo]+0x2f0): undefined reference to `boost::math::tools::promote_args<double, float, float, float, float, float>::type bernoulli_model_namespace::make_odds<double>(double const&, std::ostream*)'
collect2: error: ld returned 1 exit status
make: *** [examples/external-function/bernoulli] Error 1

There is now an output file, but I guess it is not complete.

11

Ah that’s interesting it looks like the function block of the stan program places your forward decleration in the bernoulli_model_namespace, I think that’s a wiff in the stanc compiler (idk if i’d call it a bug but I think I would call it undesirable behavior)

If you wrap wrap your custom C++ function in

namespace bernoulli_model_namespace {
// your c++ code
}

And try again I think that will work

@rok_cesnovar maybe in the compiler we should have this more standardized. Like functions with no body in the functions block could be in a namespace stan { namespace custom { } } namespace. That would make it easier to use the same C++ code in multiple models. In the compiler we’d probably have to add a catch for user defined functions with no body so that the compiler knows to call stan::custom::cool_func(). But then I think the instructions become simpler for users. Do we have docs somewhere explaining the rules for adding user defined C++ functions?

12

Thank you very much for your suggestion. I did’t know whether I correctly understood, so I tried different things. You can see below the different external functions I used. Unfortunately none of them worked.
First

  namespace bernoulli_model_namespace{
  #include <stan/math.hpp>

    namespace bernoulli_model_namespace{

    template <typename T0__> 
    inline typename stan::promote_args_t<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
    	return theta/(1-theta);
    }
    }
    }

Second

 #include <stan/math.hpp>

namespace bernoulli_model_namespace{
namespace bernoulli_model_namespace{

template <typename T0__> 
inline typename stan::promote_args_t<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
	return theta/(1-theta);
}
}
}

Third (and original)

#include <stan/math.hpp>
namespace bernoulli_model_namespace{

template <typename T0__> 
inline typename stan::promote_args_t<T0__>::type make_odds(const T0__& theta, std::ostream* pstream__){
	return theta/(1-theta);
}
}

In the first case, I have countless errors. With a double wrap as the second case, I got the same error as yesterday (my previous answer). The third one works exactly as the second one.
I followed this document also, but it does not refer to anything like this.

1 Like
13

I think the custom C++ functions should just automatically be added to the model namespace.
See Including .hpp files and namespaces · Issue #712 · stan-dev/stanc3 · GitHub for more on that.

1 Like