# Trouble including an integrand as part of the model

I’m trying to estimate a model in which the DGP involves integrating a custom function. When I try to compile the model (below), I get a `Error in sink(type = "output") : invalid connection` error (full error message below). The issue is very clearly coming from the call to integrate_1d().

Sample model below. Anyone have an idea of what might be going wrong?

``````functions{
real cost_density(real x,          // Function argument
real xc,         // Complement of function argument
//  on the domain (defined later)
real[] theta,    // parameters
real[] x_r,      // data (real)
int[] x_i) {     // data (integer)
real mu = theta;
real sigma = theta;
real max_c = theta;
int num_bidders = x_i;
real out;

out = (num_bidders - 1)*(lognormal_lcdf(x | mu, sigma) - lognormal_lcdf(max_c | mu, sigma));
return(exp(out));
}

vector getGenerativeTotalAuctionHazard(int N,
int[] J, // num bidders in each auction
vector costs,
matrix X,
vector beta_costs,
real sigma,
real c_max
){
vector[N] mus;
vector[N] numerators;
vector[N] denominators;
vector[N] out;

mus = X * beta_costs;

// NOTE: This assumes homogeneous bidders so that g_j(b_i) = g(b_i | X_j) = g(b_i | X_i) = g_i(b_i)
for (n in 1:N){

numerators[n] = integrate_1d(cost_density,
0.0, // left limit
c_max, // right limit
{ mus[n], sigma, costs[n] }, //theta
{ 0.0 }, //x_r
{ J[n] }, //x_i
0.001
);

denominators[n] = exp(
(J[n] - 1)*(lognormal_lcdf(costs[n] | mus[n], sigma) - lognormal_lcdf(c_max | mus[n], sigma))
);

out[n] = numerators[n] / denominators[n];
}

return(out);
}

}

data {
int<lower=0> N; // Number of auction-bidder obs
int<lower=0> K; // number of auction-bidder feaures
int J[N]; // Number of bidders in each auction
matrix[N, K] X; // Auction-bidder features
vector[N] bids;
}

parameters {
vector[K] beta;
real<lower=0> sigma;
real<lower = 0> c_max;
vector[N] costs;
}

model {
vector[N] inv_hazards;
//prior
to_vector(beta) ~ normal(0,1);
sigma ~ normal(1,0.01);
c_max ~ normal(3, 1);

//likelihood
for(i in 1:N){
costs[i] ~ normal(X[i] * beta, sigma) T[,c_max]; //empirical bid distribution
}
//
inv_hazards = getGenerativeTotalAuctionHazard(
N,
J, // num bidders in each auction
costs,
X,
beta,
sigma,
c_max
);
(bids - costs) ~ normal(inv_hazards , 1);

}

``````

Full error message:

``````Error in compileCode(f, code, language = language, verbose = verbose) :
Compilation ERROR, function(s)/method(s) not created! In file included from file3d89436904cd.cpp:8:
In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/l
In system(cmd, intern = !verbose) :
running command '/Library/Frameworks/R.framework/Resources/bin/R CMD SHLIB file3d89436904cd.cpp 2> file3d89436904cd.cpp.err.txt' had status 1
Error in sink(type = "output") : invalid connection
``````
1 Like

You need to call it with `verbose = TRUE` to see the actual compiler error.

Ah! Error below:

``````/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/stan/math/rev/arr/functor/integrate_1d.hpp:120:1: note: candidate function template not viable: no known conversion from 'vector<local_scalar_t__>' to 'const vector<double>' for 5th argument
integrate_1d(const F &f, const T_a &a, const T_b &b,
^
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/stan/math/prim/arr/functor/integrate_1d.hpp:195:15: note: candidate function template not viable: no known conversion from 'const stan::math::var' to 'const double' for 3rd argument
inline double integrate_1d(
^
file4f60bc4249b.cpp:535:45: error: no matching function for call to 'getGenerativeTotalAuctionHazard'
stan::math::assign(inv_hazards, getGenerativeTotalAuctionHazard(N, J, costs, X, beta, sigma, c_max, pstream__));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/src/stan/model/log_prob_grad.hpp:44:28: note: in instantiation of function template specialization 'model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model::log_prob<true, false, stan::math::var>' requested here
^
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/src/stan/services/util/initialize.hpp:166:31: note: in instantiation of function template specialization 'stan::model::log_prob_grad<true, false, model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model>' requested here
^
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/src/stan/services/optimize/newton.hpp:54:21: note: in instantiation of function template specialization 'stan::services::util::initialize<false, model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014, 0, 2147483563>, boost::random::linear_congruential_engine<unsigned int, 40692, 0, 2147483399> > >' requested here
= util::initialize<false>(model, init, rng, init_radius, false,
^
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/rstan/include/rstan/stan_fit.hpp:488:35: note: in instantiation of function template specialization 'stan::services::optimize::newton<model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model>' requested here
= stan::services::optimize::newton(model, *init_context_ptr,
^
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/rstan/include/rstan/stan_fit.hpp:1201:11: note: in instantiation of function template specialization 'rstan::(anonymous namespace)::command<model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014, 0, 2147483563>, boost::random::linear_congruential_engine<unsigned int, 40692, 0, 2147483399> > >' requested here
ret = command(args, model_, holder, names_oi_tidx_,
^
file4f60bc4249b.cpp:741:140: note: in instantiation of member function 'rstan::stan_fit<model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014, 0, 2147483563>, boost::random::linear_congruential_engine<unsigned int, 40692, 0, 2147483399> > >::call_sampler' requested here
&rstan::stan_fit<model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model, boost::random::ecuyer1988>::call_sampler)
^
file4f60bc4249b.cpp:108:1: note: candidate template ignored: substitution failure [with T2__ = stan::math::var, T3__ = double, T4__ = stan::math::var, T5__ = stan::math::var, T6__ = stan::math::var]
getGenerativeTotalAuctionHazard(const int& N,
^
In file included from file4f60bc4249b.cpp:730:
In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/rstan/include/rstan/rstaninc.hpp:3:
In file included from /Library/Frameworks/R.framework/Versions/3.6/Resources/library/rstan/include/rstan/stan_fit.hpp:36:
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/StanHeaders/include/src/stan/model/log_prob_propto.hpp:45:28: error: no matching member function for call to 'log_prob'
~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Frameworks/R.framework/Versions/3.6/Resources/library/rstan/include/rstan/stan_fit.hpp:1124:40: note: in instantiation of function template specialization 'stan::model::log_prob_propto<false, model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model>' requested here
return Rcpp::wrap(stan::model::log_prob_propto<false>(model_, par_r, par_i, &rstan::io::rcout));
^
file4f60bc4249b.cpp:759:140: note: in instantiation of member function 'rstan::stan_fit<model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model, boost::random::additive_combine_engine<boost::random::linear_congruential_engine<unsigned int, 40014, 0, 2147483563>, boost::random::linear_congruential_engine<unsigned int, 40692, 0, 2147483399> > >::log_prob' requested here
&rstan::stan_fit<model4f60717efaac_generative_model_namespace::model4f60717efaac_generative_model, boost::random::ecuyer1988>::log_prob)
^
file4f60bc4249b.cpp:464:9: note: candidate template ignored: substitution failure [with propto__ = true, jacobian__ = false, T__ = stan::math::var]
T__ log_prob(std::vector<T__>& params_r__,
^
file4f60bc4249b.cpp:550:8: note: candidate function template not viable: requires at most 2 arguments, but 3 were provided
T_ log_prob(Eigen::Matrix<T_,Eigen::Dynamic,1>& params_r,
^
18 warnings and 3 errors generated.
make: *** [file4f60bc4249b.o] Error 1

ERROR(s) during compilation: source code errors or compiler configuration errors!
``````

shosh.stan (3.0 KB) compiles. The parser shouldn’t be generating C++ code that can’t compile, but it has been known to happen. Thus, I ended up having to pass `x_r` from `transformed data` to your function with the `data` argument qualifier to get it to work.

2 Likes

Thanks very much, Ben :)