# Calling integrate_1d() reports parsing error (PARSER EXPECTED ")")

Is it permitted to call integrate_1d() inside a function called by algebra_solver()? My code is below. I’m getting this error

error in '/takeup_header.stan' at line 102, column 139
included from 'model14806083057_takeup_struct' at line 1
-------------------------------------------------
100:       int dummy_x_i[0];
101:
102:       real delta = integrate_1d(expected_delta, negative_infinity(), positive_infinity(), { v_cutoff, u_sd }, dummy_x_r, dummy_x_i, 0.00001);
^
103:
-------------------------------------------------

PARSER EXPECTED: ")"

My code:

real expected_delta(real u, real xc, real[] theta, real[] x_r, real[] x_i) {
real v = theta[1];
real u_sd = theta[2];

return reputational_returns_normal(v - u, [ 1 ]', [ 0 ]', [ 1 ]') * exp(normal_lpdf(u | 0, u_sd));
}

vector v_fixedpoint_solution_normal(vector model_param, vector theta, real[] x_r, int[] x_i) {
real v_cutoff = model_param[1];

real benefit_cost = theta[1];
real mu = theta[2];

int num_v_mix = x_i[1];

vector[num_v_mix] lambda = theta[3:(3 + num_v_mix - 1)];
vector[num_v_mix] mix_mean = theta[(3 + num_v_mix):(3 + 2 * num_v_mix - 1)];
vector[num_v_mix] mix_sd = theta[(3 + 2 * num_v_mix):(3 + 3 * num_v_mix - 1)];
real u_shock = theta[3 + 3 * num_v_mix];
real u_sd = theta[3 + 3 * num_v_mix + 1];

if (u_sd > 0) {
real dummy_x_r[0];
int dummy_x_i[0];

real delta = integrate_1d(expected_delta, negative_infinity(), positive_infinity(), { v_cutoff, u_sd }, dummy_x_r, dummy_x_i, 0.00001);

return [ v_cutoff + benefit_cost + mu * delta ]';
} else {
return [ v_cutoff + benefit_cost + mu * reputational_returns_normal(v_cutoff - u_shock, lambda, mix_mean, mix_sd) ]';
}
}

Thanks!

I’m not sure that’s a good question. I think maybe @bbbales2 or @charlesm93 will know.

Edit: I meant “I’m not sure. That’s a good question.”!

Sorry, is it not a good question because I’m jumping to the wrong conclusion about what the cause of this error is? How can I make the question better?

Haha, missing semicolon. Will check this later today. Certainly integrate_1d should work in there but maybe there’s a buggo.

I think he meant “I’m not sure. That’s a good question” haha

1 Like

Haha!

This doesn’t seem to be a problem with calling integrate_1d() inside algebra_solver(). It doesn’t work anywhere. FYI, I’m using rstan 2.21.2.

I meant “I’m not sure. That’s a good question”. Sorry!

1 Like

Give this a try:

real expected_delta(real u, real xc, real[] theta, data real[] x_r, data int[] x_i) {
real v = theta[1];
real u_sd = theta[2];

return reputational_returns_normal(v - u, [ 1 ]', [ 0 ]', [ 1 ]') * exp(normal_lpdf(u | 0, u_sd));
}

vector v_fixedpoint_solution_normal(vector model_param, vector theta, data real[] x_r, data int[] x_i, data real tol) {
real v_cutoff = model_param[1];

real benefit_cost = theta[1];
real mu = theta[2];

int num_v_mix = x_i[1];

vector[num_v_mix] lambda = theta[3:(3 + num_v_mix - 1)];
vector[num_v_mix] mix_mean = theta[(3 + num_v_mix):(3 + 2 * num_v_mix - 1)];
vector[num_v_mix] mix_sd = theta[(3 + 2 * num_v_mix):(3 + 3 * num_v_mix - 1)];
real u_shock = theta[3 + 3 * num_v_mix];
real u_sd = theta[3 + 3 * num_v_mix + 1];

if (u_sd > 0) {
real delta = integrate_1d(expected_delta, negative_infinity(), positive_infinity(), { v_cutoff, u_sd }, x_r, x_i, 0.00001);

return [ v_cutoff + benefit_cost + mu * delta ]';
} else {
return [ v_cutoff + benefit_cost + mu * reputational_returns_normal(v_cutoff - u_shock, lambda, mix_mean, mix_sd) ]';
}
}

I couldn’t test all the code (no definition for reputational_returns), but I think the error message you’re getting is because Stan is very picky about data arguments (they need to be declared this way in the arguments in this case).

Full error looks something like:

SYNTAX ERROR, MESSAGE(S) FROM PARSER:
fifth argument to integrate_1d, the real data, must be data only and not reference parameters.
error in 'model8d162bc2383_example' at line 27, column 129
-------------------------------------------------
25:       int dummy_x_i[0];
26:
27:       real delta = integrate_1d(expected_delta, negative_infinity(), positive_infinity(), { v_cutoff, u_sd }, dummy_x_r, x_i, tol);
^
28:
-------------------------------------------------

PARSER EXPECTED: ")"

which is a lot to parse but hidden above the code is a more useful error.

So things like x_r and x_i as arguments to the higher order functions have to be declared in transformed data or data.

The declarations:

real dummy_x_r[0];
int dummy_x_i[0];

are not counted as data and can’t be used as the x_r/x_i arguments. Since you aren’t using x_r and x_i in the integral, just pass along the ones from the algebra_solver. That’s an awkward solution though. I’ll make an issue in the language about this (edit: issue made https://github.com/stan-dev/stanc3/issues/756).

That seems to fix the parser error (the “data” qualifier in Stan must be new because I haven’t seen it before in any of the documentation).

I’m not getting a runtime error that I’m not sure how to decipher:

Chain 1: Unrecoverable error evaluating the log probability at the initial value.
Chain 1: Exception: Exception: Error in function boost::math::quadrature::sinh_sinh<double>::integrate: The sinh_sinh quadrature evaluated your function at a singular point, leading to the value inf.
sinh_sinh quadrature cannot handle singularities in the domain.
If you are sure your function has no singularities, please submit a bug against boost.math
(in '/takeup_header.stan' at line 117; included from 'modelefa45057ba7d_takeup_struct' at line 1)
(in 'modelefa45057ba7d_takeup_struct' at line 283)

My functions are now defined as below (you don’t need any other functions).

real expected_delta(real u, real xc, real[] theta, data real[] x_r, data int[] x_i) {
real cutoff = theta[1] - u;
real u_sd = theta[2];

return exp(normal_lpdf(cutoff | 0, 1) - normal_lcdf(cutoff | 0, 1) - normal_lccdf(cutoff | 0, 1) + normal_lpdf(u | 0, u_sd));
}

vector v_fixedpoint_solution_normal(vector model_param, vector theta, data real[] x_r, data int[] x_i) {
real v_cutoff = model_param[1];

real benefit_cost = theta[1];
real mu = theta[2];

int num_v_mix = x_i[1];
int use_u_in_delta = x_i[2];

real u_sd = theta[3 + 3 * num_v_mix];

real delta = integrate_1d(expected_delta, negative_infinity(), positive_infinity(), { v_cutoff, u_sd }, x_r, x_i, 0.01);

return [ v_cutoff + benefit_cost + mu * delta ]';
}

The problem seems to be the normal_lcdf() and normal_lccdf() calls in expected_delta(). If I remove them or add them instead of subtract them from the result the code runs smoothly. Is there something I should be looking into for initialization that is throwing off the integration?

I don’t know if this helps, but I’m basically trying to calculate

\int_u (E[V \mid V > w^* - u] - E[V \mid V < w^* - u]) p(u)\,\textrm{d}u = \int_u \frac{\phi(w^* - u)}{\Phi(w^* - u)(1 - \Phi(w^* - u))} p(u)\,\textrm{d}u\\ \\ V \sim \mathtt{Normal}(0,1) \\ U \sim \mathtt{Normal}(0, \sigma) \\

I think the problem is with what I’m trying to do. Part of my integrand is U-shaped and integrating to infinity might be a problem. I have to rethink this part. Thanks for fixing the code though!

No problem. integrate_1d can be finicky – if it keeps giving you trouble report back. We found a bug in it recently and there could easily be others.

The ticket for knowing Stan is doing something wrong seems to be exposing functions through expose_stan_functions and checking against R’s integrate. It’s a bit tedious but better than not having the integrals you need :D.

1 Like

I think I fixed my integrand and I did confirm that it works fine using R’s integrate() function. I’m not hitting a complier error when I use stan_model() on my model file. This is what my functions now look like:

real expected_delta_part(real v, real xc, real[] theta, data real[] x_r, data int[] x_i) {
real w = theta[1];
real u_sd = theta[2];

return v * exp(normal_lcdf(w - v | 0, u_sd) + normal_lpdf(v | 0, 1));
}

real expected_delta(real w, real u_sd) {
real delta_part = integrate_1d(expected_delta_part, negative_infinity(), positive_infinity(), { w, u_sd }, { 0.0 }, { 0 }, 0.01);
real F_w = Phi_approx(w / u_sd);

return - delta_part / (F_w * (1 - F_w));
}

vector v_fixedpoint_solution_normal(vector model_param, vector theta, data real[] x_r, data int[] x_i) {
real cutoff = model_param[1];

real benefit_cost = theta[1];
real mu = theta[2];

int num_v_mix = x_i[1];
int use_u_in_delta = x_i[2];

vector[num_v_mix] lambda = theta[3:(3 + num_v_mix - 1)];
vector[num_v_mix] mix_mean = theta[(3 + num_v_mix):(3 + 2 * num_v_mix - 1)];
vector[num_v_mix] mix_sd = theta[(3 + 2 * num_v_mix):(3 + 3 * num_v_mix - 1)];
real u_sd = theta[3 + 3 * num_v_mix];

if (use_u_in_delta && u_sd > 0) {
return [ cutoff + benefit_cost + mu * expected_delta(cutoff, u_sd) ]';
} else {
return [ cutoff + benefit_cost + mu * reputational_returns_normal(cutoff, lambda, mix_mean, mix_sd) ]';
}
}

I’m getting the below error message. It goes away if I remove the line that calls expected_delta() above.

> stan_model("stan_models/takeup_struct.stan")
DIAGNOSTIC(S) FROM PARSER:
Info: integer division implicitly rounds to integer. Found int division: num_elements(to_index) / num_indices
Positive values rounded down, negative values rounded up or down in platform-dependent way.
Info: integer division implicitly rounds to integer. Found int division: sample_size - 1 * quants[quant_index] / 100
Positive values rounded down, negative values rounded up or down in platform-dependent way.
Info: integer division implicitly rounds to integer. Found int division: sample_size - 1 * quants[quant_index] / 100
Positive values rounded down, negative values rounded up or down in platform-dependent way.
Info: integer division implicitly rounds to integer. Found int division: 1 - 2 * reverse * to - from / by
Positive values rounded down, negative values rounded up or down in platform-dependent way.

When you compile models, you are also contributing to development of the NEXT
Stan compiler. In this version of rstan, we compile your model as usual, but
also test our new compiler on your syntactically correct model. In this case,
the new compiler did not work like we hoped. By filing an issue at
or a minimal example that shows this warning you will be contributing
valuable information to Stan and ensuring your models continue working. Thank you!
This message can be avoided by wrapping your function call inside suppressMessages()
or by first calling rstan_options(javascript = FALSE).
Error in context_eval(join(src), private$context, serialize) : 0,248,Frontend.Errors.SyntaxError,178,1,Could not find include file takeup_header.stan in specified include paths. ,0,string,1,0,0 hash mismatch so recompiling; make sure Stan code ends with a blank line make cmd is make -f '/opt/R/4.0.3/lib/R/etc/Makeconf' -f '/opt/R/4.0.3/lib/R/share/make/shlib.mk' -f '/home/karim/.R/Makevars' CXX='$(CXX14) $(CXX14STD)' CXXFLAGS='$(CXX14FLAGS)' CXXPICFLAGS='$(CXX14PICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX14LDFLAGS)' SHLIB_LD='$(SHLIB_CXX14LD)' SHLIB='file21adf7c88bef6.so' OBJECTS='file21adf7c88bef6.o' make would use g++ -std=gnu++14 -I"/opt/R/4.0.3/lib/R/include" -DNDEBUG -I"/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/Rcpp/1.0.5/125dc7a0ed375eb68c0ce533b48d291f/Rcpp/include/" -I"/media/karim/Code Drive/karimn-code/takeup/renv/library/R-4.0/x86_64-pc-linux-gnu/RcppEigen/include/" -I"/media/karim/Code Drive/karimn-code/takeup/renv/library/R-4.0/x86_64-pc-linux-gnu/RcppEigen/include/unsupported" -I"/media/karim/Code Drive/karimn-code/takeup/renv/library/R-4.0/x86_64-pc-linux-gnu/BH/include" -I"/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/StanHeaders/2.21.0-6/9da8818e00c8e3fc7c94d707523a0009/StanHeaders/include/src/" -I"/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/StanHeaders/2.21.0-6/9da8818e00c8e3fc7c94d707523a0009/StanHeaders/include/" -I"/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/RcppParallel/5.0.2/551d50addb18c7b99637aa377a25afc8/RcppParallel/include/" -I"/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/rstan/2.21.2/52772d81aa532a6331fd535701882c12/rstan/include" -DEIGEN_NO_DEBUG -DBOOST_DISABLE_ASSERTS -DBOOST_PENDING_INTEGER_LOG2_HPP -DSTAN_THREADS -DBOOST_NO_AUTO_PTR -include '/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/StanHeaders/2.21.0-6/9da8818e00c8e3fc7c94d707523a0009/StanHeaders/include/stan/math/prim/mat/fun/Eigen.hpp' -D_REENTRANT -DRCPP_PARALLEL_USE_TBB=1 -I/usr/local/include -fpic -O3 -march=native -mtune=native -fPIC -DSTAN_THREADS -pthread -c file21adf7c88bef6.cpp -o file21adf7c88bef6.o if test "zfile21adf7c88bef6.o" != "z"; then \ echo g++ -std=gnu++14 -shared -L"/opt/R/4.0.3/lib/R/lib" -L/usr/local/lib -o file21adf7c88bef6.so file21adf7c88bef6.o '/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/rstan/2.21.2/52772d81aa532a6331fd535701882c12/rstan/lib//libStanServices.a' -L'/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/StanHeaders/2.21.0-6/9da8818e00c8e3fc7c94d707523a0009/StanHeaders/lib/' -lStanHeaders -L'/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/RcppParallel/5.0.2/551d50addb18c7b99637aa377a25afc8/RcppParallel/lib/' -ltbb -L"/opt/R/4.0.3/lib/R/lib" -lR; \ g++ -std=gnu++14 -shared -L"/opt/R/4.0.3/lib/R/lib" -L/usr/local/lib -o file21adf7c88bef6.so file21adf7c88bef6.o '/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/rstan/2.21.2/52772d81aa532a6331fd535701882c12/rstan/lib//libStanServices.a' -L'/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/StanHeaders/2.21.0-6/9da8818e00c8e3fc7c94d707523a0009/StanHeaders/lib/' -lStanHeaders -L'/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/RcppParallel/5.0.2/551d50addb18c7b99637aa377a25afc8/RcppParallel/lib/' -ltbb -L"/opt/R/4.0.3/lib/R/lib" -lR; \ fi Error in compileCode(f, code, language = language, verbose = verbose) : /home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/StanHeaders/2.21.0-6/9da8818e00c8e3fc7c94d707523a0009/StanHeaders/include/stan/math/rev/mat/fun/multiply.hpp:131:9: required from ‘void stan::math::multiply_mat_vari<Ta, Ra, Ca, Tb, Cb>::chain() [with Ta = stan::math::var; int Ra = -1; int Ca = -1; Tb = stan::math::var; int Cb = -1]’/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/StanHeaders/2.21.0-6/9da8818e00c8e3fc7c94d707523a0009/StanHeaders/include/stan/math/rev/mat/fun/multiply.hpp:107:16: required from here/media/karim/Code Drive/karimn-code/takeup/renv/library/R-4.0/x86_64-pc-linux-gnu/RcppEigen/include/Eigen/src/Core/DenseCoeffsBase.h:55:30: warning: ignoring attributes on template argument ‘Eigen::internal::packet_traits<double>::type’ {aka ‘__m256d’} [-Wignored-attributes] 55 | >::type PacketReturnType; | ^~~~~~~~~~~~~~~~make: *** [/opt/R/4.0.3/lib/R/etc/Makeconf: Error in sink(type = "output") : invalid connection The integration I’m doing can be replicated in R like this: delta_part <- function(v, w, u_sd, lower.tail = TRUE) v * pnorm(w - v, sd = u_sd, lower.tail = lower.tail) * dnorm(v) integrate(delta_part, w = 2, u_sd = 2, -Inf, Inf) I appreciate all your help! Please let me know if I can help investigating this problem. Also, I’m kind of re-using an old discussion. Let me know if you want me to start a new one. I believe the error is here: Error in context_eval(join(src), private$context, serialize) :
0,248,Frontend.Errors.SyntaxError,178,1,Could not find include file takeup_header.stan in specified include paths.
,0,string,1,0,0

It can’t find the additional stan code takeup_header.stan that you’re trying to include

It finds it fine if I don’t call the expected_delta(). I already use a lot of other functions in that included file.

Getting rid of all includes and putting everything in one .stan file, I’m still getting an error:

make cmd is
make -f '/opt/R/4.0.3/lib/R/etc/Makeconf' -f '/opt/R/4.0.3/lib/R/share/make/shlib.mk' -f '/home/karim/.R/Makevars' CXX='$(CXX14)$(CXX14STD)' CXXFLAGS='$(CXX14FLAGS)' CXXPICFLAGS='$(CXX14PICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX14LDFLAGS)' SHLIB_LD='$(SHLIB_CXX14LD)' SHLIB='file23d4d27b09142.so' OBJECTS='file23d4d27b09142.o'

make would use
if test  "zfile23d4d27b09142.o" != "z"; then \
fi

That’s not an error, just a summary of the compiler arguments. Can you re-run with verbose=TRUE and post the output?

1 Like

Also, if you’re able to post the full Stan model, then we can debug locally and get you an answer faster as well

Sorry, I must have cut off some of the error message:

Error in compileCode(f, code, language = language, verbose = verbose) :
/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/StanHeaders/2.21.0-6/9da8818e00c8e3fc7c94d707523a0009/StanHeaders/include/stan/math/rev/mat/fun/multiply.hpp:131:9:   required from ‘void stan::math::multiply_mat_vari<Ta, Ra, Ca, Tb, Cb>::chain() [with Ta = stan::math::var; int Ra = -1; int Ca = -1; Tb = stan::math::var; int Cb = -1]’/home/karim/.local/share/renv/cache/v5/R-4.0/x86_64-pc-linux-gnu/StanHeaders/2.21.0-6/9da8818e00c8e3fc7c94d707523a0009/StanHeaders/include/stan/math/rev/mat/fun/multiply.hpp:107:16:   required from here/media/karim/Code Drive/karimn-code/takeup/renv/library/R-4.0/x86_64-pc-linux-gnu/RcppEigen/include/Eigen/src/Core/DenseCoeffsBase.h:55:30: warning: ignoring attributes on template argument ‘Eigen::internal::packet_traits<double>::type’ {aka ‘__m256d’} [-Wignored-attributes]   55 |                      >::type PacketReturnType;      |                              ^~~~~~~~~~~~~~~~make: *** [/opt/R/4.0.3/lib/R/etc/Makeconf:[test2.stan|attachment](upload://t6wEHdguz1zD9nEthgHCuGEo3cs.stan) (64.9 KB)

I’ve uploaded the Stan file that causes this.

1 Like

Ah that error. Unfortunately that’s a legitimate bug with that version of the Math library that we see sometimes. I’ll do some investigation with your model to try and find what’s causing it, but in the interim your only course of action here is to use the cmdstanR package to run your model: https://mc-stan.org/cmdstanr/

1 Like