Compilation error on rstan

General
,
#1

I run into a compilation error since upgrade to R4.0. Also, this error is generated for some models (most likely the models that use the integrate_1d) function but other models run perfectly fine.

I am using Ubuntu 20.04.1 LTS and R version 4.0.2

This is the error I am getting

Compilation ERROR, function(s)/method(s) not created!
Error in compileCode(f, code, language = language, verbose = verbose) : 
  /usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/mat/functor/adj_jac_apply.hpp:535:10:   required from ‘void stan::math::adj_jac_vari<F, Targs>::chain() [with F = stan::math::internal::softmax_op; Targs = {Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>}]’/usr/local/lib/R/site-library/StanHeaders/include/stan/math/rev/mat/functor/adj_jac_apply.hpp:531:8:   required from here/usr/local/lib/R/site-library/RcppEigen/include/Eigen/src/Core/DenseCoeffsBase.h:650:34: warning: ignoring attributes on template argument ‘Eigen::internal::packet_traits<double>::type’ {aka ‘__vector(4) double’} [-Wignored-attributes]  650 |   return internal::first_aligned<int(unpacket_traits<DefaultPacketType>::alignment),Derived>(m);      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~make: *** [/usr/lib/R/etc/Makeconf:176: filea863c55825ef.o] Error 1
#2

The error looks like it has a softmax in it.

Can you try this model:

transformed data {
  int N = 5;
}
parameters {
  vector[N] theta;
}
model {
  1 ~ categorical(softmax(theta));
}

If this doesn’t cause an error, can you post the model that causes the problem?

#3

(Don’t worry about running that model just see if it compiles)

#4

@bbbales2
Hi Ben,

Thanks for your suggestion. The model that you suggested compiled fine.
So, I have simplified my model and attached here -->
stantester_fit.R (2.7 KB) stantester_expose.R (455 Bytes) stan_tester.stan (7.8 KB) artf_data.csv (1.0 KB)

I think the problem is coming from use of integrate_1d() function. In the attached model I have included only one function that uses integrate_1d. In the actual model I use integrate_1d 5 more times in different functions , which are omitted here. This simplified model still fails to compile.
The functions within the model are working fine and are verified using the expose function (see the stantester_expose.R file).

Additionally, I am getting this message:

Ill-typed arguments supplied to function 'integrate_1d'. Available signatures: 
((real, real, real[], data real[], data int[]) => real, real, real, real[], data real[], data int[]) => real
((real, real, real[], data real[], data int[]) => real, real, real, real[], data real[], data int[], data real) => real
Instead supplied arguments of incompatible type: (real, real, real[], real[], int[]) => real, real, real, real[], real[], int[], real.

let me know :)
Thanks

#5

So for the second error, can you try compiling your model with this code for N_total (it isn’t correct, I just want to know if it compiles without the first error):

real N_total(real[] parms, real t0) {
  real y_solve;
  real time = parms[5];   // time (host age) input as a parameter

  y_solve = integrate_1d(N_total_integrand, 0.0, time, parms, { 0.0 }, { 0 }, 1E-4);
  return y_solve;
}

The real data and integer data arguments to integrate_1d are kindof tricky.

In the previous code you passed t0 in the real data part. However, t0 is declared as real in the argument as N_total. If you wanted this to be data, you would need to declare the t0 argument as data real t0.

This causes an error elsewhere in a model as something marked not-data is passed in.

Anyway, the easiest thing to do is probably just pass t0 as a parameter.

I don’t think there’s a way to just declare a variable in any block or function as a data variable. I think the only way to declare a non-argument variable as data, you gotta just defined a variable in the data or transformed data blocks and then pass it around. @rybern is this correct?

1 Like
#6

@bbbales2 Thanks for that clarification.

I never wanted to define t0 as real data. I couldn’t figure out how to run the integrate_1d function without parsing a real[] data argument. If one defines the real data argument as rdata[0], then Stan throws an error of missing argument. Anyway, I did modify the code and now got rid of the second error. But the main problem still persists. The model doesn’t compile. I get the same error.
stan_tester_modified.stan (5.8 KB)

Error in compileCode(f, code, language = language, verbose = verbose) : 
  /home/sanket/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat/functor/adj_jac_apply.hpp:535:10:   required from ‘void stan::math::adj_jac_vari<F, Targs>::chain() [with F = stan::math::internal::softmax_op; Targs = {Eigen::Matrix<stan::math::var, -1, 1, 0, -1, 1>}]’/home/sanket/R/x86_64-pc-linux-gnu-library/4.0/StanHeaders/include/stan/math/rev/mat/functor/adj_jac_apply.hpp:531:8:   required from here/home/sanket/R/x86_64-pc-linux-gnu-library/4.0/RcppEigen/include/Eigen/src/Core/DenseCoeffsBase.h:650:34: warning: ignoring attributes on template argument ‘Eigen::internal::packet_traits<double>::type’ {aka ‘__vector(2) double’} [-Wignored-attributes]  650 |   return internal::first_aligned<int(unpacket_traits<DefaultPacketType>::alignment),Derived>(m);      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~make: *** [/usr/lib/R/etc/Makeconf:176: file7f5e34a489fe.o] Error 1
Error in sink(type = "output") : invalid connection

Let me know :)
Sanket

#7

Is this Rstan, by the way? Did you rebuild your rstan once you upgraded to R 4.0? That might be worth a shot.

There’s another R interface called cmdstanr: https://mc-stan.org/cmdstanr/ . I’m curious if this would build there.

I can try upgrading my R to 4.0 tomorrow and report back as well. (Edit: gonna be Sunday or Monday before I get to this)

#8

I just checked this model throws an error in rstan for me (in R 3.6):

> model = stan_model("stan_tester.stan")
Error in compileCode(f, code, language = language, verbose = verbose) : 
  Compilation ERROR, function(s)/method(s) not created! file7e7217b0649.cpp:842:37: error: no matching function for call to 'integrate_1d'
        stan::math::assign(y_solve, integrate_1d(N_total_integrand_functor__(), 0.0, time, parms, static_cast<std::vector<local_scalar_t__> >(stan::math::array_builder<local_scalar_t__ >().add(0.0).array()), static_cast<std::vector<int> >(stan::math::array_builder<int >().add(0).array()), *pstream__, 1E-4));

It compiles in cmdstanr though which makes me think this is a bug that has since been fixed. rstan is a little behind cmdstanr right now – hopefully it catches up soon but can you use cmdstanr for your analysis for now: https://mc-stan.org/cmdstanr/ ?

1 Like
#9

(Also I suspect this wasn’t a problem in 2.19, but when you upgraded to R you had to rebuild rstan which upgraded it to 2.21 and this is why the problem popped up now)

#10

@bbbales2 yes it does compile in cmdstanr. hoping rstan would catch up soon. Its hard to troubleshoot in cmdstanr. thanks for your help.

#11

You might also try downgrading to rstan 2.19 for the time being if that makes life easier. I imagine expose_stan_functions is quite handy for the model you’re working on.

#12

@bbbales2
Downgrading to rstan 2.19 didn’t solve the issue but I am happy with cmdstan for now.
However, I am running into some serious error messages and warnings while fitting the models in which I use the integrate_1D function suggesting that the metropolis proposal is about to be rejected. I checked my code thoroughly and verified that none of my functions were discontinuous or running into singularities.
To troubleshoot further I wrote a stan program that numerically integrates a simple exponential decay,

\int_0^t \phi_0 \, e^{-\nu \, t},

which definitely is smooth and should not run in any issue while integrating.
integ_tester.stan (1.0 KB) integtester_fit.R (728 Bytes) integ_data.csv (885 Bytes)

This is the error message I get for every chain and quite repetitively (4 long pages).

Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: Exception: Error in function tanh_sinh<double>::integrate: The tanh_sinh quadrature evaluated your function at a singular point and got inf. Please narrow the bounds of integration or check your function for singularities. (in '/tmp/RtmpuljRnk/model-5e7132d4f9ef.stan', line 7, column 4 to column 17) (in '/tmp/RtmpuljRnk/model-5e7132d4f9ef.stan', line 7, column 4 to column 17)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3

What does this mean? Should I just ignore it?

#13

Can you say more about what makes it hard to troubleshoot in CmdStanR? If you have suggestions we can hopefully work on them to make it easier. Thanks!

#14

Because they happen early in adaptation and go away later, they’re probably fine to ignore if you have to. They do mean something is a little fragile in the model though.

Sometimes it’s not possible to get rid of these and you just have to live with it. For this model though, if you put an upper bound on log_phi the errors go away.

Something that happens a lot in Stan is exponentiated things blow up.

I didn’t dig all the way down to find what was happening exactly, but if phi0 is something crazy like 1e200 (which might happen early in adaptation), maybe the integrals blow up and NaNs appear and the integrator gets confused.

This integral might work better by either modeling phi0 on a linear scale or moving it outside the integral. I realize this is a test though and so the second thing might not be realistic.

Also you can make time data in your function:

real phi_time(data real time, real[] parms){
  real y_solve = integrate_1d(phi_integrand, 0.0, time, parms, {0.0}, {0});
  return y_solve;
}

It didn’t change anything for me but may as well code it this way.

2 Likes
#15

That makes sense! Irrespective of the warnings my models ran fine and gave reasonable fits. I was under the impression that the algorithm doesn’t wander far off from the priors in the parameter space.

What’s the difference between data real versus real? Is it explained in the updated manual? I will check.

Thanks so much!

#16

It might be hard to search for, but it should be in there somewhere. An argument marked data is a special indicator that basically means we’ll never need to take a derivative with respect to (like things in the data block or transformed data blocks).

Variables without data might either be something we need to compute a gradient with respect to or otherwise we need to pass autodiff information through to compute gradients of something else. There’s some extra hidden work on these things.

Especially in early warmup it can do some wild things. It is surprising sometimes. Exponentiated things blow up quick too. Sometimes there are tricks to avoid these things. I remember this thread was interesting: Using integration to fit the difference of gamma distributed random variable

1 Like
#17

In my experience, the warning and error messages that I got from cmdstanr were a bit ambiguous. For example, I had forgotten to define a parameter but the program did not point to it but just showed that all chains had failed. In some other cases, it did point out what was wrong but associated it with a wrong line in the Stan code. I understand its gonna improve.

I really like it. Its quite neat and clean. Allows for customization. I definitely see a great potential. Especially, when integrating custom functions written in CPP.

#18

I’ll add to that the lack of expose_stan_functions hits hard for stuff like this.

It can be hard to debug stuff like this in R itself – to have fancy functions like an integrator nested inside another fairly complicated thing like Stan makes life even harder.

#19

Oh yeah that’s something we need to work on. For now you can get more information for a particular chain using fit$output(chain_id). So fit$output(1) for chain 1.

#21

@jonah
Is it possible to recover the parameter values and initial conditions for an iteration for which the sampler is failing?

Chain 3 Rejecting initial value:
Chain 3   Error evaluating the log probability at the initial value.
Chain 3 Exception: Exception: Exception: Exception: integrate: error estimate of integral above zero 5.00586e-09 exceeds the given relative tolerance times norm of integral above zero (in '/tmp/RtmpAnWvKa/model-3fad4361b76ae.stan', line 254, column 4 to column 19) (in '/tmp/RtmpAnWvKa/model-3fad4361b76ae.stan', line 254, column 4 to column 19) (in '/tmp/RtmpAnWvKa/model-3fad4361b76ae.stan', line 254, column 4 to column 19) (in '/tmp/RtmpAnWvKa/model-3fad4361b76ae.stan', line 254, column 4 to column 19)
Chain 3 Exception: Exception: Exception: Exception: integrate: error estimate of integral above zero 5.00586e-09 exceeds the given relative tolerance times norm of integral above zero (in '/tmp/RtmpAnWvKa/model-3fad4361b76ae.stan', line 254, column 4 to column 19) (in '/tmp/RtmpAnWvKa/model-3fad4361b76ae.stan', line 254, column 4 to column 19) (in '/tmp/RtmpAnWvKa/model-3fad4361b76ae.stan', line 254, column 4 to column 19) (in '/tmp/RtmpAnWvKa/model-3fad4361b76ae.stan', line 254, column 4 to column 19)
Chain 3 Initialization between (-2, 2) failed after 100 attempts. 
Chain 3  Try specifying initial values, reducing ranges of constrained values, or reparameterizing the model.
Chain 3 Initialization failed.
Warning: Chain 3 finished unexpectedly!

sanket