Error estimate of integral in some chains with different number of looping

Modeling
1

@bbbales2 @nhuurre @jonah @Bob_Carpenter
Hello everyone,

I am trying to fit my model with these versions
R version 4.0.2
Rstan 2.19.3
Platform: x86_64-w64-mingw32/x64 (64-bit)

In my case I am trying to looping the likelihood function(user defined function) over 1:25 with 3 chains. Initially, I tried to fit from 1 to 5 loops, and then 1: 10 loops…For those cases I could do sampling for all chains, but I failed for all loops (i.e. 1:25), it runs only chain 1 and 2 and for chain 3, it gives following error message,

SAMPLING FOR MODEL ‘585f9f35f8c48980a605c47c29c98779’ NOW (CHAIN 3).
Chain 3: Exception: integrate: error estimate of integral 7.80258e-16 exceeds the given relative tolerance times norm of integral

Could anyone help me here? I put seed =1234 in each cases.

Thanks

2

The error says some integral is too small to estimate reliably. Assuming your relative tolerance is 1e-8 then the integral evaluates to less than 7.8e-8.
The error doesn’t necessarily prevent sampling. Does the model produce a fit result? Does it warn about divergent transitions?

From your previous threads, your integral is

\int_{t_{1}}^{t_{2}}\frac{0.3}{x\sqrt{2\pi\sigma^{2}}}e^{-\frac{\left(\log x-\mu\right)^{2}}{2\sigma^{2}}-\lambda\left(t_{2}-x\right)^{\alpha}}dx

This is equal to

\frac{0.3}{\sqrt{2\pi\sigma^{2}}}e^{-\frac{\left(\mu-\log t_{2}\right)^{2}}{2\sigma^{2}}-\frac{\left(\mu-\log t_{2}\right)\log t_{2}}{\sigma^{2}}}\int_{t_{1}}^{t_{2}}\frac{1}{x}e^{-\frac{\left(\log x-\log t_{2}\right)^{2}}{2\sigma^{2}}+\frac{\left(\mu-\log t_{2}\right)\log x}{\sigma^{2}}-\lambda\left(t_{2}-x\right)^{\alpha}}dx

which may be a bit more stable when \mu and \log t_2 are several \sigma apart.

3

@nhuurre I replaced the integral as what you have suggested here. Thank you.
But, I still same error is giving when I am trying to run my whole loop and it prevent the sampling. The error is something like that, (relative tolerance 1e-8 )

Chain 1: Exception: integrate: error estimate of integral 2.20131e-11 exceeds the given relative tolerance times norm of integral
[1] "Error in sampler$call_sampler(args_list[[i]]) : "
[2] " Exception: integrate: error estimate of integral 2.20131e-11 exceeds the given relative tolerance times norm of integral

It was fine when looping over some rows of my data table and in this case I was able to fit results, but it mentioned some divergent transitions too. I tried to work with defining initial parameter values, so then it’s a mess.

Could you please help me?

4

When the integrate_1d function fails, the thing to do first is figure out exactly what function it is failing to integrate.

Then we can work through if it is overflow/underflow, or some issue with the function.

The way to do this in Stan is add print statements near where your integrals are. See what values the parameters took when the integrate failed. Once we have the parameter values we can make plots of the functions and stuff and think about if it’ll ever be possible for the integrate to work.

1 Like