Model trains fails with message, but no indication which part of the code

Modeling
1

I get error message during model training.

Rejecting initial value:
Log probability evaluates to log(0), i.e. negative infinity.
Stan can’t start sampling from this initial value.

Initialization between (-2, 2) failed after 100 attempts.
Try specifying initial values, reducing ranges of constrained values, or reparameterizing the model.
Initialization failed.

There is no where in the code which uses the parameter values 2.0.

I am not able to figure out where to look, because the message gives too little clue. Is there a way to find out which part of the model code caused this error?

2

This message means that the sampler cannot get off the ground - in needs a starting set of parameter values - a starting point for sampling, but attempts to randomly initialize the parameters to values close to zero fail, given the data and your model.

The values (-2,2) are not in your code - they’re the defaults for HMC. you can change these - see the Stan reference manual and doc for the interface you’re using.

1 Like
3

it seems inserting print statements is still the best way to find the problem area.

4

absolutely!

5

am I correct that if I set constraints on the parameter with something like param ~normal(mean, sigma), then HMC would no longer use uniform initialization between [-2,2]?

6

no, the (-2,2) range is controlled by the “init” param - here’s the Rstan doc:

The default is to randomly generate initial values between -2 and 2 on the unconstrained support. The optional additional parameter init_r can be set to some value other than 2 to change the range of the randomly generated inits

check your data - are you trying to take the log of a negative input or otherwise doing a transform that results in negative infinity?

1 Like
7

I did find the area where the log(0) happened. I am just wondering, if I already declared a normal prior, why is HMC still sampling based on uniform[-2,2]?

8

The normal prior does not directly determine sampling. The prior (and all other distributions) in your model only construct the loglikelihood (target).
The sampling on uniform[-2,2] is only for the initial values of the parameters. The uniform is also only on [-2,2] in the unconstrained space. The idea is that if you have log(parameter) somewhere you want to declare the parameter > 0. Stan will (implicitly) sample from [exp(-2), exp(2)] for the initial value of this parameter

2 Likes
9

could you give the same instruction for cmdstan? is it simply init= or init=?

and in the textfile, I can specify different init ranges for each parameter?

10

> ./bernoulli sample data file=bernoulli.data.R init=0.5

don’t understand “textfile”?

init is an argument to the interface command that invokes the sampler.

in a Stan program, you can and should put constraints on data variables - these constraints will be checked at the point where the model is instantiated with the data. e.g. if you have input vector y with values between 0 and 1:

data {
  vector<lower=0, upper=1>[10] y;
  ...

this will allow values on the boundaries, as these can arise due to finite-precision arithmetic rounding errors, so to properly insure strictly > 0 data values, you’d need to add a check in the transformed data section:

transformed data {
 for (i in 1:10) {
   if (y[i] == 0) {
     print("bad value in y at index: ", i);
     reject;
   }
 }
....

hope this helps.