Negative value from exponential distribution

Modeling
1

Hi there,

I am using stan to write an adaptive lasso model as in Figueiredo (2003). I write the prior with a non-centered parameterization.

The code is the following:

data {
 ...
}

parameters {
  vector[K] eta;            // variable for non-centered parameterization
  vector[K] tau;            // for the hierarchical structure
  real delta;                // parameter for lasso
}

transformed parameters {
  vector[K] beta = eta .* sqrt(tau);
}

model {
  eta ~ normal(0, 1);
  delta ~ inv_gamma(3, 1);
  tau ~ exponential(delta/2);

  ...
}

When I run this model, I got the following errors:

RuntimeError: Error during sampling:
Exception: exponential_lpdf: Random variable[3] is -0.839665, but must be nonnegative! (in 'adaptivelasso.stan', line 21, column 2 to column 29)
	Exception: exponential_lpdf: Random variable[12] is -1.80133, but must be nonnegative! (in 'adaptivelasso.stan', line 21, column 2 to column 29)

I am confused by the errors themselves. May I know what the errors refer to and how could I solve this problem?

Thanks in advance!

2

The fix is to declare constraints:

parameters {
  vector[N] eta;
  vector<lower=0>[N] tau;
  real<lower=0> delta;
}

Explanation:

The model block defines a probability density function, not a sampling procedure.
For examples, when you write

data {
  real x;
} parameters {
  real mu;
} model {
  x ~ normal(mu, 1);
}

it wouldn’t make sense to sample x which has a known value. Instead the “distribution statement” only calculates the probability of observing the particular values of data and parameters. All parameters have been assigned their values before the model block runs.

The parameter values come from the HMC algorithm. For every sample, HMC guesses values for the parameters, runs the model block to evaluate the probability of that guess, then produces a new guess based on what it learned, evaluates that guess, and so on until it decides to accept one of these candidates.
(The number of candidates evaluated for each accepted sample is recorded as n_leapfrog__ column in Stan’s output.)

HMC looks only at the parameters block when creating the initial guess. If you don’t declare lower bound constraints there, some parameters may be assigned negative values, and the probability evaluates to zero. HMC cannot learn from this sort of “impossible” candidate value.

5 Likes
3

Hi nhuurre,

Thanks a lot for your reply. It makes sense to me.

I am concerned that if I assign a lower bound for delta and tau, will that cause unnecessary Jacobian transformation?

4

Stan automatically applies the relevant Jacobian adjustments for the constraining transforms associated with bounds declarations.

5

Thanks for your reply. Yes, I think I figure it out. When the prior distribution is bounded (although by construction), I always need to declare it. Stan will implement its procedure on the unconstrained parameter space anyway.