Manage warnings: a "production" mode for warnings and errors managing, for the end-user

Hello,

as per title, I have a simple model involving overdispersed NB data, for example with

mu ~ 1000
phi ~ 0.1

data {
	int<lower=1> G;
	int<lower=1> S;
  int<lower=0> y[S,G]; // data. gene mupression (integer counts) 
  real<lower=0> mu_mu_mu;
}
parameters {
	real<lower=0> mu[S];
	real<lower=0> mu_mu;
	real<lower=0> mu_si;
	real<lower=0> si[S];
}
model {
	si ~ cauchy(0,2.5);
	mu_si ~ cauchy(0,10);

	mu ~ normal(mu_mu, mu_si);
	mu_mu ~ normal(mu_mu_mu, mu_mu_mu/5);
	
	for(s in 1:S) y[s] ~ neg_binomial_2(mu[s], si[s]);

}

Although I get good convergence I would like to avoid these warnings.

Exception thrown at line 22: neg_binomial_2_lpmf: Precision parameter is 0, but must be > 0!          2
Exception thrown at line 22: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     2
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2

Any Idea?

Thanks

Those may well come from the warmup period anyway. si can overflow or underflow, which is not surprising with a Cauchy prior and the flakiness of the negative binomial distribution.

Thanks Ben,

yes I agree that is not a problem at least in my case.

But nonetheless:

  1. is it possible that those are a signal for wrong parametrization?

  2. is it possible that re-parametrizing I will get better performances (i.e., execution time)?

  3. This message tends to be hard to collect in a pipe and hide it from the end user, which will be worried seeing it

Thanks

The parameterization may be wrong in any particular Stan program but a couple of overflow or underflow warnings does not indicate that either way. But I can’t think of a reason why anyone should be using the Cauchy PDF for a prior; the Stan User Manual has an subsection of section 27.6 entitled “Reparameterizing the Cauchy”.

Those messages are hard to hide because what if an end user inputs data where 200 such messages are thrown?

Thanks Ben,

For the warning messages, would be nice to have advanced options for production product, where the developer (with enough experience on that kind of data) can establish that for less than 20 the warning is hidden.

For example, I am running stan in parallel for many data sets (for particular reason)

What the end user sees in R is

Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     1
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!      1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                                  count
Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     1
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!           1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                                 count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                                  count
Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     1
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!      1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                                  count
Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                                 count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 3
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                             count
Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 3
                                                                                                 count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                                 count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.

And so on.

I know that 1 chain error doesn’t mean much in my implementation but the user get thrown 100s of warning messages. And I can imagine the reaction in RStudio where an incredibly long output full of warnings get shown.

Thanks

Yeah I agree that we should ideally provide more options in this regard. Like Ben said we definitely don’t want to make it at all easy for people to ignore these messages but it’s certainly true that there are cases when it would be useful to turn them off (or at least easily capture them and report them in a less messy way).

Many frameworks have a default “development” mode, and an optional “production” mode.

The needs for these two can be quite different, with the latter one taking into account the user experience, including: collecting and managing warnings, making decisions for the user knowing the risks (e.g., giving warning conditionally).

For example: since my parallelism I would like to collect all warnings, parse them, and present a unique warning to the user, in a custom fashion.

Would be nice to have this implemented, as I think a big part of the diffusion of Stan at academic level is to produce tools that are user friendly for the end-user.

A github feature request maybe?

Thanks. Bw.

If there’s a feature more specific than a “production” mode, then yes, a feature request on GitHub is the way to go. We want requests there to have a tight enough functional and technical spec to be implementable—GitHub isn’t the place to have design discussions.

For “production”, we tend to recommend CmdStan as it’s less memory intensive and you get a clean output in the form of a CSV file per chain which can then be imported into RStan or PyStan.