Manage warnings: a "production" mode for warnings and errors managing, for the end-user

stemangiola · August 17, 2017, 4:20am

Hello,

as per title, I have a simple model involving overdispersed NB data, for example with

mu ~ 1000
phi ~ 0.1

data {
	int<lower=1> G;
	int<lower=1> S;
  int<lower=0> y[S,G]; // data. gene mupression (integer counts) 
  real<lower=0> mu_mu_mu;
}
parameters {
	real<lower=0> mu[S];
	real<lower=0> mu_mu;
	real<lower=0> mu_si;
	real<lower=0> si[S];
}
model {
	si ~ cauchy(0,2.5);
	mu_si ~ cauchy(0,10);

	mu ~ normal(mu_mu, mu_si);
	mu_mu ~ normal(mu_mu_mu, mu_mu_mu/5);
	
	for(s in 1:S) y[s] ~ neg_binomial_2(mu[s], si[s]);

}

Although I get good convergence I would like to avoid these warnings.

Exception thrown at line 22: neg_binomial_2_lpmf: Precision parameter is 0, but must be > 0!          2
Exception thrown at line 22: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     2
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2

Any Idea?

Thanks

bgoodri · August 17, 2017, 4:31am

Those may well come from the warmup period anyway. si can overflow or underflow, which is not surprising with a Cauchy prior and the flakiness of the negative binomial distribution.

stemangiola · August 17, 2017, 5:14am

Thanks Ben,

yes I agree that is not a problem at least in my case.

But nonetheless:

is it possible that those are a signal for wrong parametrization?
is it possible that re-parametrizing I will get better performances (i.e., execution time)?
This message tends to be hard to collect in a pipe and hide it from the end user, which will be worried seeing it

Thanks

bgoodri · August 17, 2017, 5:46am

The parameterization may be wrong in any particular Stan program but a couple of overflow or underflow warnings does not indicate that either way. But I can’t think of a reason why anyone should be using the Cauchy PDF for a prior; the Stan User Manual has an subsection of section 27.6 entitled “Reparameterizing the Cauchy”.

Those messages are hard to hide because what if an end user inputs data where 200 such messages are thrown?

stemangiola · August 18, 2017, 12:31am

Thanks Ben,

For the warning messages, would be nice to have advanced options for production product, where the developer (with enough experience on that kind of data) can establish that for less than 20 the warning is hidden.

For example, I am running stan in parallel for many data sets (for particular reason)

What the end user sees in R is

Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     1
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!      1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                                  count
Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     1
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!           1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                                 count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                                  count
Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     1
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!      1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                                  count
Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                                 count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 3
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                             count
Exception thrown at line 28: neg_binomial_2_lpmf: Precision parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 3
                                                                                                 count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 2
                                                                                            count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is 0, but must be > 0!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.
The following numerical problems occurred the indicated number of times on chain 1
                                                                                                 count
Exception thrown at line 29: neg_binomial_2_lpmf: Location parameter is inf, but must be finite!     1
When a numerical problem occurs, the Hamiltonian proposal gets rejected.
See http://mc-stan.org/misc/warnings.html#exception-hamiltonian-proposal-rejected
If the number in the 'count' column is small, there is no need to ask about this message on stan-users.

And so on.

I know that 1 chain error doesn’t mean much in my implementation but the user get thrown 100s of warning messages. And I can imagine the reaction in RStudio where an incredibly long output full of warnings get shown.

Thanks

jonah · August 18, 2017, 4:16pm

Yeah I agree that we should ideally provide more options in this regard. Like Ben said we definitely don’t want to make it at all easy for people to ignore these messages but it’s certainly true that there are cases when it would be useful to turn them off (or at least easily capture them and report them in a less messy way).

stemangiola · August 21, 2017, 1:27am

Many frameworks have a default “development” mode, and an optional “production” mode.

The needs for these two can be quite different, with the latter one taking into account the user experience, including: collecting and managing warnings, making decisions for the user knowing the risks (e.g., giving warning conditionally).

For example: since my parallelism I would like to collect all warnings, parse them, and present a unique warning to the user, in a custom fashion.

Would be nice to have this implemented, as I think a big part of the diffusion of Stan at academic level is to produce tools that are user friendly for the end-user.

A github feature request maybe?

Thanks. Bw.

Bob_Carpenter · August 28, 2017, 8:06pm

If there’s a feature more specific than a “production” mode, then yes, a feature request on GitHub is the way to go. We want requests there to have a tight enough functional and technical spec to be implementable—GitHub isn’t the place to have design discussions.

For “production”, we tend to recommend CmdStan as it’s less memory intensive and you get a clean output in the form of a CSV file per chain which can then be imported into RStan or PyStan.

Topic		Replies	Views
Exception: neg_binomial_2_log_glm_lpmf: Precision parameter is 0, but must be positive finite! brms fitting-issues , brms	1	222	March 13, 2024
Metropolis rejection proposal due to incorrect numerical values for derived parameter Modeling	4	2190	March 23, 2018
Text for warning message General	46	6019	August 11, 2020
Positive definite warnings during beta binomial model (brms) Interfaces brms	3	842	February 10, 2023
Simple Example of an Improper Posterior Without Warnings from Stan General	22	2786	December 7, 2019

Manage warnings: a "production" mode for warnings and errors managing, for the end-user

Related topics