Log(0) and Stan Arithmetic

General
1

I have been making frequent use of log_sum_exp() and log_diff_exp() in my modelling. But the following struck me as curious:

log(0) = -inf
log_sum_exp(log(1), log(0)) = 0
log_diff_exp(log(1), log(0)) = 0
log_sum_exp(log(0), log(0)) = nan
log_diff_exp(log(0), log(0)) = nan
log_diff_exp(log(1), log(1)) = nan

In the first three cases, Stan properly treats log(0) = -inf as a number and allows arithmetic to be performed.

For the last three cases I would also expect that the proper return should also be -inf, but instead we get nan. I’ve gotten around this for my immediate purpose by writing a custom function, but I’m wondering if this should be “fixed” on the developer side.

1 Like
2

I think arithmetic on infs can turn into NaNs (for instance “NaNs are produced by these operations” here: https://www.doc.ic.ac.uk/~eedwards/compsys/float/nan.html).

I agree the last one should probably be an inf, but is there a way to avoid infs in your code? Depending on infs seems really fragile, cause I’m sure there are a lot more confusing examples like the ones you brought up (and who knows what the autodiff will do).

3

I encountered it when I was “slicing” a (log)normal CDF. The last case, log(1 - 1), is what I had in mind when I asked the question.

I think this should replicate that last case:
log_diff_exp(normal_lcdf(9, 1, 1), normal_lcdf(8.5, 1, 1));

I now have code to catch it when it occurs in my model. And I suppose I could also try log(normal_cdf() - normal_cdf()).

4

That’s a bug. I filed an issue:

Hopefully we’ll get that fixed by 2.18 (no quotes necessary!).

We ran into a similar issue with our truncation that I fixed by doing what you suggest, log(normal_cdf() - normal_cdf()) (and @bgoodri suggested for the patch).

1 Like
5

I’m back at it. In Rstan 2.19.3 I get:
log(0) * 0 = nan
0^0 = 1
log(0^0) = 0

Seems like the first case should also be 0. Infinity times zero is zero, yes?

(Patiently waiting for passionate mathematical debate to follow my concluding innocent question).

1 Like
6

Nope. Not in general. In measure theory, yes.