How to use 'log_sum_exp' function to deal with a negative value

General
,
1

Hello everyone,

I have some trouble using “log_sum_exp” function as I may have some negative inputs inside the “log_sum_exp()”. How can I still use this function to make the calculation numerically stable for this case? Thanks!

1 Like
2

Just to clarify first, if I understand your question correctly, you have a vector of values where some are negative which you need to first take the log of and then accumulate with log_sum_exp, is that right?

Can you provide any more detail about what you’re trying to do?

2 Likes
3

Just to clarify further: log_sum_exp already works and should be numerically stable for negative inputs. @andrjohns wonders whether your problem arises because you are really asking about log_sum_exp(log(x)), which cannot handle negative values. But it’s not supposed to be able to handle negative values. If that’s your case, it would be helpful to provide more information about what you’re trying to do and why.

1 Like
4

While I do agree we should clarify the OP’s goals, I have to say I do have use cases where I’d like to do summation of arrays with negative values in real space. Of course this involves storing the signs separately.

1 Like
5

Could you give an example? Partly so I can make sure I understand what operation you’re referring to.

6

I actually had to deal with this exact situation recently, where I had to operate on the log scale but also track the signs on the input arguments in case they were negative (for an infinite sum).

I’ll recreate below in stan pseudo-code, since this was in c++ originally

So it ends up in the situation of:

// Store the sign of the inputs
sign_v = sign(v);
// Take absolute value before log-ing
log_v = log(fabs(v));

Then to accumulate on the log scale, I just had to recreate the log_sum_exp function but with the signs handled:

// Only interested in the largest value for positive arguments
max_val = max(sign_v .* log_v);

// Scale by largest value then accumulate on natural scale
sum_exp_val = sum(exp(log_v - max_val) .* sign_v);

// Take the absolute value before logging and rescaling by maximum
log_sum_exp_val = max_val + log(fabs(sum_exp_val));

// Get the returned sign to continue tracking
log_sum_exp_sign = sign(sum_exp_val);
2 Likes
7

I guess I’m having trouble seeing why it’s useful to work on the log scale here. Is the issue that v/max(v) underflows and sum(v) overflows?

8

I think it is always useful to work on the log scale, no? From a purely numerical point of view? At any rate an example of where this could be useful is when computing something like

\begin{equation*} f(t \mid 0, 1, w) = \pi\sum_{k=1}^\infty k \exp \left(\frac{-k^2\pi^2t}{2}\right)\sin(k\pi w), \end{equation*}

as in Navarro & Fuss, 2009 and this thread here.

1 Like
9

For my use-case I was working with rising factorials in an infinite sum (a Kampe de Feriet function).

So for a function that involves iterating over rising_factorial(1, n), once n reaches 10, the result is already at ~3.6 million. On the log scale, that comes down to just ~15.10. This lets the infinite sum iterate much further before the individual components overflow

2 Likes