I have been making frequent use of log_sum_exp() and log_diff_exp() in my modelling. But the following struck me as curious:
log(0) = -inf
log_sum_exp(log(1), log(0)) = 0
log_diff_exp(log(1), log(0)) = 0
log_sum_exp(log(0), log(0)) = nan
log_diff_exp(log(0), log(0)) = nan
log_diff_exp(log(1), log(1)) = nan
In the first three cases, Stan properly treats log(0) = -inf as a number and allows arithmetic to be performed.
For the last three cases I would also expect that the proper return should also be -inf, but instead we get nan. I’ve gotten around this for my immediate purpose by writing a custom function, but I’m wondering if this should be “fixed” on the developer side.
I agree the last one should probably be an inf, but is there a way to avoid infs in your code? Depending on infs seems really fragile, cause I’m sure there are a lot more confusing examples like the ones you brought up (and who knows what the autodiff will do).