Jacobian correction to likelihood of transformed response

I recently read @avehtari’s FAQ on cross-validation, and have been working through the Mesquite example from ROS, where the outcome variable is log transformed. In that case study, \log (Y) is subtracted from the log likelihood of the log-transformed model before comparing log-likelihoods with the untransformed model. I’m struggling to see where this comes from.

In the cross-validation FAQ, there’s a comment that you need to correct by the Jacobian after transformation, which makes sense to me. However, I’m confused about where the \log in particular comes from. If we have some normal random variable X and Y = \log X, then f_y(y) = f_x(x(y)) \cdot \left \vert \frac{\partial x(y)}{\partial y} \right \vert = f_x(\exp y) \cdot \exp(y) where f_x is the Gaussian density for x, and x(y) = \exp(y). It seems we’d want to correct the density for each point by \exp(y) and correct the log density by \log \exp (y) = y rather than \log y, so I’m a bit lost.

This SO question mentions a paper by Akaike that arrives at the same transformation, but also does not show the derivation itself. I imagine I’m missing something obvious about the setup of the problem and would love some pointers.

Hi, first let me say that I really don’t know what I’m talking about :). But in the ROS example you linked, the response transformation is y = log(weight) so it seems to match your derivation.

Applying the log to the Jacobian adjustement gives log( f_y(y) ) = log( f_x(exp(y)) ) + y as you said, which in the example is log(f_y) = log( f_x ) + log(weight). And so to make the loo comparison, they remove log(weight) from log(f_y) in order to compare with log(f_x)

1 Like

I got it. :)

I also confused the equation

\text{log}(f_y(y)) = \text{log}(f_x(\text{exp}(y))) + y

with

\text{log}(f_y(y)) = \text{log}(f_x(\text{exp}(y))) + x.

1 Like

Adding here couple references