Thanks, @jsocolar. This is super cool. I didn’t do the math to figure out the correction and I don’t even think I reviewed the PR.
There is a Jacobian, but it’s constant (i.e., doesn’t depend on parameters).
The Jacobian is always square in a change of variables but it’s not always apparent what those variables are (and sometimes there’s a choice). For example, if I have variables (x, y) and want a transformed variable \exp(x + y), then I can’t just put a prior on \exp(x + y) and get a distribution over (x, y)—the result is the same for (x - c, \ y + c), so I need more constraint. For example, I could put a prior on x or y in addition to \exp(x + y) and get a proper prior on (x, y). The typical fix is to think of the transform as mapping both x and y, which means you need two outputs. A simplex choice is (x, y) \rightarrow (\exp(x + y), y) and you get a nice triangular Jacobian and easy determinant calculation and then you can put a prior on \exp(x + y) and on y and get a proper prior on (x, y).
Let’s look at a simple example, where we map (x, y) \rightarrow (x, y, -(x + y)). This is indeed not square. We have too many degrees of freedom in the output. So instead we need to pick and choose. (x, y) \rightarrow (-(x + y), \ y) works. Then what’s happening in this model is what @betanalpha is complaining about—we’re overconstraining the solution by putting priors on all of x, y, and -(x + y). That’s why we need the adjustment to the scale of the prior indicated in the user’s guide.
Is there an example you had in mind?