Jacobian of softmax tansformation of a (n-1 degrees of freedom) unbounded parameter

[stemangiola]

@betanalpha is it possible the the whole mistake comes from the fact that we are mapping K-1 to a K vector? Although it is not the softmax that does that per-se, I though to just make sure.

@betanalpha

Change of variables follows from probability theory which is airtight in what you can and cannot do.

1. If you have a one-to-one transformation then the transformed density is given by the original density evaluated at the transform inverse augmented with the absolutely value of the Jacobian determinant.
2. If you have a one-to-many transformation then you do the above but sum over all of the possible values of the transform inverse. This behaves well except in cases where there are infinite number of values, for example when you try to map a circle into the real line by transforming (0, 2pi) to (-\infty, \infty), in which case the transform becomes ill-posed.

Our case I assume is number 2 (one-to many; i.e. K-1 vector to K simplex)