In Stan, we have named parameters and we also have parameters on the underlying (typically unconstrained) scales. I have some questions/issues:
When we do optimizing we will often get draws using Laplace approx on underlying scale. What will happen once we add this new feature in which linear transformations can depend on other parameters? In this case, the meaning of the underlying or unconstrained parameters changes in each iteration, hence if we draw on the underlying scale, we’d need to be able to convert back, using all the parameters in the model. All this info is there (it would seem to me to have the form of “generated quantities” in that it could be seen as a form of postprocessing) but I expect we’d need to put it in Stan to make this really work.
Same issue for ADVI as well as future algs such as GMO and EP: We often want to work on the unconstrained space but then we have to return to reality at some point.
Currently we do some of this in the interfaces (in rstan and rstanarm) but I’m thinking this is really fundamental and important enuf that it should be in Stan itself. Especially given that we’ve been talking about removing certain features in rstan such as the ability to access unconstrained parameters directly.