In “Estimating Parameters of a Mixture”: “The location parameter
mu is declared to be an ordered vector in order to identify the model.”
If the mixture components have the same location (e.g. 0) but different scales, can we order the scales to identify the model?
simplex[K] theta; // mixing proportions
positive_ordered[K] sigma; // scales of mixture components
Yes. @betanalpha wrote a nice case study for mixtures and identifiability that goes through the (surprisingly simple once you see it) math on why this is OK.
And in this particular case, it might be easier to think about what’s happening on the unconstrained scale, where \sigma_k gets transformed to \tau_k = \log \sigma_k. There, you can order the \tau_k, which is the same as ordering the \sigma_k once you apply \exp(\tau_k).