Gibbs post-processing to find unknown K in mixture model

Modeling
2

Note however that for not-so-big H you should be able to marginalize h out. The idea is that you model the probabilities p_k = P(h = k) as parameters in the model and compute the mixture probability over all H possible models. A sketch in code (please double check, I do make mistakes quite frequently):

parameters {
  simplex[H] p; //probabilities need to sum to 1
}

model {
  for(h in 1:H) {
    sub_likelihood[h] = ... //Compute likelihood assuming h components
  }
  //Mixture likelihood
  target += log_sum_exp(p + sub_likelihood);
}

I however need to admit that I do not have experience running those with mixture models neither have I seen it done before. (But if it works, please let us know, would be nice for the manual :-) The only mention I could find quickly was a thread by @ldeschamps (Mixture model with unknown k), but there is no conclusion whether this approach has been succesful.

Note that you would probably need completely separate model parameters for each value of h, as there is no reason to believe that components of the smaller models are related to the components of the larger models (e.g. that \alpha^{h=2}_1 = \alpha^{h=3}_1).

The methods described in the Latent discrete parameters chapter of the Stan User’s Guide are somewhat related, but not very instructive for your case.

Hope that helps!

2 Likes