Multinomial mixture model

Modeling
1

I’m new to Stan.
I coded Poisson mixture model by copying stan tutorial.
My code is here.

data {
  int N;
  int K;
  int x[N];
}

parameters {
  ordered[K] theta;
  simplex[K] pi;
}

model {
  real lp[K];
  for(n in 1:N){
    for (k in 1:K){
    lp[k] = log(pi[k]) + poisson_lpmf(x[n] | theta[k]);
    }
    target += log_sum_exp(lp);
  }
}

I learned to define theta as an order variable to escape label switching problem from this code.
Next, I want to model multinomial mixture model.
A situation what I thought is below.
There are two dices. One is normal dice whose probability of getting one particular value is all 1/6. Another one is cheating dice whose probability is not all same. I want to infer which dice is thrown from the result.
So, I modified my code like below.

data {
  int N;
  int S;
  int K;
  int x[S,N];
}

parameters {
  simplex[S] theta[K];
  simplex[K] pi;
}

model {
  real lp[K];
  for(n in 1:N){
    for (k in 1:K){
      lp[k] = log(pi[k]) + poisson_lpmf(x[,n] | theta[k]);
    }
    target += log_sum_exp(lp);
  }
}

I don’t know how to define theta as order variable because theta is not scalar but vector.
As a result of modeling, inferred parameters may not be convergenced.
Please someone tell me how to fix my code. Thank you!

2

I think the problem is not with the code, but with the model. For Stan (or any other Bayesian tool) to work, you need to avoid labelling non-identifiabilities, those arise when for some 1 < i,j <= K; i != j switching pi[i] with pi[j] and theta[i] with theta[j] at the same time results in the same likelihood. Ordering is one way to avoid this but, as you observed it does not generalize to simplices. However, I don’t think there is a general way to order simplices, so you need to make more assumptions (e.g. that ordering only the first component of the simplex will identify the model).

What is the final scenario you are aiming at? If it is checking that a dice is fair, you might explicitly model the fair dice with a single parameter and only introduce simplex for the non-fair case, which could avoid the non-identifiability.

3

Thank you for your comment.

I’ll check about label non-identifiabilities.
As you said, my model have some problems due to lack of consideration about the model.
I don’t decide my goal because this task is just for practice.
As you said, if I want to check two dice programs, another model would be better than this model.

Anyway, I appreciate your advice. Thank you!

4

I see two poisson models and no multinomial mixture. An ordered positive theta parameter can be made by using the cummulative_sum and simplex. We had that topic one or two years ago.
Also worth checking out the dirichlet multinomial as prior.