# Non-identifiability of mixing weights in a 2-component Gaussian model

**URL:** https://discourse.mc-stan.org/t/non-identifiability-of-mixing-weights-in-a-2-component-gaussian-model/4630
**Category:** Modeling
**Created:** [June 25, 2018, 10:11pm UTC](https://discourse.mc-stan.org/t/non-identifiability-of-mixing-weights-in-a-2-component-gaussian-model/4630 "2018-06-25T22:11:35Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Jaslene\_Lin](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@Jaslene\_Lin](https://discourse.mc-stan.org/u/Jaslene_Lin)
#### Post date: [June 25, 2018, 10:11pm UTC](https://discourse.mc-stan.org/t/non-identifiability-of-mixing-weights-in-a-2-component-gaussian-model/4630/1 "2018-06-25T22:11:35Z")

</div>

[Rplot.pdf](https://canada1.discourse-cdn.com/flex030/uploads/mc_stan/original/2X/5/5af2a855f296f30f8d2bfe381bf26cee415b5266.pdf) (5.3 KB)  
Dear stan users:  
I am currently constructing a 2-component time-varying Gaussian mixture model using Rstan,the full model is a bit complex and thus right now I just construct a smaller model to examine the model behaviour.  
My model is like:  
E(y\_it) = \lambda\_t \* N(\beta\_{t1}, \sigma) + (1-\lambda\_t) \* N( \beta\_{t2}, \sigma) for all observations i =1: n ( in the real data, n=12 and there is some missing observations, for the simulation study n is set be 50 with the same missing patterns as in the real data)  
and t is the time index, currently t=1:72.  
the mixing weights and mixture locations are all time-varying with time-invariant and component-shared scale \sigma. Right now, I only focus on estimating \lambda\_t and \sigma ( supplying “true” values of \beta\_{t1} and \beta\_{t2}, to ensure identifiability, all \beta\_{t2} \>\beta\_{t1})  
the mixing weights are believed to be slowly and smoothly changing over time and thus I use model it with a bspline with a RW(1) penalty prior ( seen post by [http://mc-stan.org/users/documentation/case-studies/splines\_in\_stan.html](http://mc-stan.org/users/documentation/case-studies/splines_in_stan.html)).  
Right now the model estimates sigma well. But in fact, the “true” \lambda\_t is the 1-\lambda\_t in the posterior samples. I do not understand why as I already supply the mixture locations \beta\_{t1} and \beta\_{t2} as data, why there is still label switching? \lambda\_t is modelled by inverse logit of \eta\_t where \eta\_t is the product of the Bspline basis matrix eta\_bspline and the coefficients \beta\_{\eta}. Can someone please provide some insights? The posterior mean of (1-\lambda\_t) is attached, the red line is the simulated “true” values while the black line is the posterior means for (1-\lambda\_t)  
The stan code is attached below

* * *

```stan
data {
  
  int<lower=1> N; // number of items that are defined
  
  int<lower=1> K; //number of columns==72 hours in a week 
  
  int<lower=1> D;// indicating number of days fitted
  //for simulation D=3, for real full 2-week data D=14
  
  int<lower=1> Ncol;
  
  int<lower=1,upper=Ncol> kk[N];//hour of week index 1:72
  
  vector[N] dat_complete;//defined LOG-TRANSFORMED observations
  
  matrix[K,2] ordered_overall_mean;
  
  int<lower=0> num_basis;
  
  matrix[K, (num_basis-1)] eta_bspline;

   
  
}

parameters {
  vector[(num_basis-1)] beta_eta_raw;
  
  real<lower=0> tau; \\RW penalty parameter for eta 
  
  real<lower=0> sigma;

  
}

transformed parameters{
  vector[(num_basis-1)] beta_eta;
  
  vector[K] eta;
  
  vector[K] lambda;
  
  beta_eta[1] =beta_eta_raw[1];
  
  for(i in 2:(num_basis-1)){
    beta_eta[i] = beta_eta[i-1] + beta_eta_raw[i] * tau;
  }
  
  eta = eta_bspline* beta_eta;
  
  lambda = inv_logit(eta);
  
}

model{
  
  beta_eta_raw~normal(0,1);
  
  tau~normal(0,1);
  
  sigma~cauchy(0,2.5);
 
  //likelihood  
  
   for(n in 1:N){
      
      
      target +=log_mix(lambda[kk[n]], normal_lpdf(dat_complete[n]| ordered_overall_mean[kk[n],1], sigma),
                     normal_lpdf(dat_complete[n]| ordered_overall_mean[kk[n],2], sigma));
  }
    
 
  
  }

```

* * *

Thank you for your feedback.

---

<div class="post-metadata">

### Author: ![bbbales2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bbbales2/32/77_2.png) [@bbbales2](https://discourse.mc-stan.org/u/bbbales2)
#### Post date: [June 26, 2018, 1:26pm UTC](https://discourse.mc-stan.org/t/non-identifiability-of-mixing-weights-in-a-2-component-gaussian-model/4630/2 "2018-06-26T13:26:27Z")

</div>

> [@Jaslene\_Lin](#):
>
> The posterior mean of (1−λt)(1-\lambda\_t) is attached, the red line is the simulated “true” values while the black line is the posterior means for (1−λt)

Can you plot a bunch of posterior samples of (1 - \lambda t)? What about this seems bad though? They look like they’re doing the same thing.

---

<div class="post-metadata">

### Author: ![Jaslene\_Lin](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@Jaslene\_Lin](https://discourse.mc-stan.org/u/Jaslene_Lin)
#### Post date: [June 26, 2018, 9:41pm UTC](https://discourse.mc-stan.org/t/non-identifiability-of-mixing-weights-in-a-2-component-gaussian-model/4630/3 "2018-06-26T21:41:47Z")

</div>

I plotted posterior samples for \lambda\_t and 1-\lambda\_t the red line is the simulated truth.

I think 1-\lambda\_t posterior samples are quite good and looking like they estimate the “truth” fairly well.

I am quite confused why it is 1-\lambda\_t rather than \lambda\_t that does a good job.

The first plot is for 101 posterior samples of 1-\lambda\_t while the second plot is for \lambda\_t  
[post\_sample1.pdf](https://canada1.discourse-cdn.com/flex030/uploads/mc_stan/original/2X/8/8ad2a1b3b943b1a7fc8f2003287e3c53746a3b8e.pdf) (36.7 KB)  
[post\_sample2.pdf](https://canada1.discourse-cdn.com/flex030/uploads/mc_stan/original/2X/7/7fb6a0f245c888d30af9e3b45dfc9215949d5f08.pdf) (36.3 KB)

---

<div class="post-metadata">

### Author: ![bbbales2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bbbales2/32/77_2.png) [@bbbales2](https://discourse.mc-stan.org/u/bbbales2)
#### Post date: [June 27, 2018, 12:50pm UTC](https://discourse.mc-stan.org/t/non-identifiability-of-mixing-weights-in-a-2-component-gaussian-model/4630/4 "2018-06-27T12:50:36Z")

</div>

My gut feeling is there is some sort of indexing problem somewhere. You can use print statements in Stan code, so maybe print a few values of ordered\_overall\_mean and see if they’re what you’d expect?

If that doesn’t get you anywhere, maybe try generating the simulated data with the Stan model? Maybe there’s a difference in the splines you’re using to generate your data and the ones you’re fitting with?

---

<div class="post-metadata">

### Author: ![jaslenelin](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/jaslenelin/32/5267_2.png) [@jaslenelin](https://discourse.mc-stan.org/u/jaslenelin)
#### Post date: [June 27, 2018, 11:31pm UTC](https://discourse.mc-stan.org/t/non-identifiability-of-mixing-weights-in-a-2-component-gaussian-model/4630/5 "2018-06-27T23:31:34Z")

</div>

Thank you for your suggestions. I figured out, the error actually comes from the way I simulate the data. in my simulated data \lambda\_t actually corresponds to 2nd component so the model is doing the right thing.

---

<div class="post-metadata">

### Author: ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)
#### Post date: [July 20, 2018, 4:44pm UTC](https://discourse.mc-stan.org/t/non-identifiability-of-mixing-weights-in-a-2-component-gaussian-model/4630/6 "2018-07-20T16:44:58Z")

</div>

There’s also a nice case study on mixture models by Michael Betancourt on the Stan web site.