Mixture Model Fitting

Modeling
#1

I’m trying to fit a mixture model to data below.
data.R (55.0 KB)

This is what I’ve got so far. How can I improve even more? I’m not sure what to do on the left part of the plot. Any help is appreciated.

plot
plot700×432 3.81 KB

data {
  int <lower = 0> N;
  vector[N] y;
}

parameters {
  ordered[2] mu;
  real <lower = 0> sigma[2];
  real <lower = 0, upper = 1> theta;
}

model {
  sigma ~ normal(0, 0.5);
  mu[1] ~ normal(0, 2);
  mu[2] ~ normal(4, 1);
  
  mix_weight ~ beta(5, 5);

  for (i in 1:N) {
    target += log_mix(theta,
                      normal_lpdf(y[i] | mu[1], sigma[1]),
                      normal_lpdf(y[i] | mu[2], sigma[2]));
  }
}
#2

Your data looks like it is truncated. Maybe modeling the truncation would improve the fit

  target += log_mix(theta,
         normal_lpdf(y[i] | mu[1], sigma[1]) - normal_lccdf(0| mu[1], sigma[1]),
         normal_lpdf(y[i] | mu[2], sigma[2]));
1 Like
#3

Hey @xhackerz, welcome! To me your data doesn’t look Gaussian (normal). First thing I notice is that your data is positive only; and that it looks spiky on the left. Also, I guess mix_weight is suppose to be theta.

Try a mixture of gamma distributions instead. If you have a look at the wikipedia article, it looks to me like you can reparameterise a gamma to use mean and variance, so then you can use your ordering trick like you did with the normals. E.g.

\mu=\frac{\alpha}{\beta}, \sigma=\frac{\alpha}{\beta^2}, so \beta=\frac{\mu}{\sigma},\alpha=\frac{\mu^2}{\sigma}

If youre luckly a mixture of 3 gamma’s may also account for that big spike you see on the left too. Here’s a 2 component mixture example to get you started. I haven’t run it but it’s something.

data {
  int <lower = 0> N;
  vector[N] y;
}

parameters {
  positive_ordered[2] mu;
  vector<lower=0>[2] sigma;
  vector<lower=0, upper=1> theta;
}

transformed parameters {
  vector<lower=0>[2] alpha= mu .* mu ./ sigma;
  vector<lower=0>[2] beta= mu ./ sigma;
}

model {
  sigma ~ normal(0, 1000);
  mu ~ normal(0, 1000);
  for (i in 1:N) {
    target += log_mix(theta,
                      gamma_lpdf(y[i] | alpha[1], beta[1]),
                      gamma_lpdf(y[i] | alpha[2], beta[2]));
  }
}

Forgive any mistakes; I wrote this super quickly. Hope it helps.

2 Likes
#4

Yeah, it could be a gamma instead of a truncated normal.

I think you meant to have gamma_lpdf in both mixture components instead of that normal_lpdf in the second. EDIT: fixed

2 Likes
#5

i’m trying to run this model but i got

SYNTAX ERROR, MESSAGE(S) FROM PARSER:
No matches for: vector ./ real[ ]

at real<lower=0> alpha[2]= mu .* mu ./ sigma;

#6

Can’t mix arrays and vectors. mu is vector because it’s ordered so make sigma a vector too. Also I think mu has to be positive.

positive_ordered[2] mu;
vector<lower = 0>[2] sigma;
1 Like
#7

Change the type of mu to “positive_ordered” and the type of all the other arrays to “vector”. As the error indicates, real[] look like they are not supported in elementwise operations