Non-centered multivariate normal - in a mixture?

Modeling
1

When using the non-centered parameterisation for the multivariate normal:

parameters {
  vector[N] beta_raw;
  vector[N] mu;
  cholesky_factor_corr[N] L;
}

transformed parameters {
  vector[N] beta = mu + L * beta_raw;
}

model{
  beta_raw ~ normal(0,1);
  ...
}

Is there an equivalent for where the parameter (beta) comes from a mixture of multivariate normals? For example, this is the centered parameterisation for what I’m referring to:

parameters {
  simplex[K] theta;
  vector[N] beta;
  vector[N] mu[K];
  cholesky_factor_corr[N] L[K];
}

model {
vector[K] log_theta = log(theta);

  for (k in 1:K)
     log_theta[k] += multi_normal_cholesky_lpdf(beta | mu[k], L[k]);
  target += log_sum_exp(log_theta);
}

Thanks!

2

You can do the non-centering thing with any location-scale distribution, so if you parameterize your mixture that way, sure thing.

If you’re actually having to sample something that comes from a mixture though, life is going to be really difficult unless the modes are sitting on top of each other (cause the sampler would need to hop between modes). I doubt the intersection of what-non-centering-fixes and problems-mixtures-create is very big.

But! Either way, you can work out the math. As an example, for a two variable, scalar case, the regular pdf of a mixture of two normals would look like (with p_N being the pdf of a normal, and \phi_0 + \phi_1 = 1):

p(y | \mu_0, \sigma_0, \mu_1, \sigma_1) = \phi_0 p_N(y | \mu_0, \sigma_0) + \phi_1 p_N(y | \mu_1, \sigma_1)

You could instead define the pdf of your mixture like:

p(y | \mu_1, \sigma_1) = \phi_0 p_N(y | 0, 1) + \phi_1 p_N(y | \mu_1, \sigma_1)

And then make a transformed variable:

z = y \sigma_0 + \mu_0

The pdf of z would be:

p(z | \mu_0, \sigma_0, \mu_1, \sigma_1) = \phi_0 p_N(y | \mu_0, \sigma_0) + \phi_1 p_N(y | \sigma_0 \mu_1 + \mu_0, \sigma_0 \sigma_1)

You could do the same thing for the multivariate normal, just the things are matrices n’ such now.

1 Like
3

Thanks for that Ben! I couldn’t get my head around how it would look but seeing the math written out made it ‘click’.

Much appreciated!