Weighted loglikelihoods in mixture model

Interfaces PyStan
2

Hey the general guidance is not to do this kind of weighting for Bayesian models because the resultant model can’t really be said to be generative. I.e. once you’ve fitted it, it can’t generate any data since that requires weights.

That given, there’s a discussion here. You can do it by:

(A) just multiplying the target by the weight. I.e.

 for (n in 1:N)
       target += log_mix(lambda,
                           normal_lpdf(x[n] | mu[1], sigma[1]),
                           normal_lpdf(x[n] | mu[2], sigma[2]))) * w[n];

(B) or you may be able to model as if your variance is heteroscedastic and shrink the variance for points with bigger weights. I.e.

 for (n in 1:N)
       target += log_mix(lambda,
                           normal_lpdf(x[n] | mu[1], sigma[1]/w[n]),
                           normal_lpdf(x[n] | mu[2], sigma[2]/w[n])));
1 Like