Reduce_sum and finite mixture (log_mix)

General
,
1

Hello, I did reduce_sum before following the wonderful case study tutorial. In that example, the likelihood can be vectorized. Is it still possible to use reduce_sum when the likelihood cannot be vectorized? Specifically, I can use log_mix to run a 2-component finite mixture model. But the likelihood cannot be vectorized (as far as I know). Thanks.

2

Not being able to vectorize does not mean you can’t use reduce sum. Can you write your log likelihood as a big sum? If yes, then things are promising…if not…then maybe it’s the sum over a few big sub sums?

It’s about big and costly sums.

3

Thank you for your reply. The finite mixture log-likelihood for 2 classes is like this:

for (n in 1:N) {
  target += log_mix(lambda,
                    normal_lpdf(y[n] | mu[1], sigma[1]),
                    normal_lpdf(y[n] | mu[2], sigma[2])) 
  };

A for-loop is needed and can’t be vectorized (according to the Stan manual). If there are more than 2 classes, there will be another for-loop instead of log_mix.

So I don’t know how one can express this log-likelihood as a big sum or a few big sub sums.

4

This for-loop is a big sum over N terms. That’s what the += is doing; it’s summing another term into the target for each iteration of the loop.

5

Oh. Thanks for pointing me to the right direction. Would this work?

functions {
  real partial_sum_ll(real[] y_slice,
                        int start, int end,
                        vector mu,
                        vector sigma,
                        real lambda) {
    sum=0;
    for (i in start:end) {
        sum += log_mix(lambda,
                    normal_lpdf(y_slice[i] | mu[1], sigma[1]),
                    normal_lpdf(y_slice[i] | mu[2], sigma[2])) 
  };
      return sum;
}
1 Like
6

Almost. The indexing is wrong. Maybe have a look at the within chain parallelization case study?

1 Like
7

Could you tell me where I was wrong?

I read the tutorial again.
It seems to me I should just loop from start to end.

8

The y slice runs from 1 to end - start + 1.

1 Like
9

Oh, I get it now. Thank you very much!