# :: Parallelization :: Pystan :: Covariance Matrix

**URL:** <https://discourse.mc-stan.org/t/parallelization-pystan-covariance-matrix/26007>\
**Category:** Modeling\
**Created:** [January 18, 2022, 7:09am UTC](https://discourse.mc-stan.org/t/parallelization-pystan-covariance-matrix/26007 "2022-01-18T07:09:58Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![oliverk](https://avatars.discourse-cdn.com/v4/letter/o/b2d939/32.png) [@oliverk](https://discourse.mc-stan.org/u/oliverk)\
**Post date:** [January 18, 2022, 7:09am UTC](https://discourse.mc-stan.org/t/parallelization-pystan-covariance-matrix/26007/1 "2022-01-18T07:09:58Z")

</div>

Dear STAN community,

I need to parallelize a model run. Reading the docs (incl. threads) I learned that (pystan (I use 2.19) and Windows (use Windows 10)) is not the best combination for parallel runs with stan.

Reading the docs I guess I have 2 options:

1. install cmdstan and use sum\_reduce
2. use pystan and use map\_rect

I would highly appreaciate your comments:  
Q1: In case I use sum\_reduce with cmdstan, is the code below likely to be successful?  
Q2: Can you hint me to a source, which describes how covariance matrices can be used with map\_rect. I did not find something that helped me.  
Q3: What is longterm the best horse: cmdstan or pystan (system: windows & python)?

```stan
// in functions:
        real partial_sum1( real[,] W, int start, int end, real[] W_mean, real[,,] L_COVMAT ){
           // used for sum_reduce ( which is not yet ready in pystan )
           // W.. matrix with [N,4]
           // W_mean... vector with [4]
           // L_COVMAT... array with matrices [4,4]
            return multi_normal_cholesky_lpdf( W[start:end,:] | W_mean, L_COVMAT[start:end,:,:] );
        }

// in model:
    target += reduce_sum( partial_sum1, W, grain_size, W_mean, L_COVMAT ); 

    // the above should parallelize this:
    // for (i in 1: data_n) {
    // W[i,:] ~ multi_normal_cholesky_lpdf(W_mean, L_COVMAT[i,:,:] );
    // }

```

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [February 1, 2022, 11:29pm UTC](https://discourse.mc-stan.org/t/parallelization-pystan-covariance-matrix/26007/2 "2022-02-01T23:29:57Z")

</div>

> [@oliverk](#):
>
> Q3: What is longterm the best horse: cmdstan or pystan (system: windows & python)?

If you’re committed to Windows and can’t use the Linux subsystem, then I’d suggest using `cmdstanpy`. It plays more nicely with Windows because it doesn’t need to run Stan’s C++ in the same environment as Python.

> [@oliverk](#):
>
> Q1: In case I use sum\_reduce with cmdstan, is the code below likely to be successful?

It looks OK, but anything that evaluates a bunch of multivariate normals with different covariance matrices is going to be slow.

> [@oliverk](#):
>
> Q2: Can you hint me to a source, which describes how covariance matrices can be used with map\_rect.

It’s just a matter of packing matrices into arrays and then unpacking them. It’s a lot of index fiddling, I’m afraid. But you should probably stick to `sum_reduce` if you can.

---

<div class="post-metadata">

**Author:** ![oliverk](https://avatars.discourse-cdn.com/v4/letter/o/b2d939/32.png) [@oliverk](https://discourse.mc-stan.org/u/oliverk)\
**Post date:** [February 2, 2022, 11:57am UTC](https://discourse.mc-stan.org/t/parallelization-pystan-covariance-matrix/26007/3 "2022-02-02T11:57:41Z")

</div>

Thanks you Bob,  
If I like it or not I am bound to Windows and since I decided to continue with cmdstanpy things changed positively for me.  
Kind regards