# Reduce\_sum cores, chains, threads

**URL:** <https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114>\
**Category:** Interfaces\
**Tags:** cmdstanr\
**Created:** [May 13, 2020, 11:56pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114 "2020-05-13T23:56:04Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![shira](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/shira/32/14161_2.png) [@shira](https://discourse.mc-stan.org/u/shira)\
**Post date:** [May 13, 2020, 11:56pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/1 "2020-05-13T23:56:04Z")

</div>

Following [Reduce Sum: A Minimal Example](https://mc-stan.org/users/documentation/case-studies/reduce_sum_tutorial.html) and wondering how to use a cluster of cores for speed.

1. `num_chains` \* `num_threads` = # physical cores ?

2. `num_cores` = `num_chains` ?  
or  
`num_cores` = `num_chains` \* `num_threads` ?

To parallelize Stan over different chains (not within chains), I think [this discourse post](https://discourse.mc-stan.org/t/buying-a-new-computer-best-hardware-for-fast-stan-performance/3349/18) recommended:

1. `num_chains` = # physical cores ?
2. `num_cores` = `num_chains` ?

thank you !!

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [May 14, 2020, 11:54am UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/2 "2020-05-14T11:54:01Z")

</div>

I would go with the first option 1, but if you find out something different …then let us know.

Edit: To only consider physical cores works best.

---

<div class="post-metadata">

**Author:** ![shira](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/shira/32/14161_2.png) [@shira](https://discourse.mc-stan.org/u/shira)\
**Post date:** [May 14, 2020, 2:14pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/3 "2020-05-14T14:14:40Z")

</div>

Thank you !

What do you mean by the first option ? To be more clear, I have two separate questions:

1. Do I choose `num_chains` (set in `sample()`) and `num_threads` (set in `set_num_threads()`) such that `num_chains` \* `num_threads` = # physical cores ?

2. Do I choose `num_cores` (set in `sample()`)  
(a) such that `num_cores` = `num_chains` ?  
or  
(b) such that `num_cores` = `num_chains` \* `num_threads` ?

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [May 14, 2020, 8:04pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/4 "2020-05-14T20:04:34Z")

</div>

I meant 1, but feel free to explore other options, but this is what works knowingly well. In case you gain valuable experience, please share.

---

<div class="post-metadata">

**Author:** ![shira](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/shira/32/14161_2.png) [@shira](https://discourse.mc-stan.org/u/shira)\
**Post date:** [May 14, 2020, 8:41pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/5 "2020-05-14T20:41:22Z")

</div>

thanks ! so you meant to say “yes” to question 1, i.e. we should take `num_chains` \* `num_threads` = # physical cores.

Do you happen to know the answer to question 2, i.e. what should `num_cores` be set to ? whether we want (a) or (b) ? thanks again !

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [May 14, 2020, 8:52pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/6 "2020-05-14T20:52:38Z")

</div>

> [@shira](#):
>
> so you meant to say “yes” to question 1

Correct

> [@shira](#):
>
> what should `num_cores` be set to ?

Equal to the number of physical cores.

At least that’s my experience. Stan needs a lot of cpu cache to work well such that hyper threading does not help.

---

<div class="post-metadata">

**Author:** ![Joran](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/joran/32/9806_2.png) [@Joran](https://discourse.mc-stan.org/u/Joran)\
**Post date:** [May 16, 2020, 10:02am UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/7 "2020-05-16T10:02:50Z")

</div>

Hi @wds15,

I’m a little confused, so sorry if this is a stupid question ;).  
In “Reduce Sum: A Minimal Example” it also says to set the number of `chains*threads` (4 \* 2 in the example) equal to the number of physical cores ( 8 in the example), but the number of cores is left equal to the number of chains (4 in the example). Should the number of cores have been set to 8? Or is there something special going on on that example?

Best,  
Joran

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [May 16, 2020, 10:58am UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/8 "2020-05-16T10:58:42Z")

</div>

It would not matter. Cores sets the number of parallel chains running. So how many concurrent chains. Threads controls within chain cpu use.

---

<div class="post-metadata">

**Author:** ![shira](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/shira/32/14161_2.png) [@shira](https://discourse.mc-stan.org/u/shira)\
**Post date:** [May 16, 2020, 3:04pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/9 "2020-05-16T15:04:27Z")

</div>

I get no speed-up at all, following [Reduce Sum: A Minimal Example](https://mc-stan.org/users/documentation/case-studies/reduce_sum_tutorial.html). I tried my local machine and a cluster of cores, different values for chains, cores, threads, but I rarely get any speed-up at all, and never anywhere close to the 2.7 speed-up in the case study.

Attempting in regular CmdStan, to see if any speed-up is possible there, see [Cmdstanr reduce sum case study, but: unused argument (threads = TRUE)](https://discourse.mc-stan.org/t/cmdstanr-reduce-sum-case-study-but-unused-argument-threads-true/15105/5).

---

<div class="post-metadata">

**Author:** ![Joran](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/joran/32/9806_2.png) [@Joran](https://discourse.mc-stan.org/u/Joran)\
**Post date:** [May 16, 2020, 3:52pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/10 "2020-05-16T15:52:39Z")

</div>

Got it! :). Thanks again @wds15

---

<div class="post-metadata">

**Author:** ![shira](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/shira/32/14161_2.png) [@shira](https://discourse.mc-stan.org/u/shira)\
**Post date:** [May 20, 2020, 6:49pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/11 "2020-05-20T18:49:00Z")

</div>

are others able to replicate the case study ? thank you !

---

<div class="post-metadata">

**Author:** ![djgustafson](https://avatars.discourse-cdn.com/v4/letter/d/3da27b/32.png) [@djgustafson](https://discourse.mc-stan.org/u/djgustafson)\
**Post date:** [May 21, 2020, 4:25pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/12 "2020-05-21T16:25:29Z")

</div>

I’m able to replicate the case study, but my local machine only has 4 cores, so I updated to `cores=1`, `chains=1`, and `set_num_threads(4)`. I had a speedup of ~2.5 with that setup.

---

<div class="post-metadata">

**Author:** ![shira](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/shira/32/14161_2.png) [@shira](https://discourse.mc-stan.org/u/shira)\
**Post date:** [May 27, 2020, 8:59pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/13 "2020-05-27T20:59:07Z")

</div>

thanks @djgustafson ! unfortunately this didn’t work for me either, still unable to replicate the speed-up in the case study (or any speed-up at all !).

---

<div class="post-metadata">

**Author:** ![Michael\_Peck](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/michael_peck/32/5450_2.png) [@Michael\_Peck](https://discourse.mc-stan.org/u/Michael_Peck)\
**Post date:** [May 28, 2020, 2:22pm UTC](https://discourse.mc-stan.org/t/reduce-sum-cores-chains-threads/15114/14 "2020-05-28T14:22:02Z")

</div>

I was able to replicate the case study on a home assembled PC built around an Intel i9-7960X cpu (16 cores, 32 threads). I followed the case study exactly except I set the number of threads to 4 per chain. The timings were - unthreaded base model:

```R
 fit0$time()
$total
[1] 291.7603

$chains
  chain_id warmup sampling total
1 1 143.0066 147.9823 291.7224
2 2 139.7483 129.8698 270.4000
3 3 138.5047 123.3652 262.6477
4 4 140.6788 127.5929 269.0466

```

with reduce\_sum:

```R
fit1$time()
$total
[1] 81.71897

$chains
  chain_id warmup sampling total
1 1 41.84857 36.96113 79.13284
2 2 39.39713 38.37734 78.19834
3 3 39.60059 41.78128 81.69504
4 4 40.41441 38.59576 79.42406

```

so about a factor 3.6 improvement in total execution time. I haven’t experimented much with different chain/thread combinations except there appears to be no incremental improvement with chains\*threads \> number physical cores.

I’m running Windows 10 with R 4.0.0, Rtools 4.0, cmdstan 2.23.0, cmdstanr 0.0.0.9000.