# Threading and mpi and tbb and gpu in cmdstan

**URL:** <https://discourse.mc-stan.org/t/threading-and-mpi-and-tbb-and-gpu-in-cmdstan/11660>\
**Category:** CmdStan\
**Created:** [October 30, 2019, 2:53pm UTC](https://discourse.mc-stan.org/t/threading-and-mpi-and-tbb-and-gpu-in-cmdstan/11660 "2019-10-30T14:53:42Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![linas](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/linas/32/5342_2.png) [@linas](https://discourse.mc-stan.org/u/linas)\
**Post date:** [October 30, 2019, 2:53pm UTC](https://discourse.mc-stan.org/t/threading-and-mpi-and-tbb-and-gpu-in-cmdstan/11660/1 "2019-10-30T14:53:42Z")

</div>

Hi,

In order to parallelize sampling I initially used mpi. As I understand as long as you are not using more than 1 node threading is not less effective. Right?

There is a thread [Benefits of parallelization with a threadpool of the Intel TBB](https://discourse.mc-stan.org/t/benefits-of-parallelization-with-a-threadpool-of-the-intel-tbb/9704/5) where it is shown that TBB is more efficient than MPI. Is there any way to invoke different TBB methods as demonstrated in the thread?

I have compiled cmdstan with the following flags in make/local  
STAN\_OPENCL=true  
OPENCL\_DEVICE\_ID=0  
OPENCL\_PLATFORM\_ID=0

Does it mean that cmdstan for sampling uses GPU exclusively for Cholesky and CPU all other times? Does it use GPU for matrix operations?

Thanks

---

<div class="post-metadata">

**Author:** ![rok\_cesnovar](https://avatars.discourse-cdn.com/v4/letter/r/7bcc69/32.png) [@rok\_cesnovar](https://discourse.mc-stan.org/u/rok_cesnovar)\
**Post date:** [October 30, 2019, 4:25pm UTC](https://discourse.mc-stan.org/t/threading-and-mpi-and-tbb-and-gpu-in-cmdstan/11660/2 "2019-10-30T16:25:32Z")

</div>

Hi Linas,

If you are using `map_rect`, which I presume you are since you are using MPI, the only thing you need to do in order to use TBB for threading is:

- make sure you are using Cmdstan 2.21
- turn on threading by adding `CXXFLAGS += -DSTAN_THREADS` to make/local (see [https://github.com/stan-dev/math/wiki/Threading-Support](https://github.com/stan-dev/math/wiki/Threading-Support) for more)

Regarding OpenCL support:

If you are using Cmdstan 2.21 the OpenCL the following functions will be run on the GPU if the input sizeis large enough:

- cholesky\_decompose
- matrix mutliplication
- mdivide\_left\_tri\_low
- mdivide\_right\_tri\_low
- gp\_cov\_exp\_quad

The plan is to support most Stan functions with GPUs for the 2.22 release but that doesnt help you here.

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [October 30, 2019, 5:02pm UTC](https://discourse.mc-stan.org/t/threading-and-mpi-and-tbb-and-gpu-in-cmdstan/11660/3 "2019-10-30T17:02:12Z")

</div>

> [@linas](#):
>
> There is a thread [Benefits of parallelization with a threadpool of the Intel TBB](https://discourse.mc-stan.org/t/benefits-of-parallelization-with-a-threadpool-of-the-intel-tbb/9704/5) where it is shown that TBB is more efficient than MPI. Is there any way to invoke different TBB methods as demonstrated in the thread?

At the moment the TBB is used to parallelise `map_rect` and I would expect that MPI will give you the same performance. The thread you refer to doesn’t even compare against MPI as I can see. From my experience, the TBB `map_rect` is now just as fast as MPI. So threading was lacking in speed behind MPI, but that slowness of threading is now gone with the use of the TBB.

---

<div class="post-metadata">

**Author:** ![linas](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/linas/32/5342_2.png) [@linas](https://discourse.mc-stan.org/u/linas)\
**Post date:** [October 30, 2019, 7:43pm UTC](https://discourse.mc-stan.org/t/threading-and-mpi-and-tbb-and-gpu-in-cmdstan/11660/4 "2019-10-30T19:43:03Z")

</div>

Thanks for the help. Should I expect a difference between threading and MPI (in the favor of MPI)?

---

<div class="post-metadata">

**Author:** ![wds15](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wds15/32/908_2.png) [@wds15](https://discourse.mc-stan.org/u/wds15)\
**Post date:** [October 30, 2019, 9:24pm UTC](https://discourse.mc-stan.org/t/threading-and-mpi-and-tbb-and-gpu-in-cmdstan/11660/5 "2019-10-30T21:24:33Z")

</div>

No