Optimal num_stan_threads when using multiple chains

[eee]

Thought i was having the same issue. Running Stan in Docker (DigitalOcean) with 6 vCPU (Intel Xeon Gold 6140 @ 2300Hz, 25MB cache). Was hopping to have 16 vCPUs, but even 6 doesnt get more than 66% CPU utilization.

I made sure i had the correct Makevars file:
CXXFLAGS=-O3 -mtune=native -march=native -Wno-unused-variable -Wno-unused-function -flto -ffat-lto-objects -Wno-unused-local-typedefs -Wno-ignored-attributes -Wno-deprecated-declarations in $HOME/.R/Makevars
and
options(mc.cores = parallel::detectCores())
rstan_options(auto_write = TRUE)
Sys.setenv(LOCAL_CPPFLAGS = ‘-march=native’)

I get 4 chains/4 cpu at 100%, but 2 cpu’s totally unused.

This seems to be a limitation of Stan as noticed by @Mike_Terrell :

Plans for parallelization

Hi, I’m a huge fan of Stan, but (what appears to be) a limit to use 4 cores only is pretty restrictive in the world of parallel cloud computing. I saw a few posts discussing increases to this from Feb 2017, but I wondered if there was an update on progress/plans in this area. Also, it seems to me that if the dataset is “too big” then only one core will be utilised. Has anyone else encountered this? I’ve tested and if I reduce the dataset size I get to a point where all cores will be utilised. …

It would save a lot of time if we could split 1 chain over 2 cpu’s (or even 3) to hopefully split the sampling time. But as @Bob_Carpenter points out:

@Bob_Carpenter

The chains can’t be spread over more CPUs because each chain is serial (hence the name).
But as the other responders point out, what can be parallelized is the implementation of the log density and gradient.