RStan startup message - guidelines on "excess RAM"

Interfaces
1

here’s the current RStan startup message:

Loading required package: ggplot2
Loading required package: StanHeaders
rstan (Version 2.16.2, packaged: 2017-07-03 09:24:58 UTC, GitRev: 2e1f913d3ca3)
For execution on a local, multicore CPU with excess RAM we recommend calling
rstan_options(auto_write = TRUE)
options(mc.cores = parallel::detectCores())

2 things:

  1. any guidelines on excess RAM? some rule of thumb for GB or RAM w/r/t complexity of model, number of params?

  2. is auto_write = TRUE useful in a non-multicore setting?

2

Memory requirements per chain:

  • 8 bytes per post-warmup iteration per parameter to save the parameters in memory.

  • roughly 16 bytes plus 8 or 16 bytes per operand for each operation involved in calculating the log density

There may be extra dynamic memory overhead for some operations like ODEs and matrix factoring, but it shouldn’t be a huge factor.

3

auto_write = TRUE writes a serialized version of the compiled model to the disk so that you can avoid recompilation in a subsequent R session. It doesn’t really have anything to do with multicore or serial.

4

my recommendation would be to change the message to something like this:

To save the compiled model and avoid recompilation in a later session, call
rstan_options(auto_write = TRUE)
For execution on a local, multicore CPU with excess RAM we recommend calling
options(mc.cores = parallel::detectCores())
5

The problem with parallel::detectCores() is that it overreports cores on hyperthreaded Intel hardware (that’s why Mitzi’s 2-core i5 was trying to use 4 cores during our class in Halifax).