Is it possible to speed up rstan in Windows 64bit system

Hi,

Recently I found my Rstan runs slower as my input data size increases, I used ‘cores = 8’ to take full advantage of all the available cores when I call stan() and I am wondering whether it’s all possible to let R use all the memory I have on my laptop (32GB).

I found some command like memory.limit() but not sure whether it will work in this situation. Also I remembered that when I install Rstan for the first time, there was some code (sample installation code on Rstan website) trying to optimize some settings but after that I never used similar code again.

Thanks!

Run things under wsl.

2 Likes

Thanks so much! May I ask that what is wsl?

Linux in windows

Unfortunately that’s expected behaviour as the sample size increases.

When you call cores = 8, all it does is specify the number of MCMC chains that can be run in parallel. So if you specify cores = 8, but are using 4 chains, then you will only be using 4 cores. If you want to take advantage of extra CPU cores that you have available then you’ll need to update your model to use one of the parallelisation frameworks (reduce_sum or map_rect). Note that if you want to use reduce_sum you’ll need to run your models through cmdstanR, since that functionality is not yet available in RStan.

You don’t need to do anything to manually change how R uses memory, it will automatically use as much memory as it needs.

Run things under wsl.

To expand on Sebastians answer a little, we commonly see models running much faster under Linux than Windows (on the same hardware), there’s more information on that in this thread. However, you don’t need to switch to a new operating system to take advantage of this. With the Windows Subsystem for Linux (WSL) framework, you can operate a Linux environment from within your existing Windows OS. Running Stan under WSL has seen speeds comparable to native Linux.

The most user-friendly way to take advantage of this is through installing RStudio Server on your WSL install, which lets you interact with the WSL R installation through your web browser. The RStudio support pages have a great guide for setting this up which has worked really well for me:

3 Likes