Getting Stan running in AWS Sagemaker

Interfaces CmdStan
1

Hello all,

I’m quite new to both Sagemaker and Stan, but am attempting make use of the platforms’ processing power to speed up some simulation studies I’m working on. I’ve run into roadblocks just getting the Stan language installed and running, however. I was curious to see what approaches people have used and/or if anyone has advice, I would love to hear it.

Please do not be afraid to explain basic concepts either; assume I know very little about the inner operations of Sagemaker and/or Stan.

2

In case anyone runs into similar issues in the future, here’s the solution I ended up using to get it all working:

  • Start by making sure everything is up to date in your SageMaker instance.
!apt-get update

!apt-get install -y build-essential
  • Install cmdstanpy directly from github.
!pip install -e git+https://github.com/stan-dev/cmdstanpy#egg=cmdstanpy
  • Restart your kernel manually at this point. You can do that by clicking “Kernel → Restart Kernel” in the upper bar.
    restart_kernel
    restart_kernel983×250 34.6 KB
  • Import cmdstanpy, then use it to install cmdstan.
import cmdstanpy
cmdstanpy.install_cmdstan()

IMPORTANT: At least in my experience, the instance needs to be large enough to support the installation or it will fail. Attempting the install on a ml.t3.medium (2 vCPU + 4 GiB) instance failed. However, attempting the install on a ml.m5.large (2vCPU + 8GiB) instance worked.

Finally, the installation will take 10-15 minutes to do its thing. When it completes, it will leave a cmdstanpy folder in your sagemaker workspace. Sometimes, when you relaunch the instance, the install method will recognize this folder and skip the installation process. Sometimes it won’t. I haven’t yet figured out why this only happens some of the time - but hey, at least it works!

1 Like