CmdstanR compilation behavior

General
1

I would like to check my understanding on how CmdStanR works in terms of compilation.

I want to run cmdstan R on a server. I know that if I use

mod <- cmdstan_model("mystanfilename.stan")

to compile the codes, it’ll automatically generate an executable file in the same folder. My question is, in this case, if I then do slurm array runs on the server. Every time it executes the same line of codes, is it true that it no longer needs to recompile the stan codes?

I usually use rstan. With rstan, I need to save the compiled object, then later read it back in order for my slurm array jobs not to recompile again. I’m not sure how to do that with CmdStanr.

I tried to google about this and searched in the forum, but couldn’t find an answer. Let me know if there’s already a post on this that I should read!

I decided to try CmdStanR because, somehow the rstan runs have been unstable. I’m not sure whether it’s because of my server or rstan … for some simple models, it can finish all the runs without problem. But for a bit complicated models, say I’m doing 60 runs of the same model, but with different parameterization, then maybe 10 out of the 60 runs would die out of no reason. I had to keep rerunning those failed jobs until everything is complete, which is pretty annoying…

Many thanks!

2

CmdStanR is much simpler than RStan because it’s just a thin R wrapper around CmdStan as an executable. This should make it more stable on clusters. CmdStan uses make to build and save the model. If the code doesn’t change between calls, make won’t try to recompile it.

Having said that, if you have a cluster with heterogeneous compute nodes and you’re trying to distribute work over multiple nodes, you’ll have to compile it on each node.

P.S. As of today (April 2024), CmdStanPy should be even more stable as it’s had more developer attention. They’re trying to bring CmdStanR up to the feature set of CmdStanPy right now.

1 Like
3

I’m not entirely sure, but I think I’m in the “trying to distribute work over multiple nodes” scenario. In this case, is there a way that I can read back the compilation file and use it for each run, just like for rstan?

Thanks!!

4

I’m not sure. Unless you’re running really short compute per node, compiling once per node shouldn’t be an issue. In some cases, clusters are set up to let you cross-compile for the node architecture and distribute binaries. If the nodes are all the same configuration, you should be able to do this just once. Our executables are pretty big, though, so I’d be careful that the communication cost doesn’t outweigh the recompilation cost.