Summary: We would like to simplify the installation of CmdStan for CmdStanPy. In particular, we would like to use the conda package manager to install CmdStan automatically when CmdStanPy is installed.
Why? Because conda can handle the additional requirements for compilers and other unix tools automatically. This means Windows users for example won’t need to separately install RTools.
Current Status:
We have various attempts which succeed in building cmdstan in the docker test images conda-forge uses. The main problem we’ve encountered is that the pre-compiled stanc3 seems to segfault when called. A combination of infamiliarity with conda recipes and with the specifics of the cmdstan installation make debugging this rather difficult.
@bparbhu I’d love to chat sometime about your similar effort. The most recent road block we’ve encountered seems to be related to some of the c++ libraries stan math uses (see this comment). Someone more experienced with the compilation here would probably have a much easier time making sense of this
Happy to discuss anytime on Wednesday this week. If you want I can set up some facetime for us to talk more about what we’re both doing. The route I’m taking is more of a graphical user install using a combination of NSIS & PowerShell for windows, Bash scripting for Linux, and most likely some other route for mac installs. My email is brian.parbhu@gmail.com, I’ll send out an invite for Wednesday just let me what time works best for you and also be sure to send me your email.
In terms of the tbb library, I think that is in relation to the tbb/intel math library that CmdStan can use as well. Though, I’m not too sure about the specifics of what is going on there. Though, when installing CmdStan, we do make sure to run an install command around that library. So it would be something like make install-tbb or mingw32-make install-tbb on Windows 10.
In the future this should be able to support non-x86 architectures once we have stanc3 builds for them, which I know is being worked on. We should also be able to support MPI builds via conda.
I know nothing about R packaging, but I have learned quite a bit about Conda and conda-forge doing this. If anyone wants to try to add cmdstanr to conda-forge I’m happy to help/be a resource
Cool, yeah we still want to get cmdstanr on CRAN. @rok_cesnovar and I should probably have a chat about which issues we absolutely need to close before then. We have a 1.0 milestone but I’m not sure if that has to be exactly equivalent to the CRAN release milestone.
For it to be available on conda, cmdstan needs to package them in a release on github. I noticed that the linux-arm build already has been, so I can try to learn more about the conda-forge architecture settings using that
Great! Are there plans for similar support for native builds for M1 macs? (Not sure if OCaml even supports this, we’d probably need a newer version than we have pinned)
Native builds for M1 are a bit tricky since there aren’t any M1 Github runners we can use to build the binaries, and there isn’t the incentive to use an EC2 instance (like we previously did for linux-aarch64) since the Stanc3 binaries will run through Rosetta.
One approach that I was thinking of investigating is using x86 OSX runners to build Universal binaries: Apple Developer Documentation, but I have no idea what that would entail for our build process at the moment