@sakrejda asked for a write-up for how we thought about which language to choose. Here are the desiderata for language features when choosing a language to write a compiler in, some of which are specific to the needs of Stan given that we essentially create desktop apps (vs cloud something something).
- Pattern matching - this makes tree transformations first-class in a language, enabling you to write compiler phases much more easily.
- Fun - we want something that will inspire programmers to volunteer and help us build the compiler.
- Amenable to research - it would be great if the language enabled easy collaboration with the broader academic Programming Language Theory community.
- Solid, modern tooling - testing frameworks, package and dependency management, code formatting, build systems… these little things count for a lot.
- Distribution - must be capable of producing statically-linked native binaries containing all of their own dependencies.
- Community - it’d be great if there was a good community we could turn to for help and programmers.
- Production use - we want something that will be battle-tested and maintained by other people because it’s important to their business.
C++ has the last three pretty well, but the first four it mostly fails. OCaml and Rust both do a good job on all of these. Choosing between those two, it’s so far mostly coming down to three things:
- Parsing libraries - Rust doesn’t really have good parsing libraries for real PL development. OCaml has the awesome Menhir parser.
- Community and inertia - Rust and OCaml are both smaller than C++, but Rust has a lot of inertia and OCaml has a long history and solid base with Jane Street and INRIA still actively maintaining it.
- Pattern matching - in Rust, pattern matching is ruined a little bit by their
borrow checker, and generally programming in Rust has a bit of overhead from this.
This decision is not taken lightly. We’ve implemented a simple arithmetic expression interpreter end-to-end here in both OCaml and Rust, and you can (and should if you have time!) compare for yourself the relative complexity. Going on gut and experience here (along with Matthijs enthusiasm for Menhir), it seems like the parsing library and lack of borrow checker are pointing us in favor of OCaml. Not having to write our own recursive descent parser (like we would need to in Rust) will save us a lot of time and let us minimize time spent maintaining two systems.