Blog post about scaling a language


#1

Came across an interesting blog post by Rust’s initial creator, Graydon Hoare on the limits to growth and how to think about scaling a language and project.

https://graydon2.dreamwidth.org/263429.html

cc @Bob_Carpenter @Matthijs @mitzimorris


#2

Some of my favorite snippets:
His TL;DR is pretty good at explaining what the post is about:

I’m writing to make a recommendation about paying attention to – acknowledging, planning, making explicit mechanisms and policy around the limits of – the growth of two things in the project:

  1. Necessarily-shared technical artifacts, specifically the language definition itself.
  2. The strains on people participating in conversations about those artifacts.

In particular, I want to draw attention (as others have done, with other words) to the fact that unlimited growth in both of these things is neither possible nor desirable, that there are natural limits, and that as with all natural systems reaching their growth limits, it will be better for everyone if the encounter with the limits is made in a controlled, planned fashion.

From “Suggested Controls”

  1. Set a learning-rate and book-length target. Try to work backwards from the amount of time and number of pages it “should take” to learn the language, or become expert in it, then cut things that go beyond that. If “teach yourself Rust in 21 days” isn’t going to work, figure out what should. Three months? Six? A year? Think about languages that “definitely take too long”, and pick a number that’s less than that. Is a thousand page manual healthy? Five hundred? Three hundred?

#3

Thanks for forwarding! It’s a good read.

I also like his idea of planning negatively : deciding what you want the language not to be. I feel like that might help us as we often seem to be having similar discussions about which direction to grow the language in. In my mind, it’s really cool that there are so many ideas on how to extend Stan at the moment. At some point, though, presumably we’ll need to make some decisions as a community on which directions to pursue and which ones not to. I wonder if it’d be helpful to try to write a negative Stan road map.


#4

re: the language definition itself and how it can be designed to evolve over time with a community, there is a classic Guy Steele talk “Growing a Language” with specific tradeoffs and recommendations:

It starts off a little weird, but literally is my favorite technical talk ever, so I promise its worth sticking with it!

There is also a transcript: https://www.cs.virginia.edu/~evans/cs655/readings/steele.pdf

The tl;dr is perhaps:

  • To meet the demands of modern programming tasks, you have to design a “big” language with many complex built-in primitives and a large standard library
  • But you can’t make it too big (as @seantalts your article pokes at C++/Haskell)
  • And you can’t foist it on the world all at once; the world has a limited absorptive capacity
  • So you must explicitly design it to be grown and extended by users in ways that make user-defined extensions look identical to built-in constructs
  • Fast-to-market and working will beat slow-to-market and elegant, so you have to ship with some “warts” but only those you can easily remove later

The talk is from 1998 so it’s instructive to note 20 years later, all of Guy Steele’s then-for-Java recommendations indeed came to pass in either Java or Scala (all thanks to Martin Odersky), including Java Generics, Scala’s enhanced generics, operator overloading in Scala, and other extensions of Guy’s “meta” ask, such as Scala “implicits” that generalize and make user-extensible Haskell’s in-built idea of type classes. And we do now have Guy’s foretold giganto-does-everything-as-primitive language; it’s called Mathematica (and you can see the implications of that choice).

Anyway, even for Stan I have to agree with his premise and think you all have done a wonderful job. Sometimes we curse Stan for being so “large” – this morning I was finally adding support for the “Specialized Products” matrix functions in our Stan Interpreter – but I only had to do that b/c we are using those constructs in real models!!


#5

@ryan-richt Whoa, this is great! Thanks for posting. I’d seen this floating around the internet before but never actually watched or read it. Funnily enough, I did undergrad research with Dr. Evans, who has, as you’ve found, this talk in his readings for cs655 :P

I think there’s a really powerful message in here that I’ve occasionally tried to convince @betanalpha of over the past year or so. I’m not sure if this talk will speak to him and his experiences, but it might be worth a shot… I’m hoping that it proves convincing for the need to develop a language that users can augment themselves in a first-class way. This was, I believe, the primary motivation for @mgorinova’s SlicStan and our continued iterations on what that might mean for Stan. We’ll have to run some of our ideas for this past you (and the rest of the community here on discourse) and see what you all think. I dream of the day when not only can a user import models or math functions from user-developed libraries of them, but that our own Math library can be written mostly or possibly entirely within Stan without losing too much performance. Maybe that last part is a pipe dream but even if we got to 80% I would be happy.