Hi–I’ve been talking with various journalists about Stan and coronavirus, Stan and election polling, etc., and I have conversations that go like this:
Q: What is Stan?
A: It’s a computer program that does Bayesian inference?
Q: ?
A: So, you have a model that you are fitting to data, the model has parameters . . .
Q: What’s a parameter?
Etc.
When I talk about any particular application, it’s all fine, I can talk about public opinion and urns with black and white balls, or I can talk about infection fatality rate etc., or whatever, and that’s all fine. But when I start to speak more generally, it’s too much for non-technical reporters who have not heard of parameters or propagation of uncertainty or any of our other favorite concepts.
Also, then the reporter asked, What is Stan? When was it created? Etc. Again, I did not have a pithy answer.
This gets back to something that’s come up before, that it would be good to have a What is Stan? page with different answers that will be targeted to people with different levels of technical knowledge.
Parameters and model-fitting seem too technical for talking to a reporter. When I explain Stan to someone who is non-technical I would look to connect what makes it exceptional to the specific application. For COVID I’d say something like:
Stan is a tool for transforming a conceptual (or mental) model into a quantitative model. Epidemiologist have models of how disease spreads from person to person but to understand how many lives are at risk or when hospitals will be over-run with patients these models need to produce numbers. Using Stan and a modern statistical workflow you can build a quantitative version of the conceptual model and also quantify for decision-makers how much trust should be placed in the model.
Quantitative models abound, but many tools are too complicated and make researchers get bogged down or overwhelmed by the details of software engineering. Others are too simple and make it difficult to represent the real world accurately enough to be trustworthy. Stan was designed for the Goldilocks zone where a researcher or analyst has access to the math needed to represent mental models but is shielded from the low-level computation needed to make them practical.
Stan is a state-of-the-art Bayesian scientific modeling package for combining what is already known with data while preserving uncertainty about conclusions in mathematically optimal ways. The conclusions then serve as input to the next iteration of the process which include changes to the model, model checking via simulation and evaluation on new data. Rinse, lather, repeat. Very important in all this is that the models are understandable by humans because researchers can watch and learn as the model works.
This is one of those topics that gets kicked around every year and dropped because it is quite hard to do. There is a “what is Stan” youtube video (https://youtu.be/k9sH7x8O0Y8)
Here’s one idea of how to move forward. Someone (Krzysztof?) could write a proposed What is Stan? page and post it in this thread, then we could all make suggestions on how to improve it, then we could send it to the SGB and propose they add it to the webpage.
It won’t be perfect, but I expect we can construct something that’s better than what we currently have.
Krzysztof: Regarding your “Goldilocks zone” idea: that’s good, and also we should make it clear that Stan can be valuable for researchers who don’t have that mathematical background, as they can run Stan programs written by others. Just as millions of people run linear or logistic regression without being able to really work with the math for those models.
My explanation is probably too folksy & wrong in lot of aspects, but I like to use the metaphor of a landscape when I’m explaining Stan/MCMC alogorithms to a friend or a family member. That is, the posterior is a little bit like a landscape of probability, usually with one big hill in the middle & huge flat areas extending in all directions. As Bayesians, we want to be able to survey the shape of the hill & summarize the properties of the shape (e.g. between what lattitudes/longitudes is the bulk of the hill located). Trouble is, figuring out the shape of the hill is really mathematically challenging, outside of a small set of convenient examples. MCMC samplers allow us to map out the shape of the hill by sending out a robot to randomly walk around the hill & report altitude at different coordinates. However, the standard MCMC robot can be slow sometimes, because it tends to forget where it had walked just few moments ago and so it will just blindly meander around. Stan allows us to overcome that hurdle by allowing the robot to memorize the past landscape to some degree, and achieve much faster exploration.
I’ve adapted the example from Ben Lambert’s Student’s Guide to Bayesian statistics, from memory, and I’ve read it when I had zero understanding of Bayes, so maybe something like that might also work for journalist/uninitiated people?
I like that landscape thing. Part of the challenge is to go back and forth between different levels of abstraction:
At the top level is Krzysztof’s formulation in terms of models, where the cool thing about Stan its flexibility in allowing scientists to write models that make sense. This connects to Bayesian inference and probabilistic programming more generally.
At a more granular level there is the idea of a landscape, where the cool thing about Stan is how it’s like a flexible dune buggy navigating this high-dimensional space. This connects to NUTS and autodiff and other high-tech features.
I think a big draw for scientists might also be that Stan allows the users to fit raw, untransformed data through appropriate likelihood (e.g. lognormal, categorical, etc…). Lot of time, scientists have to jump through hoops to get their data to fit the model assumptions (that is, if they don’t say “ANOVA is robust to violation of assumptions” and leave it at that), and that’s often due to the fact that many sensible choices of likelihood for specific data just aren’t available in standard software packages. Additionally, heterogeneity of variance is another common issue, and Stan allows the users to explicitly model it, instead of 1) having to use robust standard errors, 2) ignoring it.
It does seem very important that there’s no one answer to ‘What is Stan’, and that trying to answer it for all people in one go will be confusing. The answer to tempt a current JAGS user is going to be very different to that for a tabloid news journalist. A page with different answers for different people would be an amazing resource. (I’m vaguely inspired by this description of harmony at five different levels: https://www.youtube.com/watch?v=eRkgK4jfi6M)
I will email you, I looked around and reached out to one of the writers involved. The quantum computing one was my favorite. https://www.youtube.com/watch?v=OWJCfOvochA&list=PLibNZv5Zd0dyCoQ6f4pdXUFnpAIlKgm3N&index=6, question is who would be our explainer that can work from an elementary school kid to fellow researcher. I’ll see what Maggie thinks, our youtube presenter.
It’d be great to get something I could give to people. I like the idea of explaining things at multiple levels. The YouTube videos @breckbaldwin links chose child, teen, undergrad, grad student, and professional (?). Do you think the last one was supposed to be professor but they said “professional” instead and just kept going with it? Because professional is not the logical progression from grad student. Also, aren’t a lot of undergrads still teenagers?
Seriously, though, I think our choices cut slightly differently—explain it to a computer scientist, statistician, doctor, reporter, and philanthropist, say. I was just called upon to do this recently. One thing that works for everyone is an overview of who’s using and how big it is.
Any sell sheet is going to want to include a bunch of the material in this, but right from the start, this is pitched at insiders.
I wish that were true! We’re trying to build mathematically consistent tools, but there’s o sense in which NUTS is known to be optimal for sampling or even that sampling is optimal for calculating high-dimensional expectations.
One of the analogies I like to use is that Stan is a bit like Excel. Excel lets businesspeople explore cashflows very efficiently and flexibly pretty much no matter what they need to do and it’s very programmable under the hood. Stan’s like that, only for statisticians to write down statistical models. It’s a crude analogy, but you have to start somewhere.
I also try to stress that Stan opened up a horizon of possibilities in models that can be fit that simply couldn’t be fit before, at least not easily in a way that lets you explore multiple models in a workflow.
I also try to stress how great the larger Stan community of users, teachers and developers is. We couldn’t do this without the constructive energy of the community as a whole. There are huge network effects, as evidenced by the remaining popularity of BUGS/JAGS.