Hi,
I’ve had BDA3 sitting on my shelf for years now, and have only ever read part of it. I’ve been reading some of Gelman’s recent blogs and see that he still links to some chapters. Perhaps I should read this? I’m sure there are some great lessons for me to learn!
However, I’m aware that 10 years+ is a long time in this field and perhaps some sections have been superseded.
thanks
BDA3 is still relevant. My notes on BDA3 Chapters 1-18 provide additional explanations, mentions of which computational approaches are outdated, and links to complementary material. The basic theory has not changed and will not change, but there are advances in computation, diagnostics, and prior and workflow recommendations.
BDA4 is on our to-do list, but there are two other books in queue before that, so it will take a long time to before we have time to work on that.
Sounds like I have my summer reading sorted then. Your notes are exactly what I was looking for!
I think that was already true with a lot of the sampling and modeling advice in BDA3. In particular, there’s a ton of up front overhead doing things analytically, which isn’t really a thing people do much of any more. Similarly, if you look at how things like hierarchical models are handled in chapter 5, it’s an old-fashioned combination of moment matching, deriving priors through change of variables and not connecting to the Pareto distribution that winds up being used, etc.
Metropolis, Gibbs, and EM are all much less important than they were when Andrew wrote BDA (though you’ll need to learn them to follow all the literature). I’m also not a huge fan of how it presents HMC or variational inference. We don’t get a presentation of NUTS, so it stops short of state-of-the-art MCMC. There’s basically zero MCMC theory along the lines of, say, Roberts and Rosenthal’s survey article, which feels like a missed opportunity to roll into the same style as the rest of the book. The ADVI section doesn’t mention black-box VI or make the connection to minimizing KL from approximation to target (Andrew tells me he doesn’t like that approach, but I don’t know how you can expect someone to understand VI without mentioning it).
For the regression modeling, there are two other books which go into more detail, Andrew’s original regression book with Jennifer Hill, and a revision of the first half of that and doubling of its size with Aki.
I also found BDA3 rough going because it presupposed a lot of math stats which you really don’t need so much of these days, and at the same time it doesn’t clearly separate random variables from bound variables in notation, which makes discussions of things like expectations and cdfs very awkward (I get why Andrew uses that notation, but still contend it’s rough on beginners). I tried to rectify the missing probability theory in my own intro to Stan (much lower level and more hands-on than BDA):
Mainly I filled this out because I work in a math department and my colleagues kept asking for more precision on basic notions from probability theory until I got to this.
I also added a bunch of stuff about understanding concentration of measure and volumes of high probability mass which is also rather lacking in BDA3. I also added a bit more about how I (and Laplace!) think about random variables epistemically and about calibration. If there’s going to be a BDA4, please include more on calibration!
The main thing that’s lacking from BDA3 is an explicit discussion of workflow, which Aki and Andrew are trying to rectify with yet another new book! Until then, you can check out the arXiv paper: