This is a very general post.I have been trying to learn Bayesian Statistics for a while(~1 year on and off). It’s great that there are many good materials both online and offline. For reading and general reference, I have referred both Kruschke and Ben Lambert. My goal is to roughly cover the applied statistics material for the master’s course @Columbia by myself within the next 8 months.
What is a good learning strategy for studying Bayesian? I’m noticing that I don’t remember a lot of things /properties of techniques off the top of my head and don’t really feel like I know the subject even after spending quite some months with it and always have to refer to recall. Is this normal?
McElreath’s book is excellent. His 2019 lecture series is, I think, better because it covers more.
However, some people I know struggle with McElreath’s book because he tries to teach Bayesian methods within a general modelling/scientific context. I enjoy this style, but some want to get a grip of the basics more directly first, and McElreath’s use of metaphors to teach the rationale of (Bayesian) statistical modelling actually departs from their learning. If you’re like this, then I’d recommend John Kruschke’s Doing Bayesian Data Analysis instead, which is how I started.
Which ever text you choose, the best way is to do the exercises. Then do some more. Try to write your own code and don’t depend on the books. Try to write out exercises and examples for things you are interested in. Simulate data and fit simple models in your own time. Otherwise, you’re likely to continue to forget things if you don’t have this hands-on practice.
I did go through Krushke’s and went through McElreath lectures.I have covered few exercises in both of them.Though, I feel like I don’t understand the subject enough to explain it convincingly to anyone or don’t feel confident in my approach while approaching problems @franzsf@mitzimorris
I would suggest getting the actual book of Statistical Rethinking (there is a new edition just out) to compliment the lectures. Reading through the first chapters does give a more detailed understanding of the fundamentals and it is actually quite engaging to read. I think it helps to treat it like an interesting/thought provoking book rather than an assignment to motor through, too. I would also really recommend having your laptop ready and running through the different bits of code he presents so you really see things happening as you’re doing it.
reading two books and doing a few exercises is rarely enough to get a good understanding of the essential underlying problems, tools, and available techniques for any field. statistics requires a certain amount of math and general numerical reasoning. and practice.
is there a particular domain that you’re interested in? do you have a dataset that you’d like to analyze? can you try to abstract the data generating process?
I started the same way. I thought Kruschke’s book is a great way to start, then McElreath’s book.
Recently I read David MacKay’s Information Theory, Inference, and Learning Algorithms. I think its a little more challenging than those but it might be a good next step for you. It has great practice problems and answers which will help get a stronger grasp of Bayesian inference than what you’ve been into so far. These are pencil and paper problems to make these problems really concrete for you. You can follow his own guide for how to use the book to cover Bayesian inference (and skip the other topics if you want). Its free online but I can’t imagine trying to study it that way.
That one has a great bibliography too. A couple papers I found that provide a good philosophical and practical introduction, which also generally follow the philosophy behind Statistical Rethinking, are:
Both are elaborating on the work by Richard Cox and E. T. Jaynes (Bretthorst was Jaynes’ student). I think their work is always worth reading. Cox’s Algebra of Probable Inference has some challenging material (just depends how much time you’re willing to put into it) but its also well written, short, and some key arguments are presented clearly in the first few pages. The first few chapters of Jaynes’ Probability Theory are another great place to look if you want to delve into the foundations (at least of this school of thought).
edit: Also, I think my answer is a little different from what you asked, so more directly, I think what you describe is absolutely the norm. And what I’ve listed here is kind of like my strategy—solve problems by hand, and go for the underlying concepts first rather than (or alongside) the fancy stuff. Bayesian inference is all about building on the basics in my view.
@mitzimorris I don’t have a particular dataset I’m looking at. I did my bachelor’s in Maths and at least the ‘math’ part that I have across in these books are mostly rudimentary. I’m understanding that I’m hugely lacking the practice part. And would you suggest some possible ways to improve this aspect? @cmcd Hi, thank you very much for that suggestion.I came across this few weeks back.I went through the index and thought it was so much of theory and I would lose whatever I knew until now, so held back. I will go through that.Thank you again :)
If your background is in maths then indeed, something like Rethinking might not be quite what you are after, though I think for the practical uses of Bayes it is excellent. The other suggestions above seem great. If you are strong with stats and maths then perhaps the classic Gelman et al. textbook on Bayes would provide some more insight? For me, I got totally lost but I am unfortunately not so strong with maths. It also would depend on what your goals are - e.g., to use it practically? In that case finding some open source data sets (perhaps related to covid like the others suggest if it’s your interest) and then thinking what exactly you want to do with it via Bayes would be good. Once you know what you want to achieve, the path might be clearer for you or for others to point out.
thank you for these links, I will check try them out :)
also, the username is just random, more because it’s popping out in every conversation/ feed than out of interest…
@JimBob thank you for the insights.I think my problem is I don’t know what my goal is…I started bayes because of my two roomates who kept throwing p-values and jazz in the kitchen.So, last year I randomly picked up the Kruskhe’s BDA book to just go through it.
I like the subject , as I mentioned I completed few stuff around and would like to learn.Don’t really have a specific goal.It’s more like a usefull hobby at the moment. So, may be like how @mitzimorris suggested I will take the covid or some other data set and work through…I also don’t know if I’m getting the subject properly at the moment…I haven’t worked through real data.
Yes, I’m still not close to being super proficient with Bayes, but by going through the Kruschke book and even more so Statistical Rethinking I have since been able to publish papers using a Bayesian estimation approach. I did find that actually going through my own study with a purpose was tremendously helpful in forcing me to learn the programming and sort of questions you might ask and really get to grips with it. I think everyone would agree that there is always more to learn, but the best way is probably giving it a proper go on something you find interesting once you’ve got some of the basic building blocks!
I love both Krushke and McElreath, but my students needed something a little less technical. Hence, the book and the package. Check it out and please give me feedback as to whether its helpful. Thx.
