What would be a good resource to understand the method of two sample Bayesian hypothesis testing?

Modeling
1

I’m looking for a resource (blog post, tutorial, paper) that takes the reader step by step through an example of two sample Bayesian hypothesis testing.

I found some but looking for others:

  1. Kruschke, J. K. (2013). Bayesian estimation supersedes the t test. Journal of Experimental Psychology: General, 142(2), 573-603.
  2. Robust bayesian estimation, the Stan version (https://github.com/mvuorre/bestan)
2

You should be aware that most Stan developers have an unfavorable opinion toward what you refer to as “two sample Bayesian hypothesis testing”. That said, the “Tutorial on Bridge Sampling” linked at
https://cran.r-project.org/web/packages/bridgesampling/index.html
along with the bridgesampling R package (which has methods for objects produced by Stan) is probably the best available implementation for producing the posterior probability that an “alternative” model is correct relative to a “null” model that assumes there is literally no difference between in the data-generating processes that yielded the two samples.

1 Like
3

Many thanks, … very new to this Bayesian thing – life was so good as a Frequentist.

4

Luckily for you, life’s even better here in Bayes land if what you care about is building rich scientific and measurement models.

5

just in case you still need it. In the second edition of Doing Bayesian Data Analysis by Kruschke, in Chapter 16 the author updated his original code in JAGS from 2013 into Stan. It is quite easy to understand (a little bit more simplified as in his original paper). In the following link (at the very bottom of the page) you can download a zip file containing all the programs used in the new edition of the book.

https://sites.google.com/site/doingbayesiandataanalysis/software-installation

If you don’t have access to the new book, the procedure is very straight forward:

Copy the following files into any folder in your PC:

Stan-Ymet-Xnom2grp-MrobustHet.R (file containing stan model basically and translation of stan fits into coda readable outputs)
Stan-Ymet-Xnom2grp-MrobustHet-Example.R (file for running the given example)
TwoGroupIQ.csv (data)
DBDA2E-utilities.R (source file for generating plots as shown in the paper from 2013)

Then, set your working directory as the one where you just copied all the files above, run the program and have fun (you may lack some packages for running the code at once, so be aware of warning messages when running the first times)

P.S. The code is plenty of notes that simplify the understanding of what is each part doing

6

Check out this vid and the one that follows it for a walk through. The two-samples case starts around minute 52 in the first vid.

1 Like