Reference for SBAC validation?

Modeling
1

Hi – I want to code up Cook-Gelman-Rubin-style testing of my model, and I saw a couple of threads on here which refer to some improvement on this called SBAC. This one calls it the “new cook-gelman-rubin”, and this one says that the original paper was somehow flawed. But these seem to be the only two mentions on here, and neither say what SBAC is (or at least what the acronym stands for) nor give any references. Also the manpage for rstanarm::pp_validate only points to the CGR paper (although I didn’t study the implementation thoroughly enough yet to see if/how it deviates from that original paper).

Can anyone point me in the right direction with this? I also tried Googling, but the results are all about some standardized testing consortium in the US educational system :-)

2

Not sure what is the best reference, but SBAC is short for Simulation-Based Algorithmic Calibration.

1 Like
3

Ah I see, thanks for defining this!

4

There will be a paper out in a few days. We’re just finishing it off.

3 Likes
5

That’s great news, thanks for the tip. I will keep an eye out for it.

6

There is a better than average chance that it’ll be up tonight.

7

Presuming Dan ran an appropriate ensemble of simulations to calculate that average chance.

1 Like
8

You know me, everything is Monte Carlo

9

Blog post (with paper link) here: http://andrewgelman.com/2018/04/18/better-check-yo-self-wreck-yo-self/

It’ll be up on the arxiv tonight (for referencing purposes)

5 Likes
10

Here’s the arxiv link: https://arxiv.org/abs/1804.06788

2 Likes