I came across a paper describing “Bean Machine” a new thing from FB built on PyTorch for declarative probabilistic programming,
It seems a major innovation is explicit use of model dependencies to do various things that aren’t otherwise possible e.g. block updates which are better for correlated variables (?). What caught my eye was the custom proposers which they suggest are beneficial for difficult multimodal problems.
Oddly enough they mention using PyTorch as beneficial compared to Stan, but their benchmarks show Stan has higher neff/s for the tested models.
Oddly enough they mention using PyTorch as beneficial compared to Stan, but their benchmarks show Stan has higher neff/s for the tested models.
It is odd, because the differences are pretty extreme in some cases. They pointed to Figure 9 / Table 3 as showing the Bean Machine performing better in runtime:
Which gives neff/s of:
Bean Machine: 164/4293 = 0.038
Stan: 1486/13903 = 0.107
Which has Stan has almost 3x more ‘effective’!
I also couldn’t find the Stan syntax that they used, so we don’t know whether they used the basic bernoulli(logit(mu)) or the bernoulli_logit_glm, which could have resulted in a bigger performance difference.
But I suppose pure performance wasn’t really the focus of the paper, so I’m probably getting caught up on the wrong thing here.
It seems a major innovation is explicit use of model dependencies to do various things that aren’t otherwise possible e.g. block updates which are better for correlated variables (?). What caught my eye was the custom proposers which they suggest are beneficial for difficult multimodal problems.
For what it’s worth, the Gen system that this paper uses as a baseline does in fact explicitly capture dependencies between random variables models (the experiment in Figure 2 does not utilize the features of Gen that enable this), and also allows for a number of highly customizable inference techniques including custom block proposals and arbitrary reversible jump moves.
Thanks for the link to Gen. it looks like an interesting system to work with. Stepping back though I would be hard pressed to know what to try first in Julia. There’s Mamba, Turing, DynamicHMC and Gen. I’d bet there’re even others upcoming or lurking.
