Program Transformations for PPL

It seems like there could be something useful for Stan devs in


" This workshop will bring together researchers in the fields of AD, probabilistic programming, programming languages, compilers, and ML , with the goal of understanding the commonalities between disparate approaches and views, and sharing ways to make these techniques broadly available. It would enable ML practitioners to iterate faster on novel models and architectures"

There seems to be only abstracts available, but at least it provides overview of the activity in these interesting topics.

3 Likes

Did anybody attend?

Was there something in particular you thought looked interesting, @avehtari? This seems like a mix of very general theory and specific implementations for Python.

There’s a lot of work going on in stanc3 on program transforms from @matthijs, @seantalts, @rybern, and others like Michael Thomas (don’t know if he’s on Discourse), but I don’t think anyone has the time or inclination to submit NeurIPS abstracts.

There’s been a lot of work on this in the past in both the AD literature and in the PPL literature. Particularly around the Hakaru system from U. Indiana.

I’m not an expert on these, and just assumed that it might be worth someone more knowledgeable of these to check if there happens to be anything new. In some workshops in topics I know better there was something new, and in some workshops they were practically repeating the last year talks.

I wish I’d known this was happening. I’m surprised by the jet-based higher-order AD paper that doesn’t cite @betanalpha.

1 Like

You mean this one https://openreview.net/forum?id=SkxEF3FNPH ?

Yes, exactly. It seems like the same idea as https://arxiv.org/pdf/1812.11592.pdf .

Even the reviewers who accepted it acknowledged it’s a known technique in the autodiff literature. Now I have no idea if they’re all talking about the same thing. When I used to submit a lot of logic/programming language stuff in CS, I was continually dinged for people thinking the results were already known.

The worst case of this for me was an Assoc for Comp Ling submission by me and a grad student. I was on the program committee, so was excluded from commenting on my own paper. It was rejected by the committee who went along with reviewers who said it was proved in my book. But I hadn’t proved the theorem in my book—it was a conjecture I never managed to prove that Gerald Penn finally figured out.