Interactive dashboard visualizing various MCMC sampling algorithms including HMC and NUTS
https://chi-feng.github.io/mcmc-demo/app.html#EfficientNUTS,banana
9 Likes
Neat, but I don’t get what the vectors are, nor what all those algorithms are. naive NUTS vs. efficient NUTS? What’s that mean?
I think the vectors are the initial momenta (although you run into the issue that their magnitude is on a different scale than the positions because they’re not the same spaces!). Also the NUTS animations are misleading because very smooth trajectories are shown instead of the more-obviously discrete trajectories we use in practice.
No idea what half the algorithms referenced are.
I like this dashboard also, but I’m not really clear about what the Gibbs sampler is showing.
Do you have a specific question about what is being shown? The Gibbs sampler conditionally updates one parameter, then the other, and thereby produces a step.
Sorry, I wasn’t clear in my question. I know what Gibbs does, it just wasn’t clear why the visualization app draws a right triangle for each update.
Because it updates one parameter and then the other.
Let me try to be more specific. The visualization starts with drawing a line from a point with out a dark dot to a point with a dark dot. Then from the first dark dot it draws a line to a second dark dot, then draws a green arrow from the point without the dot to the end of the second line. The draws are then from the initial point to the end of the second line. Hard to explain in text, but this what I see when I slow down the app as much as possible. I presume the line starts at say \theta_1^{s+1} \sim p(\theta_1|\theta_2^{s}), where s is the start value, and the second dot is \theta_2^{s+1} \sim p(\theta_2|\theta_1^{s+1}).
Thanks,
David
The line starts at some initial point. Then it updates \theta_1 via
which leads to the intermediate point at the terminus of the first black line. Then it updates \theta_2 via
which leads to the final point. The overall transition is represented by the green line. Thus, the transition (green arrow) can be thought of as the outcome of a path with two steps (black arrows), just as in static HMC the transition can be thought of as the outcome of a path with many steps. Because the parameters are updated one-at-a-time, the path is always a right triangle on a representation of parameter space where the parameters are orthogonal to one another.
1 Like
Got it. Thanks!
David