This discussion is motivated by Bayesian Data Analysis, a conversation I had with Trivellore Raghunathan (Raghu), and a thread from April 2013 from that Stan Users Google group on “Reasons to switch from BUGS to Stan”.
From BDA
There are two ways in which ideas of the Gibbs sampler fit into Hamiltonian Monte Carlo.
First, it can make sense to partition variables into blocks, either to simplify computation or to speed convergence. Consider a hierarchical model with J groups, …. In this case, … it may be more effective—in computation speed or convergence—to cycle through J + 1 updating steps, altering each η(j) and then φ during each cycle. Parameter expansion can be used to facilitate quicker mixing through the joint distribution.
BDA then goes on to say that the Gibbs sampler can also be used with HMC for updating of discrete variables.
My conversation with Raghu suggested that Stan actually uses partitioning of variables into blocks for fitting complicated models.
My first question is does this partitioning actually occur in Stan or does Stan actually update the entire parameter vector at once?
My second question is whether what BDA suggests even possible in Stan?
Finally, as BDA suggests, parameter expansion/marginal augmentation has been used in hierarchical models to speed convergence. BDA suggests that Gibbs sampling with parameter expansion works fine for these models. In the on “Reasons to switch from BUGS to Stan” thread, Bob Carpenter stated that a student had been recruited to do comparisons. Are there any results from these comparisons? I have not been able to find them.