# What acceptance probability does `adapt delta` target

I’m trying to better understand what adapt_delta does and in particular, which acceptance probability it targets. Neither NUTS, nor the dynamic HMC implemented in Stan have an accept/reject step.

One could still compute either (i) the probability of accepting the final point of the trajectory in a hypothetical accept/reject step, (ii) the probability of accepting the sampled point in the trajectory, or (iii) the the average probability of accepting any point along a trajectory.

With NUTS and dynamic HMC, we generate a multiplicative trajectory expansion, but my understanding is that all the points generated along the way should appear in the final trajectory.

From the NUTS paper (Hoffman & Gelman 2014),

[The targeted probability] is the average acceptance probability that HMC would give to the position-momentum states explored during the final doubling iteration.

Ok, this makes sense and it seems consistent with what is described in Section 5 of @betanalpha’s conceptual introduction to HMC. But I would love some confirmation that this is indeed what we do for dynamic HMC and what is currently implemented in Stan.

3 Likes

adapt_delta is the target acceptance rate during warmup. It is the target the dual averaging algorithm tries to hit. The difference between historical acceptance rate and this target (similar to loss function. It’s an optimization process after all) is used to adjust step size. See Stan src here. delta_ in the code corresponds to adapt_delta. So yes, it’s in NUTS.

1 Like

Right, but how do you compute the current acceptance rate (options (i) - (iii) in my question)? That’s the question.

it’s iii. See transition and build_tree function in base_nuts. But also see this thread: Improve step size adaptation target · Issue #3105 · stan-dev/stan · GitHub.

3 Likes

Thank you @yizhang for the answer. Lots of interesting discussion there.