Hello everyone, I am reading the paper of A Conceptual Introduction to Hamiltonian Monte Carlo, and have questions on typical set. My understanding is that , the whole point of designing HMC is to efficiently explore the typical set in high dimensional space. And in high dimensional space, typica…

You guess could always say that there is only one typical set, but for multimodal distributions it is disconnected. And this is a problem for HMC, as it is difficult to move from the region close to one mode to another.

do we have multiple typical sets? One way to interpret this is in the sense that @jtimonen suggests – is the typical set guaranteed to be connected, or could it consist of multiple disconnected regions. Another (probably far less useful) way to interpret this could be to ask – is the concept of …

I discuss the typical set in far more detail in Section 2 of Probabilistic Computation . As @rfc notes I stick to a qualitative definition of the typical set as the quantitative definition from information theory doesn’t, in my opinion, provide much insight. In particular I present the typical set …

Explore different typical sets in HMC

General

jtimonen December 2, 2021, 10:07am 4

There has been some discussion about the concept of typical set: The typical set and its relevance to Bayesian computation that you might find interesting

Topic		Replies	Views
The typical set and its relevance to Bayesian computation Algorithms	40	5776	April 12, 2021
HMC samples not from the posterior, but from the typical set? General	8	1980	May 7, 2019
Developing a Deeper Intuition for HMC: Connecting the Underlying Concepts with the Hamiltonian Equations Algorithms techniques , mcmc , math	10	625	January 17, 2025
Markov chain and typical set General	7	1321	April 10, 2020
Concentration of measure and typical sets General stan-math	29	4895	January 6, 2018

Explore different typical sets in HMC

Related topics