Markov chain and typical set

Dear Stan community,

I’m trying to understand how probabilistic computatuon works. I have a conceptual idea of what typical set is and why it is importan to explore it. However, need help in trying to connect the typical and transition functions.
How does a markov chain with a given transition function find this narrow typical set and then remains inside it?
I would like to undestand this before getting into Hamiltonian monte carlo.

Take a look here and here and please come back with questions if you have them.

1 Like

Is there any threads showing how to generate correlated samples (Markov transitions) that preserve a target density and see how these samples are atracted to the typical set and not to where the density lies.

I still do not clearly understand why the correlated samples are atracted to the probability MASS and not to the region where there is more density.

That’s a task for @betanalpha and his geometrical arguments.

2 Likes

See introduction to probabilistic comptutation and then introduction to Markov chain Monte Carlo. I have plenty more writing in this direction on my website, https://betanalpha.github.io/writing/.

2 Likes

Is this correct? Due to the nature (they concentrate on the typical set) and interdependency of the samples (MCMC) we can say that we are exploring the typical set of the target probability distribution.

Yes. But you have to be careful what “exploring” means. It only explores enough to compute expectations. It’s not going to explore the whole posterior. This is easy to see in a high dimensional standard normal. Let’s say there are 20 dimensions. Then there are over a million quadrants (choice of signs in each dimension). You’re not going to explore every quadrant in a few thousand or even a few million draws.

3 Likes

Hamiltonian monte carlo comes in because we need a transition mechanism that is better in the sense that it doesn’t get easily stuck in any region or it does longer jumps between samples across the typical set, allowing the exploration of more regions of this typical set?