Difficulty in converging to gobal minimum region given multiple local minimum regions

I find a convergence problem if there are multiple local minimal likelihood regions and the distance between 2 local minimal regions is much larger than step sizes.

For example, given a single parameter X, in the 1st likelihood region, the local minimum is at the location X_1 = 2, whereas in the 2nd likelihood region, the local minimum is at the location X_2 = 20. A chain starting from locations near X_1 = 2 will converge to 1st likelihood region, whereas another chain starting from locations near X_2 = 20 will converge to 2nd likelihood region. However, only the 1st likelihood region is the global minimum region with reasonable parameter values.

If we use a flat prior between 0 and 100 and random initial values, in my model some chains will converge to the 1st likelihood region, while some chains will converge to the 2nd likelihood region. This makes me confused because the sampler cannot jump between different local minimal regions?

Do you know some methods to solve this problem? Thanks in advance!

1 Like

Most samplers won’t be able to move between the two modes even if you find them. The right answer is to sample each proportional to their probability mass. There are some samplers that can do this with a very small number of modes, but nothing can handle the combinatorially intractable multimodality of something like k-means or latent Dirichlet allocation.

1 Like