Hmm, well the posterior itself is probably the best probabilistic solution under some assumptions, but that’s beyond what I know. Presumably I unwittingly assume this a lot.

I guess “most likely” reminds me of having-an-MLE. That doesn’t have to exist cuz things can go off to infinity or whatnot.

I think the way I colloquially talk about identifiability around the forums is something like: the marginal distribution of a parameter needs to be in a small range.

But I guess this isn’t something we’re optimizing for, or whatever.

Good point, but presumably fitting them a bunch would. Eventually you’d get to:

And yeah I guess nothing is guaranteed here.

Interesting link. I don’t know how to think about it when algorithms sometimes fail/sometimes work (other than being annoyed, at least).

Like this isn’t just an MCMC thing. Happens with anything. I guess short of changing algorithms, you’re changing random initializations and tightening up priors.

This can look like you’re carefully regularizing your problem (and you can find other justifications for this after the fact), or it looks like cheating (whatever that means). Either way you’re just trying to not get tricked by your inferences (or rather, trick yourself into tricking the calculations).

Maybe the best you can do is try to understand how often calculations fail when they are repeated and work from there.

Like, what fraction of chains fail? Does this fraction change with smaller initialization? Is this specific to one set of generated data.

And then there’s Andrew’s folk theorem that if a model fits badly then there’s probably something wrong with the model (rather than the computation).