Simple HMM does not converge

I personally don’t use traceplots much for diagnosing issues, but I don’t see anything particularly concerning, especially if the R-hat values look good.

All the chains seem to have found the same mode, and the differences in the exact densities could just be a result of finite samples and kde settings. Try running longer chains and see if the plots look better. The long tail to the left reads, to me, like a reflection of true uncertainty in the parameter.

In short, looks good to me, for what that’s worth. I would try to test the model using simulated data. Does the model recover the true parameters of the data-generating process? Does uncertainty decrease with more and more data? Etc. This can be made more rigorous: Simulation-Based Calibration

As for the prior, I don’t think there’s anything necessarily wrong with your choice of distribution. You could always put the prior directly on theta and simplify things a little. If you can capture your true prior knowledge with a distribution that behaves better numerically, that’s a win in my book.

I can’t answer your question on the initial state. I found this previous post: HMM : how to specify the initial distribution - #2 by martinmodrak and it makes sense to me that the influence of the initial state should decay over time, so you can test your hypothesis by increasing T.