Semi-markov process

I wonder if there is any example for semi-markov processes in Stan. Semi-markov processes are those in which the transition between states is given by a state-transition matrix and a dwell (waiting) time in the state.

I think a semi-markov process would be a great fit for the analysis I am trying to carry out. I wish to study a sequence of events (categorical values) in terms of their transition probability and the temporal length of each event.

For example, in the figure below, I simulated 150 steps in the semi-markov process. Each step, I sampled the state and the dwell time in that state according to the transition matrix

P=\begin{matrix} &s_0&s_1&s_2\\ s_0&0.0&0.9&0.1\\ s_1&0.9&0.0&0.1\\ s_2&1.0&0.0&0.0\\ \end{matrix}

and the dwell time distributions

t_{s_0} \sim lognormal(1, 0.5)\\ t_{s_1} \sim weibull(1, 1)\\ t_{s_0} \sim lognormal(2, 0.1)

I assume that I know the type of dwell time distribution in each state but I would like to use Stan to infer the parameters of each distribution and the state transition matrix. Is this doable?

This seems like a hidden markov model where the observation is the dwell time. The interface here should be useful:

There’s an example here:

Interesting way of looking at it. Would it be possible to set the hidden state? Is my case, the (not so) hidden state is the value of the categorical variable.

1 Like

Oh if you know the states then HMM is not the thing, cause the whole trick there is getting around not knowing the states.

If you know the states then your observations are transitions, so I guess you are estimating p(A | B), p(C | B) – stuff like that.

The only thing that strikes me here is state transition matrix may be a way to think about this mathematically (like from B, the thing ends up somewhere, so a column or row sums to 1, or something like that), but there’s probably a way to do modeling here.

Like, maybe p(A | B) = 0 cuz you know that, or maybe the transition probabilities have common terms that are a function of other covariates:

logit_pAB = X * beta + something;
logit_pCB = X * beta + something_else;
state_transitions ~ categorical_logit(logit_scale_probabilities);

And I guess you’re saying the transition times are correlated with what states you end up with, so there must be some sort of shared component between the regression of transition probabilities and times

log_tAB = X * beta
dwell_times ~ exponential(exp(log_scale_times));

I’m really not sure how to code up the shared component, but I think that’s what you want to do here.

If you don’t have the correlations in some sort of shared component in the regression, then you’d have to have some sort of parameterized distribution for join (transition, dwell_time) observations. I don’t know what that would be, not that it doesn’t exist.

1 Like

are you suggesting to model the transition matrix (e.g., with a Dirichlet regression) and a mixture of time distributions in which each component is associated with a state? Now that I think about it, if the transition matrix is independent on the dwell times, I can model transitions and dwell times separately right?
Thanks for your input. I realized I need to think a bit longer and harder on the problem.

Hmm, I guess not your current state, but whichever transition you take will be correlated with your next dwell time.

But since you measure your states directly then I guess yeah, these two models come apart.

Yeah yeah, this. As long as you build a valid transition matrix in the end, you can do whatever you want.

Mixture models can be really hard to fit, so if possible something else, but otherwise yeah

1 Like

Thanks, I’ll put something together soon. I am planning to use brms as much as I can. Let’s see how far I get. Thanks!

1 Like