How can I enforce ordering on normally distributed parameters

Modeling
1

Hi, I was wondering if anyone could help me with this.
I am wanting to enforce some ordering on a set of normally distributed parameters to avoid some potential multi-modality in the posteriors. I am still learning Stan and (new-ish) to everything so forgive me if this is a silly question:

I have some parameters as follows, (pseudo-code, I have not coded this yet but trying to plan out how I would)

Pos_1 ~ normal (mu1, s1)
Pos_2 ~ normal (mu2, s2)
Pos_3 ~ normal (mu3, s3)
Pos_3 ~ normal (mu4, s4)

where pos_1<pos_2<pos_3<pos_4
So how do I enforce ordering on these parameters? mu1 < mu2 < mu3 < mu4 are the means for the priors on pos_1, …, pos_4, and s1, …, s4 are the prior variances. Can I do this using some sort of truncation? How can I do this within Stan?

2

Stan has ordered parameters.

defined in the Stan Reference Manual here: 5.4 Vector and matrix data types | Stan Reference Manual

examples of use on the Stan User’s Guide here: 1.8 Ordered logistic and probit regression | Stan User’s Guide

3

Hi, thanks for this. To clarify could I do something like:

ordered[4] pos; where this would be a vector of the 4 positions

and then in the model section, something like:

pos[1] ~ normal(mu1, s1)…
pos[2] ~ norma(mu2 , s2)
etc?

Should something like this work, or do I have the completely wrong idea?

4

without knowing more about what you’re trying to model, I can’t say.
perhaps you should try to write up the problem at a high level first.

5

Thanks, I am just trying to account for very small differences in mean positions such that the sampler will not struggle as much to identify the parameters, since they are very similar (<0.5 difference in means). My thought is if I order those close positions then that should alleviate some things. Thank you anyway, I will try to have more of a think about how I can do this.

6

If the ordered type, implemented in Stan, does not work for you for some reason, a reparametrization could help, where you model not the quantities directly, but differences between them:

declare positive differences:
real<lower=0> diff_2;
real<lower=0> diff_3;

sample them:
pos_1 ~ normal(mu1,s2);
diff_2 ~ normal(0,s);
diff_3 ~ normal(0,s);

transform parameters:
pos_2 = pos_1 + diff_2;
pos_3 = pos_2 + diff_3;

2 Likes
7

Hi,

Thank you, I managed to get what I wanted by creating an ordered vector of means, which seems(?) to work well with no issues in convergence like I was having before! Thank you for this other way of doing things, I will also try that out and compare

8

Just a note: when you impose the ordering constraint, the marginal posterior distributions for your Pos_1 through Pos_4 will no longer be the normal distributions that you are assigning to them. It’s not easy to formulate a prior for the joint distribution of a set of ordered parameters. If you need to use a prior that doesn’t naturally respect the ordering constraint, then a close examination of the prior pushforward density may be warranted to ensure that your prior is reasonable.

2 Likes
9

Hi, Thank you for the heads up… Yeah, I realised this after some thought and I think explicitly coming up with a joint prior is going to be quite difficult especially as I wanted to evaluate its density. In this case, I have three options: try to examine the pushforward density as you suggested, use strong priors, or model the differences. I have been using option 3 of strong priors so far, but I was thinking forward to cases where a strong prior can’t be used and what can be done instead, which is why I was thinking of ordering these parameters but I think modelling the differences as was suggested earlier may be the safest approach. Thanks again .