# What is actually happening when sampling ordered parameters?

**URL:** <https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263>\
**Category:** Modeling\
**Created:** [November 13, 2020, 7:19pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263 "2020-11-13T19:19:17Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![kgoldfeld](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/kgoldfeld/32/6833_2.png) [@kgoldfeld](https://discourse.mc-stan.org/u/kgoldfeld)\
**Post date:** [November 13, 2020, 7:19pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/1 "2020-11-13T19:19:17Z")

</div>

A reviewer has asked about my specification of a set of parameters for ordinal logistic regression. This is basically what I’ve done.

```stan
parameters {
...
ordered[K-1] tau;
...
}

...

model {
...
tau ~ student_t(3, 0, 5);
...
y ~ ordered_logistic(..., tau);
}

```

The reviewer asked if the sampling is using a truncated t-\text{distribution} for \tau in the sampling. I suspect not, but I don’t really know how to explain it. I saw that Bob wrote this on a related [Google groups discussion](https://groups.google.com/g/stan-users/c/aN6eFv7Qmo8):

```
... tau is a parameter declared as ordered[C-1],
Stan will use (C - 1) unconstrained parameters, all but the first
of which are log transformed differences from the previous parameter. 

```

This provides a little clue, but can anyone provide even more information about what is going on so that I can explain it to someone else?

---

<div class="post-metadata">

**Author:** ![mitzimorris](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/mitzimorris/32/29_2.png) [@mitzimorris](https://discourse.mc-stan.org/u/mitzimorris)\
**Post date:** [November 13, 2020, 7:40pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/2 "2020-11-13T19:40:15Z")

</div>

perhaps your reviewer will be happy with this reference?

> **[10.7 Unit Simplex | Stan Reference Manual](https://mc-stan.org/docs/2_25/reference-manual/simplex-transform-section.html)**
>
> Stan reference manual specifying the syntax and semantics of the Stan programming language.

---

<div class="post-metadata">

**Author:** ![kgoldfeld](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/kgoldfeld/32/6833_2.png) [@kgoldfeld](https://discourse.mc-stan.org/u/kgoldfeld)\
**Post date:** [November 13, 2020, 8:42pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/3 "2020-11-13T20:42:42Z")

</div>

Thanks - very helpful. I assume you meant this?

[https://mc-stan.org/docs/2\_25/reference-manual/ordered-vector.html](https://mc-stan.org/docs/2_25/reference-manual/ordered-vector.html)

Clarification just to make sure I’ve got this correct: In terms of the notation in the description where y is unconstrained and x is ordered (constrained), when I specify the distribution as t(3, 0 , 2,5) I am specifying the distribution of x. In the background, stan is drawing from p\_Y(y) based on the transform, and then the inverse transform is applied to generate the desired x. Is that basically correct?

---

<div class="post-metadata">

**Author:** ![bbbales2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bbbales2/32/77_2.png) [@bbbales2](https://discourse.mc-stan.org/u/bbbales2)\
**Post date:** [November 14, 2020, 4:22pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/4 "2020-11-14T16:22:55Z")

</div>

> [@kgoldfeld](#):
>
> In the background, stan is drawing from p\_Y(y) based on the transform

Just to be clear you shouldn’t think about Stan drawing numbers from random number generators anywhere. You write down a log density proportional to the log density you want to sample and it generates draws from that. It is working on an unconstrained space and transforming to a constrained space though.

For the parameters:

```stan
parameters {
  ordered[K-1] tau;
}
model {
  tau ~ student_t(3, 0, 5);
}

```

the way to think about this is first the constraint, and then the distribution.

So `tau` is a random variable constrained to live in `ordered[K - 1]` space and Stan will generate draws via MCMC from:

p(\tau) \propto \text{student\_t\_density}(\tau)

The student\_t distribution itself is defined on unconstrained spaces, so the fact that you’re constraining `tau` makes this a bit weird to talk about (cuz your distribution here is constrained – so it’s not really a student\_t).

The example to think about is if you have a constrained variable and put a normal on it:

```stan
parameters {
  real<lower = 0.0> x;
}
model {
  x ~ normal(1, 1);
}

```

So `x` doesn’t have a normal distribution here – you have a variable `x` constrained to be positive and the actual distribution of `x` in this case would be the density of `normal(1, 1)` above zero re-normalized to integrate to 1. Does that make sense?

In this case you could think about drawing from a `normal(1, 1)` and rejection sampling to get x. That doesn’t work in your model above, because you have more than just the prior (there is the likelihood as well) that will affect what `tau` is, so the rejection sampling picture breaks.

The way Stan generates these draws is MCMC on an unconstrained space – so it isn’t working with `x` or `tau` directly but something that transforms to `x` and `tau` in a way that gives the expected distributions.

It’s a bit confusing to get it all together, and also somewhat unconvincing. You might try to justify your priors by generating prior predictives and talking about why those are sane.

---

<div class="post-metadata">

**Author:** ![kgoldfeld](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/kgoldfeld/32/6833_2.png) [@kgoldfeld](https://discourse.mc-stan.org/u/kgoldfeld)\
**Post date:** [November 14, 2020, 4:58pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/5 "2020-11-14T16:58:30Z")

</div>

Thanks for the details - makes more sense. I wanted to go ahead and generate the prior predictive data, and got an error. This is what I did:

```
prior_pred.stan = 
"data {
  int<lower=1> N;
  int<lower=1> L;
}

generated quantities {
  ordered[L-1] tau;
  
  for (l in 1:L)
    tau[l] = student_t_rng(3, 0, 5);
}"

fit = stan(model_code=prior_pred.stan, data=list(N=10, L= 5), iter=1000)

```

Here is the error message:

```
SAMPLING FOR MODEL 'e8e73083f939a64f9b16ea06904e6d35' NOW (CHAIN 1).
[1] "Error in sampler$call_sampler(args_list[[i]]) : "                      
[2] " Must use algorithm=\"Fixed_param\" for model that has no parameters."
error occurred during calling the sampler; sampling not done

```

Also, the example in the rstan documentation does not seem complete - that also generated an error:

```
pois.stan <-
"data {
  int<lower = 0> N;
  vector[N] x;
}
generated quantities {
  real alpha = normal_rng(0, 1);
  real beta = normal_rng(0, 1);
  real y_sim[n] = poisson_log_rng(alpha + beta * x);
}"

fit = stan(model_code=pois.stan, data=list(N=5, x= rnorm(5)), iter=1000)

```

```
Variable "n" does not exist.
 error in 'model83275db511af_844481826f2cec1338df952452505ed5' at line 8, column 14
  -------------------------------------------------
     6: real alpha = normal_rng(0, 1);
     7: real beta = normal_rng(0, 1);
     8: real y_sim[n] = poisson_log_rng(alpha + beta * x);
                     ^
     9: }
  -------------------------------------------------

Error in stanc(file = file, model_code = model_code, model_name = model_name, : 
  failed to parse Stan model '844481826f2cec1338df952452505ed5' due to the above error.

```

---

<div class="post-metadata">

**Author:** ![mitzimorris](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/mitzimorris/32/29_2.png) [@mitzimorris](https://discourse.mc-stan.org/u/mitzimorris)\
**Post date:** [November 14, 2020, 5:17pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/6 "2020-11-14T17:17:53Z")

</div>

whenever you run the sampler on a Stan program that doesn’t have a model block, then you need to specify `algorithm=fixed_param` -

cf: [9 MCMC Sampling using Hamiltonian Monte Carlo | CmdStan User’s Guide](https://mc-stan.org/docs/2_25/cmdstan-guide/mcmc-config.html#algorithm)

> The fixed parameter sampler generates a new sample without changing the current state of the Markov chain. This can be used to write models which generate pseudo-data via calls to RNG functions in the transformed data and generated quantities blocks.

the syntax error is just capitalization - `n` should be `N`, no? (update - doh! rstan doc error - nevermind ) (also, to get latest Stan goodness, switch to cmdstanr)

---

<div class="post-metadata">

**Author:** ![kgoldfeld](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/kgoldfeld/32/6833_2.png) [@kgoldfeld](https://discourse.mc-stan.org/u/kgoldfeld)\
**Post date:** [November 14, 2020, 6:18pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/7 "2020-11-14T18:18:40Z")

</div>

So, I am still missing something - the following code is generating an error message indicating that I am violating the ordered nature of the vector. This makes sense, since there is no reason to believe that the random number generator is aware of this constraint. Do I need to implement the inverse transformation manually to mimic what is going on in the sampling process? If so, that would make sense.

```
library(rstan)
prior_pred.stan = 
  "data {
  int<lower=1> L;
}

generated quantities {
  ordered[L] tau;

  for (i in 1:L) {
    tau[i] = student_t_rng(3, 0, 5);
  }
}"

fit <- stan(model_code=prior_pred.stan, data=list(L= 5), iter=1000, algorithm = "Fixed_param")

```

```
Chain 1: Iteration: 1000 / 1000 [100%] (Sampling)
Chain 1: Exception: model83274a127ad9_b379d9fdb6e90cf4eae78caf47434293_namespace::write_array: tau is not a valid ordered vector. The element at 2 is -12.427, but should be greater than the previous element, 0.162201 (in 'model83274a127ad9_b379d9fdb6e90cf4eae78caf47434293' at line 6)

```

---

<div class="post-metadata">

**Author:** ![mitzimorris](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/mitzimorris/32/29_2.png) [@mitzimorris](https://discourse.mc-stan.org/u/mitzimorris)\
**Post date:** [November 14, 2020, 10:25pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/8 "2020-11-14T22:25:02Z")

</div>

Betancourt has a nice tutorial on ordered logistic regression in Stan: [https://betanalpha.github.io/assets/case\_studies/ordinal\_regression.html](https://betanalpha.github.io/assets/case_studies/ordinal_regression.html)

---

<div class="post-metadata">

**Author:** ![bbbales2](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bbbales2/32/77_2.png) [@bbbales2](https://discourse.mc-stan.org/u/bbbales2)\
**Post date:** [November 16, 2020, 12:49pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/9 "2020-11-16T12:49:50Z")

</div>

> [@kgoldfeld](#):
>
> the following code is generating an error message indicating that I am violating the ordered nature of the vector

With constrained random variables like this, use MCMC to generate random draws (even ones from the prior).

```stan
parameters {
  ordered[L-1] tau;
}

model {
  tau ~ student_t(3, 0, 5);
}

```

Cause you’re right otherwise we’d have to do a bunch of math to be able to sample from this strange new distribution.

---

<div class="post-metadata">

**Author:** ![kgoldfeld](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/kgoldfeld/32/6833_2.png) [@kgoldfeld](https://discourse.mc-stan.org/u/kgoldfeld)\
**Post date:** [November 16, 2020, 2:10pm UTC](https://discourse.mc-stan.org/t/what-is-actually-happening-when-sampling-ordered-parameters/19263/10 "2020-11-16T14:10:19Z")

</div>

For some reason, I thought I already tried that, but I guess not - it totally did the trick. Thanks so much. And Mitzi, I had read that Betancourt tutorial before, but I was very glad to revisit, because it made much more sense to me after spending all this time on it. So, thanks for the recommendation.