# Multi-level ordered logistic regression: where to put the varying part of the model?

**URL:** <https://discourse.mc-stan.org/t/multi-level-ordered-logistic-regression-where-to-put-the-varying-part-of-the-model/996>\
**Category:** Modeling\
**Tags:** techniques, specification\
**Created:** [June 21, 2017, 10:54pm UTC](https://discourse.mc-stan.org/t/multi-level-ordered-logistic-regression-where-to-put-the-varying-part-of-the-model/996 "2017-06-21T22:54:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![tyatabe](https://avatars.discourse-cdn.com/v4/letter/t/3e96dc/32.png) [@tyatabe](https://discourse.mc-stan.org/u/tyatabe)\
**Post date:** [June 21, 2017, 10:54pm UTC](https://discourse.mc-stan.org/t/multi-level-ordered-logistic-regression-where-to-put-the-varying-part-of-the-model/996/1 "2017-06-21T22:54:02Z")

</div>

Hello there,

Firstly, congrats on the new forum. Looks very streamlined and stylish.  
Now my query:

I’m fitting a multi-level ordered logistic regression model, which seems to converge well. But when I started checking on the old forum, and noticed that the varying intercept part is not on the lineal model part, but rather on the cutpoints part (hope that’s understandable). The model I fitted though has the varying intercepts on the lineal model part, something like this:

![](https://canada1.discourse-cdn.com/flex030/uploads/mc_stan/original/1X/d46f4532653af79bc3d760b8f6a178dfcd7a9814.png)

where the last bit, b1i, is the varying incercept. This kind parameterization (and picture) I took from a frequentist book though (Applied Longitudinal Analysis, by Fitzmaurice). Is this an OK way to add varying intercepts to a ordered logistic regression model?  
Please see below the full code for my model:

data{  
int\<lower=1\> N;  
int\<lower=1\> N\_sample;  
int\<lower=1\> N\_cage;  
int histo[N];  
real AGD[N];  
real Des[N];  
real cyst[N];  
real PRV[N];  
real pox[N];  
real Ten[N];  
int sample[N];  
int cage[N];  
real Temp[N];  
real time\_w[N];  
matrix[N\_sample,N\_sample] Dmat;  
}  
parameters{  
ordered[3] cutpoints;  
real bt;  
real bwater\_t;  
real btemp\_timew;  
real bagd;  
real bdes;  
real bcyst;  
real bprv;  
real bpox;  
real bten;  
real bagd\_des;  
real bagd\_cyst;  
real bagd\_prv;  
real bagd\_pox;  
real bagd\_ten;  
vector[N\_sample] a\_sample;  
vector[N\_cage] a\_cage\_raw;  
real\<lower=0\> sigma\_cage;  
real\<lower=0\> etasq;  
real\<lower=0\> rhosq;  
real\<lower=0\> sigma;  
}  
transformed parameters{  
vector[N\_cage] a\_cage;  
a\_cage = a\_cage\_raw\*sigma\_cage;  
}

model{  
matrix[N\_sample,N\_sample] SIGMA\_Dmat;  
vector[N] phi;  
sigma ~ cauchy( 0 , 1 );  
sigma\_cage ~ cauchy( 0 , 1 );  
rhosq ~ cauchy( 0 , 1 );  
etasq ~ cauchy( 0 , 1 );  
for ( i in 1:(N\_sample-1) )  
for ( j in (i+1):N\_sample ) {  
SIGMA\_Dmat[i,j] = etasq_exp(-rhosq_pow(Dmat[i,j],2));  
SIGMA\_Dmat[j,i] = SIGMA\_Dmat[i,j];  
}  
for ( k in 1:N\_sample )  
SIGMA\_Dmat[k,k] = etasq + sigma;  
a\_sample ~ multi\_normal( rep\_vector(0,N\_sample) , SIGMA\_Dmat );  
a\_cage\_raw ~ normal(0, 1);  
cutpoints ~ normal( 0 , 5 );  
bagd\_ten ~ normal( 0 , 1 );  
bagd\_pox ~ normal( 0 , 1 );  
bagd\_prv ~ normal( 0 , 1 );  
bagd\_cyst ~ normal( 0 , 1 );  
bagd\_des ~ normal( 0 , 1 );  
bten ~ normal( 0 , 1 );  
bpox ~ normal( 0 , 1 );  
bprv ~ normal( 0 , 1 );  
bcyst ~ normal( 0 , 1 );  
bdes ~ normal( 0 , 1 );  
bagd ~ normal( 0 , 1 );  
btemp\_timew ~ normal( 0 , 1 );  
bwater\_t ~ normal( 0 , 1 );  
bt ~ normal( 0 , 1 );  
for ( i in 1:N ) {  
phi[i] = bt \* Temp[i] + bwater\_t \* time\_w[i] + btemp\_timew \* Temp[i] \* time\_w[i] + bagd \* AGD[i]

- bdes \* Des[i] + bcyst \* cyst[i] + bprv \* PRV[i] + bpox \* pox[i] + bten \* Ten[i]
- bagd\_des \* AGD[i] \* Des[i] + bagd\_cyst \* AGD[i] \* cyst[i]
- bagd\_prv \* AGD[i] \* PRV[i] + bagd\_pox \* AGD[i] \* pox[i] + bagd\_ten \* AGD[i] \* Ten[i]
- a\_sample[sample[i]] + a\_cage[cage[i]];  
}  
for ( i in 1:N )  
histo[i] ~ ordered\_logistic( phi[i] , cutpoints );  
}  
generated quantities{  
matrix[N\_sample,N\_sample] SIGMA\_Dmat;  
vector[N] phi;  
real log\_lik[N];

for ( i in 1:(N\_sample-1) )  
for ( j in (i+1):N\_sample ) {  
SIGMA\_Dmat[i,j] = etasq_exp(-rhosq_pow(Dmat[i,j],2));  
SIGMA\_Dmat[j,i] = SIGMA\_Dmat[i,j];  
}  
for ( k in 1:N\_sample )  
SIGMA\_Dmat[k,k] = etasq + sigma;

for ( i in 1:N ) {  
phi[i] = bt \* Temp[i] + bwater\_t \* time\_w[i] + btemp\_timew \* Temp[i] \* time\_w[i] + bagd \* AGD[i]

- bdes \* Des[i] + bcyst \* cyst[i] + bprv \* PRV[i] + bpox \* pox[i] + bten \* Ten[i]
- bagd\_des \* AGD[i] \* Des[i] + bagd\_cyst \* AGD[i] \* cyst[i]
- bagd\_prv \* AGD[i] \* PRV[i] + bagd\_pox \* AGD[i] \* pox[i] + bagd\_ten \* AGD[i] \* Ten[i]
- a\_sample[sample[i]] + a\_cage[cage[i]];  
}

for ( i in 1:N )  
log\_lik[i] = ordered\_logistic\_lpmf( histo[i] | phi[i] , cutpoints );  
}  
"

Thanks,

Tada

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [July 13, 2017, 7:46pm UTC](https://discourse.mc-stan.org/t/multi-level-ordered-logistic-regression-where-to-put-the-varying-part-of-the-model/996/2 "2017-07-13T19:46:04Z")

</div>

Not our design—just took it from Discourse. But I keep losing track of posts like this one, so for me, it’s been a big step backwards. I do like the ability to drop in markdown—it’d be even better if it gave us MathJax.

The main thing you have to worry about is identifiability. If you have too many intercepts floating around, you get additive non-identifiabilities where adding a number to all the cut points and to the intercept leaves the likelihood unchanged.

You can vectorize that calculation of `phi` now and it should be more efficient. It just requires all the continuous containers to be vectors.

Is this right?

```
for ( k in 1:N_sample )
SIGMA_Dmat[k,k] = etasq + sigma;
a_sample ~ multi_normal( rep_vector(0,N_sample) , SIGMA_Dmat );

```

It looks like you set a different element each run through. If that’s the case, there’s not much you can do efficiency-wise (unless you could figure out how this would factor down to Cholesky factors).

---

<div class="post-metadata">

**Author:** ![tyatabe](https://avatars.discourse-cdn.com/v4/letter/t/3e96dc/32.png) [@tyatabe](https://discourse.mc-stan.org/u/tyatabe)\
**Post date:** [July 14, 2017, 6:08am UTC](https://discourse.mc-stan.org/t/multi-level-ordered-logistic-regression-where-to-put-the-varying-part-of-the-model/996/3 "2017-07-14T06:08:56Z")

</div>

Thanks, Bob. I’m glad to know it’s a valid parameterization for the varying intercepts part.  
I think I had too many them in the model though, which gave poor model convergence. After taking the highest level out, the model ran well. Even with the Gaussian process not being so efficient.

Thanks for getting back to me on this.

Cheers,

Tada