# Dirichlet regression

**URL:** <https://discourse.mc-stan.org/t/dirichlet-regression/2747>\
**Category:** Modeling\
**Tags:** techniques\
**Created:** [December 13, 2017, 7:49am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747 "2017-12-13T07:49:30Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![stemangiola](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stemangiola/32/260_2.png) [@stemangiola](https://discourse.mc-stan.org/u/stemangiola)\
**Post date:** [December 13, 2017, 7:49am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/1 "2017-12-13T07:49:30Z")

</div>

Hello Community,

I am trying to find a way to implement a Dirichlet regression in stan.

My problem is a multiple version of a beta regression (that didn’t work when the information content of the data is low, being a 2 levels hierarchical model, [Regression of proportions](http://discourse.mc-stan.org/t/regression-of-proportions/2375/13))

I went through some material

- [http://r-statistics.co/Dirichlet-Regression-With-R.html](http://r-statistics.co/Dirichlet-Regression-With-R.html)
- [https://stats.stackexchange.com/questions/263202/what-are-the-differences-between-dirichlet-regression-and-log-ratio-analysis](https://stats.stackexchange.com/questions/263202/what-are-the-differences-between-dirichlet-regression-and-log-ratio-analysis)
- [https://eeecon.uibk.ac.at/psychoco/2011/slides/Maier\_hdt.pdf](https://eeecon.uibk.ac.at/psychoco/2011/slides/Maier_hdt.pdf)

but I could not implement anything concrete yet

Could someone give a dummy possible model as example, or point to some more clear direction?

Thanks a lot.

---

<div class="post-metadata">

**Author:** ![stemangiola](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stemangiola/32/260_2.png) [@stemangiola](https://discourse.mc-stan.org/u/stemangiola)\
**Post date:** [December 13, 2017, 9:05pm UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/2 "2017-12-13T21:05:42Z")

</div>

A resource I am finding helpful: [http://epub.wu.ac.at/4077/1/Report125.pdf](http://epub.wu.ac.at/4077/1/Report125.pdf)

I found this discussion of Bob some time ago that also tries to deal with a hierarchical dirichlet regression.

[https://groups.google.com/forum/#!topic/stan-users/3Gh1qvfoogY](https://groups.google.com/forum/#!topic/stan-users/3Gh1qvfoogY)

In particular deals with the intercept/angular coefficients as such:

```
 parameters {
     vector[C] coef;
     real<lower=0> prior_count;
   }
   model {
     vector<lower=0>[N] alphavec;
     alphavec <- prior_count * softmax(xm * coef);
     prior_count ~ exp(1);
     ...
   } 

```

If I understand well:

- shouldn’t coef be a matrix (number\_of\_covariates / number\_of\_components\_of\_the\_dirichlet)?
- If we softmax, then coef would not need to be a constrained variable to avoid indeterminability?

For example if I have two time points where nothing has changed

- T0: Dirichlet(softmax([2, 2, 2, 2, 2) \* 40)
- T1: Dirichlet(softmax([2 + 1 \* coef[1], 2+ 1 \* coef[2], 2+ 1 \* coef[3], 2+ 1 \* coef[4], 2+ 1 \* coef[5]) \* 40)

Then coef could be any (single) number and will be unconstrained. Is this correct? Would a 0 centered prior be enough?

- Shouldn’t a logit link function be used on xm \* coef , similarly to the beta regression?

Thanks a lot!

---

<div class="post-metadata">

**Author:** ![stemangiola](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stemangiola/32/260_2.png) [@stemangiola](https://discourse.mc-stan.org/u/stemangiola)\
**Post date:** [December 14, 2017, 2:52am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/3 "2017-12-14T02:52:48Z")

</div>

I might be able eventually to answer my own question, hopefully.

But I leave here some working notes for discussion.

I created a generative process to simulate my probabilities from a dirichlet regression. To solve the indeterminability of angular coefficients I constrained them to be a simplex multiplied by a positive real number

```
# DIRICHLET

n_samples = 500

## from http://tr.im/hH5A
logsumexp <- function (x) {
	y = max(x)
	y + log(sum(exp(x - y)))
}
softmax <- function (x) {
	exp(x - logsumexp(x))
}
set.seed(123)

phi <- 40;
x <- runif(n_samples);
props <- matrix(rep(NA, n_samples*2), ncol=2);

X = model.matrix(~cov , data = data.frame(cov=x))

coef_ang = c(0.95, 0.05)
kappa = 5
beta = matrix(c(1, 5, coef_ang*kappa), ncol=2, byrow=T)
	
y_hat = X %*% beta
for (n in 1:n_samples) y_hat[n,] = softmax( y_hat[n,])
for (n in 1:n_samples) props[n,] = rdirichlet(1,y_hat[n,] * phi)

plot(X[,2],props[,1], ylim=c(0,1))
points(X[,2],props[,2], col='red')

```

![image](https://canada1.discourse-cdn.com/flex030/uploads/mc_stan/original/2X/a/acd1640806e02996bfae9b080dd93e279a5146bf.png)

I will try to compile a stan model soon

---

<div class="post-metadata">

**Author:** ![andre.pfeuffer](https://avatars.discourse-cdn.com/v4/letter/a/e480ec/32.png) [@andre.pfeuffer](https://discourse.mc-stan.org/u/andre.pfeuffer)\
**Post date:** [December 15, 2017, 4:05am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/4 "2017-12-15T04:05:11Z")

</div>

You named the tool already, Dirichlet regression. I’m with Bob: I’ve never seen a Dirichlet regression like this, so take the following with a grain of salt. "  
[https://groups.google.com/forum/#!topic/stan-users/3Gh1qvfoogY](https://groups.google.com/forum/#!topic/stan-users/3Gh1qvfoogY)

Did you thought about a Wishart distribution as multivariate extension of a gamma distribution?  
Or if you don’t want to model correlations between the Ys go with gamma distributions and predefine  
the variances the each gamma, similar the alphas prior in the Dirichlet distribution.  
Hint: The Dirichlet distribution can be defined as ratio of gamma distributions. Since you don’t need  
the normalization of the Dirichlet and later scale it, the normalization becomes obsolete, I feel not so  
ok with it.

---

<div class="post-metadata">

**Author:** ![stemangiola](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stemangiola/32/260_2.png) [@stemangiola](https://discourse.mc-stan.org/u/stemangiola)\
**Post date:** [December 17, 2017, 10:27pm UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/5 "2017-12-17T22:27:12Z")

</div>

Thanks Andre,

yes I agree that multiplication by factors before a softmax sounds a bit strange, in fact I am trying to eliminate them when I can. I will give update on the model soon.

About the Gamma alternative, I have to research a bit, as it is not clear to me the conversion between Dirichlet framework and Gamma.

---

<div class="post-metadata">

**Author:** ![andre.pfeuffer](https://avatars.discourse-cdn.com/v4/letter/a/e480ec/32.png) [@andre.pfeuffer](https://discourse.mc-stan.org/u/andre.pfeuffer)\
**Post date:** [December 18, 2017, 1:45am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/6 "2017-12-18T01:45:49Z")

</div>

In section Random number generation its noted how to simulate a Dirichlet distribution  
from gamma distributions.  
[https://en.wikipedia.org/wiki/Dirichlet\_distribution#Gamma\_distribution](https://en.wikipedia.org/wiki/Dirichlet_distribution#Gamma_distribution)

A good practical paper about multinominal and Dirichlet using JAGS can be  
found here:  
[http://www3.jouy.inra.fr/miaj/public/nosdoc/rap2012-5.pdf](http://www3.jouy.inra.fr/miaj/public/nosdoc/rap2012-5.pdf)

In your case you might omit the normalization, thus independent gammas  
with gamma(mu\*sigma, sigma) could be used to fit your data. If you struggle  
with the gamma, read the mean and variance part in Wikipedia. The mean and  
variance just needs this transformation and confusingly the gamma has two  
forms.

---

<div class="post-metadata">

**Author:** ![stemangiola](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stemangiola/32/260_2.png) [@stemangiola](https://discourse.mc-stan.org/u/stemangiola)\
**Post date:** [December 19, 2017, 7:27am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/7 "2017-12-19T07:27:14Z")

</div>

Thanks I will approach it

---

<div class="post-metadata">

**Author:** ![stemangiola](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stemangiola/32/260_2.png) [@stemangiola](https://discourse.mc-stan.org/u/stemangiola)\
**Post date:** [December 19, 2017, 9:45am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/8 "2017-12-19T09:45:19Z")

</div>

In the meanwhile, I have two questions.

1. Why the gamma representation should be better than the “beta regression”

> [@Regression of proportions](https://discourse.mc-stan.org/t/regression-of-proportions/2375/5):
>
> matrix[P,S] mu;  
> mu = inv\_logit( alpha \* x’);  
> to\_vector(y) ~ beta(to\_vector(mu) \* phi, (1 - to\_vector(mu)) \* phi);

1. How come something quite common such as softmax/dirichlet regression has never been encountered/established in stan-like languages?

---

<div class="post-metadata">

**Author:** ![andre.pfeuffer](https://avatars.discourse-cdn.com/v4/letter/a/e480ec/32.png) [@andre.pfeuffer](https://discourse.mc-stan.org/u/andre.pfeuffer)\
**Post date:** [December 19, 2017, 11:52am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/9 "2017-12-19T11:52:01Z")

</div>

> [@stemangiola](#):
>
> Why the gamma representation should be better than the “beta regression”

beta “limits” you to [0,1], its based upon gamma distribution.  
See for understanding the beta distribution:

[http://varianceexplained.org/r/beta\_binomial\_baseball/](http://varianceexplained.org/r/beta_binomial_baseball/)

> How come something quite common such as softmax/dirichlet regression has never been encountered/established in stan-like languages?

softmax is implemented in Stan and used by some models in Stan. I have no time to do a  
research, but I used it myself. Dirichlet regression I have never seen.

BTW, there are interpretations of exponential, gamma, weibull, lognormal distributions and connections  
eg. the exponential and the poisson. Normally we define a problem and then we think about, but you came  
with the tool first and then ask how you can use it, like how I can nail that screw with my hammer.  
Sorry, if I sound judging. Just my 2 cents.

[https://cran.revolutionanalytics.com/web/packages/betareg/vignettes/betareg.pdf](https://cran.revolutionanalytics.com/web/packages/betareg/vignettes/betareg.pdf)

---

<div class="post-metadata">

**Author:** ![stemangiola](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/stemangiola/32/260_2.png) [@stemangiola](https://discourse.mc-stan.org/u/stemangiola)\
**Post date:** [December 22, 2017, 6:34am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/10 "2017-12-22T06:34:57Z")

</div>

Hello,

please have a look to an attempt to approach the problem in form of a R package testable for feedback

> [@R Package dirReg (beta): an attempt to use stan for improving "softmax regression" inference. Please test if you wish for feedback](http://discourse.mc-stan.org/t/r-package-dirreg-beta-an-attempt-to-use-stan-for-improving-softmax-regression-inference-please-test-if-you-wish-for-feedback/2831):
>
> Hello, I was approaching the problem of inferring regression for a simplex. After a week of work I came out with a package that works possibly better than public R pachages such as DirichletReg. Here what it does better (fully explained on Github): [https://github.com/stemangiola/dirReg](https://github.com/stemangiola/dirReg) dirReg This is an R package that infers trend of change in proportions with the condition that they have to sum to 1. The Challenging part is represented by this example: The problem There are 4 species of…

---

<div class="post-metadata">

**Author:** ![increasechief](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/increasechief/32/2039_2.png) [@increasechief](https://discourse.mc-stan.org/u/increasechief)\
**Post date:** [August 21, 2018, 2:50pm UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/11 "2018-08-21T14:50:54Z")

</div>

You may find this paper helpful [Bayesian Regression for a Dirichlet Distributed Response using Stan]:

[https://arxiv.org/abs/1808.06399](https://arxiv.org/abs/1808.06399)

---

<div class="post-metadata">

**Author:** ![Bob\_Carpenter](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bob_carpenter/32/9230_2.png) [@Bob\_Carpenter](https://discourse.mc-stan.org/u/Bob_Carpenter)\
**Post date:** [August 22, 2018, 9:07am UTC](https://discourse.mc-stan.org/t/dirichlet-regression/2747/12 "2018-08-22T09:07:04Z")

</div>

> [@increasechief](#):
>
> You may find this paper helpful

Thanks for sharing. I really appreciate papers that lay out the model they’re using cleanly and then include code.

I don’t think rstanarm supports Dirichlet regression yet, though I know they did a bunch of work trying to get beta regression to work, which must have similar issues (but probably not the same identifiability issue, as there are only two categories).
