I am currently writing up an experiment where we asked people on a 0-10 scale how much they expected three different beverages, water, decaf, and coffee, to reduce their caffeine withdrawal symptoms. Initially, using Bayesian parameter estimation, I modelled this as a within-subjects regression beverage type as the categorical predictor. I got great estimates that matched the mean differences from the descriptive stats, but the posterior predictive check of this model returned draws from the joint posterior that were outside the actual range of the data.
So to remedy this, and constrain the draws from the posterior to the actual range of the data I modeled the outcome variable as an ordinal variable, using ordered logistic regression (see this post).
I wanted to get estimates of the difference in log odds between the three beverage types. To do this I simply calculated, at each step in the markov chain, the difference in estimated log odds between each of the three pairwise comparisons of the three groups (water vs decaf, water vs coffee, decaf vs coffee). The parameter estimates from this model are expressed in log odds, which I can graph as histogram/density plots to show the mean and 95% Highest Posterior Density Interval for each pairwise comparison. However log-odds are difficult to interpret even at the best of times, let alone in an ordered logistic regression. So what I did was exponentiate each chain of log-odds estimates for each pairwise comparison to get odds ratios. These are the histograms for log-odds of the difference scores and odds ratios for each pairwise comparison
Each row graphs the histogram/density plots for each pairwise comparison as (i) log-odds (left) and (ii) odds ratios (right). Each graphs also has the mean and HPDI of each markov chain for each comparison.
To my eyes the odds ratio graphs look strange, of course this doesn’t mean they’re wrong, just that I am inexperienced. I have two questions
Are the mean and the 95% HPDI the right statistics to use to summarise the distribution of odds ratios and/or log-odds? Mode or mean would make little difference for the log-odds but in the odds ratio graph the mean is considerably to the right of the the peak of the density plot, due to the heavy skew in the data. Perhaps the mode would be better?
Do these graphs look ok? I understand exponentiating log-odds transforms distributions in unintuitive ways, but I don’t have much experience in reporting distributions in these ways and just wondered if the right-hand column graphs look sensible, given the log odds distributions on their left?