I’m trying to find the best framework for dealing with the following problem. With ordinal Y there is no link function that works the majority of times even within a specific subject matter field. I often compare goodness of fit for a proportional odds ordinal logistic model with a log-log link ordinal model (proportional hazards) because they are so different. I sense that it’s best to allow “link function uncertainty”, but I don’t seek a final result that mixes the two links but instead one that results in a single effect measure I can communicate to the researcher, e.g., I’d like to get a posterior distribution for a certain odds ratio or hazards ratio, and be able to communicate exactly which one this is.
Is there a way to do this that will reflect in the final posterior distribution the uncertainty in choosing that model? Are there other ways to think about this?
As a side note I don’t typically use the probit ordinal model because of the lack of an easy interpretation for the model parameters.
Compute quantity of interest for various link functions, using the posterior predictive distribution under counterfactuals when it is not computable directly from the posterior distribution
Weight the quantity of interest using stacking weights (M-open) or model weights (M-closed)
Report the weighted distribution of the quantity of interest
The idea of using the posterior predictive distribution to compute things like this has come up before
but it probably has the highest ratio of usefulness to utilization. I guess with an ordinal model you are interested in the ratio of the probability that the outcome is less than or equal to y divided by the probability that the outcome is greater than y?
Sorry for the delayed response. Thanks for the suggestions. There may be some subtle problems for which workarounds may be difficult. Here’s an example. If one wants to obtain an odds ratio and there are no interactions between covariates on the scale of the link function, then the odds ratio is ‘universal’, i.e., it applies to any subject. If a second model is entertained, it is likely to have the odds ratio (for say B:A treatment effect) that is a function of all the covariates in the model. It’s hard to compare a ‘universal’ estimate with one that needs to be covariate-specific.
Since no one took the bait on the last point, perhaps the conclusion is the following. In a discrete proportional hazards model (log-log link ordinal model), only the hazard ratio can be independent of the other covariates, and in a proportional odds model only the odds ratio can be independent of other covariates. That is to say if there are no interactions within the true model, whatever that is, then we have a single parameter to describe the effect of a single predictor, if it acts linearly. If we want a general measure of effect for a single predictor, with allowance for link-uncertainty, then perhaps the quantity of interest needs to always be covariate-specific. If the quantity of interest were to be an odds ratio and the proportional odds model was in effect with high probability, a series of covariate-specific odds ratios would have almost the same posterior mean and credible intervals, so making the estimates covariate-specific would not be inefficient, just a bit confusing for the consumer of the research. Make sense?
When the quantity of interest is a risk difference, that always needs to be covariate-specific, so there’s less of an issue.
Interesting because I find the probit link the easiest to interpret – a unit change in the predictor X_1 shifts the latent variable, which underlies the ordinal responses, by \beta_1 standard deviations.
I think the linearity and additivity of the effects on the normally distributed latent scale is helpful for understanding. When the results are plotted — as below — the sizes of the effects with respect to the ordinal categories are easy to understand.
Yes - I’ve used 3 displays: predicted exceedance probabilties as a function of a covariate, holding other covariates constant; inter-quartile-range odds ratios; predicted mean Y|X varying one covariate. These are very easy to do with the R rms package functions orm, Predict, Mean, summary.