Bradley-Terry model: Stan vs. iterative conjugate fit

Modeling
1

Hey everyone, as a side project I have been working on creating a rating system for pool players from the 1980s.

I have about 2500 match results from major tournaments and fit a Bradley-Terry model using a Gamma prior as laid out on page 6 of this paper: https://www.stats.ox.ac.uk/~doucet/caron_doucet_bayesianbradleyterry.pdf

The iterative process in equation 7 is super fast and seems to work well.

I tried replicating this setup in Stan with the following:

data {
  int<lower=0> N; // N games
  int<lower=0> P; // P players
  int player1[N]; // Indicator arrays for player 1
  int player2[N]; // Indicator arrays for player 2
  int player1_wins[N]; // Number of wins by player 1
  int match_length[N]; // Match length
  int alpha; // Gamma paramater
}
parameters {
  simplex[P] ratings; // Vector of ratings for each player
}
model {
  ratings ~ gamma(alpha, 100); 
  
  for (i in 1:N) {
    player1_wins[i] ~ 
    binomial_logit(
      match_length[i], 
      ratings[player1[i]] / (ratings[player1[i]] + ratings[player2[i]])
    );
  }
}

This compiles, runs, and gives predictions that are similar but not identical to my older model. It also results in worse performance compared to the older model across a variety of priors with some obviously misrated players.

Any ideas about what might be going on? Am I setting up what I think I’m setting up?

2

It turns out I should have used “binomial” rather than “binomial_logit”. The second term is the modeled actual chance of success while the binomial_logit setup wants the logit-scaled chance of success.

1 Like