|
| 1 | +# ADR-0017: Adopt Bradley–Terry MM and additive-ties BRATT for pairwise ranking |
| 2 | + |
| 3 | +Status: **Accepted** |
| 4 | +Date: 2026-08-16 |
| 5 | +Supersedes: none |
| 6 | +Superseded by: none |
| 7 | + |
| 8 | +## Context |
| 9 | + |
| 10 | +Protected main already ships paired-comparison ranking kernels, including |
| 11 | +Bradley–Terry MM (`bradley_terry_mm`) and an additive-ties variant |
| 12 | +(`bratt_mm`). Implementation comments and changelog notes recorded that |
| 13 | +Bradley and Terry (1952) and Hunter (2004) were unread at port time and that |
| 14 | +`choix` / VGAM sources were the computational references. Those notes are |
| 15 | +source-governance history. They are not a reason to leave the shipped |
| 16 | +estimators without a primary-literature decision record, and they are not a |
| 17 | +license to claim Rao–Kupper or Davidson ties models. |
| 18 | + |
| 19 | +The implemented objects are psychometric ranking estimators. They are not |
| 20 | +security controls. |
| 21 | + |
| 22 | +## Decision drivers |
| 23 | + |
| 24 | +- Pairwise win matrices are a reusable measurement primitive for rater, |
| 25 | + model, and item comparisons. |
| 26 | +- The shipped MM update is Hunter's algorithm for the Bradley–Terry model; |
| 27 | + the decision record must say so. |
| 28 | +- Ties are observed in some designs; the implemented ties likelihood is the |
| 29 | + additive-`alpha0` BRATT model, not Rao–Kupper or Davidson. |
| 30 | +- Score-use interpretation, when worths are treated as educational or |
| 31 | + psychological scores, is governed by AERA, APA, and NCME (2014). |
| 32 | + |
| 33 | +## Ownership and dependency direction |
| 34 | + |
| 35 | +`fast-mlsirm` owns the reusable ranking kernels. Hosted leaderboards, |
| 36 | +participant identity, and operational selection policy remain downstream. |
| 37 | +This ADR does not change the ADR-0001 repository boundary. |
| 38 | + |
| 39 | +## Decision |
| 40 | + |
| 41 | +Adopt the Bradley–Terry paired-comparison model (Bradley & Terry, 1952) |
| 42 | +fitted by Hunter (2004) MM as the repository's tie-free pairwise-ranking |
| 43 | +estimator (`fast_mlsirm.bradley_terry_mm`). |
| 44 | + |
| 45 | +Adopt the implemented additive-ties BRATT variant (`fast_mlsirm.bratt_mm`) |
| 46 | +for data that contain ties: |
| 47 | + |
| 48 | +```text |
| 49 | +P(i beats j) = alpha_i / (alpha_i + alpha_j + alpha0) |
| 50 | +P(i ties j) = alpha0 / (alpha_i + alpha_j + alpha0) |
| 51 | +``` |
| 52 | + |
| 53 | +Do not claim Rao–Kupper or Davidson unless a later model-design change |
| 54 | +implements those likelihoods, gradients, tests, and documentation together. |
| 55 | + |
| 56 | +LSR / I-LSR, Rank Centrality, and Plackett–Luce ranking remain separate |
| 57 | +estimators. They are not aliases of Bradley–Terry MM. |
| 58 | + |
| 59 | +Method documentation: [`../bradley_terry_mm.md`](../bradley_terry_mm.md). |
| 60 | + |
| 61 | +## Invariants / acceptance evidence |
| 62 | + |
| 63 | +1. `wins` is a square nonnegative matrix with a zero diagonal; non-finite or |
| 64 | + negative counts are rejected. |
| 65 | +2. `bradley_terry_mm` rejects an all-zero matrix and, at `alpha = 0`, any |
| 66 | + object with zero wins (no finite log-worth). |
| 67 | +3. `bratt_mm` rejects tie-free data and any contestant with zero wins. |
| 68 | +4. Numeric work is Rust-owned; Python validates and marshals. |
| 69 | +5. Documentation names Bradley and Terry (1952) and Hunter (2004) as the |
| 70 | + method basis and does not name Rao–Kupper or Davidson as implemented |
| 71 | + product behavior. |
| 72 | + |
| 73 | +## Non-goals and claims not made |
| 74 | + |
| 75 | +- Not Rao–Kupper (1967) or Davidson (1970) ties models. |
| 76 | +- Not a Ford (1957) connectivity pre-check; disconnected graphs fail at |
| 77 | + estimation. |
| 78 | +- Not an IRT, MLSIRM, or many-facet severity model. |
| 79 | +- Not a causal ranking or high-stakes selection rule. |
| 80 | +- `choix` and VGAM `bratt()` are computational comparison sources, not |
| 81 | + scientific oracles. |
| 82 | + |
| 83 | +## Consequences and trade-offs |
| 84 | + |
| 85 | +### Benefits |
| 86 | + |
| 87 | +- Pairwise ranking has a named model and a named MM algorithm. |
| 88 | +- The ties path is explicitly the additive-`alpha0` variant. |
| 89 | +- Callers can choose `bradley_terry_mm` versus `bratt_mm` from the data |
| 90 | + contract (ties present or absent) rather than from package folklore. |
| 91 | + |
| 92 | +### Costs / risks |
| 93 | + |
| 94 | +- MM can fail to converge on sparse or one-sided graphs. |
| 95 | +- Fractional counts are accepted as weights; they are a derived extension of |
| 96 | + integer pair lists and must not be described as a different model. |
| 97 | +- Users may treat worths as IRT abilities unless the interpretation boundary |
| 98 | + stays adjacent to the API. |
| 99 | + |
| 100 | +## Alternatives considered |
| 101 | + |
| 102 | +### Claim Rao–Kupper or Davidson because the function mentions ties |
| 103 | + |
| 104 | +Rejected. The implemented likelihood is additive `alpha0` in the denominator. |
| 105 | +Those alternative ties models are not coded. |
| 106 | + |
| 107 | +### Treat changelog "NOT READ" notes as the bibliographic record |
| 108 | + |
| 109 | +Rejected. The estimators implement the named methods. Primary papers are the |
| 110 | +method basis; unread-at-port-time notes are historical. |
| 111 | + |
| 112 | +### Collapse all ranking kernels into one "Bradley–Terry" product claim |
| 113 | + |
| 114 | +Rejected. LSR, Rank Centrality, and Plackett–Luce are different estimators. |
| 115 | + |
| 116 | +## Failure, degraded, and recovery behavior |
| 117 | + |
| 118 | +Invalid shapes, non-finite or negative counts, nonzero diagonals, empty |
| 119 | +comparison graphs, zero-win objects (where the MLE is outside the positive |
| 120 | +parameter space), and non-convergence return errors. `bratt_mm` rejects |
| 121 | +tie-free data; callers must use `bradley_terry_mm` rather than treating |
| 122 | +`alpha0 = 0` as a valid BRATT fit. |
| 123 | + |
| 124 | +## Security and privacy implications |
| 125 | + |
| 126 | +Win/tie matrices can encode identifiable rater or candidate comparisons. |
| 127 | +Purpose limitation follows ADR-0012. This ADR adds no new credential or |
| 128 | +provider surface and is not a CWE/OWASP/NIST control. |
| 129 | + |
| 130 | +## Compatibility, migration, and rollback |
| 131 | + |
| 132 | +Public Python/Rust entry points are unchanged. This ADR records the |
| 133 | +scientific identity of already-shipped kernels. Retiring or replacing the |
| 134 | +likelihood requires a superseding model-design ADR. |
| 135 | + |
| 136 | +## Verification and release evidence |
| 137 | + |
| 138 | +- Rust unit oracles and Python contract tests for `bradley_terry_mm` and |
| 139 | + `bratt_mm` on protected main. |
| 140 | +- Cross-algorithm agreement with I-LSR at `alpha = 0` is supplementary |
| 141 | + numerical evidence, not a substitute for the Bradley–Terry / Hunter |
| 142 | + citations. |
| 143 | +- This ADR does not authorize a formula change. |
| 144 | + |
| 145 | +## Research and standards basis |
| 146 | + |
| 147 | +Bradley, R. A., & Terry, M. E. (1952). Rank analysis of incomplete block |
| 148 | +designs: I. The method of paired comparisons. *Biometrika, 39*(3/4), 324–345. |
| 149 | +https://doi.org/10.2307/2334029 |
| 150 | + |
| 151 | +Hunter, D. R. (2004). MM algorithms for generalized Bradley–Terry models. |
| 152 | +*The Annals of Statistics, 32*(1), 384–406. |
| 153 | +https://doi.org/10.1214/aos/1079120141 |
| 154 | + |
| 155 | +American Educational Research Association, American Psychological |
| 156 | +Association, & National Council on Measurement in Education. (2014). |
| 157 | +*Standards for educational and psychological testing*. American Educational |
| 158 | +Research Association. |
| 159 | + |
| 160 | +## Follow-ups |
| 161 | + |
| 162 | +Rao–Kupper, Davidson, or other generalized paired-comparison likelihoods |
| 163 | +remain Proposed unless implemented as a complete model path. LSR input-bound |
| 164 | +doctoring remains a separate operational control |
| 165 | +([`../doctoring/lsr_ranking_input_bounds.md`](../doctoring/lsr_ranking_input_bounds.md)). |
| 166 | + |
| 167 | +## Reversal / supersession conditions |
| 168 | + |
| 169 | +Supersede this ADR if the repository retires the MM kernels, changes the |
| 170 | +Bradley–Terry or additive-ties likelihood, or implements a different ties |
| 171 | +model as the default product behavior. |
0 commit comments