You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 3332d96
Browse filesBrowse the repository at this point in the historyBrowse files
|[AGENTS.md](AGENTS.md)| Integration guide for AI coding assistants |
165
190
191
+
## Prior work
192
+
193
+
TRACER builds on and connects three research threads:
194
+
195
+
**Learn to Defer (L2D)** -- the algorithmic backbone. The surrogate + acceptor gate architecture is grounded in Mozannar & Sontag ([ICML 2020](https://proceedings.mlr.press/v119/mozannar20b.html)), Madras et al. ([NeurIPS 2018](https://papers.nips.cc/paper_files/paper/2018/hash/09d37c08f7b129e96277388757530c72-Abstract.html)), and Mao et al. ([NeurIPS 2023](https://proceedings.neurips.cc/paper_files/paper/2023/hash/7e1bf09d2e4e6e5c5a0b63b66b47ef42-Abstract.html)).
196
+
197
+
**Post-hoc calibration** -- the threshold sweep that enforces the parity guarantee follows the post-hoc deferral estimator framework of Narasimhan et al. ([ICML 2022](https://proceedings.mlr.press/v162/narasimhan22a.html)).
198
+
199
+
**LLM routing** -- the problem setting is shared with Hybrid LLM (Ding et al., [ICLR 2024](https://openreview.net/forum?id=02f3mUtqnM)) and RouteLLM (Ong et al., [ICLR 2025](https://openreview.net/forum?id=sSt9fROCn2)). TRACER's distinction: it targets classification specifically, uses the teacher's own trace distribution rather than preference data, and enforces a formal parity guarantee rather than a quality heuristic.
200
+
201
+
**Interpretability** -- the qualitative audit design is informed by Slice Finder (Chung et al., [ICDE 2019](https://ieeexplore.ieee.org/document/8731353)) and the contrastive explanation literature (Miller, [AIJ 2019](https://www.sciencedirect.com/science/article/pii/S0004370217301126)).
202
+
166
203
## Paper
167
204
168
205
A research paper detailing the approach, formal guarantees, ablation studies, limitations, and reproducible experiment tooling is in preparation. It will be linked here upon publication.
Copy file name to clipboardExpand all lines: docs/concepts.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,13 +38,13 @@ A second binary classifier that predicts: "will the surrogate's prediction match
38
38
39
39
### Threshold calibration
40
40
41
-
The acceptor threshold τ is calibrated on a held-out set (not used for training). The calibration objective: find the threshold that maximizes coverage subject to teacher agreement ≥ target. This is a conformal-style guarantee: on the calibration distribution, you get the promised agreement rate.
41
+
The acceptor threshold τ is calibrated on a held-out set (not used for training). The calibration objective: find the threshold that maximizes coverage subject to teacher agreement ≥ target. This is a conformal-style guarantee: on the calibration distribution, you get the promised agreement rate. The approach follows the post-hoc deferral estimator framework of Narasimhan et al. ([ICML 2022](https://proceedings.mlr.press/v162/narasimhan22a.html)).
42
42
43
43
---
44
44
45
45
## Three pipeline families
46
46
47
-
TRACER fits three candidate architectures and picks the one with highest coverage at your target teacher agreement.
47
+
TRACER fits three candidate architectures and picks the one with highest coverage at your target teacher agreement. The L2D and RSB families are grounded in the Learn-to-Defer literature (Mozannar & Sontag, [ICML 2020](https://proceedings.mlr.press/v119/mozannar20b.html); Madras et al., [NeurIPS 2018](https://papers.nips.cc/paper_files/paper/2018/hash/09d37c08f7b129e96277388757530c72-Abstract.html); Mao et al., [NeurIPS 2023](https://proceedings.neurips.cc/paper_files/paper/2023/hash/7e1bf09d2e4e6e5c5a0b63b66b47ef42-Abstract.html)).
48
48
49
49
### Global
50
50
@@ -136,7 +136,7 @@ Coverage grows monotonically as more traces accumulate. Easy intents (tight embe
136
136
137
137
## Qualitative audit
138
138
139
-
Every fit produces a structured explanation of the routing policy:
139
+
Every fit produces a structured explanation of the routing policy. The slice and boundary-pair design is informed by Slice Finder (Chung et al., [ICDE 2019](https://ieeexplore.ieee.org/document/8731353)) and the contrastive explanation literature (Miller, [AIJ 2019](https://www.sciencedirect.com/science/article/pii/S0004370217301126)).
140
140
141
141
**Slice summaries**: per-label and per-length-bucket handled/deferred rates. Answers: "which parts of my label space are well-covered?"
0 commit comments