22
33Thank you for your interest in contributing.
44
5- TRACER is a research-grade routing library with formal parity guarantees. The
6- project lives or dies on those guarantees holding up under measurement, so the
7- bar for changes is evidence, not intuition. Please read the standards below
8- before opening an issue or a PR. They exist to keep contributions focused and
9- to respect everyone's time, including yours.
5+ TRACER is a routing library with formal parity guarantees, and those guarantees
6+ only mean something if they hold up under measurement. So for anything that can
7+ change routing behavior, the most useful thing you can bring is evidence. The
8+ standards below exist to keep contributions focused and to make the best use of
9+ your time and ours. None of this is meant to gatekeep good ideas. it's meant to
10+ help us evaluate them fairly.
1011
1112## Contribution standards (read first)
1213
13- TRACER is benchmark-driven. Every change that can affect routing behavior must
14- be justified with numbers, not opinion . Three rules follow from that.
14+ TRACER is benchmark-driven, so changes that affect routing behavior are easiest
15+ for us to act on when they come with numbers . Three guidelines follow from that.
1516
16- 1 . ** Adding a model needs a benchmark, not a name.** "We should add model X"
17- is not a contribution. A model proposal must show, on a reproducible eval,
18- that the new surrogate improves the coverage-vs-agreement frontier over the
19- models already in the zoo (see "Adding a new surrogate model" below). An
20- issue or PR that proposes a model without that evidence will be closed with
21- a pointer back to this section. You are welcome to reopen it once you have
22- the numbers .
17+ 1 . ** Adding a model: bring a benchmark, not just a name.** A request that only
18+ names a model to add is hard for us to evaluate, because we can't tell
19+ whether it would actually help. A model proposal is most useful when it
20+ shows, on a reproducible eval, how the new surrogate compares to the models
21+ already in the zoo on the coverage-vs-agreement frontier (see "Adding a new
22+ surrogate model" below). Without that, we'll usually ask for a benchmark
23+ before we can move forward, and we're glad to point you at how to run one .
2324
24- 2 . ** Changing a core parameter, threshold, or gate needs an ablation.** The
25+ 2 . ** Changing a core parameter, threshold, or gate: bring an ablation.** The
2526 acceptor threshold, the parity gate, the calibration logic, the default
2627 ` FitConfig ` values, and the split strategy are load-bearing for the parity
27- guarantee. A change to any of them must include before/after numbers on the
28- eval showing the frontier does not regress (and, ideally, improves). "It
29- feels like it should be configurable" is not a justification. If you believe
30- a parameter should be user-tunable, show a case where a different value wins
31- on the benchmark.
28+ guarantee. A change to any of them is much easier to accept with before/after
29+ numbers on the eval showing the frontier does not regress (and, ideally,
30+ improves). If you think a parameter should be user-tunable, the most
31+ convincing case is one where a different value measurably wins.
3232
33- 3 . ** Bug reports need a reproduction.** The strongest reports name the affected
34- files and lines and include a minimal script that reproduces the behavior.
35- See issues #29 , #30 , and #31 for the standard we are looking for .
33+ 3 . ** Bug reports: include a reproduction.** The reports we can act on fastest
34+ name the affected files and lines and include a minimal script that
35+ reproduces the behavior. Issues #29 , #30 , and #31 are good examples .
3636
3737If your idea is exploratory ("how would TRACER handle X?", "is this a good fit
3838for Y?"), open a ** GitHub Discussion** rather than an issue. Issues are for
@@ -47,11 +47,12 @@ Run against a fixed, public eval so results are reproducible by a maintainer:
4747- Report the metric that matters: ** coverage at the target teacher agreement**
4848 (how much traffic the surrogate can take while holding parity), plus the
4949 realized teacher agreement on a held-out split (not the calibration split).
50- - Compare against the current behavior on ` main ` , not against nothing .
50+ - Compare against the current behavior on ` main ` , so the delta is clear .
5151- Include the exact command and config you ran so the numbers can be checked.
5252
53- A change that does not move (or that hurts) the frontier will not be merged,
54- no matter how clean the code is.
53+ If a change does not move the frontier (or regresses it), we'll usually hold
54+ off on merging even when the code itself is clean, since the numbers are what
55+ we're optimizing for.
5556
5657## Setup
5758
@@ -107,8 +108,8 @@ src/tracer/
107108
108109## Adding a new surrogate model
109110
110- The surrogate zoo is not a popularity contest. We add a model only when it
111- earns a place on the frontier .
111+ We're happy to grow the zoo when a model earns its place on the frontier. The
112+ benchmark is what tells us it does .
112113
1131141 . Add a factory to the ` _candidates() ` dict in ` src/tracer/fit/surrogate.py ` .
114115 The model must implement the scikit-learn ` fit ` / ` predict ` / ` predict_proba `
@@ -119,8 +120,8 @@ earns a place on the frontier.
1191203 . Account for the dependency cost. A new heavy dependency needs to pay for
120121 itself in the numbers, and should be optional where possible.
121122
122- A PR that adds a model without (2) will be closed with a request for the
123- benchmark .
123+ If a PR adds a model without (2), we'll ask for the benchmark before reviewing
124+ further. It's usually quick to run, and it's the thing that lets us say yes .
124125
125126## Changing the gate, calibration, or default config
126127
0 commit comments