Skip to content

Commit bb0252c

Browse files
adridaclaude
andcommitted
Refine contributing guidelines and issue template wording
Reword the contribution guidelines and issue forms for a clearer, more welcoming tone. Requirements are unchanged: a reproducible benchmark for model proposals, an ablation for core-parameter or gate changes, and a reproduction for bug reports. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 498c383 commit bb0252c

4 files changed

Lines changed: 41 additions & 39 deletions

File tree

‎.github/ISSUE_TEMPLATE/config.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@ blank_issues_enabled: false
22
contact_links:
33
- name: Question or open-ended idea
44
url: https://github.com/adrida/tracer/discussions
5-
about: "Exploratory questions (how would TRACER handle X?, is this a fit for Y?) belong in Discussions. Issues are for actionable, evidenced work."
5+
about: "Exploratory questions (how would TRACER handle X?, is this a fit for Y?) are a great fit for Discussions. Issues work best for actionable, evidenced work."

‎.github/ISSUE_TEMPLATE/feature_request.yml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,10 @@ body:
66
attributes:
77
value: |
88
If your request changes a core parameter, threshold, gate, calibration,
9-
or default config, it must include an ablation showing the frontier does
10-
not regress on a reproducible eval. See CONTRIBUTING.md. Open-ended
11-
questions belong in Discussions, not here.
9+
or default config, an ablation showing the frontier does not regress on
10+
a reproducible eval is what makes it easiest for us to act on (see
11+
CONTRIBUTING.md). For open-ended questions, Discussions is usually a
12+
better fit than an issue.
1213
- type: textarea
1314
id: problem
1415
attributes:

‎.github/ISSUE_TEMPLATE/model_proposal.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,11 @@ body:
55
- type: markdown
66
attributes:
77
value: |
8-
The surrogate zoo is not a popularity contest. We add a model only when
9-
it earns a place on the coverage-vs-agreement frontier. A proposal
10-
without a reproducible benchmark against the existing zoo will be closed
11-
with a pointer to CONTRIBUTING.md. You are welcome to reopen it with the
12-
numbers.
8+
Thanks for proposing a model. We're glad to grow the zoo when a model
9+
earns its place on the coverage-vs-agreement frontier, and a reproducible
10+
benchmark against the existing zoo is what lets us tell. Without one we'll
11+
usually ask for the numbers before we can move forward (see
12+
CONTRIBUTING.md). It's quick to run, and we're happy to help.
1313
- type: input
1414
id: model
1515
attributes:

‎CONTRIBUTING.md‎

Lines changed: 31 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -2,37 +2,37 @@
22

33
Thank you for your interest in contributing.
44

5-
TRACER is a research-grade routing library with formal parity guarantees. The
6-
project lives or dies on those guarantees holding up under measurement, so the
7-
bar for changes is evidence, not intuition. Please read the standards below
8-
before opening an issue or a PR. They exist to keep contributions focused and
9-
to respect everyone's time, including yours.
5+
TRACER is a routing library with formal parity guarantees, and those guarantees
6+
only mean something if they hold up under measurement. So for anything that can
7+
change routing behavior, the most useful thing you can bring is evidence. The
8+
standards below exist to keep contributions focused and to make the best use of
9+
your time and ours. None of this is meant to gatekeep good ideas. it's meant to
10+
help us evaluate them fairly.
1011

1112
## Contribution standards (read first)
1213

13-
TRACER is benchmark-driven. Every change that can affect routing behavior must
14-
be justified with numbers, not opinion. Three rules follow from that.
14+
TRACER is benchmark-driven, so changes that affect routing behavior are easiest
15+
for us to act on when they come with numbers. Three guidelines follow from that.
1516

16-
1. **Adding a model needs a benchmark, not a name.** "We should add model X"
17-
is not a contribution. A model proposal must show, on a reproducible eval,
18-
that the new surrogate improves the coverage-vs-agreement frontier over the
19-
models already in the zoo (see "Adding a new surrogate model" below). An
20-
issue or PR that proposes a model without that evidence will be closed with
21-
a pointer back to this section. You are welcome to reopen it once you have
22-
the numbers.
17+
1. **Adding a model: bring a benchmark, not just a name.** A request that only
18+
names a model to add is hard for us to evaluate, because we can't tell
19+
whether it would actually help. A model proposal is most useful when it
20+
shows, on a reproducible eval, how the new surrogate compares to the models
21+
already in the zoo on the coverage-vs-agreement frontier (see "Adding a new
22+
surrogate model" below). Without that, we'll usually ask for a benchmark
23+
before we can move forward, and we're glad to point you at how to run one.
2324

24-
2. **Changing a core parameter, threshold, or gate needs an ablation.** The
25+
2. **Changing a core parameter, threshold, or gate: bring an ablation.** The
2526
acceptor threshold, the parity gate, the calibration logic, the default
2627
`FitConfig` values, and the split strategy are load-bearing for the parity
27-
guarantee. A change to any of them must include before/after numbers on the
28-
eval showing the frontier does not regress (and, ideally, improves). "It
29-
feels like it should be configurable" is not a justification. If you believe
30-
a parameter should be user-tunable, show a case where a different value wins
31-
on the benchmark.
28+
guarantee. A change to any of them is much easier to accept with before/after
29+
numbers on the eval showing the frontier does not regress (and, ideally,
30+
improves). If you think a parameter should be user-tunable, the most
31+
convincing case is one where a different value measurably wins.
3232

33-
3. **Bug reports need a reproduction.** The strongest reports name the affected
34-
files and lines and include a minimal script that reproduces the behavior.
35-
See issues #29, #30, and #31 for the standard we are looking for.
33+
3. **Bug reports: include a reproduction.** The reports we can act on fastest
34+
name the affected files and lines and include a minimal script that
35+
reproduces the behavior. Issues #29, #30, and #31 are good examples.
3636

3737
If your idea is exploratory ("how would TRACER handle X?", "is this a good fit
3838
for Y?"), open a **GitHub Discussion** rather than an issue. Issues are for
@@ -47,11 +47,12 @@ Run against a fixed, public eval so results are reproducible by a maintainer:
4747
- Report the metric that matters: **coverage at the target teacher agreement**
4848
(how much traffic the surrogate can take while holding parity), plus the
4949
realized teacher agreement on a held-out split (not the calibration split).
50-
- Compare against the current behavior on `main`, not against nothing.
50+
- Compare against the current behavior on `main`, so the delta is clear.
5151
- Include the exact command and config you ran so the numbers can be checked.
5252

53-
A change that does not move (or that hurts) the frontier will not be merged,
54-
no matter how clean the code is.
53+
If a change does not move the frontier (or regresses it), we'll usually hold
54+
off on merging even when the code itself is clean, since the numbers are what
55+
we're optimizing for.
5556

5657
## Setup
5758

@@ -107,8 +108,8 @@ src/tracer/
107108

108109
## Adding a new surrogate model
109110

110-
The surrogate zoo is not a popularity contest. We add a model only when it
111-
earns a place on the frontier.
111+
We're happy to grow the zoo when a model earns its place on the frontier. The
112+
benchmark is what tells us it does.
112113

113114
1. Add a factory to the `_candidates()` dict in `src/tracer/fit/surrogate.py`.
114115
The model must implement the scikit-learn `fit` / `predict` / `predict_proba`
@@ -119,8 +120,8 @@ earns a place on the frontier.
119120
3. Account for the dependency cost. A new heavy dependency needs to pay for
120121
itself in the numbers, and should be optional where possible.
121122

122-
A PR that adds a model without (2) will be closed with a request for the
123-
benchmark.
123+
If a PR adds a model without (2), we'll ask for the benchmark before reviewing
124+
further. It's usually quick to run, and it's the thing that lets us say yes.
124125

125126
## Changing the gate, calibration, or default config
126127

0 commit comments

Comments
 (0)