|
| 1 | +--- |
| 2 | +id: "052" |
| 3 | +status: Accepted |
| 4 | +date: 2026-10-17 |
| 5 | +category: composition |
| 6 | +supersedes: [] |
| 7 | +superseded_by: [] |
| 8 | +--- |
| 9 | + |
| 10 | +# ADR-052: Hybrid is the delivery floor |
| 11 | + |
| 12 | +## Context |
| 13 | + |
| 14 | +A generated context file delivers each rule one of three ways: copied in |
| 15 | +(inline), named as a file to read (reference), or both, with the rules that |
| 16 | +matter copied in and the rest named (hybrid). The interview offers all |
| 17 | +three, and `agents.md` recommends hybrid without ruling reference out. |
| 18 | + |
| 19 | +| Force | What it shows | |
| 20 | +| ----- | ------------- | |
| 21 | +| No loading mechanism | A referenced template is not loaded at startup, on relevance, or by any cue. It is read only when the agent decides to open it (#712) | |
| 22 | +| A consumer it failed | Under a reference-model file, a project's agent never opened the deferred templates across a session of git, testing and review work. A type-check gate that lived only there was missed and CI went red, and the defer list itself drifted unnoticed (#712) | |
| 23 | +| Adherence | In round 3 hybrid adhered to the project's conventions best of every arm: +21.6 pp against no file, where the short inline file read +11.1 pp (#1893) | |
| 24 | +| Security under a thin spec | In round 4, with a spec that states the contract and not the defences, hybrid scored highest of every arm on the judge's security and data-protection rows (`docs/audits/2026-10-17-round-4.md`). Rescored with the secret key the spec had omitted, two of its three trials read 0.82 and 1.00 on the security probes; the third was not measured (`rounds/2026-10-16-thin-sensitivity/score-py` in `braboj/tariff-runs`) | |
| 25 | +| A cost of length | Under a weaker model in round 4, every template file but the hand-written one lowered task success against no file | |
| 26 | +| A gap in the evidence | No round has run a reference-only arm, so the benchmark has not compared reference with hybrid directly | |
| 27 | + |
| 28 | +## Decision |
| 29 | + |
| 30 | +A generated context file delivers rules inline or hybrid, never by reference |
| 31 | +alone: a rule whose breaking is silent and that no tool enforces is inlined, |
| 32 | +and any other rule may stay a reference. |
| 33 | + |
| 34 | +## Rationale |
| 35 | + |
| 36 | +- A reference has no loading mechanism, so a rule that exists only as a |
| 37 | + reference is a rule the agent may never read; the consumer failure shows |
| 38 | + the cost lands where nothing catches it. |
| 39 | +- Placement by damage and enforcement keeps every rule an agent must not miss |
| 40 | + in context, while a rule whose breaking shows itself, or that a test, |
| 41 | + linter or CI step checks, can stay outside it and keep the file short. |
| 42 | +- Hybrid is the arm the benchmark reads best on adherence and on security, so |
| 43 | + making it the floor moves generation towards the measured result rather |
| 44 | + than away from it. |
| 45 | +- The direct comparison is missing, so the record names the round that |
| 46 | + supplies it and what result would reopen it. |
| 47 | + |
| 48 | +## Design |
| 49 | + |
| 50 | + |
| 51 | + |
| 52 | +1. **The floor** — Generation MUST NOT produce a file that delivers its |
| 53 | + rules by reference alone. Inline and hybrid remain. |
| 54 | +2. **Inline the silent, unenforced rules** — A rule MUST be inlined when its |
| 55 | + breaking is silent, showing no error to the agent or the user, and no |
| 56 | + test, linter or CI step enforces it. |
| 57 | +3. **Reference the rest** — A rule whose breaking fails visibly, or that a |
| 58 | + test, linter or CI step enforces, MAY be referenced instead of inlined. |
| 59 | +4. **Where the rubric lives** — `agents.md` states the rubric, and the |
| 60 | + interview applies it; neither offers the reference model. |
| 61 | +5. **Length is a cost, not a cap** — Inlining SHOULD stop at the rules rule 2 |
| 62 | + requires. This record sets no size limit. |
| 63 | + |
| 64 | +| Breaking the rule... | Enforced by a tool | Not enforced by a tool | |
| 65 | +| -------------------- | ------------------ | ---------------------- | |
| 66 | +| fails visibly | MAY reference | MAY reference | |
| 67 | +| is silent | MAY reference | MUST inline | |
| 68 | + |
| 69 | +## Alternatives considered |
| 70 | + |
| 71 | +| Alternative | Why not | |
| 72 | +| ----------- | ------- | |
| 73 | +| Keep three models and recommend hybrid | Reference stays on offer, and the consumer failure is what it produces | |
| 74 | +| Inline everything | The weaker-model result shows that length costs; a project vendoring the templates duplicates them | |
| 75 | +| Allow reference delivered as a skill | A skill is loaded on demand from its description, which a plain reference is not, but no round has tested one; it is left to a later record | |
| 76 | +| Wait for the reference arm before deciding | The missing mechanism is known, and generating reference files meanwhile ships the failure it causes | |
| 77 | + |
| 78 | +## Consequences |
| 79 | + |
| 80 | +| Consequence | Where | |
| 81 | +| ----------- | ----- | |
| 82 | +| `agents.md` "Choosing a model" states the rubric and drops reference | #712 | |
| 83 | +| The interview stops offering the reference model | #712 | |
| 84 | +| Skills as a delivery layer are reconsidered against this floor | #414 | |
| 85 | +| Round 5 runs a reference-only arm beside hybrid. If it is no worse on adherence and on the judge's security and data-protection rows, this record is reopened | #1957 | |
| 86 | +| Whether inlined length needs a bound for weaker models stays open until more than one project and K = 3 measure it | — | |
| 87 | + |
| 88 | +## Compatibility |
| 89 | + |
| 90 | +A project whose context file uses the reference model keeps working: nothing |
| 91 | +it names is removed. Its next regeneration produces a hybrid file. The |
| 92 | +reference model is listed under `Deprecated` in the CHANGELOG with hybrid as |
| 93 | +its replacement, and is removed from the interview and `agents.md` in the |
| 94 | +release that implements this record. |
| 95 | + |
| 96 | +## Security |
| 97 | + |
| 98 | +Protects: a security rule whose breaking is silent, such as output escaping, |
| 99 | +a CSRF token, or a secret kept out of logs, is always in the agent's context, |
| 100 | +where a reference-only file could leave it unread. |
0 commit comments