Skip to content

Latest commit

 

History

History
65 lines (53 loc) · 4 KB

ROUTING-FACT-QUALIFICATION.md

File metadata and controls

65 lines (53 loc) · 4 KB

Routing-fact ergonomics qualification

The interface-45 focused routing replacement is a navigation change. It does not require inventing a standards proposal or publishing policies. Automated checks, configured-client conformance and actual model behavior are separate evidence.

Automated checks

Use the supported existing hash-locked Python 3.12 environment and complete CI. The targeted suites are test_routing_fact_inputs, test_routing_fact_transport, test_required_fact_lookup, test_compact_routing and test_route_content; run native query/preview and retained-authoring regressions too. The generator, registry and plan verifier retain their existing owners. Do not broaden a dependency lock or claim a Python version that was not run.

The six-type/state differential compares final canonical validity, not identical static parsing. Literal exists/truth/unknown expectations, real distinct captured vocabularies and error-code/field checks supply independent oracles. No later negative fact is inferred from omission. A bound declaration is not approval to read private application content.

Configured-client conformance (no model)

After installing the coordinated source, restart the already authorized registration and refresh/reconnect. Keep its purpose and output-delivery settings. Run the existing codex_navigation_client with the server and output delivery configured on that host. It now exercises raw assertions, explicit uncertainty, snapshot-bound vocabulary, content continuations and full explanations. This is a scripted host test; do not call it a model-authored comparison or use its fixture data to prefill a model task.

Matched fresh model tasks

Use the same supported model, client surface, starting standards authority and task intent for the old interface-44 baseline and the candidate. Configure one version per fresh context; do not teach the candidate the old request syntax or supply ready-made schema payloads. Let each model retrieve input shapes/vocabulary through its own connected tools as needed. Do not require discovery as a ritual.

First ask for the standards relevant to an implementation task, with application forms explicitly known empty and other categories unstated. Require the agent to show which categories remain unanswered rather than manufacturing empty values. Then provide one real additional fact from the task (for example a declared IPC boundary) and ask it to refine against the same snapshot, retrieve selected content, and follow a page and full explanation without recapturing authority.

In an authorized disposable fixture with declared boolean, nullable string, enum, set and aliases, cover false, known null, explicit absence, explicit unknown and normalized alias/array continuations. Production vocabulary must not be changed just to manufacture this case. Check malformed type/alias requests separately and verify that the agent can repair the input without treating it as a missing policy or internal tool failure. Include distinct Unicode spellings that normalize into one set entry: the focused input must reject until the duplicate is removed. No authoring operation is authorized by this test.

Preserve raw call arguments/results, actual runtime/catalog/model/client identity, rejections and repairs, repeated reads, discovery counts and exact authority. Score correctness separately from request bytes and total tool/model turns. Compare selected guidance and unanswered questions to independent expected facts. Source or observer-only schemas must not be inserted into the model context. Record unavailable host-rendered context or billing metrics as unavailable, not inferred from JSON.

The acceptance owner uses these observations and independent source review to close R-A7/R-A8. The future review-packet generator is not a prerequisite. Historical baseline evidence remains unchanged and bound to its original interface; no old-shape fallback is added to the candidate. Unsupported independently deployed consumers need their own deliberate rollout or support decision before cutover.