Goal
Export calibrated primary-versus-shadow comparison datasets with immutable provenance for reviewed Router Model experiments.
Proposed Workgroup
Router Models & Inference Runtime
Scope
- Convert approved shadow observations into a versioned dataset manifest.
- Preserve prompt/reference identity, primary and shadow model identities, output hashes, judge or label provenance, sampling policy, and redaction state.
- Define deterministic inclusion, exclusion, deduplication, balancing, and train/eval split rules.
- Validate schema, lineage, privacy policy, and artifact immutability before export.
- Support pluggable storage destinations without committing datasets to the source repository.
Non-goals
- Capturing live shadow traffic; tracked by the shadow dispatch issue.
- Training or promoting a model.
- Treating an unreviewed judge score as ground truth.
Completion signal
- One approved shadow corpus exports reproducibly from the same inputs to the same manifest and split identities.
- Every example is traceable to immutable, privacy-reviewed evidence.
- Invalid lineage, redaction, or split leakage fails before publication.
Parent Epic: #2975
Goal
Export calibrated primary-versus-shadow comparison datasets with immutable provenance for reviewed Router Model experiments.
Proposed Workgroup
Router Models & Inference Runtime
Scope
Non-goals
Completion signal
Parent Epic: #2975