Skip to content

Freeze target metadata in audit manifests and comparisons - #62

Merged
SushantGautam merged 2 commits into
mainfrom
feature/61-freeze-target-metadata
Oct 28, 2026
Merged

SushantGautam merged 2 commits into
mainfrom
feature/61-freeze-target-metadata

Conversation

@SushantGautam

Copy link
Copy Markdown
Collaborator

Summary

Closes #61.

  • Discover optional nested JSON model metadata on GET /v1/models/{id}, with a fallback to GET /v1/models (also recognizes Open WebUI's info.meta).
  • Capture it exactly once per new run as frozen target metadata; record discovery status, source, and timestamp. Retries and historical views only read the frozen snapshot.
  • Preserve arbitrary nested prompt bodies, versions, retrieval settings, tool definitions, and framework-specific fields, with a 256 KiB and depth limit plus secret-key redaction.
  • Show one flattened, dot-path row per metadata key in /runs/<id>/ Frozen Manifest, revealing long values on expansion.
  • Add per-key metadata comparisons to the Compare Runs Input Parameters table, including missing-key and changed-value highlighting.
  • Document the custom compatibility extension and add tests for discovery, fallback, sanitization, freezing, and comparisons.

Validation

  • Pure metadata discovery/flattening unit tests: 8 passing with isolated dependency shims.
  • Source-edit anchors checked against current main and smoke-tested.
  • Full Django test suite / GitHub CI: not run here (repo checkout and Django environment unavailable).

Compatibility

  • Official OpenAI Model objects have no documented metadata field. This is optional and does not affect targets without the extension.
  • No database migration; existing target snapshot JSON field is reused.

Closes #61.

- Discover optional nested JSON model metadata on GET /v1/models/{id},
  with fallback to GET /v1/models (also recognizes Open WebUI info.meta).
- Capture once per new run as frozen target metadata; record discovery
  status, source, and timestamp. Retries and historical views read the
  frozen snapshot only.
- Preserve arbitrary nested prompt bodies, versions, retrieval settings,
  tool definitions, and framework-specific fields, with a 256 KiB and
  depth limit plus secret-key redaction.
- Show one flattened dot-path row per metadata key in Frozen Manifest,
  revealing long values on expansion.
- Add per-key metadata comparisons to Compare Runs Input Parameters,
  including missing-key and changed-value highlighting.
- Document the custom compatibility extension; add tests for discovery,
  fallback, sanitization, freezing, and comparisons.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@SushantGautam
SushantGautam merged commit 1a3cc4d into main Oct 28, 2026
1 check passed
@SushantGautam
SushantGautam deleted the feature/61-freeze-target-metadata branch October 28, 2026 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Retrieve meta from agentic target and freeze

1 participant