You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 4e6b025
Browse filesBrowse the repository at this point in the historyBrowse files
feat(evaluators): add extensions/metadata for additional fields that we want to pass as context to execution of evaluator (#276)
## Summary
Pass the authenticated execution context through Agent Control to
Galileo evaluators so scorer invocation can receive the Orbit-authorized
organization, caller, project, and target information.
- Preserve upstream authorization response extras as opaque
`Principal.extensions` metadata.
- Add an optional opaque `extensions` claim to newly minted runtime
JWTs. Existing claims, expiration behavior, and tokens without
extensions remain compatible.
- Pass verified principal extensions through a provider-neutral
evaluator hook. Existing evaluators continue using their current
evaluation path.
- In the Galileo evaluator only, translate the trusted extension
envelope into Galileo's `execution_context`. For the current Orbit
contract, `caller_id` is the authenticated user ID. A `log_stream`
target's authenticated target ID becomes `run_id`.
- Return a clear evaluator error when required authenticated fields are
missing.
## Scope
- API/runtime contract: additive opaque `extensions` claim; no named
Galileo fields are added to generic principal or JWT models.
- Internal: carry verified extensions from auth exchange through
runtime-token verification and engine evaluation.
- Galileo: serialize the local execution context with scorer invocation
requests.
- Out of scope: scorer grants, scorer authorization/execution, API-key
forwarding to Runners, record factory changes.
## Trust and Compatibility
Execution metadata comes only from the verified runtime principal.
Evaluator step payloads are not used to construct it, and user-supplied
tenant identity is not trusted. No scorer grant is acquired or cached,
and the application `GALILEO_API_KEY` is not sent to Runners. Legacy
runtime tokens without extensions and evaluators that do not override
the extension hook remain supported.
## Testing and Coverage
- [x] Engine tests: 120 passed, including opaque extension pass-through
and legacy evaluator fallback.
- [x] Built-in evaluator tests: 311 passed, including default
extension-hook delegation.
- [x] Galileo tests: 156 passed, including context serialization,
missing caller handling, and API-key non-forwarding.
- [x] Focused server auth tests: 6 passed, including malformed extension
handling and signed-token round-trip.
- [x] `make check` passed in PR CI; Codecov patch check passed.
- [x] `make lint` and `make typecheck` passed locally.
0 commit comments