Skip to content

feat(examples): add offline SDK-local per-control timing demo - #289

Closed
josjeon wants to merge 3 commits into
mainfrom
codex/sao-18023-sdk-local-timing-demo
Closed

josjeon wants to merge 3 commits into
mainfrom
codex/sao-18023-sdk-local-timing-demo

Conversation

@josjeon

@josjeon josjeon commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds a runnable, offline Python SDK example for the local-control timing work discussed in Part A of the Agent Control load-test thread. It uses the per-control execution_duration_ms added by #283, runs five synthetic built-in control cases (allow, block, steer, SQL, JSON), validates their outcomes, and reports per-control time beside the full SDK-call time. An optional --concurrency mode runs a bounded single-process throughput smoke and records observed calls per second. An HTTP mock rejects accidental network requests and SDK observability is disabled for the run.

This PR is stacked on #283 because that PR supplies the measured field. Retarget it to main after #283 merges.

Verification

  • python -m pytest examples/sdk_local_timing/test_demo.py -q: 2 passed using the exact feat(engine): capture per-control execution duration [SAO-18023] #283 SDK environment.
  • ruff check examples/sdk_local_timing and ruff format --check examples/sdk_local_timing: passed.
  • Ran demo.py --iterations 4 --warmup 2 and --iterations 12 --warmup 2 --concurrency 4: all five cases produced finite, nonnegative control durations and a tabular comparison.
  • Repository make prepush could not complete because the configured Splunk Artifactory Python index returned HTTP 401 while resolving google-re2; no test failure was reported before dependency resolution stopped.

Scope

This is a fixture-driven SDK-local timing example. The throughput smoke uses one Python process and event loop, and it does not add an evaluator-only timer, instrument a decorated agent, or establish SDK/server load capacity. Server-side GLoad work is tracked separately in Orbit.

@josjeon josjeon changed the title feat: add offline SDK-local per-control timing example feat(examples): add offline SDK-local per-control timing demo Oct 8, 2026
@codecov

codecov Bot commented Oct 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Base automatically changed from codex/sao-18023-control-duration to main October 8, 2026 22:26
@josjeon josjeon closed this Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant