You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: integrations/aws-strands/ARCHITECTURE.md
+10-1Lines changed: 10 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -148,6 +148,15 @@ This document explains how the AWS Strands integration inside `integrations/aws-
148
148
- Where the two fail on a bad policy differs, and cannot not. Python's `UrlFetchPolicy` is a frozen dataclass validating in `__post_init__`, so an unusable one raises `ValueError` where the host constructs it and no run starts. A TypeScript interface has no constructor, so the adapter validates the configured policy itself, once per run before the first attachment is fetched, and reports `URL_FETCH_POLICY_INVALID`. Checking it per fetch alone would be swallowed: the history-replay conversion catches a throw and falls back to text, turning a configuration mistake into attachments quietly stripped per message.
149
149
-**Citations**
150
150
- A provider citation arrives between the text deltas of the answer it annotates. The adapter folds it into that message's `metadata` under the `citations` key rather than emitting it separately, republishing the whole list each time so a client holds a prefix rather than a fragment, and carries the final list on `TEXT_MESSAGE_END` and in the following `MESSAGES_SNAPSHOT`. `citations.py` and `citations.ts` normalise the two SDKs' shapes onto one discriminated, empty-free form. The package READMEs carry the field-by-field account, including where the two bridges cannot agree because their SDKs report different things.
151
+
-**Token usage**
152
+
- `RUN_FINISHED.usage` and `RUN_ERROR.usage` are populated from Strands' per-model-call metadata event, one entry per model invocation, folded into one entry per `(provider, model)` at whichever terminal event ends the run. The fold is the published SDK helper (`aggregate_token_usage`, `aggregateTokenUsage`) rather than a local sum, so every AG-UI producer groups identically, and an empty aggregate omits the field rather than sending `[]`. The terminal `AgentResult.metrics.accumulated_usage` is deliberately not the source: it is pre-summed and seeded with zeros, so it cannot tell a provider that reported nothing apart from one that reported zero, and a client showing `0` tokens for an unmeasured run is showing a number nobody gave it. Reading the metadata event does not consume it: it still forwards as `RAW` afterwards, because the latency metrics beside the counts have no AG-UI equivalent. The mapper is `token-usage.ts` on the TypeScript side and a block near the top of `agent.py` on the Python one; only the aggregation is shared, because the Python bridge consumes the published protocol package and a new core mapper would not reach it until the next SDK release.
153
+
- Four counts map: `inputTokens`, `outputTokens`, `totalTokens` and `cacheReadInputTokens`. `cacheWriteInputTokens` is dropped rather than folded into a neighbouring count, which would overstate the count that received it, and Strands reports no reasoning-token count at all, so the reasoning field is never set from this channel. Nothing but the counts and the two labels is copied, since this shape feeds anonymous telemetry; an entry that would carry labels and no count is not usage and is not appended at all. The accumulator is a local of each run's generator, so a second sequential run in one stream cannot inherit the first run's counts.
154
+
- Usage rides the terminals a model call can precede and no others: the normal `RUN_FINISHED`, the interrupt-variant `RUN_FINISHED` (an interrupted run is a finished run, the calls that raised the interrupt were real, and the resume reports its own), the forced-stop `RUN_ERROR`, the post-stream session gates, `FRONTEND_TOOL_IDENTITY_ERROR`, and both paths' catch-all `RUN_ERROR`. The preflight gates, the idempotent-replay finish and `MEDIA_RESOLUTION_FAILED` fire before any model has run and carry nothing rather than claiming a measured nothing. So do the `INTERRUPT_RECONCILIATION_ERROR` refusals raised while correcting history, which sit ahead of the stream on both bridges; TypeScript has one further site for that code after the stream, and that one carries usage.
155
+
- Every count passes a guard before it is accepted: finite, non-negative, whole, and no larger than `2**53 - 1`. A count outside that is DROPPED and the rest of the entry survives, never clamped (a clamp reports a number no provider gave) and never zeroed (a zero claims a measurement nobody made). The upper bound is worth stating precisely because it is counterintuitive and was verified rather than assumed: `TokenUsageSchema` constrains counts to non-negative integers and sets no upper bound, so an oversized count validates fine and then throws inside the protobuf transport's `int64` decoder. The failure is the transport's rather than validation's, which means the same run would break at its final event on the binary wire and be served correctly over SSE, so bounding at the source is what keeps the two transports reporting the same thing. Python settles integers before any float check, because `math.isfinite` coerces to float and raises `OverflowError` on a large int, which would abort the run from inside the guard that exists to protect it.
156
+
- Both agent paths report. On the orchestrator path each entry is labelled with the model of the node that actually spent the tokens, so a multi-model `Graph` keeps its models apart, and the two bridges reach that pairing differently because that is where each SDK exposes it: Python resolves the node id the metadata event arrives under through `orchestrator.nodes[node_id].executor.model`, while TypeScript reads each node's inner `beforeModelCallEvent`, which carries the `Model` and arrives ahead of that node's metadata event. Only the labels are kept, never the model. A nested orchestrator node has no model of its own, so its entries stay label-less and still aggregate.
157
+
- The one real divergence is the provider label table, and it is the SDKs' rather than a decision: the two ship different model providers. Every provider both SDKs have maps to the same canonical label, Gemini and Google included, since Python's `GeminiModel` and the TypeScript SDK's `GoogleModel` are one vendor and both label `google`. Python additionally covers `litellm`, `llamaapi`, `llamacpp`, `mistral`, `ollama`, `sagemaker` and `writer`; TypeScript additionally covers `vercel`. Both tables are keyed on the model class name rather than derived from it, because a derivation is exactly what would split one vendor across the two bridges without anyone noticing. A class a table does not name omits the provider label rather than guessing, which also covers an integrator's own `Model` subclass: a subclass of `BedrockModel` reports its own class name, and inventing `bedrock` for it would attribute the spend to a provider nobody named.
158
+
- A sub-agent's own spend is not counted, and the gap is proven rather than suspected. A generator tool wrapping another `Agent` re-yields the inner agent's whole stream as tool-stream payloads, which the parent loop routes to its inner tool-call forwarder and never through the metadata branch, so the inner model calls are absent from the parent run's usage. That is real spend going unreported. `python/tests/test_terminal_event_token_usage.py` pins the boundary rather than asserting it is right, and widening it has to land on both bridges at once, or the same run would report different totals depending on which one served it.
159
+
- There is no capabilities flag for this, deliberately. `DEFAULT_CAPABILITIES` is served from the TypeScript bridge alone, so advertising a behaviour both bridges have from the one document only one of them serves would create exactly the drift the error-code table and the resume contract were settled to close.
151
160
-**Unmapped events**
152
161
- A Strands stream event with no AG-UI translation is forwarded as `RawEvent(event=..., source="strands")` rather than dropped, which is how Bedrock's per-turn `metadata` reaches a client. The payload is filtered, never coerced: per-run invocation-state keys are stripped, and anything that will not survive a strict JSON round trip is dropped with a warning rather than stringified, since coercing it would ship the serialized live `Agent` (system prompt and conversation history included) to every connected client. Both bridges do this, from the same two positions in the loop.
153
162
- What rides `event` is a framework-shaped payload the SDK is free to change in any release. Neither adapter treats its shape as part of its own contract, and neither one's README invites a client to depend on a field found in it.
@@ -353,7 +362,7 @@ Both example sets double as integration tests, but they do not reach every confi
353
362
| Terminal result whose `stop_reason` is abnormal | Emit `CustomEvent(name="AgentStopped")`, then finish normally | UI can explain a truncated, filtered or guardrailed answer instead of reading it as success. |
354
363
| Stream sends `complete` or adapter decides to halt | Close text/reasoning envelopes and emit `RunFinishedEvent`| Signals the UI that the run ended; frontends may start follow-up runs or show idle states. |
355
364
|`stream_async` yields `{"force_stop": True, ...}`| Record the reason, drain the stream, emit `RunErrorEvent` with `code="STRANDS_FORCE_STOP"`| Frontend sees a failed run rather than a short success; no final state or finish arrives. |
356
-
| Any event the adapter does not map | Strip invocation-state keys, drop what will not survive a strict JSON round trip, emit the rest as `RawEvent(source="strands")`| A client can read SDK-shaped detail (Bedrock usage metadata, say) at its own risk; the shape is the SDK's, not this adapter's. |
365
+
| Any event the adapter does not map | Strip invocation-state keys, drop what will not survive a strict JSON round trip, emit the rest as `RawEvent(source="strands")`| A client can read SDK-shaped detail (Bedrock's latency metrics, say) at its own risk; the shape is the SDK's, not this adapter's. The token counts on that same metadata event have a mapped channel and do not have to be read this way: see Token usage above. |
357
366
| Exceptions anywhere in the stack | Emit `RunErrorEvent` with the exception message, under `code="ADAPTER_BUG"` when the escaping exception is a `TypeError`, `AttributeError` or `NameError` and `code="STRANDS_ERROR"` for anything else | Frontend surfaces the failure and can offer retries, and can tell an adapter defect from a provider or SDK one. |
358
367
359
368
The table above covers the run loop, not every terminal code: preflight and transport failures carry their own, listed under Lifecycle framing and under Additions Beyond the Python Adapter.
0 commit comments