test(aws-strands): read the documents that keep drifting

The truth pass on the four Strands documents corrected roughly fifty false
claims, in four recurring classes: counted quantities stated in prose, quoted
signatures, example-table claims about which hooks a demo uses, and claims
about what a package exports. The one class that did not drift is the one
class the repository already had a lever for.

Six levers, each in the existing test file for its subject. Every value is
derived, from the document by parsing a structured region of it and from the
source by enumerating it, so none of them carries an expected count or name
list beside the test.

- The TypeScript README route table against the DEMOS registry, by set and by
  length, mirroring the Python route-table test that held.
- The README's runner and script numerals against the files that carry a
  standalone runner and the package.json scripts that point at one, plus the
  named complement so the sentence and the numeral cannot disagree.
- Named imports in every ts fence against the real entry namespaces, and the
  install fence against this package's non-optional peers and the installed
  Strands SDK's own, with both fence numerals derived from those manifests.
- The ARCHITECTURE packaging-surface block against __all__ three ways: the
  name set, the block size and the prose numeral, so editing the sentence
  alone cannot satisfy it.
- Every signature the documents quote against inspect.signature, as ordered
  sequences, with the surface taken from __all__.
- Each example-table row's identifiers against the primitives the demo
  actually uses, by AST walk, set-equal so both directions are covered.

The example-table lever found two rows wrong by omission, which is the
direction a one-way check misses, and both are corrected here:
predictive_state_updates.py also uses state_from_args and
state_context_builder, and a2ui_recovery.py sets StrandsAgentConfig.a2ui.

Every lever was driven red before green, including the cases a weaker test
would pass: a duplicated row that is set-equal but the wrong length, a
stripped runner whose numeral stays correct, a prose numeral edited to match
a shortened block, and a parameter reorder a set comparison accepts.

Author: ranst91

integrations/aws-strands/ARCHITECTURE.md

@@ -309,23 +309,23 @@ Behaviors the Python adapter does not currently implement, added to match TypeSc
 
 The repository includes fifteen runnable FastAPI apps that showcase different features. Each example builds a Strands SDK agent, or a `Graph` of them, wraps it with `StrandsAgent`, and exposes it via `create_strands_app`. `server/settings.py` holds the route table and `server/__init__.py` mounts each app as a sub-application of the dojo:
 
-| Module                        | Focus                                                                   | Relevant Configuration                                                                                                                    |
-| ----------------------------- | ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
-| `agentic_chat.py`             | Baseline text generation with a frontend-only `change_background` tool. | No custom config; demonstrates automatic text streaming and frontend tool short-circuiting.                                               |
-| `agentic_chat_reasoning.py`   | Reasoning/thinking event streaming with extended thinking models.       | No custom config; demonstrates REASONING\_\* event emission. Pins a reasoning-capable model mode in `model_factory.py`.                   |
-| `agentic_chat_citations.py`   | Answers carrying the sources they came from.                            | No custom config; asks for OpenAI's Responses API with the built-in `web_search` tool, which is honoured only on `MODEL_PROVIDER=openai`. |
-| `agentic_chat_multimodal.py`  | Multimodal image/document analysis with vision-capable model.           | No custom config; demonstrates automatic multimodal content conversion.                                                                   |
-| `backend_tool_rendering.py`   | Backend-executed tools (`render_chart`, `get_weather`).                 | Shows how tool results become `ToolCallResultEvent`s and can be rendered directly in the UI.                                              |
-| `shared_state.py`             | Collaborative recipe editor that streams server-side state.             | Uses `state_context_builder` and `state_from_args` to keep the UI's recipe object synchronized.                                           |
-| `agentic_generative_ui.py`    | Predictive and reactive state updates for generative UI surfaces.       | Demonstrates `PredictStateMapping` alongside `state_context_builder` and `state_from_result`.                                             |
-| `predictive_state_updates.py` | Document editor painted from a frontend tool's streaming arguments.     | A `PredictStateMapping` on `write_document` projects the streaming args into `state.document` before the result arrives.                  |
-| `tool_based_generative_ui.py` | Frontend-rendered tool (`generate_haiku`) auto-registered as a proxy.   | No custom config; exercises the `TOOL_CALL_*` stream the dojo's page consumes.                                                            |
-| `human_in_the_loop.py`        | Human-in-the-loop confirmation flow with frontend tools.                | Explicitly configures `generate_task_steps` with `continue_after_frontend_call=False`; the shared frontend remains unchanged.             |
-| `interrupt.py`                | A backend tool pauses itself mid-body to ask the user for a time.       | No custom config; the tool calls `tool_context.interrupt(...)` and reads the user's choice back from that same call on resume.            |
-| `multi_agent.py`              | A `Graph` of agents, streamed as steps.                                 | Built with `GraphBuilder`; the adapter detects the orchestrator and drives its stream rather than cloning a per-thread agent.             |
-| `a2ui_dynamic_schema.py`      | A2UI surfaces composed on the fly.                                      | Sets `StrandsAgentConfig.a2ui` to name the catalog; `generate_a2ui` is auto-injected rather than wired here.                              |
-| `a2ui_fixed_schema.py`        | A2UI from fixed-layout backend tools.                                   | Backend tools return an `a2ui_operations` envelope directly, so nothing is auto-injected.                                                 |
-| `a2ui_recovery.py`            | A2UI validate-and-retry recovery loop.                                  | Same auto-injection as the dynamic demo; the injected tool validates each surface and retries up to three attempts before failing.        |
+| Module                        | Focus                                                                   | Relevant Configuration                                                                                                                                                                                                                                                                                  |
+| ----------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `agentic_chat.py`             | Baseline text generation with a frontend-only `change_background` tool. | No custom config; demonstrates automatic text streaming and frontend tool short-circuiting.                                                                                                                                                                                                             |
+| `agentic_chat_reasoning.py`   | Reasoning/thinking event streaming with extended thinking models.       | No custom config; demonstrates REASONING\_\* event emission. Pins a reasoning-capable model mode in `model_factory.py`.                                                                                                                                                                                 |
+| `agentic_chat_citations.py`   | Answers carrying the sources they came from.                            | No custom config; asks for OpenAI's Responses API with the built-in `web_search` tool, which is honoured only on `MODEL_PROVIDER=openai`.                                                                                                                                                               |
+| `agentic_chat_multimodal.py`  | Multimodal image/document analysis with vision-capable model.           | No custom config; demonstrates automatic multimodal content conversion.                                                                                                                                                                                                                                 |
+| `backend_tool_rendering.py`   | Backend-executed tools (`render_chart`, `get_weather`).                 | Shows how tool results become `ToolCallResultEvent`s and can be rendered directly in the UI.                                                                                                                                                                                                            |
+| `shared_state.py`             | Collaborative recipe editor that streams server-side state.             | Uses `state_context_builder` and `state_from_args` to keep the UI's recipe object synchronized.                                                                                                                                                                                                         |
+| `agentic_generative_ui.py`    | Predictive and reactive state updates for generative UI surfaces.       | Demonstrates `PredictStateMapping` alongside `state_context_builder` and `state_from_result`.                                                                                                                                                                                                           |
+| `predictive_state_updates.py` | Document editor painted from a frontend tool's streaming arguments.     | A `PredictStateMapping` on `write_document` projects the streaming args into `state.document` before the result arrives; `state_from_args` then publishes the finished document as authoritative state, and `state_context_builder` feeds that document back into the prompt so edits stay incremental. |
+| `tool_based_generative_ui.py` | Frontend-rendered tool (`generate_haiku`) auto-registered as a proxy.   | No custom config; exercises the `TOOL_CALL_*` stream the dojo's page consumes.                                                                                                                                                                                                                          |
+| `human_in_the_loop.py`        | Human-in-the-loop confirmation flow with frontend tools.                | Explicitly configures `generate_task_steps` with `continue_after_frontend_call=False`; the shared frontend remains unchanged.                                                                                                                                                                           |
+| `interrupt.py`                | A backend tool pauses itself mid-body to ask the user for a time.       | No custom config; the tool calls `tool_context.interrupt(...)` and reads the user's choice back from that same call on resume.                                                                                                                                                                          |
+| `multi_agent.py`              | A `Graph` of agents, streamed as steps.                                 | Built with `GraphBuilder`; the adapter detects the orchestrator and drives its stream rather than cloning a per-thread agent.                                                                                                                                                                           |
+| `a2ui_dynamic_schema.py`      | A2UI surfaces composed on the fly.                                      | Sets `StrandsAgentConfig.a2ui` to name the catalog; `generate_a2ui` is auto-injected rather than wired here.                                                                                                                                                                                            |
+| `a2ui_fixed_schema.py`        | A2UI from fixed-layout backend tools.                                   | Backend tools return an `a2ui_operations` envelope directly, so nothing is auto-injected.                                                                                                                                                                                                               |
+| `a2ui_recovery.py`            | A2UI validate-and-retry recovery loop.                                  | Sets `StrandsAgentConfig.a2ui` the way the dynamic demo does; the auto-injected tool validates each surface and retries up to three attempts before failing.                                                                                                                                            |
 
 ### TypeScript (`typescript/examples/server/api/*.ts`)
 

integrations/aws-strands/python/tests/test_docs_contract.py

@@ -1,4 +1,4 @@
-"""The README's load-bearing claims, checked against the code.
+"""The load-bearing claims of README.md and ARCHITECTURE.md, checked against the code.
 
 Review of this package repeatedly found documentation that named a constant the
 adapter does not emit, or a shape it does not produce. Prose drifts because
@@ -12,14 +12,19 @@
 
 from __future__ import annotations
 
+import inspect
 import re
 from pathlib import Path
 
-from ag_ui_strands import INTERRUPT_CANCELLED
+import ag_ui_strands
+from ag_ui_strands import INTERRUPT_CANCELLED, __all__
 
 _ROOT = Path(__file__).resolve().parent.parent
 README = (_ROOT / "README.md").read_text()
 SOURCE = (_ROOT / "src" / "ag_ui_strands" / "agent.py").read_text()
+ARCHITECTURE = (_ROOT.parent / "ARCHITECTURE.md").read_text()
+
+_PACKAGING_HEADING = "### Packaging Surface"
 
 
 def test_the_readme_never_names_a_near_miss_of_a_real_error_code():
@@ -96,3 +101,183 @@ def test_the_readme_documents_every_approval_metadata_key_published():
     assert undocumented == [], (
         f"the README does not document published approval metadata keys: {undocumented}"
     )
+
+
+def _packaging_surface_region() -> str:
+    """The Packaging Surface subsection of ARCHITECTURE.md, heading to next heading."""
+    start = ARCHITECTURE.find(_PACKAGING_HEADING)
+    assert start != -1, (
+        f"the packaging-surface heading {_PACKAGING_HEADING!r} moved or was reworded"
+    )
+    end = ARCHITECTURE.find("\n### ", start + 1)
+    return ARCHITECTURE[start:] if end == -1 else ARCHITECTURE[start:end]
+
+
+def _packaging_surface_names() -> list[str]:
+    """Every name listed in the grouped fence, in document order.
+
+    The fence lines are either ``label<gap>Name / Name /`` or an indented
+    continuation carrying bare names, so a label is whatever precedes the first
+    run of two or more spaces on an unindented line.
+    """
+    region = _packaging_surface_region()
+    fence = re.search(r"```\n(.*?)```", region, re.DOTALL)
+    assert fence is not None, "the packaging-surface group listing is no longer fenced"
+
+    names: list[str] = []
+    for line in fence.group(1).splitlines():
+        if not line.strip():
+            continue
+        if line[0].isspace():
+            listed = line
+        else:
+            parts = re.split(r"\s{2,}", line.strip(), maxsplit=1)
+            assert len(parts) == 2, f"packaging-surface line carries no names: {line!r}"
+            listed = parts[1]
+        names.extend(name.strip() for name in listed.split("/") if name.strip())
+
+    assert names, "the packaging-surface fence parsed to no names"
+    return names
+
+
+def test_architecture_lists_exactly_the_names_the_package_exports():
+    """Set, block size and prose numeral, all three against ``__all__``.
+
+    The audited drift was a fence listing 8 of 32 names beside a prose numeral
+    that was wrong on its own. Asserting only the set would let the numeral rot;
+    asserting only the numeral would let someone satisfy the test by editing one
+    digit while the fence stays short. Nothing here is a literal name list, so
+    adding an export fails until the fence and the sentence both catch up.
+    """
+    listed = _packaging_surface_names()
+    exported = list(__all__)
+
+    assert set(listed) == set(exported), (
+        "the packaging-surface fence and `__all__` disagree: "
+        f"only in the document {sorted(set(listed) - set(exported))}, "
+        f"only in `__all__` {sorted(set(exported) - set(listed))}"
+    )
+    assert len(listed) == len(exported), (
+        f"the fence lists {len(listed)} names but `__all__` carries {len(exported)}"
+    )
+
+    prose = re.search(
+        r"`__all__` is the exact surface and currently carries (\d+) names",
+        _packaging_surface_region(),
+    )
+    assert prose is not None, (
+        "the sentence stating how many names `__all__` carries moved or was reworded"
+    )
+    claimed = int(prose.group(1))
+    assert claimed == len(listed) == len(exported), (
+        f"the prose claims {claimed} names, the fence lists {len(listed)}, "
+        f"`__all__` carries {len(exported)}"
+    )
+
+
+def _code_parameter_names() -> dict[str, list[str]]:
+    """Declared parameter names of every callable in ``__all__``, in order.
+
+    Varargs are dropped because a document spells them inconsistently and they
+    say nothing about the call a reader will write. Protocol classes reduce to
+    nothing here and are simply never matched by a documented span.
+    """
+    names: dict[str, list[str]] = {}
+    for name in __all__:
+        obj = getattr(ag_ui_strands, name)
+        if not callable(obj):
+            continue
+        try:
+            signature = inspect.signature(obj)
+        except (TypeError, ValueError):
+            continue
+        names[name] = [
+            parameter.name
+            for parameter in signature.parameters.values()
+            if parameter.kind not in (parameter.VAR_POSITIONAL, parameter.VAR_KEYWORD)
+        ]
+    return names
+
+
+def _split_top_level(text: str) -> list[str]:
+    """Split on commas outside brackets and quotes, so defaults stay intact."""
+    parts: list[str] = []
+    current: list[str] = []
+    depth = 0
+    quote = ""
+    for char in text:
+        if quote:
+            current.append(char)
+            if char == quote:
+                quote = ""
+            continue
+        if char in "\"'":
+            quote = char
+        elif char in "([{":
+            depth += 1
+        elif char in ")]}":
+            depth -= 1
+        elif char == "," and depth == 0:
+            parts.append("".join(current))
+            current = []
+            continue
+        current.append(char)
+    parts.append("".join(current))
+    return [part.strip() for part in parts if part.strip()]
+
+
+def _documented_parameter_names(arguments: str) -> list[str]:
+    names = []
+    for argument in _split_top_level(arguments):
+        if argument in {"*", "/"} or argument.startswith("*"):
+            continue
+        names.append(argument.split("=", 1)[0].split(":", 1)[0].strip())
+    return names
+
+
+# A documented signature is a code span that opens a list item. Both looser
+# rules admit an example call, which legitimately passes a variable or a chosen
+# value where the signature names a parameter, so reading one as a signature
+# would fail against correct prose: dropping the list marker admits a sentence
+# that happens to start with a call span, and dropping the line-start anchor
+# admits a call quoted mid-sentence. A genuine signature written some other way
+# is not silently skipped either, because the set of callables this finds is
+# asserted below.
+_SIGNATURE_SPAN = re.compile(
+    r"^[ \t]*[-*+][ \t]+`([A-Za-z_][A-Za-z0-9_]*)\((.*?)\)`",
+    re.MULTILINE,
+)
+
+# Which callables the documents currently quote a signature for. Pinned so a
+# signature quietly vanishing from the prose fails here instead of shrinking the
+# scan to nothing. Parameter names are never pinned; those come from the code.
+DOCUMENTED_SIGNATURES = {"add_strands_fastapi_endpoint", "create_strands_app"}
+
+
+def test_every_quoted_signature_matches_the_parameters_the_code_declares():
+    """Ordered names, because a reader passes positional arguments by position.
+
+    The audited drift was a quoted signature missing ``invocation_state_provider``
+    in two documents while the parameter had existed for releases. Comparing
+    sets rather than sequences would have caught that one and missed a reorder,
+    which misleads a reader in exactly the same way.
+    """
+    declared = _code_parameter_names()
+    found: dict[str, set[str]] = {}
+
+    for label, document in (("ARCHITECTURE.md", ARCHITECTURE), ("README.md", README)):
+        for match in _SIGNATURE_SPAN.finditer(document):
+            name, arguments = match.group(1), match.group(2)
+            if name not in declared:
+                continue
+            found.setdefault(name, set()).add(label)
+            documented = _documented_parameter_names(arguments)
+            assert documented == declared[name], (
+                f"{label} documents {name} as {documented} but the code declares "
+                f"{declared[name]}"
+            )
+
+    assert set(found) == DOCUMENTED_SIGNATURES, (
+        f"the documents quote signatures for {sorted(found)}, expected "
+        f"{sorted(DOCUMENTED_SIGNATURES)}"
+    )