test(aws-strands): bind the test servers to the address the probes dial - #2621
Merged
Conversation
The CORS suite failed roughly one run in six under load, and the failure moved around: a wrong header, an agent that never ran, an HTTP parse error carrying a body no route in this package emits, a bare close with nothing read. All four were the same defect. `app.listen(0)` with no host binds the IPv6 wildcard, while every probe in these suites dials `127.0.0.1`. Those are not the same port. An unrelated process already holding the IPv4 wildcard on the number the kernel hands out keeps receiving the loopback traffic, so the probe is answered by that process and the test's own server never sees a connection. Confirmed by `lsof` at the moment of failure, which showed a foreign IPv4 listener and the test's IPv6 listener on one port, and reproduced deliberately with a stand-in process: an unspecified bind hands the reply to the stranger, a loopback bind reaches the intended app. Pin the bind address in the shared transport harness and in the seven sibling suites that bound the same way, and add a harness self-test on the address so dropping it fails here rather than silently reappearing as a flake somewhere else. A specific bind takes precedence over a wildcard holder, so a probe now reaches its own app or nothing at all. No behaviour under test changes: no assertion is weakened, no retry or timeout is added, and no runtime string is touched. Verified: a stress rig that failed three times in 2000 cycles now runs 6000 clean; 35 and then 32 consecutive green runs of the CORS suite under ten busy cores; the full package green at 1612 passing; typecheck and Prettier clean.
Contributor
Python Preview PackagesVersion
Install with uvAdd the TestPyPI index to your [[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
explicit = trueThen install the packages you need: # Core SDK
uv add 'ag-ui-protocol==0.0.0.dev1788438802' --index testpypi
# Integrations (each already depends on the matching ag-ui-protocol preview)
uv add 'ag-ui-langgraph==0.0.0.dev1788438802' --index testpypi
uv add 'ag-ui-crewai==0.0.0.dev1788438802' --index testpypi
# NOTE: ag-ui-agent-spec depends on pyagentspec (git-only, not on PyPI).
# You will need to install pyagentspec separately from its git repo.
uv add 'ag-ui-agent-spec==0.0.0.dev1788438802' --index testpypi
uv add 'ag_ui_adk==0.0.0.dev1788438802' --index testpypi
uv add 'ag_ui_strands==0.0.0.dev1788438802' --index testpypiInstall with pippip install \
--index-url https://test.pypi.org/simple/ \
--extra-index-url https://pypi.org/simple/ \
ag-ui-protocol==0.0.0.dev1788438802
Commit: 81cd60d |
@ag-ui/a2a-middleware
@ag-ui/a2ui-middleware
@ag-ui/event-throttle-middleware
@ag-ui/mcp-apps-middleware
@ag-ui/mcp-middleware
@ag-ui/a2a
@ag-ui/adk
@ag-ui/ag2
@ag-ui/agno
@ag-ui/aws-strands
@ag-ui/claude-agent-sdk
@ag-ui/claude-managed-agents
@ag-ui/crewai
@ag-ui/langchain
@ag-ui/langgraph
@ag-ui/llamaindex
@ag-ui/mastra
@ag-ui/pydantic-ai
@ag-ui/vercel-ai-sdk
@ag-ui/watsonx
@ag-ui/a2ui-toolkit
create-ag-ui-app
@ag-ui/client
@ag-ui/core
@ag-ui/encoder
@ag-ui/proto
commit: |
AlemTuzlak
approved these changes
Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The flake
integrations/aws-strands/typescript/src/__tests__/cors.test.tsfailed roughly one run in six when the machine was under load. The failure moved around between runs, which is why it read as several unrelated problems:agent.runsstill0after a POST that returned a responseHTTPParserError: Response does not match the HTTP/1.1 protocolwhose payload was{"type":"Tier1","version":"1.0"}, a body no route in this package emitsSocketError: other side closedwithbytesWritten: 334, bytesRead: 0All four are the same defect.
Root cause
app.listen(0)with no host binds the IPv6 wildcard (::). Every probe in these suites dials127.0.0.1, an IPv4 address. Those are not the same port.When an unrelated process already holds the IPv4 wildcard on the port number the kernel hands out, that process keeps receiving the loopback traffic and answers the probe. The test's own server never sees a connection, so it emits no
connectionevent and its agent never runs. What the suite asserts on is whatever the stranger happened to reply.lsofat the moment of failure, captured by a stress harness:Two listeners on one port number, one per address family. The
{"type":"Tier1","version":"1.0"}body in the parser error is that other process's local API answering, not anything this package serves.Reproduced deliberately with a stand-in process holding the IPv4 wildcard, which isolates the mechanism from any particular application being installed:
This also explains why no listen port was ever duplicated inside the test process, and why the collision rate tracks machine load: the more of the ephemeral range is churning, the likelier a test draws a number some other process is already sitting on.
The change
Pin the bind address to
127.0.0.1in the shared transport harness and in the seven sibling suites that bound the same way. A specific bind takes precedence over a wildcard holder, so a probe now reaches its own app or nothing at all.The harness self-test gains an assertion on the bind address, so dropping the host fails there rather than reappearing silently as a flake in an unrelated suite.
No behaviour under test changes: no assertion is weakened, no retry or timeout is added, and no runtime string, error code, or logged message is touched.
Verification
/capabilities, preflight, close) failed 3 times in 2000 cycles before the change and ran 6000 cycles clean after it, both under ten busy cores.cors.test.tsunder load, then 32 more after the self-test was added.tsc --noEmitclean; Prettier clean on every changed file.Unrelated to the documentation-contract work in #2620.