Skip to content

Commit 942d621

Browse files
authored
feat(server): add multi-replica cluster mode (#1281)
Adds a2a.server.cluster, key interfaces (__VersionedTaskStore__ and __TaskEventStream__) for running the A2A server as several replicas, with no need to route a task's requests back to the replica that started it, and wires DefaultRequestHandlerV2 to use them so any replica can serve any request for a remote task. __Opt-in and backward compatible:__ with no event stream and a plain (non-versioned) TaskStore, pre-existing behavior is preserved. __Key changes:__ - VersionedTaskStore - a task store that keeps track of which revision of a task a replica last read. When that replica saves, the store checks the task has not changed in the meantime and refuses the write / retries if it has, instead of quietly overwriting whatever another replica just did. VersionedDatabaseTaskStore is the database-backed implementation. - TaskEventStream - a shared record of the events a task produces, so a replica that is not running the agent can still see the event stream. This lets a client reconnect to any replica and keep receiving updates for a task running elsewhere. DatabaseTaskEventStream is the database-backed implementation. - Schema for both: a new task_versions table and a new task_events table. The tasks table is unchanged. - New optional event_stream argument; task_store now accepts a plain or a versioned store and is still exposed unchanged as self.task_store. - Cancel works from any replica. The cancelling replica records the cancellation in the shared store, and the replica actually running the agent notices on its next save and stops. - Resubscribe works from any replica. It returns the current task, then follows the shared event stream from that point on, so the client sees no gap and no duplicates. - Alembic revision b5e3d1c8a2f7 (revises 38ce57e08137) creates the task_versions table and the task_events table with an index on task_id. It only creates tables and does not alter existing ones. - A runnable sample showing how to deploy the server as several replicas: it starts N replicas as separate processes sharing one database, and a driver starts a task on one replica, watches it from a second, and cancels it from a third. Defaults to a shared SQLite file so it runs with no external services; set A2A_CLUSTER_DSN to point every replica at Postgres or MySQL instead. __Tested:__ - Integration tests run two DefaultRequestHandlerV2 replicas against one real Postgres or MySQL database, each replica on its own engine. They cover send, resubscribe and cancel handled by a replica other than the one running the agent, and a multi-turn input_required conversation that moves between replicas. - An integration test sends follow-ups to the same task from both replicas at once and asserts that neither turn is lost. - Unit tests cover the same cross-replica flows without a database, using in-memory versions of the shared store and event stream. - Unit tests for the migration's full up/down cycle, idempotency and offline SQL generation. - Adds the database-backed cluster tests to scripts/run_db_tests.sh. Run the sample from the repository root: ``` python -m samples.clustermode.run_cluster --replicas 3 --base-port 41241 python -m samples.clustermode.exercise --host 127.0.0.1 --ports 41241,41242,41243 ``` __Design discussion:__ #1224 Release-As: 1.2.0
1 parent dc5a5da commit 942d621

34 files changed

Lines changed: 5315 additions & 59 deletions

‎.github/actions/spelling/allow.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ buf
3434
bufbuild
3535
cla
3636
cls
37+
clustermode
3738
coc
3839
codegen
3940
coro
@@ -45,6 +46,7 @@ deepwiki
4546
denormal
4647
denormals
4748
drivername
49+
dsn
4850
DSNs
4951
dunders
5052
ES256
@@ -111,13 +113,15 @@ openapiv
111113
openapiv2
112114
opensource
113115
otherurl
116+
outerjoin
114117
pb2
115118
podman
116119
Podman
117120
poolclass
118121
postgres
119122
POSTGRES
120123
postgresql
124+
procs
121125
proot
122126
proto
123127
protobuf
@@ -144,6 +148,7 @@ signings
144148
SLF
145149
socio
146150
sourced
151+
sqlstate
147152
sse
148153
starlette
149154
Starlette
@@ -162,5 +167,7 @@ TResponse
162167
typ
163168
typeerror
164169
UIDs
170+
versioned
171+
Versioned
165172
vulnz
166173
whl

‎samples/clustermode/README.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# Cluster-mode sample (multi-replica A2A)
2+
3+
Runs several A2A server replicas that share one durable task store and one event
4+
stream, so any replica can serve any request for any task. This demonstrates the
5+
multi-server support in `a2a.server.cluster`: send, resubscribe, and cancel all
6+
work regardless of which replica a request lands on — no sticky routing needed.
7+
8+
## What it shows
9+
10+
- **Shared, versioned task store** (`VersionedDatabaseTaskStore`) — concurrent
11+
writes across replicas are serialized by optimistic concurrency (CAS); no lost
12+
updates.
13+
- **Shared event stream** (`DatabaseTaskEventStream`) — a subscription on one replica
14+
streams events produced by an agent running on another.
15+
- **Cancel via CAS** — a cancel on replica C stops an agent running on replica A
16+
(A's next save fails the compare-and-swap and aborts).
17+
18+
## Requirements
19+
20+
Nothing extra by default: it uses a file-backed SQLite database
21+
(`/tmp/a2a_cluster_demo.db`) shared by all replica processes.
22+
23+
For a realistic setup, point every replica at a shared Postgres/MySQL:
24+
25+
```bash
26+
export A2A_CLUSTER_DSN='postgresql+asyncpg://user:pass@localhost/a2a'
27+
```
28+
29+
## Run
30+
31+
Start a 3-replica cluster (ports 41241, 41242, 41243):
32+
33+
```bash
34+
python -m samples.clustermode.run_cluster --replicas 3 --base-port 41241
35+
```
36+
37+
In another terminal, exercise it (one task, spread across replicas):
38+
39+
```bash
40+
python -m samples.clustermode.exercise --host 127.0.0.1 --ports 41241,41242,41243
41+
```
42+
43+
Expected output ends with:
44+
45+
```
46+
OK: one task was started, observed, and cancelled across three different replicas.
47+
```
48+
49+
## Files
50+
51+
| File | Purpose |
52+
|------|---------|
53+
| `cluster_common.py` | Shared store/stream wiring, agent card, and the demo agent |
54+
| `server.py` | One replica (a FastAPI app on one port) |
55+
| `run_cluster.py` | Launches N replicas as subprocesses |
56+
| `exercise.py` | Client that sends/gets/cancels one task across replicas |
57+
58+
## How it maps to the API
59+
60+
Every replica builds its handler the same way — the only multi-server-specific
61+
part is passing a shared `event_stream` and a shared `VersionedTaskStore`:
62+
63+
```python
64+
handler = DefaultRequestHandler(
65+
agent_executor=SlowEchoAgent(...),
66+
task_store=VersionedDatabaseTaskStore(engine=..., create_table=False),
67+
agent_card=agent_card,
68+
event_stream=DatabaseTaskEventStream(engine=..., create_table=False),
69+
)
70+
```
71+
72+
Omit `event_stream` (and use a plain `InMemoryTaskStore`) and you get the ordinary
73+
single-process behaviour — the multi-server path is fully opt-in.
74+
```

‎samples/clustermode/__init__.py‎

Whitespace-only changes.
Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
"""Shared wiring for the multi-replica (cluster mode) sample.
2+
3+
Every replica in this sample builds its handler from the SAME durable task store
4+
and the SAME event stream (pointed at one shared database). That is what lets any
5+
replica serve any request for any task - send, resubscribe, and cancel all work
6+
regardless of which replica the load balancer picks.
7+
8+
By default this uses a file-backed SQLite database so the sample runs with no
9+
external services. Set A2A_CLUSTER_DSN to a Postgres/MySQL async DSN for a more
10+
realistic setup, e.g.:
11+
12+
export A2A_CLUSTER_DSN='postgresql+asyncpg://user:pass@localhost/a2a'
13+
"""
14+
15+
import asyncio
16+
import logging
17+
import os
18+
import tempfile
19+
20+
from pathlib import Path
21+
22+
from sqlalchemy.ext.asyncio import create_async_engine
23+
24+
from a2a.helpers.proto_helpers import new_task_from_user_message
25+
from a2a.server.agent_execution.agent_executor import AgentExecutor
26+
from a2a.server.agent_execution.context import RequestContext
27+
from a2a.server.cluster import VersionedDatabaseTaskStore
28+
from a2a.server.cluster.database_event_stream import DatabaseTaskEventStream
29+
from a2a.server.events.event_queue import EventQueue
30+
from a2a.server.models import Base
31+
from a2a.server.tasks.task_updater import TaskUpdater
32+
from a2a.types import (
33+
AgentCapabilities,
34+
AgentCard,
35+
AgentInterface,
36+
AgentSkill,
37+
Part,
38+
TaskState,
39+
)
40+
41+
42+
logger = logging.getLogger(__name__)
43+
44+
45+
def default_sqlite_path() -> str:
46+
"""Path to the shared SQLite file used when no DSN is configured."""
47+
return os.environ.get(
48+
'A2A_CLUSTER_SQLITE',
49+
str(Path(tempfile.gettempdir()) / 'a2a_cluster_demo.db'),
50+
)
51+
52+
53+
def default_dsn() -> str:
54+
"""The shared-database DSN. SQLite file by default; override via env."""
55+
dsn = os.environ.get('A2A_CLUSTER_DSN')
56+
if dsn:
57+
return dsn
58+
# A file (not :memory:) so separate replica processes share one database.
59+
return f'sqlite+aiosqlite:///{default_sqlite_path()}'
60+
61+
62+
async def init_schema(dsn: str) -> None:
63+
"""Creates the shared tables (task + task_events) once, up front."""
64+
engine = create_async_engine(dsn)
65+
try:
66+
async with engine.begin() as conn:
67+
await conn.run_sync(Base.metadata.create_all)
68+
finally:
69+
await engine.dispose()
70+
71+
72+
def build_cluster_backends(
73+
dsn: str,
74+
) -> tuple[VersionedDatabaseTaskStore, DatabaseTaskEventStream]:
75+
"""Builds a versioned store + event stream over the shared database.
76+
77+
Each replica calls this with the same DSN. `create_table=False` because
78+
`init_schema` owns table creation.
79+
"""
80+
store_engine = create_async_engine(dsn)
81+
stream_engine = create_async_engine(dsn)
82+
store = VersionedDatabaseTaskStore(engine=store_engine, create_table=False)
83+
stream = DatabaseTaskEventStream(
84+
engine=stream_engine, create_table=False, poll_interval_s=0.2
85+
)
86+
return store, stream
87+
88+
89+
def build_agent_card(base_url: str) -> AgentCard:
90+
"""The agent card advertised by every replica (same logical agent)."""
91+
return AgentCard(
92+
name='Cluster Demo Agent',
93+
description='A slow agent used to demonstrate multi-replica A2A.',
94+
version='1.0.0',
95+
capabilities=AgentCapabilities(
96+
streaming=True, push_notifications=False
97+
),
98+
default_input_modes=['text'],
99+
default_output_modes=['text', 'task-status'],
100+
skills=[
101+
AgentSkill(
102+
id='cluster_demo',
103+
name='Cluster Demo',
104+
description='Echoes slowly so you can observe it across replicas.',
105+
tags=['sample', 'cluster'],
106+
examples=['hello'],
107+
input_modes=['text'],
108+
output_modes=['text', 'task-status'],
109+
)
110+
],
111+
supported_interfaces=[
112+
AgentInterface(
113+
protocol_binding='JSONRPC',
114+
protocol_version='1.0',
115+
url=f'{base_url}/a2a/jsonrpc',
116+
),
117+
],
118+
)
119+
120+
121+
class SlowEchoAgent(AgentExecutor):
122+
"""Goes WORKING, emits ticks slowly, then completes.
123+
124+
The deliberate slowness makes the multi-replica behaviour observable: you
125+
can subscribe or cancel from a different replica while a task is mid-flight.
126+
Resumability across replicas relies on this agent reading state only from
127+
the task (it does not keep anything in process memory between turns).
128+
"""
129+
130+
def __init__(self, replica_id: str, ticks: int = 10) -> None:
131+
self._replica_id = replica_id
132+
self._ticks = ticks
133+
134+
async def execute(
135+
self, context: RequestContext, event_queue: EventQueue
136+
) -> None:
137+
"""Runs the slow echo: WORKING, ticks, artifact, complete."""
138+
updater = TaskUpdater(
139+
event_queue,
140+
str(context.task_id or ''),
141+
str(context.context_id or ''),
142+
)
143+
if context.current_task is None:
144+
await event_queue.enqueue_event(
145+
new_task_from_user_message(context.message)
146+
)
147+
await updater.start_work(
148+
message=updater.new_agent_message(
149+
[Part(text=f'[{self._replica_id}] starting')]
150+
)
151+
)
152+
for i in range(self._ticks):
153+
await asyncio.sleep(1.0)
154+
await updater.update_status(
155+
TaskState.TASK_STATE_WORKING,
156+
message=updater.new_agent_message(
157+
[Part(text=f'[{self._replica_id}] tick {i + 1}')]
158+
),
159+
)
160+
await updater.add_artifact(
161+
[Part(text=f'[{self._replica_id}] done')],
162+
name='response',
163+
last_chunk=True,
164+
)
165+
await updater.complete()
166+
167+
async def cancel(
168+
self, context: RequestContext, event_queue: EventQueue
169+
) -> None:
170+
"""No-op: the stop happens via the store CAS when CANCELED is recorded."""
171+
return

0 commit comments

Comments
 (0)