Skip to content

Commit 58788bb

Browse files
fix(server): propagate caller cancellation out of ActiveTask.subscribe (a2aproject#1402)
# Description `ActiveTask.subscribe` catches `asyncio.CancelledError` together with `QueueShutDown` and breaks out of its loop. So when the task that iterates a subscription gets cancelled, the generator just finishes and the cancellation is lost. It shows up with a timeout around a blocking send. With an executor that moves the task to WORKING and keeps going, `asyncio.wait_for(handler.on_message_send(...), timeout=0.3)` returns the task in TASK_STATE_WORKING instead of raising `TimeoutError`. Calling `cancel()` on a task running `on_message_send` behaves the same way. It ends normally with `cancelled() == False` and returns whatever non-terminal state the task had at that moment. A cancelled consumer of `on_message_send_stream` or `on_subscribe_to_task` sees its stream end as if the task had finished. The fix catches only `QueueShutDown`. The `finally` block already closes the tap, drops the reference count and runs `_maybe_cleanup()`, so cleanup happens the same way when the cancellation propagates. I couldn't find anything other than the caller that would raise `CancelledError` there. The producer and the consumer only put `Exception` instances on the queue (the producer swallows its own cancellation in `_run_producer`), and shutting the queue down raises `QueueShutDown` for both `immediate=True` and `immediate=False`. The existing `test_active_task_subscribe_cancelled_during_wait` accepted either `CancelledError` or `StopAsyncIteration`, with a comment putting the second one down to a Python 3.10 generator quirk. It was really this `except` swallowing the cancel, and the test gets `StopAsyncIteration` on 3.13 as well. With the fix both versions raise `CancelledError`, so the test now requires it. I also added three handler tests using the existing `_HangingAgent`. One wraps `on_message_send` in `wait_for` and expects `TimeoutError`, one cancels a task running `on_message_send`, and one cancels a task consuming `on_message_send_stream`. With the `src` change stashed all four fail on both 3.10 and 3.13 (the three new ones with "DID NOT RAISE", the old one with `StopAsyncIteration`), and with it they pass. On 3.10 at 43d1eff the full suite gives 2494 passed, 203 skipped, 4 xfailed, 1 xpassed, and the coverage run clears the 88% gate. `./scripts/lint.sh` passes. One related thing I left alone. The unary wrapper in `_plugin_hooks.py` only calls `after_handler_run` from `except Exception`, so a cancelled `send_message` now gets `before_handler_run` with no matching after call. That already happened for a cancel at any other await, but the subscribe wait is where a blocking send usually sits when a timeout or disconnect hits, and before this change the after hook ran there with the stale task. The stream wrapper calls it from `finally`. I'm happy to make the unary side match in a separate PR if you want that. - [x] Follow the [`CONTRIBUTING` Guide](https://github.com/a2aproject/a2a-python/blob/main/CONTRIBUTING.md). - [x] Make your Pull Request title in the <https://www.conventionalcommits.org/> specification. - [x] Ensure the tests and linter pass (Run `bash scripts/format.sh` from the repository root to format) - [x] Appropriate docs were updated (if necessary) --------- Co-authored-by: Mateusz Iwaniuk <45974414+Iwaniukooo11@users.noreply.github.com>
1 parent d830036 commit 58788bb

3 files changed

Lines changed: 187 additions & 5 deletions

File tree

‎src/a2a/server/agent_execution/active_task.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -869,7 +869,7 @@ async def subscribe( # noqa: PLR0915
869869
yield event
870870
finally:
871871
tapped_queue.task_done()
872-
except (QueueShutDown, asyncio.CancelledError):
872+
except QueueShutDown:
873873
break
874874
finally:
875875
logger.debug('Subscribe[%s]: Unsubscribing', self._task_id)

‎tests/server/agent_execution/test_active_task.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -701,9 +701,9 @@ async def slow_execute(req, q):
701701

702702
task.cancel()
703703

704-
# In python 3.10+ cancelling an async generator next() might raise StopAsyncIteration
705-
# if the generator handles the cancellation by closing.
706-
with pytest.raises((asyncio.CancelledError, StopAsyncIteration)):
704+
# The cancellation belongs to the caller, so subscribe() must re-raise
705+
# it rather than end the stream as if the task had finished.
706+
with pytest.raises(asyncio.CancelledError):
707707
await task
708708

709709
await it.aclose()

‎tests/server/request_handlers/test_default_request_handler_v2.py‎

Lines changed: 183 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
)
2525
from a2a.server.agent_execution.active_task_registry import ActiveTaskRegistry
2626
from a2a.server.context import ServerCallContext
27-
from a2a.server.events import EventQueue, InMemoryQueueManager
27+
from a2a.server.events import Event, EventQueue, InMemoryQueueManager
2828
from a2a.server.events.event_queue_v2 import EventQueueSource
2929
from a2a.server.request_handlers import DefaultRequestHandlerV2
3030
from a2a.server.tasks import (
@@ -2721,6 +2721,188 @@ async def test_on_cancel_of_parked_task_is_owner_scoped():
27212721
await handler.aclose()
27222722

27232723

2724+
class _ReleaseThenCompleteAgent(_HangingAgent):
2725+
"""Like ``_HangingAgent``, but completes the task once released."""
2726+
2727+
def __init__(self) -> None:
2728+
super().__init__()
2729+
self.task_id: str | None = None
2730+
2731+
async def execute(
2732+
self, context: RequestContext, event_queue: EventQueue
2733+
) -> None:
2734+
self.task_id = context.task_id
2735+
await super().execute(context, event_queue)
2736+
updater = TaskUpdater(event_queue, context.task_id, context.context_id)
2737+
await updater.complete()
2738+
2739+
2740+
async def _wait_for_task_state(handler, task_id, ctx, state) -> None:
2741+
async def poll() -> None:
2742+
while True:
2743+
task = await handler.on_get_task(GetTaskRequest(id=task_id), ctx)
2744+
if task.status.state == state:
2745+
return
2746+
await asyncio.sleep(0.01)
2747+
2748+
await asyncio.wait_for(poll(), timeout=5)
2749+
2750+
2751+
async def _assert_task_survives(handler, agent, ctx) -> None:
2752+
"""The task keeps running after its caller goes away and still finishes."""
2753+
assert agent.task_id is not None
2754+
await asyncio.sleep(0.1)
2755+
assert not agent.cancel_called.is_set()
2756+
assert await handler._active_task_registry.get(agent.task_id) is not None
2757+
2758+
agent.release.set()
2759+
await _wait_for_task_state(
2760+
handler, agent.task_id, ctx, TaskState.TASK_STATE_COMPLETED
2761+
)
2762+
2763+
2764+
def _hanging_send_request(text: str) -> SendMessageRequest:
2765+
return SendMessageRequest(
2766+
message=Message(
2767+
message_id=f'msg-{text}',
2768+
role=Role.ROLE_USER,
2769+
parts=[Part(text=text)],
2770+
)
2771+
)
2772+
2773+
2774+
@pytest.mark.timeout(10)
2775+
@pytest.mark.asyncio
2776+
async def test_on_message_send_wait_for_times_out_while_task_works():
2777+
"""A caller timeout on a blocking send raises instead of returning the
2778+
unfinished task."""
2779+
agent = _HangingAgent()
2780+
handler = DefaultRequestHandlerV2(
2781+
agent_executor=agent,
2782+
task_store=InMemoryTaskStore(),
2783+
agent_card=create_default_agent_card(),
2784+
)
2785+
2786+
with pytest.raises(asyncio.TimeoutError):
2787+
await asyncio.wait_for(
2788+
handler.on_message_send(
2789+
_hanging_send_request('timeout'), create_server_call_context()
2790+
),
2791+
timeout=0.5,
2792+
)
2793+
2794+
agent.release.set()
2795+
await handler.aclose()
2796+
2797+
2798+
@pytest.mark.timeout(10)
2799+
@pytest.mark.asyncio
2800+
async def test_on_message_send_propagates_caller_cancellation():
2801+
"""Cancelling the task that awaits a blocking send cancels it, and only
2802+
it: the agent keeps running."""
2803+
agent = _ReleaseThenCompleteAgent()
2804+
handler = DefaultRequestHandlerV2(
2805+
agent_executor=agent,
2806+
task_store=InMemoryTaskStore(),
2807+
agent_card=create_default_agent_card(),
2808+
)
2809+
ctx = create_server_call_context()
2810+
2811+
send = asyncio.create_task(
2812+
handler.on_message_send(_hanging_send_request('cancel'), ctx)
2813+
)
2814+
await asyncio.wait_for(agent.working.wait(), timeout=5)
2815+
send.cancel()
2816+
2817+
with pytest.raises(asyncio.CancelledError):
2818+
await send
2819+
assert send.cancelled()
2820+
2821+
await _assert_task_survives(handler, agent, ctx)
2822+
await handler.aclose()
2823+
2824+
2825+
@pytest.mark.timeout(10)
2826+
@pytest.mark.asyncio
2827+
async def test_on_message_send_stream_propagates_consumer_cancellation():
2828+
"""Cancelling a stream consumer cancels it instead of ending the stream
2829+
as if the task had finished, and leaves the agent running."""
2830+
agent = _ReleaseThenCompleteAgent()
2831+
handler = DefaultRequestHandlerV2(
2832+
agent_executor=agent,
2833+
task_store=InMemoryTaskStore(),
2834+
agent_card=create_default_agent_card(),
2835+
)
2836+
ctx = create_server_call_context()
2837+
2838+
async def consume() -> None:
2839+
async for _ in handler.on_message_send_stream(
2840+
_hanging_send_request('stream'), ctx
2841+
):
2842+
pass
2843+
2844+
reader = asyncio.create_task(consume())
2845+
await asyncio.wait_for(agent.working.wait(), timeout=5)
2846+
reader.cancel()
2847+
2848+
with pytest.raises(asyncio.CancelledError):
2849+
await reader
2850+
assert reader.cancelled()
2851+
2852+
await _assert_task_survives(handler, agent, ctx)
2853+
await handler.aclose()
2854+
2855+
2856+
@pytest.mark.timeout(10)
2857+
@pytest.mark.asyncio
2858+
async def test_on_subscribe_to_task_propagates_consumer_cancellation():
2859+
"""Cancelling one resubscriber cancels only that consumer. The task and
2860+
the other subscriber keep going until the task completes."""
2861+
agent = _ReleaseThenCompleteAgent()
2862+
handler = DefaultRequestHandlerV2(
2863+
agent_executor=agent,
2864+
task_store=InMemoryTaskStore(),
2865+
agent_card=create_default_agent_card(),
2866+
)
2867+
ctx = create_server_call_context()
2868+
2869+
task_id = await _start_live_task(handler, ctx, 'resubscribe')
2870+
await asyncio.wait_for(agent.working.wait(), timeout=5)
2871+
2872+
def subscriber() -> tuple[asyncio.Task, list[Event], asyncio.Event]:
2873+
events: list[Event] = []
2874+
attached = asyncio.Event()
2875+
2876+
async def consume() -> None:
2877+
async for event in handler.on_subscribe_to_task(
2878+
SubscribeToTaskRequest(id=task_id), ctx
2879+
):
2880+
events.append(event)
2881+
attached.set()
2882+
2883+
return asyncio.create_task(consume()), events, attached
2884+
2885+
cancelled, _, cancelled_attached = subscriber()
2886+
other, other_events, other_attached = subscriber()
2887+
await asyncio.wait_for(cancelled_attached.wait(), timeout=5)
2888+
await asyncio.wait_for(other_attached.wait(), timeout=5)
2889+
2890+
cancelled.cancel()
2891+
with pytest.raises(asyncio.CancelledError):
2892+
await cancelled
2893+
assert cancelled.cancelled()
2894+
assert not other.done()
2895+
2896+
await _assert_task_survives(handler, agent, ctx)
2897+
2898+
await asyncio.wait_for(other, timeout=5)
2899+
last = other_events[-1]
2900+
assert isinstance(last, TaskStatusUpdateEvent)
2901+
assert last.status.state == TaskState.TASK_STATE_COMPLETED
2902+
2903+
await handler.aclose()
2904+
2905+
27242906
class _AsksForInputAgent(AgentExecutor):
27252907
"""Always asks for more input; counts execute() and cancel() calls."""
27262908

0 commit comments

Comments
 (0)