fix: Add workarounds for known WebKit automation bugs - #543
Merged
Merged
Conversation
…n quirks
Fixes several distinct bugs uncovered while chasing an intermittent hang in the
Automation session: a missing `domain`/`path`/`expires` on `Automation.addSingleCookie`
was silently wedging the Automation target's message queue, making every subsequent
call hang for minutes; `getWindowRect`/`computeElementLayout` could crash or return
`{0,0}` when WebKit's own response was incomplete; W3C Actions with an `Element`
origin threw `MoveTargetOutOfBoundsError` because WebKit's own element-origin
resolution is unreliable, worked around by resolving the on-screen center ourselves;
and `rpc-client.ts`'s `selectPage()` could hang forever if a page's target was
already initialized (its fallback timeout was only armed from inside the very event
handler it was meant to guard).
Also routes `sendKeys` and `click()` on checkbox/radio inputs through the bundled JS
atoms instead of native Automation-domain primitives - confirmed via cross-checking
against Apple's own safaridriver, on both iOS 27 beta and stable iOS 26.x, on
Simulator and real hardware, that native keyboard delivery and checkbox/radio touch
synthesis don't actually work there, independent of this driver's own implementation.
Adds a new per-API isolation e2e suite (not wired into CI) to make future regressions
in any single Automation API easier to isolate from the rest.
…ssion teardown AutomationSession.stop() enumerating and closing its owned browsing contexts is exactly where WebKit's Automation-domain connection can wedge for minutes with no response (https://bugs.webkit.org/show_bug.cgi?id=322937), turning a routine teardown into a multi-minute stall. forwardDidClose plus the caller's own reconnect already abandon these contexts either way, so this is now opt-in via stop({closeAllWindows: true}) / RemoteDebugger.stopAutomationSession({closeAllWindows: true}), off by default. Also references the filed WebKit bugs (322937/322938/322939) from the doc comments of the code paths that route around them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCYaAycq4xCnM2GSEgidWt
Generalize run-e2e-atoms.mjs into run-e2e.mjs, taking test file(s) as CLI args instead of a hardcoded atoms spec, and route all three e2e npm scripts through it for consistent CI timeout handling. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JiXQcr8YPYJ3C4YV6TuNr7
…okie The earlier "hang" was misdiagnosed as a getAllCookies-specific defect - it was actually the general Automation-domain connection wedge (https://bugs.webkit.org/show_bug.cgi?id=322937), reproduced identically via other commands too. Verified getAllCookies works correctly and quickly in isolation via safaridriver, so switch back to it: document.cookie could never see HttpOnly cookies or report domain/path/secure/sameSite/expiry metadata. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCYaAycq4xCnM2GSEgidWt
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JiXQcr8YPYJ3C4YV6TuNr7
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JiXQcr8YPYJ3C4YV6TuNr7
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JiXQcr8YPYJ3C4YV6TuNr7
Two real bugs conspired to make Automation.performInteractionSequence-based
click()/performW3CActions silently do nothing (no error, no effect):
1. Every state we sent omitted `mouseInteraction` (Down/Move/Up). Per
WebKit's own Automation.json, "if unmentioned and the interaction cannot
be determined through other heuristics, the state is dropped" - and
WebAutomationSession's C++ has no such heuristic, only its
WebDriver/Session.cpp reference client does. Verified byte-for-byte
against WebKit's own source and a live wire capture.
2. computeElementLayout was called with the legacy `LayoutViewport`
coordinate system ("should not be used going forward" per WebKit's own
docs) instead of the current `Viewport` system that
performInteractionSequence's own `origin: 'Viewport'` expects.
Also fixes the test fixture page, which had no <meta name="viewport"> tag -
Safari was rendering it at a fake 980x1741 desktop width instead of a real
mobile viewport, so even correctly-computed tap coordinates landed on
nothing. Adds a button e2e test that exercises raw performW3CActions
against a plain element (as opposed to the checkbox, whose checked-state
specifically still doesn't update via native touch - real, narrow WebKit
bug 322939 - left skipped with the bug reference).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JiXQcr8YPYJ3C4YV6TuNr7
…orks Native touch tapping correctly toggles a checkbox/radio's checked state once mouseInteraction and the Viewport coordinate system are set correctly (both fixed in the previous commit) - confirmed live, repeatedly, via both click() and raw performW3CActions. The WebKit bug 322939 reference was a misdiagnosis: it was actually this driver's own missing mouseInteraction field all along, not a genuine platform limitation. Removes the now-unreachable checkbox/radio branch in click() and the 'click' Automation atom it exclusively used (entry point, bundled output, and its entries in the atom name lists). Un-skips the corresponding e2e test, which now passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JiXQcr8YPYJ3C4YV6TuNr7
KazuCocoa
reviewed
Aug 30, 2026
3 tasks
- actions.ts: resolveElementOriginCenters now resolves every distinct element origin's on-screen center in two passes - scroll each into view first, then re-read all of them (without scrolling again) once that settles. A second element's scrollIntoView can move an already-resolved element's position before the interaction sequence is even dispatched (e.g. a drag between two far-apart elements); the old single-pass code cached a coordinate that could go stale relative to the page's final scroll position, silently touching the wrong spot. The two-pass version either gets it right or fails loudly with a genuine out-of-bounds error when the two elements can't both be reachable from one scroll position. - cookies.ts: addCookie now maps an omitted `expiry` to `expires: 0, session: true` (matching WebKit's own WebDriver adapter), instead of synthesizing a 400-day expiry and defaulting `session` to false - a WebDriver "session" cookie was outliving the browser session. - session.ts/windows.ts: evaluateJavaScriptFunction gained a `topLevelOnly` option; getWindowRect's DOM-fallback path now passes it, so it reads the top-level window's dimensions even when the session is currently switched into an iframe, instead of the iframe's own. - run-e2e.mjs: timeout-tolerance is now opt-in via `--tolerate-timeouts`, passed only for the atoms suite (which has intentionally-tolerated per-test timeouts). safari-e2e/automation-session-api don't carry that tolerance, so a genuine stall there still fails the job. - The checkable-input obscured-check comment is already resolved by the earlier removal of the checkbox/radio click-atom branch - click() now always runs the obscured check for every element type. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JiXQcr8YPYJ3C4YV6TuNr7
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JiXQcr8YPYJ3C4YV6TuNr7
KazuCocoa
approved these changes
Aug 30, 2026
| sessionId: this.sessionId, | ||
| ...params, | ||
| }), | ||
| this.commandTimeoutMs, |
Member
There was a problem hiding this comment.
[P1] Respect the configured page/script timeouts
This 120-second wrapper caps every Automation call, including navigation commands that pass the default pageLoadTimeoutMs of 300000 and executeAsyncScript when scriptTimeoutMs is configured above 120000. A healthy command that WebKit is still allowed to run will therefore be rejected locally two minutes early, making those public timeout settings ineffective. Please let call sites supply the effective timeout, or otherwise ensure the generic command cap cannot undercut a longer protocol-level timeout.
Contributor
Author
There was a problem hiding this comment.
This timeout is more like a safety net and should be not reached. Two minutes is a lot in a fast web world. I will think about customizing it if it ever becomes an issue
…ndable delta buildPointerTickState tracked a `pointer`-relative pointerMove's x/y as a raw delta in the per-source running state, then unconditionally re-emitted that same location/origin on every later sustain/pause/pointerDown tick (the same mechanism that correctly holds pressedButton/absolute locations). WebKit re-resolves a `Pointer`-origin state against its *current* pointer position on every state it receives, so each resend compounded the delta: a pointerMove(10,20, origin:pointer) -> pause -> pointerDown sequence would press at (30,60), not (10,20). Resolves the delta into an absolute viewport point immediately instead, so whatever gets held/resent afterward is a true no-op sustain, matching how element-origin and absolute-viewport locations already behave. Verified live against a real Simulator: a pointerdown dispatched after such a sequence now fires at the requested (10,20), not a drifted position. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JiXQcr8YPYJ3C4YV6TuNr7
github-actions Bot
pushed a commit
that referenced
this pull request
Aug 31, 2026
## [17.4.1](v17.4.0...v17.4.1) (2026-08-31) ### Bug Fixes * Add workarounds for known WebKit automation bugs ([#543](#543)) ([6f05db2](6f05db2))
Contributor
|
🎉 This PR is included in version 17.4.1 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
callAutomationnow bounds everyAutomation.*command with a configurablecommandTimeoutMs, converting a wedged connection (WebKit bug 322937) into a clearTimeoutErrorinstead of an indefinite hang.AutomationSession.stop()no longer closes owned browsing contexts by default — that enumerate-and-close step is exactly where the connection tends to wedge on teardown. Now opt-in viastop({closeAllWindows: true})/RemoteDebugger.stopAutomationSession({closeAllWindows: true}).Cookies: read via
Automation.getAllCookies(notdocument.cookie, which can't seeHttpOnlycookies or report domain/path/secure/sameSite/expiry).addCookiefills in all required fields before callingAutomation.addSingleCookie, avoiding a rejected call that could also wedge the connection.getWindowRectfalls back to reading the viewport off the DOM when WebKit reportswindowSizeas{0, 0}.Native touch tap/click was silently doing nothing. Root-caused two real bugs by reading WebKit's own source (
Automation.json,WebAutomationSession.cpp,SimulatedInputDispatcher.cpp,WebDriver/Session.cpp) and cross-checking against a live wire capture:performInteractionSequencestate we sent omittedmouseInteraction(Down/Move/Up). Per WebKit's own docs, "if unmentioned and the interaction cannot be determined through other heuristics, the state is dropped" — and there's no such heuristic in WebKit's C++, only in its reference WebDriver client. We were silently no-oping every tap.computeElementLayoutwas called with the legacyLayoutViewportcoordinate system ("should not be used going forward" per WebKit's own docs) instead ofViewport, which is whatperformInteractionSequence's ownorigin: 'Viewport'expects.With both fixed, native touch correctly clicks buttons/divs and toggles checkbox/radio state — the WebKit-bug-322939 checkbox workaround (routing clicks through a JS atom) was a misdiagnosis of the same root cause and has been removed, along with the now-dead atom it used.
Investigated
sendKeysthe same way and confirmed the opposite: its request shape already matches WebKit's reference client exactly, and tracingplatformSimulateKeySequenceinWebAutomationSessionIOS.mmshows it dispatches raw DOM keyboard events, bypassing theUITextInput/IME pipeline iOS Safari actually needs to insert text. That's a structural platform gap, not a fixable request bug — the JS-atom-basedsendKeys()stays as-is.W3C Actions with an
Elementorigin resolve the element's on-screen center themselves viacomputeElementLayout, instead of relying on WebKit's own unreliable element-origin resolution (MoveTargetOutOfBoundsErroreven for in-bounds elements).rpc-client.ts'sselectPage()fixed a bug where its fallback timeout could never fire if a page's target was already initialized.Generalized
scripts/ci/run-e2e-atoms.mjsintoscripts/ci/run-e2e.mjs, taking test file(s) as CLI args; all threee2e-test*npm scripts now route through it.Added a new per-API isolation e2e suite (
automation-session-api-e2e.spec.ts, not wired into CI) to make future regressions easier to isolate to a single Automation API.Known residual flakiness
performW3CActions/click()can intermittently throwTargetOutOfBoundsright after navigation — verified this is a transient WebKit/Simulator timing race (rejected coordinates are well within the viewport, and it reproduces on both our client-resolved and WebKit's own native element-origin resolution). Left unmitigated for now; a retry-on-TargetOutOfBoundswould be the natural fix if it becomes a real problem.Test plan
npm test) — 385 passingautomation-session-api-e2e.spec.tssuite run live against a real iOS Simulator (26.5 and 27.0)npm run e2e-test:atoms)