From a1cee02547b7aae4c72134902fe43ed007215069 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 07:35:03 -0800 Subject: [PATCH 01/15] docs: record Windows connected-session qualification Part of #4441 and #4435. Record the 505a2187b Windows Fabric / Win32 Paper qualification baseline, physical MCP request/response costs, per-case results, and explicit not-run reasons. Keep semantic activation and unregistered tool-window findings open while their fixes continue on this branch. Validation passed: - yarn markdown-link-check -c .github\markdown-link-check-config.json packages\agentic\desktop-driver\references\current-status.md - yarn markdown-link-check -c .github\markdown-link-check-config.json packages\agentic\desktop-driver-mcp\references\current-status.md - yarn markdown-link-check -c .github\markdown-link-check-config.json packages\agentic\desktop-test\references\current-status.md - yarn markdown-link-check -c .github\markdown-link-check-config.json packages\agentic\storybook-desktop\references\current-status.md - yarn markdown-link-check -c .github\markdown-link-check-config.json packages\agentic\storybook-desktop\references\README.md - yarn markdown-link-check -c .github\markdown-link-check-config.json .changeset\windows-connected-session-qualification.md - yarn oxfmt --check - yarn change:check (patch changeset added after the initial required-changeset failure) - yarn check-agent-instructions No desktop runs for this documentation-only update; full buildci not run. No push or PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- ...windows-connected-session-qualification.md | 10 ++ .../references/current-status.md | 96 +++++++++++++ .../references/current-status.md | 110 +++++++++++++++ .../desktop-test/references/current-status.md | 129 ++++++++++++++++++ .../storybook-desktop/references/README.md | 1 + .../references/current-status.md | 101 ++++++++++++++ 6 files changed, 447 insertions(+) create mode 100644 .changeset/windows-connected-session-qualification.md create mode 100644 packages/agentic/storybook-desktop/references/current-status.md diff --git a/.changeset/windows-connected-session-qualification.md b/.changeset/windows-connected-session-qualification.md new file mode 100644 index 0000000000..75b2dfefdc --- /dev/null +++ b/.changeset/windows-connected-session-qualification.md @@ -0,0 +1,10 @@ +--- +"@fluentui-react-native/desktop-driver": patch +"@fluentui-react-native/desktop-driver-mcp": patch +"@fluentui-react-native/desktop-test": patch +"@fluentui-react-native/storybook-desktop": patch +--- + +Record Windows Fabric and Win32 Paper connected-session qualification, +physical MCP click costs, explicit untested display scenarios, and the +remaining secondary-window semantic activation and registration gaps. diff --git a/packages/agentic/desktop-driver-mcp/references/current-status.md b/packages/agentic/desktop-driver-mcp/references/current-status.md index be788a2ea1..5c7d389741 100644 --- a/packages/agentic/desktop-driver-mcp/references/current-status.md +++ b/packages/agentic/desktop-driver-mcp/references/current-status.md @@ -338,3 +338,99 @@ out before the observed bridge retirement; the subsequent attempt waited for retirement. No raw image or grant is retained in this reference. Driver shutdown, app cleanup, and the failed Win32 app-run reports are recorded in `packages/agentic/desktop-test/references/current-status.md`. + +## Windows Fabric and Win32 Paper — #4441 physical MCP click measurement — 2026-11-12 UTC + +**Observed:** revision `505a2187b0a2451aed7081c9260b343d55742f93`, +Windows 11 Enterprise build 26300 x64, Node 26.13.0. Packaged Windows +Fabric reused the native build at `bdeb0b291` after confirming no native +implementation differences; Win32 Paper used REX 0.81.1. Metro/catalogs/ +bundles came from the tested checkout. Each run held the W3 campaign and +machine lock with `-RequirePhysical`. Before/after gates were eligible, +unlocked **remote RDP**, one 1728 x 1084 monitor at origin zero, 96 DPI / +100%; this was not console-session qualification. + +### Method and missing click cost + +Real MCP SDK **1.32.2** stdio connection through the private descriptor and +native-verified service; tokenizer **gpt-tokenizer 4.1.0 / o200k_base**. +After selecting `components-checkbox--default`, each endpoint made six +actual `click` actions, each followed by a checked-state assertion and an +**unscoped default observe**. Every receipt was applied/physical-pointer; +no semantic fallback counted. All six state assertions passed and the +checkbox ended unchecked. Each connection made 23 tool calls, zero error +responses/readiness rejections, and closed its session. These runs did not +exercise retries or cancellation. + +Discovery serialized the six-tool array using the November 8 baseline method: +**4,691 tokens / 20,277 UTF-8 bytes (characters)**, once per connection. +The full `tools/list` result envelope was separately 4,693 tokens / 20,287 +bytes; server instructions added 120 tokens / 591 bytes. Discovery has not +been omitted from the accounting. + +| Measurement, six samples per endpoint | Windows Fabric | Win32 Paper | +| ------------------------------------------------------ | -------------------------------------- | -------------------------------------- | +| Click response text, all blocks | **99–106 tokens / 313–314 bytes** | **96–103 tokens / 311–312 bytes** | +| Click request JSON, separately | 56–63 tokens / 186 bytes | 57–64 tokens / 186 bytes | +| Request + response-text pair | 155–169 tokens / 499–500 bytes | 153–167 tokens / 497–498 bytes | +| Click RPC p50 | **299.107 ms** | **308.003 ms** | +| Click RPC p90 / max | 369.976 ms | 318.208 ms | +| Click + checked check + default observe, response text | 1,999–2,005 tokens / 5,537–5,546 bytes | 1,585–1,592 tokens / 5,302–5,309 bytes | +| Same three RPCs, summed elapsed time | 411.849–550.601 ms | 431.774–473.477 ms | +| Default observe completeness | Incomplete: maxNodes, 13 omitted | Complete, no truncation | +| Default observe emitted / matched / visited | 100 / 113 / 115 | 99 / 99 / 124 | + +Response text blocks were independently encoded; request counts encode +user-facing `JSON.stringify({name, arguments})`, **not** JSON-RPC wire or +model-billing totals. Opaque identifiers cause normal token variation. +Percentiles use nearest rank; with six samples p90 is the maximum. +`rpcElapsedMs` stops immediately after SDK `callTool` resolves, before +tokenization, excluding service/app startup and discovery. Raw artifacts +also retain encoding-inclusive elapsed time: p50/max 299.192/370.036 ms +(Fabric), 308.128/318.281 ms (Paper). Earlier encoding-inclusive samples +were not silently substituted for this RPC distribution. + +Default budgets on both were deadline 3000 ms, maxBytes 12288, maxDepth 24, +maxNodes 100, maxStringLength 120, maxVisited 1500. No reduced scope or +budget was used to claim a saving. These six local samples complete the +previously missing physical-click row; they are not a controlled speedup +comparison with November 8's `activate` tuple or a platform ranking. + +### Commands, evidence and qualification limits + +Exact supervisor invocations from the qualification root, inside +`with-native-lock.ps1 -Lane W3 -RequirePhysical`: + +```powershell +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows --mcp-only --with-secondary +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs win32 --mcp-only +``` + +The retained supervisor owns declared Storybook driver/bundle/run commands, +exact app identity, MCP child connection and cleanup. It is local +qualification tooling, not a new published CLI surface. Artifacts are +relative to that checkout: +`artifacts\desktop\w3-qualification\runs\\mcp-click-.json`. + +| Endpoint | Qualification ID | Driver run / MCP session | Physical session gates, UTC | +| -------- | -------------------------------------- | ------------------------------------------------------------------------------- | --------------------------- | +| Fabric | `f188fae5-67c9-47e4-9800-ea584185fed9` | `57d4f6cb-2dc3-4b00-bb95-4331efd8d34b` / `b739a296-4647-4c34-bb80-a8f898fddab3` | 15:15:13.606–15:15:47.712 | +| Paper | `6d474fd4-61cb-41f6-824e-7a80df68db45` | `918930ba-e498-4e20-ad98-52e4187b5618` / `6df40ce1-32db-44d1-a9ba-64cc2dc6ec90` | 15:14:30.979–15:14:57.843 | + +Both MCP cases passed independently. Fabric's enclosing run also retained +a **failed** secondary semantic-effect assertion; its overall exit 1 is +not relabeled passed. A later exact-secondary physical click passed, but +semantic activation reproduced applied/accessibility with actual status +`Idle`. Win32's visible untitled tool-window popup was not separately +registered. These are **Observed** open gaps being fixed on this branch; +accessibility-pattern dispatch and the untitled-toolwindow registration +filter are **Inferred** causes, not proven fixes. + +200% was **not run** (100% scale; changes unauthorized), mixed DPI was +**not run** (one monitor), negative-origin layout was **not run** (only an +origin-zero monitor), and physical console Ctrl+C was **not run** (separate +implementation owner). Full per-case native results are in +`@fluentui-react-native/desktop-test/references/current-status.md`. +Sessions, applications and owned services drained; the campaign ended and +pre/post-end audits passed: +`FURN-AUDIT: no FURN desktop processes found`. diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index 507829248c..d5a83fed75 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -472,3 +472,113 @@ after the disconnect, so fresh-preview fencing remains Partial. The first probe timed out after 10 seconds; bridge retirement occurred about 19 seconds after injection. A later attempt waited for retirement and retained the same Partial result. Physical Ctrl+C and DPI changes were not exercised. + +## Windows Fabric and Win32 Paper — #4441 connected-session qualification — 2026-11-12 UTC + +**Observed baseline:** `505a2187b0a2451aed7081c9260b343d55742f93`, +Windows 11 Enterprise build 26300, x64, Node 26.13.0. All native runs held +the machine-wide campaign/lock; physical gates before and after each run +reported an active, unlocked **remote RDP** session, not a console session. +The single attached monitor was 1728 x 1084, origin `(0,0)`, 96 DPI / 100%. +No display settings changed. The packaged Fabric app's registered native +build was `bdeb0b291`; comparison of the Windows app project and native-core +sources to the tested SHA showed only native-core +version/changelog changes. **Inferred:** native-build reuse was valid. +Metro, generated catalogs and protected bundles came from the qualification +checkout. Paper used the prebuilt REX 0.81.1 host. + +### Matching-OS F16 clear contracts + +F16 is issue #4467, implemented by merged PR #4468, +`ac20660ff039f4e0dc914706ff24f548610f30aa`. From the qualification checkout, +with `$CampaignTools` identifying the private campaign helper directory: + +```powershell +$QualificationRoot = (Get-Location).Path +$env:TEMP = Join-Path $QualificationRoot 'artifacts\native-working' +$env:TMP = $env:TEMP +$env:TMPDIR = $env:TEMP +$env:FURN_NATIVE_DRIVER_TEST = '1' +pwsh -NoProfile -File "$CampaignTools\with-native-lock.ps1" ` + -Lane W3 -Label 'W3 qualification native strict-clear and fixture contracts' ` + -RequirePhysical ` + -WorkingDirectory (Join-Path $QualificationRoot 'packages\agentic\desktop-driver') -- ` + yarn test --runInBand nativeDriver win32Fixture operations.contract +``` + +**Passed:** 3 suites, 27 tests; **skipped:** 2 macOS-only signed/unsigned +build cases. This is one shared Windows-provider/Win32-fixture run, not 27 +tests per renderer. The real helper self-test negotiated `strictClear` and +covered semantic/keyboard mechanism selection and no-dispatch refusal; +the Windows native provider serves both endpoints. Helper artifact SHA-256: +`d8ee05e4796fb3aa22f7b5dbf481bd8cdfb0c20b6d5d24676d689dc7900d5b3c`. +The self-test's keyboard-denial instrumentation is not itself successful +React Native keyboard deletion. + +Separate live endpoint probes **passed** actual nonempty RN keyboard clear +(keyboard receipts 1 -> 2), accessible clear of an **initially empty** +TextInput, and readonly accessible clear refusal with effect `none` and +unchanged receipts. Nonempty semantic removal is native self-test evidence, +not a claim about the initially-empty RN probe. + +### Secondary targeting, capture and remaining gaps + +All rows below are **Observed** at the baseline SHA: + +| Case | Windows Fabric packaged | Win32 Paper REX | +| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | +| Separately registered secondary | Passed: exact new opaque handle, explicit switch, complete 11-node native observation, unique `modern-callout-focus`, physical-pointer click, `focus: confirmed`, then handle removal | Not run / partial: real Callout shown, but only main registered; no secondary targeting claim | +| Additional secondary semantic activation | Failed: applied/accessibility on the exact unique button, but six checks / 2090 ms and final native observation stayed `Idle` | Not run: no registered secondary | +| Minimized capture | Passed: `unsupported operation`, still minimized, no implicit restore; explicit restore then valid 586 x 793 capture | Passed: same refusal/restore contract; restored capture 1010 x 761 | +| 200% scale | Not run: 100% RDP scale; changing it was not authorized | Not run: same | +| Mixed DPI | Not run: only one attached monitor | Not run: same | +| Negative-origin layout | Not run: only an origin-zero monitor; no approved eligible layout | Not run: same | +| Physical console Ctrl+C | Not run: owned by the separate implementation lane | Not run: same | + +The semantic failure was reproduced **without** switching foreground back +to main: main status was read through its explicitly cited native root. +The same selector and status assertion passed with physical click in one +attempt / 79 ms. **Inferred:** the renderer/accessibility-pattern dispatch +path may acknowledge activation without reaching Pressable `onPress`. +`native/windows/src/automation.cpp` tries several patterns and accepts a +successful HRESULT; the precise selected pattern was not instrumented. +An applied receipt alone is not application-level effect proof. + +On Win32, exact-process native inventory **observed** a visible, untitled, +root `Net UI Tool Window` with ToolWindow true. **Inferred:** the deliberate +untitled `WS_EX_TOOLWINDOW` exclusion in `CollectWindow` explains the missing +registration. Modern Callout was absent from the actual Win32 catalog. +No native title/style manipulation was attempted. Both open findings are +being fixed on this connected-session branch; this entry records the +unfixed baseline, not successful fix qualification. + +Exact endpoint supervisor invocations, each inside the same W3 +`with-native-lock.ps1 -RequirePhysical` wrapper: + +```powershell +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs win32 +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows --secondary-only --modern-only +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows --secondary-only --modern-only --physical-secondary +``` + +Retained artifacts are relative to that qualification checkout, not shipped +package files: `artifacts\desktop\w3-qualification\runs\\`. + +| Evidence | Qualification ID | Report in that directory | +| --------------------------------------------------------------------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| Fabric clear / minimized capture | `5c97fc26-71ef-483c-aa2a-b454d6ff4458` | `descriptor-cases-windows-clear.json`, `descriptor-cases-windows-minimized-capture.json` | +| Paper clear / minimized capture / missing secondary | `22332065-6fa3-4061-a2b6-93f83295ca2e` | `descriptor-cases-win32-clear.json`, `descriptor-cases-win32-minimized-capture.json`, `descriptor-cases-win32-secondary.json` | +| Fabric semantic failure, driver run `a9f2e71e-71ab-40f0-a375-fd6d76a0b971` | `675160f7-e069-413c-a6c8-486de0c37b69` | `descriptor-cases-windows-secondary---modern.json` | +| Fabric physical secondary pass, driver run `16aac92b-bb36-4742-bd29-01df381d9594` | `7fbe3b0b-df33-476b-80e3-95b97bf19f02` | `descriptor-cases-windows-secondary---modern---physical.json` | + +The physical secondary pass does not erase the semantic failure or earlier +receipt-only/investigation failures. The full matrix and injected Win32 +fresh-preview fence are recorded in +`@fluentui-react-native/desktop-test/references/current-status.md`; real MCP +click costs are in +`@fluentui-react-native/desktop-driver-mcp/references/current-status.md`. +Exact owned applications, helpers, Metro and service processes drained. +The campaign ended; pre/post-end audits both exited 0: +`FURN-AUDIT: no FURN desktop processes found`. No physical Ctrl+C claim is +made from the supervisor's cooperative programmatic shutdown. diff --git a/packages/agentic/desktop-test/references/current-status.md b/packages/agentic/desktop-test/references/current-status.md index 5850b32c1d..5c9991bf20 100644 --- a/packages/agentic/desktop-test/references/current-status.md +++ b/packages/agentic/desktop-test/references/current-status.md @@ -752,3 +752,132 @@ not physical Ctrl+C. Both owned app/service processes exited, the recorded ports were free, and the per-user claim and default descriptor were absent. The separate `serve --agent` interrupt limitation above is fixed but not yet requalified on Windows. + +## Windows Fabric and Win32 Paper — #4441 connected-session qualification — 2026-11-12 UTC + +**Observed baseline:** `505a2187b0a2451aed7081c9260b343d55742f93`, +Windows 11 Enterprise build 26300 x64, Node 26.13.0. Native/GUI work ran +serially in the W3 campaign, 14:50:24–15:22:51 UTC, under the machine-wide +lock. Every before/after physical gate reported an active unlocked +**remote RDP** session. The attached display was one 1728 x 1084, +origin-zero, 96-DPI / 100% monitor; no display setting changed. + +The registered Fabric native build was `bdeb0b291`; the compared app/native +sources differed only in native-core version/changelog metadata, not Windows app/native +implementation. **Inferred:** reuse was valid. Own-checkout Metro, +generated manifests and protected bundles were used. Paper ran the prebuilt +REX 0.81.1 host. Each supervisor matched the newly launched app's exact +image, start time, title and window before probing it. + +### Per-case outcomes + +These are manual native/descriptor probes, not selected authored suite +cases. **Observed** results must not be inferred from supervisor exit alone: + +| Case | Windows Fabric packaged | Win32 Paper REX | +| ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Matching-OS F16 native contracts | Passed: shared Windows-provider/fixture run, 3 suites / 27 tests | Same shared run includes the real Win32 fixture; 2 macOS-only build cases skipped, not passes | +| Nonempty RN physical keyboard clear | Passed, keyboard receipts 1 -> 2 | Passed, keyboard receipts 1 -> 2 | +| Initially-empty accessible clear | Passed, semantic-clear receipt | Passed, semantic-clear receipt | +| Readonly accessible clear refusal | Passed, effect none, receipts unchanged | Passed, effect none, receipts unchanged | +| Separately registered secondary observation / interaction / cleanup | Passed: one exact new opaque handle, complete 11-node observation, unique control, physical-pointer click, `focus: confirmed`, then popup handle removed | Not run / partial: popup shown, but no separately registered handle | +| Additional secondary semantic activation effect | Failed: applied/accessibility receipt but actual main operation status stayed `Idle` | Not run: no registered secondary | +| Fresh native preview after lost provider context | Not requested | Passed: old preview rejected `ref-region-changed`; fresh unique native preview rejected `story-context-stale`; both effect none, not quarantined; state unchanged and old chrome ref survived | +| Physical MCP click measurement | Passed: 6 clicks and post-click state checks | Passed: 6 clicks and post-click state checks | +| Minimized capture and explicit restore | Passed: unsupported-operation refusal without implicit restore, then valid restored capture | Passed: same safety contract | +| 200% scale | Not run: 100% RDP scale; changing it not authorized | Not run: same | +| Mixed DPI | Not run: one attached monitor | Not run: same | +| Negative-origin layout | Not run: one origin-zero monitor; no approved eligible layout | Not run: same | +| Physical console Ctrl+C | Not run: separate implementation lane owns it | Not run: same | + +The native strict-clear self-test supplies nonempty **semantic** clear +evidence; the RN semantic endpoint probe started empty. Provider +keyboard-denial instrumentation is not substituted for the separate real +RN keyboard deletion. The native run skipped only the two macOS-specific +build/signing cases. + +### Commands and retained evidence + +Run from the qualification checkout. `$CampaignTools` is the separately +managed private campaign helper directory, not committed configuration: + +```powershell +$QualificationRoot = (Get-Location).Path +$env:FURN_NATIVE_DRIVER_TEST = '1' +pwsh -NoProfile -File "$CampaignTools\with-native-lock.ps1" ` + -Lane W3 -Label 'W3 qualification native strict-clear and fixture contracts' ` + -RequirePhysical ` + -WorkingDirectory (Join-Path $QualificationRoot 'packages\agentic\desktop-driver') -- ` + yarn test --runInBand nativeDriver win32Fixture operations.contract +``` + +Endpoint lifecycle commands below each ran within the same +`with-native-lock.ps1 -Lane W3 -RequirePhysical` scope. The local supervisor +awaited driver readiness, ran the declared bundle stage before app launch, +then owned probes and cleanup: + +```powershell +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs win32 +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs win32 --fence-only +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows --mcp-only --with-secondary +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs win32 --mcp-only +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows --secondary-only --modern-only +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows --secondary-only --modern-only --physical-secondary +``` + +Artifacts are retained locally under the qualification checkout at +`artifacts\desktop\w3-qualification\runs\\`, including +`qualification.json`, command logs and the named probe JSON. They are not +published package files. `artifacts\desktop\w3-qualification\qualification-summary.json` +is the machine-readable matrix. + +| Case / endpoint | Qualification ID | Probe JSON / driver run | +| ----------------------------------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| Clear and minimized, Fabric | `5c97fc26-71ef-483c-aa2a-b454d6ff4458` | `descriptor-cases-windows-clear.json`, `descriptor-cases-windows-minimized-capture.json` | +| Clear, minimized, unregistered secondary, Paper | `22332065-6fa3-4061-a2b6-93f83295ca2e` | `descriptor-cases-win32-clear.json`, `descriptor-cases-win32-minimized-capture.json`, `descriptor-cases-win32-secondary.json` | +| Fresh-preview fence, Paper | `60751352-c158-40a2-b4f9-9251e15fec95` | `descriptor-cases-win32-fence.json`; driver `64d2de39-df69-4a2b-a164-fc842f0ef64e` | +| MCP physical clicks, Fabric | `f188fae5-67c9-47e4-9800-ea584185fed9` | `mcp-click-windows.json`; driver `57d4f6cb-2dc3-4b00-bb95-4331efd8d34b` | +| MCP physical clicks, Paper | `6d474fd4-61cb-41f6-824e-7a80df68db45` | `mcp-click-win32.json`; driver `918930ba-e498-4e20-ad98-52e4187b5618` | +| Secondary semantic failure, Fabric | `675160f7-e069-413c-a6c8-486de0c37b69` | `descriptor-cases-windows-secondary---modern.json`; driver `a9f2e71e-71ab-40f0-a375-fd6d76a0b971` | +| Secondary physical pass, Fabric | `7fbe3b0b-df33-476b-80e3-95b97bf19f02` | `descriptor-cases-windows-secondary---modern---physical.json`; driver `16aac92b-bb36-4742-bd29-01df381d9594` | + +### Baseline failures, scope and cleanup + +**Observed:** Fabric semantic activation targeted the unique native popup +button and returned applied/accessibility, but six status checks over +2090 ms and a final native observation remained `Idle`. Main was read +through its cited root without changing foreground. The same test with +physical click passed in one attempt / 79 ms. **Inferred:** native +accessibility-pattern/renderer dispatch may acknowledge without executing +Pressable `onPress`; no exact pattern trace was captured. + +**Observed:** Paper's shown Callout produced a visible, untitled root +`Net UI Tool Window` with ToolWindow true, but no registered secondary. +Modern Callout was absent from its catalog. **Inferred:** the provider's +intentional untitled-toolwindow filter explains the registration gap. +Both findings are being fixed on this branch; neither is claimed fixed +by this baseline evidence. + +The Win32 fence deliberately invoked the **real** provider's bridge +retirement while preserving the socket/native preview. In one uninterrupted +session, old ref `hkbo1` was fenced, fresh unique ref `hkbo3` was fenced +in the next preview epoch, and chrome ref `hkbo2` remained usable for +presence checks. This proves injected provider-context loss, not a native +crash, real network disconnect/reconnect, or chrome mutation. + +Earlier missing-catalog/setup failures, invalid probe selectors/facts and +secondary receipt-only attempts remain in the run history; they are not +erased by independent case passes. Fabric's final MCP case passed inside +an enclosing run that failed its separate semantic-secondary assertion. +Win32 window close hid REX without ending it; the uniquely owned run CLI's +cooperative stop drained its process tree. This is not physical Ctrl+C +verification. No other lane's portable test/build workers were stopped. + +All owned apps, helpers, Metro/services, sessions and child grants drained. +Endpoint listeners were absent. Campaign end and both audits **passed**: +`FURN-AUDIT: no FURN desktop processes found`. Display-layout and console +qualification remain not run. MCP costs/budgets are recorded in +`@fluentui-react-native/desktop-driver-mcp/references/current-status.md`; +Storybook context details are in +`@fluentui-react-native/storybook-desktop/references/current-status.md`. diff --git a/packages/agentic/storybook-desktop/references/README.md b/packages/agentic/storybook-desktop/references/README.md index 2e0708a4c8..d03e1b48a4 100644 --- a/packages/agentic/storybook-desktop/references/README.md +++ b/packages/agentic/storybook-desktop/references/README.md @@ -18,3 +18,4 @@ owner_paths: | `storybook-desktop.win32` | [Win32](platforms/win32.md) | Bundling or running the REX host | | `storybook-desktop.troubleshooting` | [Troubleshooting](troubleshooting.md) | Diagnosing build, extraction, readiness, or smoke failures | | `storybook-desktop.contributing` | [Contributing](contributing.md) | Editing package code or tests | +| `storybook-desktop.current-status` | [Current status](current-status.md) | Checking endpoint evidence and remaining native gaps | diff --git a/packages/agentic/storybook-desktop/references/current-status.md b/packages/agentic/storybook-desktop/references/current-status.md new file mode 100644 index 0000000000..1cb5cfabe4 --- /dev/null +++ b/packages/agentic/storybook-desktop/references/current-status.md @@ -0,0 +1,101 @@ +--- +id: storybook-desktop.current-status +kind: instruction-reference +owner_paths: + - packages/agentic/storybook-desktop +--- + +# Native qualification status + +## Windows Fabric and Win32 Paper — #4441 connected-session qualification — 2026-11-12 UTC + +**Observed baseline:** `505a2187b0a2451aed7081c9260b343d55742f93`, +Windows 11 Enterprise build 26300 x64, Node 26.13.0. Packaged Windows +Fabric reused native build `bdeb0b291` after comparison found only +native-core metadata/version differences; **Inferred:** reuse was valid. +Win32 Paper used REX 0.81.1. Own-checkout Metro, manifests and protected +bundles served both endpoints. Native/GUI runs were serialized in the W3 +campaign with `-RequirePhysical`; before/after gates stayed eligible, +unlocked **remote RDP**, not console. One 1728 x 1084 origin-zero monitor +at 96 DPI / 100% was attached; settings were not changed. + +### Win32 fresh-preview fencing + +**Passed, Observed:** after a verified Checkbox selection, an injected +stimulus called the real orchestrator's bridge-retirement path while +deliberately preserving the socket and native preview. Old preview ref +`hkbo1` at epoch `(1,1)` rejected mutation with `ref-region-changed`. +A newly found, unique **native** checkbox ref `hkbo3` at epoch `(1,2)` +then rejected mutation with `story-context-stale`. Both were effect-none, +not quarantined; the checkbox remained unchecked. Pre-established chrome +ref `hkbo2` survived, and old/fresh chrome presence checks passed. + +This closes the earlier bounded fresh-preview-fence gap for **injected +provider-context loss**. It does not claim a native crash, actual socket +disconnect/reconnect, transparent upstream reconnection, chrome mutation, +or quarantine recovery. + +Exact local qualification invocation, inside the machine-wide +`with-native-lock.ps1 -Lane W3 -RequirePhysical` scope: + +```powershell +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs win32 --fence-only +``` + +Qualification ID `60751352-c158-40a2-b4f9-9251e15fec95`, driver run +`64d2de39-df69-4a2b-a164-fc842f0ef64e`, session +`08419564-7f43-48d2-9e08-b0f971590a95`. Artifacts relative to the tested +qualification checkout: +`artifacts\desktop\w3-qualification\runs\60751352-c158-40a2-b4f9-9251e15fec95\descriptor-cases-win32-fence.json` +and `qualification.json` in the same directory. The harness explicitly read +current-story state before injecting loss. These retained local scripts +are not a new published Storybook CLI or supported fault-injection API. + +### Secondary-window observations and open fixes + +| Case | Observed outcome | +| ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Fabric separately registered modern Callout | Passed: exact distinct opaque handle, switch, complete 11-node observation, unique `modern-callout-focus`, actual physical-pointer click, `focus: confirmed`, then popup removal | +| Fabric secondary semantic activation | Failed: exact same control returned applied/accessibility, but main status stayed `Idle` over six checks / 2090 ms and final native observation | +| Paper separately registered Callout | Not run / partial: legacy popup shown, but only main registered; modern Callout absent from actual catalog | +| 200% | Not run: current scale 100%; changing it unauthorized | +| Mixed DPI | Not run: one attached monitor | +| Negative-origin layout | Not run: origin-zero monitor only; no approved eligible layout | +| Physical console Ctrl+C | Not run: separate implementation lane owns it | + +The Fabric status assertion read main through its explicit native root +without switching foreground; physical click passed once / 79 ms. +**Inferred:** accessibility-pattern/renderer dispatch may acknowledge an +activation without reaching Pressable `onPress`; the selected pattern was +not instrumented. Paper native inventory **observed** a visible untitled +root `Net UI Tool Window`, ToolWindow true. **Inferred:** the driver's +untitled `WS_EX_TOOLWINDOW` filter prevents its registration. These two +gaps are being fixed on this branch; this entry does not qualify the fixes. + +Exact bounded follow-ups, under the same physical-gated campaign wrapper: + +```powershell +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows --secondary-only --modern-only +node .\artifacts\desktop\w3-qualification\run-endpoint.mjs windows --secondary-only --modern-only --physical-secondary +``` + +Semantic failure: qualification `675160f7-e069-413c-a6c8-486de0c37b69`, +driver `a9f2e71e-71ab-40f0-a375-fd6d76a0b971`; report +`artifacts\desktop\w3-qualification\runs\675160f7-e069-413c-a6c8-486de0c37b69\descriptor-cases-windows-secondary---modern.json`. +Physical pass: qualification `7fbe3b0b-df33-476b-80e3-95b97bf19f02`, +driver `16aac92b-bb36-4742-bd29-01df381d9594`; same artifact-root layout, +`descriptor-cases-windows-secondary---modern---physical.json`. +Paper inventory: qualification `22332065-6fa3-4061-a2b6-93f83295ca2e`, +`descriptor-cases-win32-secondary.json`. + +The supervisor ran supported driver/bundle/run stages in order, bundling +before launch and using actual readiness identity. Earlier setup, +unsupported-story, invalid-selector/fact and receipt-only failures remain +retained, not silently counted as passes. Session/child grants drained, +the exact owned apps/services stopped, and the W3 campaign ended. +Pre/post-end audits passed: +`FURN-AUDIT: no FURN desktop processes found`. +The complete native clear/capture/MCP matrix is in +`@fluentui-react-native/desktop-test/references/current-status.md`; +MCP response costs and default-observe completeness are in +`@fluentui-react-native/desktop-driver-mcp/references/current-status.md`. From cd036a2d06ae450f9c26ac0837c4fe140877ffb9 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 09:25:31 -0800 Subject: [PATCH 02/15] fix: qualify Windows activation and secondary content windows Refuse legacy-only UIA activation before dispatch and preserve public native failure metadata without delivery, quarantine, or implicit physical fallback. Admit untitled tool popups only with verified lease ownership, exact native identity, and bounded UIA content; retain conservative global discovery. Add native red/green fixtures and stable legacy Callout command identities. Record qualified Fabric/Paper results and the remaining session-blocked runs. Part of #4441 Part of #4435 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- ...ative-callout-window-command-identities.md | 5 + .../windows-connected-session-refusals.md | 7 + .../desktop-driver/fixtures/win32/README.md | 53 +++-- .../fixtures/win32/Win32Fixture.vcxproj | 4 +- .../fixtures/win32/src/fixture.cpp | 213 +++++++++++++++++- .../agentic/desktop-driver/native/PROTOCOL.md | 16 ++ .../desktop-driver/native/windows/README.md | 26 ++- .../native/windows/src/application.cpp | 72 +++++- .../native/windows/src/application.h | 4 +- .../native/windows/src/automation.cpp | 70 ++++-- .../native/windows/src/automation.h | 4 + .../native/windows/src/driver.cpp | 4 +- .../references/current-status.md | 167 ++++++++++++++ .../references/native-helpers.md | 20 ++ .../desktop-driver/references/operations.md | 6 + .../references/remaining-work.md | 22 ++ .../hosts/native/NativeDesktopHost.test.ts | 18 ++ .../src/hosts/native/NativeDesktopHost.ts | 2 +- .../windows/operations.contract.test.ts | 24 ++ .../windows/win32Fixture.contract.test.ts | 43 ++++ .../desktop-driver/src/protocol/errors.ts | 7 +- .../src/server/operations.test.ts | 34 +++ .../native-core/references/contributing.md | 5 + .../src/legacy/callout/Callout.stories.tsx | 14 +- 24 files changed, 778 insertions(+), 62 deletions(-) create mode 100644 .changeset/native-callout-window-command-identities.md create mode 100644 .changeset/windows-connected-session-refusals.md diff --git a/.changeset/native-callout-window-command-identities.md b/.changeset/native-callout-window-command-identities.md new file mode 100644 index 0000000000..5cddcd2c12 --- /dev/null +++ b/.changeset/native-callout-window-command-identities.md @@ -0,0 +1,5 @@ +--- +'@fluentui-react-native/native-core': patch +--- + +Give the legacy Callout window-command story's focus and blur buttons stable test identities for unique secondary-window input qualification. diff --git a/.changeset/windows-connected-session-refusals.md b/.changeset/windows-connected-session-refusals.md new file mode 100644 index 0000000000..3617d4d256 --- /dev/null +++ b/.changeset/windows-connected-session-refusals.md @@ -0,0 +1,7 @@ +--- +'@fluentui-react-native/desktop-driver': patch +--- + +Refuse Windows legacy-only semantic activation before dispatch, preserving machine-readable reasons without recording a delivery or implicitly retrying with physical input. + +Register untitled Windows secondary tool popups only when their exact-process ownership and bounded UIA content are verified, retaining helper-window exclusions. diff --git a/packages/agentic/desktop-driver/fixtures/win32/README.md b/packages/agentic/desktop-driver/fixtures/win32/README.md index 52b2b7da62..5ac1b25d87 100644 --- a/packages/agentic/desktop-driver/fixtures/win32/README.md +++ b/packages/agentic/desktop-driver/fixtures/win32/README.md @@ -29,14 +29,16 @@ pointer and keyboard input. ## Options -| Option | Default | Effect | -| ----------------------- | -------------------- | ---------------------------------------------------------------------------------- | -| `--title ` | `FURN Win32 Fixture` | Main window title; the contract passes a unique value per scenario. | -| `--nodes ` | `0` | Adds a tree view with this many `Item N` entries. | -| `--hang-ms ` | `20000` | How long the **Hang** button blocks the UI thread. | -| `--stuck-ms ` | `0` (no control) | Adds a control whose UI Automation `Invoke` blocks this long on a provider thread. | -| `--offscreen <0\|1>` | `0` | Places the main window off every monitor. | -| `--lifetime-ms ` | `600000` | Exits after this long even when the UI thread is blocked; `0` never exits. | +| Option | Default | Effect | +| ---------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------- | +| `--title ` | `FURN Win32 Fixture` | Main window title; the contract passes a unique value per scenario. | +| `--nodes ` | `0` | Adds a tree view with this many `Item N` entries. | +| `--hang-ms ` | `20000` | How long the **Hang** button blocks the UI thread. | +| `--stuck-ms ` | `0` (no control) | Adds a control whose UI Automation `Invoke` blocks this long on a provider thread. | +| `--no-default-action <0\|1>` | `0` | Adds MSAA and passive-UIA controls that acknowledge a legacy default action without an actionable UIA pattern. | +| `--tool-windows <0\|1>` | `0` | Adds a button opening owned application-content and excluded helper-like tool roots. | +| `--offscreen <0\|1>` | `0` | Places the main window off every monitor. | +| `--lifetime-ms ` | `600000` | Exits after this long even when the UI thread is blocked; `0` never exits. | The lifetime watchdog is the only cleanup guarantee for an abandoned run: the fixture is not in a Job Object. @@ -44,18 +46,22 @@ The lifetime watchdog is the only cleanup guarantee for an abandoned run: the fi The window is shown without activation. Ids are `AutomationId` values. -| Id | Control | Behavior | -| ----------- | ------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | -| 1004 | Status label, initially `Ready` | Reports results: `Activated N times`, `Hang finished`, `Stuck action applied N times`. | -| 1001 | **Activate** button | Counts activations in the status label. | -| 1005 | **Rename window** button | Appends ` - renamed N` to the title; the helper window id is unchanged. | -| 1006 | **Open dialog** button | Opens the owned popup `Fixture dialog` (label 3002 `Dialog open`, button 3001 `Dialog OK`). | -| 1007 | **Hang** button | Blocks the UI thread for `--hang-ms`. | -| 1008, 1002 | `Name:` label and edit | Keyboard and `clear` target. | -| 1003 | **Enabled** checkbox | Toggle pattern target. | -| 1009 | **Stuck action** (`--stuck-ms` only) | `Invoke` blocks on a provider thread, then applies once. | -| 1010 | `Items` tree view (`--nodes` only) | `Item N` entries for tree-size and retention scenarios. | -| 2001 - 2003 | Menu bar `File` (`Increment`, `Exit`), `Help` (`About`) | Menu items in the UI Automation tree. Opening a menu popup needs activation, so popups are not qualified. | +| Id | Control | Behavior | +| ----------- | ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | +| 1004 | Status label, initially `Ready` | Reports results: `Activated N times`, `Hang finished`, `Stuck action applied N times`. | +| 1001 | **Activate** button | Counts activations in the status label. | +| 1005 | **Rename window** button | Appends ` - renamed N` to the title; the helper window id is unchanged. | +| 1006 | **Open dialog** button | Opens the owned popup `Fixture dialog` (label 3002 `Dialog open`, button 3001 `Dialog OK`). | +| 1007 | **Hang** button | Blocks the UI thread for `--hang-ms`. | +| 1008, 1002 | `Name:` label and edit | Keyboard and `clear` target. | +| 1003 | **Enabled** checkbox | Toggle pattern target. | +| 1009 | **Stuck action** (`--stuck-ms` only) | `Invoke` blocks on a provider thread, then applies once. | +| 1010 | `Items` tree view (`--nodes` only) | `Item N` entries for tree-size and retention scenarios. | +| 1011 | `No default action` (`--no-default-action` only) | Advertises no default action; an attempted dispatch changes the status to `Nonactionable default dispatched`. | +| 1012 | **Open tool windows** (`--tool-windows` only) | Opens seven untitled tool roots; only the main-owned, visible, positive-size content root is eligible for registration. | +| 1013 | `No action pattern` (`--no-default-action` only) | Advertises only `ScrollIntoView`; a legacy default-action acknowledgment must not count as activation. | +| 3101, 3102 | **Tool action**, **Close tool** | Content-window activation updates the main status; closing retires the secondary identity. | +| 2001 - 2003 | Menu bar `File` (`Increment`, `Exit`), `Help` (`About`) | Menu items in the UI Automation tree. Opening a menu popup needs activation, so popups are not qualified. | The default tree has 18 nodes; `--nodes N` yields about `N + 24` in total. @@ -68,6 +74,11 @@ The default tree has 18 nodes; `--nodes N` yields about `N + 24` in total. - **Title change:** an asynchronous rename keeps the window id and publishes `EVENT_OBJECT_NAMECHANGE`. - **Owned popup:** the owner rejects clicks as `element-not-interactable` while the dialog is open; closing the dialog publishes `window-closed` and `EVENT_OBJECT_DESTROY`. +- **Legacy-only activation:** both an MSAA control without default-action metadata and a native UIA button exposing + only `ScrollIntoView` are refused before dispatch, without quarantine; an ordinary Invoke remains usable. +- **Untitled tool roots:** only visible, positive-size application content owned by the main window is registered. + Unowned roots, unrelated owner chains, empty content, hidden roots, zero-size roots, and tooltip roots stay excluded. + Closing the content root retires its identity without affecting the main window. - **Hung provider (`--hang-ms 14000`):** reads and clicks fail as `automation-failed`, windows stay listable, and the helper recovers once the app responds. - **Stuck provider (`--stuck-ms 8000`):** the first click settles as `mutation-indeterminate`, later guarded mutations @@ -101,6 +112,8 @@ operation routes rather than raw helper commands: and journals without action names or typed content. - A stuck named Invoke that quarantines later operations even after its late application effect is observed exactly once. +- A no-action activation refusal preserves its machine reason, proves no effect, and leaves input receipts and + subsequent legitimate activation intact. Physical click and crop alignment have been checked at 100% and 150% scale. Other DPI configurations and React Native applications remain separate diff --git a/packages/agentic/desktop-driver/fixtures/win32/Win32Fixture.vcxproj b/packages/agentic/desktop-driver/fixtures/win32/Win32Fixture.vcxproj index c2ada0f24d..604212bfe3 100644 --- a/packages/agentic/desktop-driver/fixtures/win32/Win32Fixture.vcxproj +++ b/packages/agentic/desktop-driver/fixtures/win32/Win32Fixture.vcxproj @@ -44,7 +44,7 @@ UNICODE;_UNICODE;%(PreprocessorDefinitions) - user32.lib;shell32.lib;comctl32.lib;uiautomationcore.lib;oleaut32.lib;ole32.lib;%(AdditionalDependencies) + user32.lib;shell32.lib;comctl32.lib;uiautomationcore.lib;oleacc.lib;oleaut32.lib;ole32.lib;%(AdditionalDependencies) Windows @@ -57,7 +57,7 @@ UNICODE;_UNICODE;NDEBUG;%(PreprocessorDefinitions) - user32.lib;shell32.lib;comctl32.lib;uiautomationcore.lib;oleaut32.lib;ole32.lib;%(AdditionalDependencies) + user32.lib;shell32.lib;comctl32.lib;uiautomationcore.lib;oleacc.lib;oleaut32.lib;ole32.lib;%(AdditionalDependencies) Windows diff --git a/packages/agentic/desktop-driver/fixtures/win32/src/fixture.cpp b/packages/agentic/desktop-driver/fixtures/win32/src/fixture.cpp index 131a6f2401..084ce3c06f 100644 --- a/packages/agentic/desktop-driver/fixtures/win32/src/fixture.cpp +++ b/packages/agentic/desktop-driver/fixtures/win32/src/fixture.cpp @@ -14,12 +14,15 @@ // --child-pid-file start one offscreen descendant copy of this fixture and // write " " to this file, so a // contract can prove which processes an owned launch reaches +// --no-default-action <0|1> add an MSAA control whose no-op default method returns S_OK +// --tool-windows <0|1> add a button that opens content and non-content tool roots #define NOMINMAX #define WIN32_LEAN_AND_MEAN #include #include #include +#include #include #include @@ -31,12 +34,15 @@ #include #include #include +#include namespace { constexpr wchar_t kMainClass[] = L"FurnWin32Fixture"; constexpr wchar_t kDialogClass[] = L"FurnWin32FixtureDialog"; constexpr wchar_t kStuckClass[] = L"FurnWin32FixtureStuck"; +constexpr wchar_t kNoActionClass[] = L"FurnWin32FixtureNoAction"; +constexpr wchar_t kToolClass[] = L"FurnWin32FixtureTool"; constexpr int kActivateButton = 1001; constexpr int kNameEdit = 1002; @@ -48,12 +54,19 @@ constexpr int kHangButton = 1007; constexpr int kNameLabel = 1008; constexpr int kStuckControl = 1009; constexpr int kTree = 1010; +constexpr int kNoActionControl = 1011; +constexpr int kToolButton = 1012; +constexpr int kPassiveControl = 1013; constexpr int kMenuIncrement = 2001; constexpr int kMenuExit = 2002; constexpr int kMenuAbout = 2003; constexpr int kDialogOk = 3001; constexpr int kDialogLabel = 3002; +constexpr int kToolAction = 3101; +constexpr int kToolClose = 3102; constexpr UINT kStuckApplied = WM_APP + 1; +constexpr UINT kNoActionDispatched = WM_APP + 2; +constexpr UINT kToolApplied = WM_APP + 3; struct Options { std::wstring title = L"FURN Win32 Fixture"; @@ -62,6 +75,8 @@ struct Options { DWORD stuckMilliseconds = 0; DWORD lifetimeMilliseconds = 600000; bool offscreen = false; + bool noDefaultAction = false; + bool toolWindows = false; std::wstring childPidFile; std::wstring stuckStartedFile; }; @@ -73,6 +88,8 @@ HWND dialogWindow = nullptr; int activations = 0; int renames = 0; int stuckApplications = 0; +int toolActivations = 0; +std::vector toolWindows; Options ParseOptions() { Options parsed; @@ -100,6 +117,10 @@ Options ParseOptions() { parsed.offscreen = _wtoi(value) != 0; } else if (name == L"--child-pid-file") { parsed.childPidFile = value; + } else if (name == L"--no-default-action") { + parsed.noDefaultAction = _wtoi(value) != 0; + } else if (name == L"--tool-windows") { + parsed.toolWindows = _wtoi(value) != 0; } } LocalFree(arguments); @@ -158,9 +179,9 @@ void CreateMenuBar(HWND window) { // Automation thread, so the UI thread stays responsive, and applies its effect // only when the wait ends: a mutation whose outcome the client cannot know, // like a wedged or very slow application. -class StuckProvider : public IRawElementProviderSimple, public IInvokeProvider { +class StuckProvider : public IRawElementProviderSimple, public IInvokeProvider, public IScrollItemProvider { public: - explicit StuckProvider(HWND window) : window_(window) {} + explicit StuckProvider(HWND window) : window_(window), passive_(GetDlgCtrlID(window) == kPassiveControl) {} ULONG STDMETHODCALLTYPE AddRef() override { return ++references_; } @@ -180,6 +201,8 @@ class StuckProvider : public IRawElementProviderSimple, public IInvokeProvider { *object = static_cast(this); } else if (id == __uuidof(IInvokeProvider)) { *object = static_cast(this); + } else if (id == __uuidof(IScrollItemProvider) && passive_) { + *object = static_cast(this); } else { *object = nullptr; return E_NOINTERFACE; @@ -201,9 +224,12 @@ class StuckProvider : public IRawElementProviderSimple, public IInvokeProvider { return E_POINTER; } *provider = nullptr; - if (pattern == UIA_InvokePatternId) { + if (pattern == UIA_InvokePatternId && !passive_) { *provider = static_cast(this); AddRef(); + } else if (pattern == UIA_ScrollItemPatternId && passive_) { + *provider = static_cast(this); + AddRef(); } return S_OK; } @@ -218,10 +244,10 @@ class StuckProvider : public IRawElementProviderSimple, public IInvokeProvider { value->lVal = UIA_ButtonControlTypeId; } else if (property == UIA_NamePropertyId) { value->vt = VT_BSTR; - value->bstrVal = SysAllocString(L"Stuck action"); + value->bstrVal = SysAllocString(passive_ ? L"No action pattern" : L"Stuck action"); } else if (property == UIA_AutomationIdPropertyId) { value->vt = VT_BSTR; - value->bstrVal = SysAllocString(std::to_wstring(kStuckControl).c_str()); + value->bstrVal = SysAllocString(std::to_wstring(passive_ ? kPassiveControl : kStuckControl).c_str()); } return S_OK; } @@ -234,6 +260,10 @@ class StuckProvider : public IRawElementProviderSimple, public IInvokeProvider { } HRESULT STDMETHODCALLTYPE Invoke() override { + if (passive_) { + PostMessageW(mainWindow, kNoActionDispatched, 0, 0); + return S_OK; + } if (!options.stuckStartedFile.empty()) { const HANDLE started = CreateFileW(options.stuckStartedFile.c_str(), GENERIC_WRITE, FILE_SHARE_READ, nullptr, CREATE_NEW, FILE_ATTRIBUTE_NORMAL, nullptr); @@ -247,9 +277,12 @@ class StuckProvider : public IRawElementProviderSimple, public IInvokeProvider { return S_OK; } + HRESULT STDMETHODCALLTYPE ScrollIntoView() override { return S_OK; } + private: std::atomic references_{1}; HWND window_; + bool passive_; }; LRESULT CALLBACK StuckProc(HWND window, UINT message, WPARAM wparam, LPARAM lparam) { @@ -269,7 +302,8 @@ LRESULT CALLBACK StuckProc(HWND window, UINT message, WPARAM wparam, LPARAM lpar GetClientRect(window, &bounds); FrameRect(context, &bounds, static_cast(GetStockObject(BLACK_BRUSH))); SetBkMode(context, TRANSPARENT); - DrawTextW(context, L"Stuck action", -1, &bounds, DT_CENTER | DT_VCENTER | DT_SINGLELINE); + DrawTextW(context, GetDlgCtrlID(window) == kPassiveControl ? L"No action pattern" : L"Stuck action", -1, &bounds, + DT_CENTER | DT_VCENTER | DT_SINGLELINE); EndPaint(window, &paint); return 0; } @@ -279,6 +313,150 @@ LRESULT CALLBACK StuckProc(HWND window, UINT message, WPARAM wparam, LPARAM lpar return DefWindowProcW(window, message, wparam, lparam); } +// Keep the normal MSAA properties without an actionable pattern. The default +// action acknowledges a call without effect, exercising the driver's refusal +// of legacy-only activation. +class NoActionAccessible : public IAccessible { + public: + explicit NoActionAccessible(IAccessible* standard) : standard_(standard) {} + ~NoActionAccessible() { standard_->Release(); } + + ULONG STDMETHODCALLTYPE AddRef() override { return ++references_; } + ULONG STDMETHODCALLTYPE Release() override { + const ULONG remaining = --references_; + if (remaining == 0) { + delete this; + } + return remaining; + } + HRESULT STDMETHODCALLTYPE QueryInterface(REFIID id, void** object) override { + if (object == nullptr) return E_POINTER; + *object = nullptr; + if (id != IID_IUnknown && id != IID_IDispatch && id != IID_IAccessible) return E_NOINTERFACE; + *object = static_cast(this); + AddRef(); + return S_OK; + } + HRESULT STDMETHODCALLTYPE GetTypeInfoCount(UINT* count) override { return standard_->GetTypeInfoCount(count); } + HRESULT STDMETHODCALLTYPE GetTypeInfo(UINT index, LCID locale, ITypeInfo** info) override { + return standard_->GetTypeInfo(index, locale, info); + } + HRESULT STDMETHODCALLTYPE GetIDsOfNames(REFIID id, LPOLESTR* names, UINT count, LCID locale, DISPID* ids) override { + return standard_->GetIDsOfNames(id, names, count, locale, ids); + } + HRESULT STDMETHODCALLTYPE Invoke(DISPID member, REFIID id, LCID locale, WORD flags, DISPPARAMS* params, + VARIANT* result, EXCEPINFO* exception, UINT* argument) override { + return standard_->Invoke(member, id, locale, flags, params, result, exception, argument); + } + HRESULT STDMETHODCALLTYPE get_accParent(IDispatch** parent) override { return standard_->get_accParent(parent); } + HRESULT STDMETHODCALLTYPE get_accChildCount(long* count) override { return standard_->get_accChildCount(count); } + HRESULT STDMETHODCALLTYPE get_accChild(VARIANT child, IDispatch** result) override { + return standard_->get_accChild(child, result); + } + HRESULT STDMETHODCALLTYPE get_accName(VARIANT child, BSTR* result) override { return standard_->get_accName(child, result); } + HRESULT STDMETHODCALLTYPE get_accValue(VARIANT child, BSTR* result) override { return standard_->get_accValue(child, result); } + HRESULT STDMETHODCALLTYPE get_accDescription(VARIANT child, BSTR* result) override { + return standard_->get_accDescription(child, result); + } + HRESULT STDMETHODCALLTYPE get_accRole(VARIANT, VARIANT* result) override { + if (result == nullptr) return E_POINTER; + VariantInit(result); + result->vt = VT_I4; + result->lVal = ROLE_SYSTEM_STATICTEXT; + return S_OK; + } + HRESULT STDMETHODCALLTYPE get_accState(VARIANT child, VARIANT* result) override { return standard_->get_accState(child, result); } + HRESULT STDMETHODCALLTYPE get_accHelp(VARIANT child, BSTR* result) override { return standard_->get_accHelp(child, result); } + HRESULT STDMETHODCALLTYPE get_accHelpTopic(BSTR* file, VARIANT child, long* topic) override { + return standard_->get_accHelpTopic(file, child, topic); + } + HRESULT STDMETHODCALLTYPE get_accKeyboardShortcut(VARIANT child, BSTR* result) override { + return standard_->get_accKeyboardShortcut(child, result); + } + HRESULT STDMETHODCALLTYPE get_accFocus(VARIANT* result) override { return standard_->get_accFocus(result); } + HRESULT STDMETHODCALLTYPE get_accSelection(VARIANT* result) override { return standard_->get_accSelection(result); } + HRESULT STDMETHODCALLTYPE get_accDefaultAction(VARIANT, BSTR* result) override { + if (result == nullptr) return E_POINTER; + *result = nullptr; + return S_FALSE; + } + HRESULT STDMETHODCALLTYPE accSelect(long flags, VARIANT child) override { return standard_->accSelect(flags, child); } + HRESULT STDMETHODCALLTYPE accLocation(long* x, long* y, long* width, long* height, VARIANT child) override { + return standard_->accLocation(x, y, width, height, child); + } + HRESULT STDMETHODCALLTYPE accNavigate(long direction, VARIANT from, VARIANT* result) override { + return standard_->accNavigate(direction, from, result); + } + HRESULT STDMETHODCALLTYPE accHitTest(long x, long y, VARIANT* result) override { return standard_->accHitTest(x, y, result); } + HRESULT STDMETHODCALLTYPE accDoDefaultAction(VARIANT) override { + PostMessageW(mainWindow, kNoActionDispatched, 0, 0); + return S_OK; + } + HRESULT STDMETHODCALLTYPE put_accName(VARIANT child, BSTR value) override { return standard_->put_accName(child, value); } + HRESULT STDMETHODCALLTYPE put_accValue(VARIANT child, BSTR value) override { return standard_->put_accValue(child, value); } + + private: + std::atomic references_{1}; + IAccessible* standard_; +}; + +LRESULT CALLBACK NoActionProc(HWND window, UINT message, WPARAM wparam, LPARAM lparam) { + if (message == WM_GETOBJECT && static_cast(lparam) == OBJID_CLIENT) { + IAccessible* standard = nullptr; + if (SUCCEEDED(CreateStdAccessibleObject(window, OBJID_CLIENT, IID_IAccessible, reinterpret_cast(&standard)))) { + auto* accessible = new NoActionAccessible(standard); + const LRESULT result = LresultFromObject(IID_IAccessible, wparam, accessible); + accessible->Release(); + return result; + } + } + return DefWindowProcW(window, message, wparam, lparam); +} + +LRESULT CALLBACK ToolProc(HWND window, UINT message, WPARAM wparam, LPARAM lparam) { + if (message == WM_COMMAND) { + if (LOWORD(wparam) == kToolAction) { + PostMessageW(mainWindow, kToolApplied, 0, 0); + return 0; + } + if (LOWORD(wparam) == kToolClose) { + DestroyWindow(window); + return 0; + } + } + return DefWindowProcW(window, message, wparam, lparam); +} + +void OpenToolWindows(HWND owner) { + if (!toolWindows.empty()) return; + INITCOMMONCONTROLSEX controls{sizeof(INITCOMMONCONTROLSEX), ICC_WIN95_CLASSES}; + InitCommonControlsEx(&controls); + RECT bounds{}; + GetWindowRect(owner, &bounds); + const auto create = [&](HWND parent, bool content, bool visible, bool empty, const wchar_t* className = kToolClass) { + const HWND tool = CreateWindowExW(WS_EX_TOOLWINDOW | WS_EX_NOACTIVATE | WS_EX_TOPMOST, className, L"", + WS_POPUP | WS_BORDER, bounds.left + Scale(30, owner), bounds.top + Scale(80, owner), + empty ? 0 : Scale(280, owner), empty ? 0 : Scale(120, owner), + parent, nullptr, GetModuleHandleW(nullptr), nullptr); + if (tool == nullptr) return static_cast(nullptr); + toolWindows.push_back(tool); + if (content) { + AddChild(tool, L"BUTTON", L"Tool action", WS_TABSTOP | BS_PUSHBUTTON, kToolAction, 10, 10, 120, 28); + AddChild(tool, L"BUTTON", L"Close tool", WS_TABSTOP | BS_PUSHBUTTON, kToolClose, 10, 50, 120, 28); + } + if (visible) ShowWindow(tool, SW_SHOWNOACTIVATE); + return tool; + }; + create(owner, true, true, false); + const HWND unowned = create(nullptr, true, true, false); + create(unowned, true, true, false); + create(owner, false, true, false); + create(owner, true, false, false); + create(owner, true, true, true); + create(owner, true, true, false, TOOLTIPS_CLASSW); + SetStatus(L"Opened " + std::to_wstring(toolWindows.size()) + L" tool windows"); +} + void CreateMainControls(HWND window) { CreateMenuBar(window); statusLabel = AddChild(window, L"STATIC", L"Ready", SS_LEFT, kStatusLabel, 10, 10, 440, 20); @@ -296,6 +474,13 @@ void CreateMainControls(HWND window) { if (options.nodes > 0) { CreateTree(window); } + if (options.noDefaultAction) { + AddChild(window, kNoActionClass, L"No default action", 0, kNoActionControl, 10, 140, 200, 28); + AddChild(window, kStuckClass, L"", 0, kPassiveControl, 10, 180, 200, 28); + } + if (options.toolWindows) { + AddChild(window, L"BUTTON", L"Open tool windows", WS_TABSTOP | BS_PUSHBUTTON, kToolButton, 10, 140, 180, 28); + } } // The dialog is owned by the main window and modal in the Win32 sense: it @@ -364,6 +549,9 @@ LRESULT CALLBACK MainProc(HWND window, UINT message, WPARAM wparam, LPARAM lpara case kDialogButton: OpenDialog(window); return 0; + case kToolButton: + OpenToolWindows(window); + return 0; case kHangButton: // A busy or suspended application: nothing is pumped until this returns. Sleep(options.hangMilliseconds); @@ -380,6 +568,13 @@ LRESULT CALLBACK MainProc(HWND window, UINT message, WPARAM wparam, LPARAM lpara stuckApplications += 1; SetStatus(L"Stuck action applied " + std::to_wstring(stuckApplications) + L" times"); return 0; + case kNoActionDispatched: + SetStatus(L"Nonactionable default dispatched"); + return 0; + case kToolApplied: + toolActivations += 1; + SetStatus(L"Tool activated " + std::to_wstring(toolActivations) + L" times"); + return 0; case WM_DESTROY: PostQuitMessage(0); return 0; @@ -404,6 +599,12 @@ void RegisterClasses(HINSTANCE instance) { windowClass.lpfnWndProc = StuckProc; windowClass.lpszClassName = kStuckClass; RegisterClassExW(&windowClass); + windowClass.lpfnWndProc = NoActionProc; + windowClass.lpszClassName = kNoActionClass; + RegisterClassExW(&windowClass); + windowClass.lpfnWndProc = ToolProc; + windowClass.lpszClassName = kToolClass; + RegisterClassExW(&windowClass); } void StartWatchdog() { diff --git a/packages/agentic/desktop-driver/native/PROTOCOL.md b/packages/agentic/desktop-driver/native/PROTOCOL.md index eb19273e4f..b591683e17 100644 --- a/packages/agentic/desktop-driver/native/PROTOCOL.md +++ b/packages/agentic/desktop-driver/native/PROTOCOL.md @@ -300,6 +300,22 @@ one input turn and refuses a display name the element does not publish exactly once, so a key never reaches a different action than the one the agent named. +### Windows accessibility activation + +For the shared Windows/Win32 provider, accessibility `click` requires an +actionable `Invoke`, `Toggle`, `SelectionItem`, or `ExpandCollapse` pattern. +LegacyIAccessible default-action metadata, even when nonempty, and a +`ScrollIntoView` pattern alone do not prove an activation can be dispatched. +Without an actionable pattern, the helper fails `unsupported-operation` with +`data.reason` `activation-unsupported` and effect `none` before mutation. Node +retains the native code and reason, records no delivery, and does not +implicitly retry with physical input. An explicit permitted physical request +is a separate operation; an indeterminate mutation must not be replayed. + +This is a provider dispatch-policy correction using the existing error and +effect fields. FDR1 framing, commands, and the 1.0 wire constants in +`protocol.json` are unchanged. + ### Clear mechanisms The mutating `clear` command takes `{ "elementId": "" }`. Helpers that diff --git a/packages/agentic/desktop-driver/native/windows/README.md b/packages/agentic/desktop-driver/native/windows/README.md index 148c8de0dd..655dad4ad9 100644 --- a/packages/agentic/desktop-driver/native/windows/README.md +++ b/packages/agentic/desktop-driver/native/windows/README.md @@ -265,6 +265,17 @@ either differs. A recycled handle of the same class on the same thread whose destroy notification was never delivered is indistinguishable. Event delivery across integrity levels is untested. +Global title discovery still excludes untitled tool-window roots. Within an +exact-process application lease, `windows` also admits an untitled tool popup +when it is visible, has positive native bounds, and its same-process owner +chain reaches the verified primary window. Known tooltip and IME helper +classes stay excluded. A bounded UIA content-view probe must verify the exact +native root handle/process and find an exact-process content child; tooltip content and unreadable content do not +qualify. This probe retains no element records or whole-subtree cache and never +quarantines the session. The helper rechecks ownership, process, visibility, +bounds, and the primary/candidate fingerprints after the provider read before +registering the opaque window identity. + ### Input authority Before physical input the helper evaluates the desktop it would inject into and @@ -402,7 +413,7 @@ read of a large tree. ### Unsettled mutations UI Automation mutations (`SetFocus`, `Invoke`, `Toggle`, `Select`, -`Expand`/`Collapse`, `DoDefaultAction`, and `SetValue`) run on a bounded worker +`Expand`/`Collapse`, and `SetValue`) run on a bounded worker with a five-second deadline because a suspended or busy application can stall its UI thread indefinitely. A deadline does not cancel the provider call: the application can still apply the effect later. The helper therefore reports @@ -422,8 +433,8 @@ transaction timeouts. A cancelled read simply stops waiting. A cancelled mutation is abandoned and quarantined exactly like one that timed out, so its effect is `unknown` and the refusal says it was cancelled before it settled. -Click tries `Invoke`, `Toggle`, `SelectionItem`, `ExpandCollapse`, then -`LegacyIAccessible`. On Win32 controls `Invoke` is posted: the provider accepts +Accessibility click tries `Invoke`, `Toggle`, `SelectionItem`, then +`ExpandCollapse`. On Win32 controls `Invoke` is posted: the provider accepts the call and the application applies the effect afterwards, so callers poll for the visible result. Only a pattern the element does not implement moves on to the next one, and only an element without any of them is unsupported; a pattern @@ -431,6 +442,15 @@ query that fails otherwise fails as `automation-failed`, and one for an element that went away as `stale-element`. Named accessibility actions and a `semantic` clear classify their pattern queries the same way. +Legacy-only activation is refused before dispatch as `unsupported-operation` +with `data.reason: activation-unsupported` and effect `none`, even when the +LegacyIAccessible bridge advertises a nonempty default action. The bridge can +acknowledge `DoDefaultAction` without delivering an actionable UIA pattern. +`ScrollIntoView` alone is not activation. Node preserves the reason and native +code, records no input delivery, and never retries physically on its own. A +caller may explicitly request physical click under the normal input policy; +an indeterminate mutation must never be replayed as a fallback. + While quarantined, `click`, `clear`, `sendKeys`, `performActions`, `performAccessibilityAction`, `activate`, and `setWindowRect` fail with `target-quarantined` before reaching the diff --git a/packages/agentic/desktop-driver/native/windows/src/application.cpp b/packages/agentic/desktop-driver/native/windows/src/application.cpp index 6d6baa0082..d8dbd1072d 100644 --- a/packages/agentic/desktop-driver/native/windows/src/application.cpp +++ b/packages/agentic/desktop-driver/native/windows/src/application.cpp @@ -70,6 +70,12 @@ struct EnumerationContext { std::vector windows; }; +bool IsVisibleSizedRoot(HWND window) { + RECT bounds{}; + return IsWindowVisible(window) != FALSE && GetAncestor(window, GA_ROOT) == window && + GetWindowRect(window, &bounds) != FALSE && !IsEmptyRect(bounds); +} + BOOL CALLBACK CollectWindow(HWND window, LPARAM parameter) { auto* context = reinterpret_cast(parameter); DWORD processId = 0; @@ -77,20 +83,13 @@ BOOL CALLBACK CollectWindow(HWND window, LPARAM parameter) { if (context->processId != 0 && processId != context->processId) { return TRUE; } - if (IsWindowVisible(window) == FALSE) { - return TRUE; - } - if (GetAncestor(window, GA_ROOT) != window) { + if (!IsVisibleSizedRoot(window)) { return TRUE; } const LONG_PTR exStyle = GetWindowLongPtrW(window, GWL_EXSTYLE); if ((exStyle & WS_EX_TOOLWINDOW) != 0 && ReadWindowTitle(window).empty()) { return TRUE; } - RECT bounds{}; - if (GetWindowRect(window, &bounds) == FALSE || IsEmptyRect(bounds)) { - return TRUE; - } context->windows.push_back(window); return TRUE; } @@ -129,6 +128,30 @@ WindowFingerprint ReadWindowFingerprint(HWND window) { return fingerprint; } +bool IsOwnedContentCandidate(HWND window, HWND primaryWindow, DWORD expectedProcessId) { + DWORD processId = 0; + GetWindowThreadProcessId(window, &processId); + if (primaryWindow == nullptr || window == primaryWindow || processId != expectedProcessId || + !IsVisibleSizedRoot(window) || (GetWindowLongPtrW(window, GWL_STYLE) & WS_POPUP) == 0) { + return false; + } + const WindowFingerprint fingerprint = ReadWindowFingerprint(window); + if (fingerprint.className.empty()) return false; + const wchar_t* const helperClasses[] = {L"tooltips_class32", L"IME", L"MSCTFIME UI", L"CiceroUIWndFrame"}; + for (const wchar_t* helperClass : helperClasses) { + if (CompareStringOrdinal(fingerprint.className.c_str(), -1, helperClass, -1, TRUE) == CSTR_EQUAL) return false; + } + HWND owner = GetWindow(window, GW_OWNER); + for (std::size_t depth = 0; owner != nullptr && depth < 32; ++depth) { + DWORD ownerProcessId = 0; + GetWindowThreadProcessId(owner, &ownerProcessId); + if (ownerProcessId != expectedProcessId) return false; + if (owner == primaryWindow) return true; + owner = GetWindow(owner, GW_OWNER); + } + return false; +} + std::string ReadFileWithRetry(const std::wstring& path, const CancellationToken& token) { for (int attempt = 0; attempt < 20; attempt += 1) { token.ThrowIfCancelled(); @@ -1585,7 +1608,9 @@ const ApplicationLease& ApplicationManager::Attach(const json::Value& params, co return attached; } -std::vector ApplicationManager::Windows(const std::string& leaseId, const CancellationToken& token) { +std::vector ApplicationManager::Windows( + const std::string& leaseId, const CancellationToken& token, + const std::function& hasContent) { const auto entry = leases_.find(leaseId); if (entry == leases_.end()) { Fail(kErrorNoSuchLease, "The requested application lease is not held by this helper."); @@ -1600,11 +1625,36 @@ std::vector ApplicationManager::Windows(const std::string& leaseId, // Closed windows leave the table before the live ones register, so a handle // reused by a later window can never keep its predecessor's identifier. RetireClosedWindows(leaseId); - for (const HWND window : EnumerateTopLevelWindows(lease.processId)) { + const auto hasPrimaryIdentity = [&]() { + const auto id = windowIdsByHandle_.find(lease.primaryWindow); + if (id == windowIdsByHandle_.end()) return false; + const auto record = windowsById_.find(id->second); + if (record == windowsById_.end() || record->second.leaseId != leaseId || IsWindowClosed(record->second)) return false; + DWORD processId = 0; + GetWindowThreadProcessId(lease.primaryWindow, &processId); + const WindowFingerprint fingerprint = ReadWindowFingerprint(lease.primaryWindow); + return processId == lease.processId && record->second.threadId == fingerprint.threadId && + record->second.className == fingerprint.className; + }; + for (const HWND window : EnumerateAllTopLevelWindows(lease.processId)) { + token.ThrowIfCancelled(); + DWORD processId = 0; + GetWindowThreadProcessId(window, &processId); + if (processId != lease.processId || !IsVisibleSizedRoot(window)) continue; + const std::wstring title = ReadWindowTitle(window); + if ((GetWindowLongPtrW(window, GWL_EXSTYLE) & WS_EX_TOOLWINDOW) != 0 && title.empty()) { + if (!hasContent || !hasPrimaryIdentity() || + !IsOwnedContentCandidate(window, lease.primaryWindow, lease.processId)) continue; + const WindowFingerprint before = ReadWindowFingerprint(window); + if (!hasContent(window, lease.processId)) continue; + const WindowFingerprint after = ReadWindowFingerprint(window); + if (!hasPrimaryIdentity() || !IsOwnedContentCandidate(window, lease.primaryWindow, lease.processId) || + before.threadId != after.threadId || before.className != after.className) continue; + } WindowInfo info; info.window = window; info.processId = lease.processId; - info.title = ReadWindowTitle(window); + info.title = title; info.primary = window == lease.primaryWindow || (lease.primaryWindow == nullptr && info.title == lease.windowTitle); info.id = RegisterWindow(window, lease.processId, leaseId, info.primary, info.title); results.push_back(std::move(info)); diff --git a/packages/agentic/desktop-driver/native/windows/src/application.h b/packages/agentic/desktop-driver/native/windows/src/application.h index 68c1bb0325..0584877a49 100644 --- a/packages/agentic/desktop-driver/native/windows/src/application.h +++ b/packages/agentic/desktop-driver/native/windows/src/application.h @@ -64,7 +64,9 @@ class ApplicationManager { const ApplicationLease& Attach(const json::Value& params, const CancellationToken& token); void CloseApplication(const std::string& leaseId, const CancellationToken& token); void Dispose(const CancellationToken& token); - std::vector Windows(const std::string& leaseId, const CancellationToken& token); + std::vector Windows( + const std::string& leaseId, const CancellationToken& token, + const std::function& hasContent = {}); WindowInfo RequireWindow(const std::string& windowId); const ApplicationLease* FindLease(const std::string& leaseId) const; diff --git a/packages/agentic/desktop-driver/native/windows/src/automation.cpp b/packages/agentic/desktop-driver/native/windows/src/automation.cpp index ce93d4b705..b9e8557002 100644 --- a/packages/agentic/desktop-driver/native/windows/src/automation.cpp +++ b/packages/agentic/desktop-driver/native/windows/src/automation.cpp @@ -467,11 +467,61 @@ void Automation::Initialize() { automation->get_RawViewWalker(rawWalker_.put())); CheckHresult(kErrorAutomationFailed, "Reading the UI Automation control walker", automation->get_ControlViewWalker(controlWalker_.put())); + CheckHresult(kErrorAutomationFailed, "Reading the UI Automation content walker", + automation->get_ContentViewWalker(contentWalker_.put())); CheckHresult(kErrorAutomationFailed, "Reading the UI Automation not-supported sentinel", UiaGetReservedNotSupportedValue(notSupportedValue_.put())); automation_ = automation; } +bool Automation::HasWindowContent(HWND window, DWORD processId, const CancellationToken& token) { + token.ThrowIfCancelled(); + const auto automation = automation_; + const auto walker = contentWalker_; + const auto content = std::make_shared(false); + try { + const HRESULT result = BoundedRead( + [automation, walker, window, processId, content]() { + winrt::com_ptr root; + HRESULT result = automation->ElementFromHandle(window, root.put()); + if (FAILED(result) || !root) return FAILED(result) ? result : E_FAIL; + int rootProcessId = 0; + result = root->get_CurrentProcessId(&rootProcessId); + if (FAILED(result)) return result; + if (rootProcessId != static_cast(processId)) return S_FALSE; + UIA_HWND rootWindow{}; + result = root->get_CurrentNativeWindowHandle(&rootWindow); + if (FAILED(result)) return result; + if (rootWindow != window) return S_FALSE; + CONTROLTYPEID rootType = 0; + result = root->get_CurrentControlType(&rootType); + if (FAILED(result)) return result; + if (rootType == UIA_ToolTipControlTypeId) return S_FALSE; + winrt::com_ptr child; + result = walker->GetFirstChildElement(root.get(), child.put()); + if (FAILED(result) || !child) return FAILED(result) ? result : S_FALSE; + int childProcessId = 0; + result = child->get_CurrentProcessId(&childProcessId); + if (FAILED(result)) return result; + BOOL isContent = FALSE; + result = child->get_CurrentIsContentElement(&isContent); + if (FAILED(result)) return result; + CONTROLTYPEID type = 0; + result = child->get_CurrentControlType(&type); + if (FAILED(result)) return result; + *content = childProcessId == static_cast(processId) && isContent != FALSE && + type != UIA_ToolTipControlTypeId; + return S_OK; + }, + "Reading the owned window content"); + token.ThrowIfCancelled(); + return SUCCEEDED(result) && *content; + } catch (const HelperError&) { + token.ThrowIfCancelled(); + return false; + } +} + winrt::com_ptr Automation::ParentOf(IUIAutomationElement* element) const { winrt::com_ptr parent; if (!rawWalker_ || FAILED(rawWalker_->GetParentElement(element, parent.put()))) { @@ -1402,7 +1452,8 @@ void Automation::AccessibilityClick(const std::string& elementId, const Cancella BoundedRead([expand, state]() { return expand->get_CurrentExpandCollapseState(state.get()); }, "Reading the expand state")); if (*state == ExpandCollapseState_LeafNode) { - Fail(kErrorUnsupported, "The element is a leaf node and cannot be activated through accessibility."); + throw HelperError(kErrorUnsupported, "The element is a leaf node and cannot be activated through accessibility.", + "activation-unsupported"); } const bool collapse = *state == ExpandCollapseState_Expanded; CheckClickResult("Expanding or collapsing the element", @@ -1410,18 +1461,11 @@ void Automation::AccessibilityClick(const std::string& elementId, const Cancella "Expanding or collapsing the element")); return; } - const auto legacyQuery = - QueryPattern(element.get(), UIA_LegacyIAccessiblePatternId); - if (RequirePatternQuery(legacyQuery, "Reading the element's LegacyIAccessible pattern")) { - const winrt::com_ptr legacy = legacyQuery.pattern; - CheckClickResult("Invoking the legacy accessibility default action", - CountedMutation([legacy]() { return legacy->DoDefaultAction(); }, - "Invoking the legacy accessibility default action")); - return; - } - Fail(kErrorUnsupported, - "The element does not implement an Invoke, Toggle, SelectionItem, ExpandCollapse, or LegacyIAccessible pattern, so an " - "accessibility click is unavailable."); + // The legacy bridge can synthesize a nonempty default action and acknowledge + // it without dispatching any actionable UIA pattern. + throw HelperError(kErrorUnsupported, + "The element does not implement an actionable Invoke, Toggle, SelectionItem, or ExpandCollapse pattern.", + "activation-unsupported"); } json::Value Automation::AccessibilityActions(const std::string& elementId, const CancellationToken& token) { diff --git a/packages/agentic/desktop-driver/native/windows/src/automation.h b/packages/agentic/desktop-driver/native/windows/src/automation.h index 163a586873..c74433d3dd 100644 --- a/packages/agentic/desktop-driver/native/windows/src/automation.h +++ b/packages/agentic/desktop-driver/native/windows/src/automation.h @@ -79,6 +79,9 @@ class Automation { void Initialize(); + // A bounded content-view probe for an already identity-checked secondary + // window. It retains no element records; unreadable content is not proven. + bool HasWindowContent(HWND window, DWORD processId, const CancellationToken& token); std::vector SnapshotWindowTree(const WindowContext& context, const CancellationToken& token); std::vector Find(const WindowContext& context, const std::string& rootElementId, const Selector& selector, const CancellationToken& token); @@ -158,6 +161,7 @@ class Automation { winrt::com_ptr rawViewCondition_; winrt::com_ptr rawWalker_; winrt::com_ptr controlWalker_; + winrt::com_ptr contentWalker_; winrt::com_ptr notSupportedValue_; ElementRegistry registry_; }; diff --git a/packages/agentic/desktop-driver/native/windows/src/driver.cpp b/packages/agentic/desktop-driver/native/windows/src/driver.cpp index acfc88a474..b5ec2232be 100644 --- a/packages/agentic/desktop-driver/native/windows/src/driver.cpp +++ b/packages/agentic/desktop-driver/native/windows/src/driver.cpp @@ -633,7 +633,9 @@ CommandResult Driver::ExecuteCommand(const std::string& command, const json::Val Fail(kErrorInvalidParams, "The windows request is missing its lease."); } result.result = json::Value::Array(); - for (const WindowInfo& info : applications_.Windows(lease->StringField("id"), token)) { + for (const WindowInfo& info : applications_.Windows( + lease->StringField("id"), token, + [&](HWND window, DWORD processId) { return automation_.HasWindowContent(window, processId, token); })) { result.result.Append(WindowSummary(info)); } return result; diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index d5a83fed75..66c28d843d 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -582,3 +582,170 @@ Exact owned applications, helpers, Metro and service processes drained. The campaign ended; pre/post-end audits both exited 0: `FURN-AUDIT: no FURN desktop processes found`. No physical Ctrl+C claim is made from the supervisor's cooperative programmatic shutdown. + +## Windows Fabric and Win32 Paper — connected-session fixes — 2026-11-12 UTC + +**Observed test base:** `a1cee02547b7aae4c72134902fe43ed007215069` plus the +then-uncommitted fixes described here. This section supplements, rather than +rewrites, the unfixed qualification above. No macOS qualification is added. +The native runs used the serialized W3 campaign/lock. Eligible runs reported +an active, unlocked **remote RDP** session on `Default` before and after; +they were not console-session tests. The packaged Fabric build at +`bdeb0b291` was reused: neither the Windows app native project nor native-core +Windows native sources differed. Metro and bundles came from the fix +checkout. Paper used the prebuilt REX 0.81.1 host; its native build remains +unsupported. + +### Observed causes and corrections + +Instrumented Fabric and Paper runs +`6488435c-6222-4c35-a670-d50655753da4` and +`ecd1cdff-6b77-4c42-ad57-89733dc154d0` found no Invoke, Toggle, +SelectionItem, or ExpandCollapse pattern. LegacyIAccessible was selected: +its default-action read returned `S_OK` and a **nonempty** action, and +`DoDefaultAction` returned `S_OK`, but the application did not change. +This occurred on main-window Pressables and agentic Buttons on both +endpoints, and on the Fabric secondary button. The saved fixed-format traces +are `native-trace-21424.jsonl` and `native-trace-38384.jsonl` under +`artifacts\desktop\w3-qualification`. + +**Inferred from that dispatch trace and the independent effect checks:** a +nonempty legacy default action is not sufficient activation evidence. +An empty-action-only guard would not repair the observed defect. The helper +therefore refuses legacy-only activation before mutation with +`unsupported-operation`, reason `activation-unsupported`, and effect `none`. +The four explicit actionable pattern paths remain unchanged. The public +error preserves its class, native code, and reason; it records no delivery, +does not quarantine, and never implicitly retries physically. Temporary +native tracing was removed. + +The Paper inventory also proved an exact-process, visible, untitled, +280 x 144 `Net UI Tool Window` popup owned/root-owned by the main window. +The strict untitled-tool exclusion originated in provider commit +`5b3f74c7b`; the available history supplied no more specific rationale. +The correction leaves global discovery conservative and qualifies only +lease-local popup roots using verified primary identity, exact-process +ownership, positive visible bounds, helper-class exclusions, and a bounded +UIA exact-root/content check, followed by native identity rechecks. + +The final helper registered that real Paper secondary. Its unique Callout +content observation was complete at eight nodes; its public `find` proved +the actual focus **button** unique, cited activation refused honestly, and +story cleanup removed the secondary handle. The host's whole-window UIA tree +also exposes the owner subtree, so this is scoped popup-content evidence, +not a claim that the default whole-window observation was complete. +Stable focus/blur test IDs were added to the legacy Callout story rather +than weakening the public uniqueness-proof requirement. + +### Red/green checks and helper identity + +All commands below ran from `packages\agentic\desktop-driver` unless stated +otherwise. Native commands used the W3 lock/campaign; interactive runs also +used `-RequirePhysical`. Test working directories were confined beneath the +ignored checkout `artifacts\desktop\w3-fix\native-working`. + +```powershell +$env:FURN_NATIVE_DRIVER_TEST = '1' +yarn test --runInBand operations.contract win32Fixture.contract --testNamePattern 'nonactionable legacy|legacy-only|untitled tool roots' +yarn test --runInBand operations.contract win32Fixture.contract nativeDriver +yarn desktop-driver build-driver --platform windows +``` + +- **Expected red:** five behavioral failures, 18 skipped, in + `native-passive-red.log`: MSAA no-action and ScrollIntoView-only UIA + activation both falsely succeeded through raw/public routes, and the + owned content tool root was missing. +- **Passed on the initial fixed helper:** five new cases, 18 filtered cases + skipped (`native-fix-focused-green.log`); then three suites, 32 tests + passed and two macOS build/signing cases skipped + (`native-fix-regressions.log`). This includes existing operation, fixture, + cancellation, retention, capture, and helper build/handshake checks. +- That initial helper was artifact + `29b67c2c775f51cd360016c6abd09eb2ca377b7868b0cd7c6feb7432799e871e`, + source digest `e99e3246963a0fa4848a31c5753a026200492d9ad92c036a132bba16f9d1b469`. +- **Passed final helper build:** after adding an exact UIA native-root + handle check, artifact + `b47212dcd9e63aba388907b930da19d9a6d602f7d3de32e364f1584890f4d6dc`, + source digest `16ecdda1e41112cf2dafaf57eaad843054fbeeb2e9add0518f9c41886e8c867c`, + build fingerprint `3628b82e4280727dce2b265db184baa52285cc9f88f571621182d3893b051397`. + The build log is `native-fix-final-build.log`. The real Paper pass below + used this final helper and story blob + `f96de651a8433fc446f9c02b7b5ee3f5075c945c`. +- The **final-helper native-contract rerun is Not run**, not a replay of + the initial helper's 32 passes: the desktop was paused after disconnect. + Logs in this list are relative to `artifacts\desktop\w3-fix`. + +### Endpoint results and session boundary + +The copied qualification supervisor ran these exact commands from the +checkout root, inside the W3 `-RequirePhysical` wrapper: + +```powershell +node artifacts\desktop\w3-qualification\run-endpoint.mjs windows --secondary-only --modern-only --activation-diagnostics +node artifacts\desktop\w3-qualification\run-endpoint.mjs win32 --secondary-only --activation-diagnostics +``` + +| Case | Outcome and evidence | +| --------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Fabric main Pressable and agentic Button | **Passed** honest refusal on the initial fixed helper; original statuses remained `Closed` and `0`, with unchanged receipts. | +| Fabric modern secondary semantic action | **Passed** honest refusal on the initial fixed helper: exact secondary handle, complete 11-node observation, unique button, no delivery/quarantine, main status stayed `Idle`, handle retired. | +| Paper main Pressable and agentic Button | **Passed** honest refusal on the final helper; original statuses remained `Native window: Shown` and `0`, with unchanged receipts. | +| Paper secondary registration, observation, semantic action, retirement | **Passed** on the final helper: main plus one secondary, complete eight-node content sample, unique focus button, `activation-unsupported`/effect `none`, no receipt/quarantine/fallback, then main alone. | +| Final-helper Fabric semantic rerun | **Not run**: first startup failed before readiness with `spawn rnx-cli ENOENT`; dependency resolution confirmed the CLI was installed. Its one retry was session-gated before the supervisor started. | +| Final-helper Fabric and Paper secondary physical clicks | **Not run (session disconnected)**. No physical input was attempted after the gate. The earlier baseline Fabric physical pass remains valid; it is not a final-helper rerun. | +| DPI, mixed-monitor, negative-origin monitor layout, physical console Ctrl+C | **Not run** by this fix campaign; no new claim beyond the baseline section. | + +Artifacts are beneath +`artifacts\desktop\w3-qualification\runs\\`: + +| Evidence | Qualification ID | Driver run ID | +| -------------------------------------------------------- | -------------------------------------- | -------------------------------------- | +| Initial fixed Fabric main/secondary semantic pass | `a583840b-7ad5-4691-9e60-f4f83d49fd48` | `0644505e-3642-4fed-b6aa-157275fb519f` | +| Final fixed Paper main/secondary semantic pass | `7edaa34a-fefd-4a8b-846b-dc8aa8c3880e` | `2d342e8c-dd72-4553-bc90-af070716add2` | +| Final Fabric startup failure, no app-level case executed | `c807faf4-31e8-4aa1-a3de-5b78e720e20f` | `07fe422f-7c94-4bbc-b6e8-379b9ee989c7` | + +Each successful directory retains `qualification.json` and +`descriptor-cases--main-activation.json` / +`descriptor-cases--secondary[---modern].json`, with cleanup +acknowledgments. Earlier failed Paper probes were qualification-harness +failures, not passes: the owner subtree exhausted the observation byte +budget, searching _inside_ the unnamed button did not prove that button, +and an observation sample was correctly refused as `sample-not-unique`. +The successful run uses a real unique `find` with the stable story test ID. +The optional managed UIA probe now serializes nonfinite geometry as null; +its bounded 32-node content result is explicitly incomplete, not a complete +window-tree claim. + +At **2026-11-12T17:03:45Z** the wrapper observed session 1 as remote, +Disconnected, locked, LogonUI running, with no input desktop. It returned +75 / `NOT RUN` before starting the retry. All native/GUI/physical work was +then paused; no reconnect or session move was attempted. The campaign ended +and both audits passed: +`FURN-AUDIT: no FURN desktop processes found`. + +### Portable checks and remaining qualification + +**Passed:** desktop-driver `yarn test --runInBand` with native opt-in unset: +73 suites / 1,201 tests, with eight native suites / 93 native tests +**skipped**, not passed. Native-core `yarn test` passed its build, story +typecheck, 18 Node checks, 56 Jest checks, and one snapshot; native-core +format/lint passed. These do not replace native qualification. + +From the checkout root, the final portable checks **passed**: + +- `yarn workspace @fluentui-react-native/desktop-driver format` +- `yarn workspace @fluentui-react-native/desktop-driver lint` +- `yarn workspace @fluentui-react-native/desktop-driver build` +- `yarn oxfmt --check` and `git diff --check` +- `yarn markdown-link-check -c .github\markdown-link-check-config.json ` + for all ten changed Markdown files, including both patch changesets: + 72 links checked. + +One permitted session check at **2026-11-12T17:21:31Z** reconfirmed remote +session 1 as Disconnected, locked, LogonUI running, and without an input +desktop. No native or GUI retry followed. + +The post-review clean-cache repository aggregate, committed changeset check, +and final native/physical reruns are still pending. Outstanding qualification +stays tracked in [remaining work](remaining-work.md) / #4441; no issue is +closed and no final-helper physical-success claim is made here. diff --git a/packages/agentic/desktop-driver/references/native-helpers.md b/packages/agentic/desktop-driver/references/native-helpers.md index 3ea2775f6e..2f2a537882 100644 --- a/packages/agentic/desktop-driver/references/native-helpers.md +++ b/packages/agentic/desktop-driver/references/native-helpers.md @@ -263,6 +263,26 @@ verify a service's process and listener before a credential is sent, and to prove a quarantined application's recorded process exited. See the [Windows provider guide](../native/windows/README.md#authority-helper-modes). +## Windows activation and secondary content windows + +The shared Windows/Win32 provider activates through actionable UIA `Invoke`, +`Toggle`, `SelectionItem`, or `ExpandCollapse` patterns. A legacy default-action +acknowledgment is not sufficient dispatch evidence. Legacy-only controls, +including controls exposing only `ScrollIntoView`, are refused before dispatch +with `unsupported-operation`, reason `activation-unsupported`, and effect +`none`. The public error retains the reason and native code, without an input +receipt, quarantine, or implicit physical retry. Callers may explicitly choose +physical click where policy permits; an unsettled mutation is not a safe +fallback opportunity. + +An application lease can register an untitled, visible, positive-size tool +popup only when its exact-process owner chain reaches the verified primary +window and a bounded UIA read proves exact-process content. Known IME/tooltip +helper classes, unrelated owner chains, hidden or zero-size roots, and +unreadable or empty content stay excluded. This does not broaden global title discovery. +See the [Windows identity rules](../native/windows/README.md#window-events-and-identity) +and [Windows qualification evidence](current-status.md). + ## Diagnostics `doctor` uses prebuilt-only resolution: diff --git a/packages/agentic/desktop-driver/references/operations.md b/packages/agentic/desktop-driver/references/operations.md index dbb6741c50..efe2828100 100644 --- a/packages/agentic/desktop-driver/references/operations.md +++ b/packages/agentic/desktop-driver/references/operations.md @@ -288,6 +288,12 @@ mutation. Provider messages quote window titles and element ids, so they are not passed on. A provider that no longer has the element or window drops the reference, as it does for a read. +An unsupported native activation keeps its `nativeCode` and any reported +machine `reason`. A rejection with effect `none` records no input delivery and +does not quarantine the target. `activate` never retries with physical input; +after such a rejection, a caller may explicitly choose `click` if the session +permits it. An indeterminate outcome is not permission to replay the action. + A session's `furn:inputMode` ([protocol](protocol.md#sessions-and-targets)) limits the mechanisms `act` may use. In an `accessible` session the runtime reports no `physicalClick` or `keyboard`, so `click` and `type` fail with diff --git a/packages/agentic/desktop-driver/references/remaining-work.md b/packages/agentic/desktop-driver/references/remaining-work.md index ae4bca63c8..b7042f122a 100644 --- a/packages/agentic/desktop-driver/references/remaining-work.md +++ b/packages/agentic/desktop-driver/references/remaining-work.md @@ -1082,6 +1082,28 @@ have evidence at the final revision, implementation changesets are present, and the final clean-cache repository aggregate passes. Only then prepare/push the PR; a hosted capability skip is not completion. +### Windows connected-session follow-up — 2026-11-12 UTC + +The legacy-only activation refusal and lease-local content-popup registration +corrections have focused native red/green evidence; the final helper also +passes the real Paper secondary semantic case. See the separate +[Windows evidence entry](current-status.md). Keep #4441 open until: + +- The final helper reruns Fabric main/secondary semantic activation and both + endpoints' secondary physical clicks. Physical receipts are not themselves + proof of the legacy Callout's native command outcome. +- The final helper reruns + `yarn test --runInBand operations.contract win32Fixture.contract nativeDriver` + with `$env:FURN_NATIVE_DRIVER_TEST = '1'`. +- Coordinator review and the clean-cache repository aggregate complete. + +These runs are paused because session 1 disconnected, locked, and lost its +input desktop. Resume native/GUI/physical work only with an Active, unlocked +session and the serialized campaign; physical work additionally requires +`-RequirePhysical`. No reconnect, session move, or skip-to-pass substitution +is authorized. Earlier eligible physical results remain evidence for their +tested revisions, not final-helper reruns. + ## 9. Qualification and efficiency gates | Lane | Required scope | diff --git a/packages/agentic/desktop-driver/src/hosts/native/NativeDesktopHost.test.ts b/packages/agentic/desktop-driver/src/hosts/native/NativeDesktopHost.test.ts index 710571be26..850b3edd32 100644 --- a/packages/agentic/desktop-driver/src/hosts/native/NativeDesktopHost.test.ts +++ b/packages/agentic/desktop-driver/src/hosts/native/NativeDesktopHost.test.ts @@ -89,6 +89,24 @@ describe('NativeDesktopHost error translation', () => { }); }); + test('preserves a no-dispatch activation refusal without making it an uncertain mutation', () => { + const translated = translateNativeError( + attachFailureEffect( + new NativeDriverError('unsupported-operation', 'No default action is available.', { reason: 'activation-unsupported' }), + 'none', + ), + ); + + expect(translated).toMatchObject({ name: 'HostUnsupportedError' }); + expect(toWebDriverError(translated)).toMatchObject({ + code: 'unsupported operation', + data: { nativeCode: 'unsupported-operation', reason: 'activation-unsupported' }, + }); + expect(readFailureEffect(translated)).toBe('none'); + expect(dispositionOfFailure(translated)).toBe('rejected'); + expect(readProviderQuarantine(translated)).toBe(false); + }); + describe('provider settlement codes', () => { test.each<[nativeCode: string, effect: string, disposition: string]>([ ['mutation-indeterminate', 'unknown', 'indeterminate'], diff --git a/packages/agentic/desktop-driver/src/hosts/native/NativeDesktopHost.ts b/packages/agentic/desktop-driver/src/hosts/native/NativeDesktopHost.ts index 0cede1771d..86f6a800ed 100644 --- a/packages/agentic/desktop-driver/src/hosts/native/NativeDesktopHost.ts +++ b/packages/agentic/desktop-driver/src/hosts/native/NativeDesktopHost.ts @@ -728,7 +728,7 @@ export function translateNativeError(error: unknown): unknown { } const effect = readFailureEffect(error); if (error.code === 'unsupported-operation') { - return attachFailureEffect(new HostUnsupportedError(error.message), effect); + return attachFailureEffect(new HostUnsupportedError(error.message, { ...error.data, nativeCode: error.code }), effect); } if (error.code === 'stale-element') { return attachFailureEffect(new HostStaleError(error.message), effect); diff --git a/packages/agentic/desktop-driver/src/native/windows/operations.contract.test.ts b/packages/agentic/desktop-driver/src/native/windows/operations.contract.test.ts index 37b25e7a6a..db400b3148 100644 --- a/packages/agentic/desktop-driver/src/native/windows/operations.contract.test.ts +++ b/packages/agentic/desktop-driver/src/native/windows/operations.contract.test.ts @@ -87,6 +87,30 @@ describe('Windows agent operations against the Win32 fixture', () => { }); }); + windowsTest.each(['1011', '1013'])( + 'rejects nonactionable legacy activation without a delivery receipt or quarantine (%s)', + async (automationId) => { + await withFixture( + async (session) => { + const before = await session.getInput(); + await expect(session.act({ action: 'activate', find: { automationId } })).rejects.toMatchObject({ + code: 'unsupported operation', + data: { + mutation: { disposition: 'rejected', effect: 'none', quarantined: false }, + nativeCode: 'unsupported-operation', + reason: 'activation-unsupported', + }, + }); + expect(await session.getInput()).toEqual(before); + expect((await session.check({ expect: { name: 'Ready' }, find: status })).result.status).toBe('passed'); + expect((await session.act({ action: 'activate', find: activate })).result.mechanism).toBe('accessibility'); + await expectStatus(session, 1); + }, + ['--no-default-action', '1'], + ); + }, + ); + inputTest('physically clicks with a hit-tested proof, types, and clears without journaling the text', async () => { await withFixture(async (session) => { const { result: found } = await session.find(activate); diff --git a/packages/agentic/desktop-driver/src/native/windows/win32Fixture.contract.test.ts b/packages/agentic/desktop-driver/src/native/windows/win32Fixture.contract.test.ts index c33b0112f2..be78d57c9a 100644 --- a/packages/agentic/desktop-driver/src/native/windows/win32Fixture.contract.test.ts +++ b/packages/agentic/desktop-driver/src/native/windows/win32Fixture.contract.test.ts @@ -237,6 +237,49 @@ describe('Windows helper against a plain Win32 application', () => { } }); + windowsTest.each(['1011', '1013'])('refuses legacy-only activation of a nonactionable control (%s)', async (automationId) => { + const session = await FixtureSession.start('no-default-action', { args: ['--no-default-action', '1'] }); + try { + const target = await session.control(session.main.id, automationId); + const { actions } = await session.call<{ actions: string[] }>('accessibilityActions', { elementId: target.id }); + expect(actions).toEqual(automationId === '1013' ? ['ScrollIntoView'] : []); + await expect(session.host.request('click', { elementId: target.id, mode: 'accessibility' })).rejects.toMatchObject({ + code: 'unsupported-operation', + data: { reason: 'activation-unsupported' }, + effect: 'none', + }); + expect(await session.status()).toBe('Ready'); + await session.click('1001'); + await waitFor('the actionable control to remain usable', async () => (await session.status()) === 'Activated 1 times'); + } finally { + await session.dispose(); + } + }); + + windowsTest('registers only owned untitled tool roots containing application content', async () => { + const session = await FixtureSession.start('tool-content', { args: ['--tool-windows', '1'] }); + try { + expect(await session.windows()).toHaveLength(1); + await session.click('1012'); + await waitFor('all content and excluded tool roots to open', async () => (await session.status()) === 'Opened 7 tool windows'); + const secondary = await waitFor('the owned content tool window', async () => { + const windows = await session.windows(); + return windows.length === 2 ? windows.find((window) => window.id !== session.main.id) : undefined; + }); + expect(secondary.title).toBe(''); + expect(await session.control(secondary.id, '3101')).toMatchObject({ name: 'Tool action', role: 'button' }); + await session.click('3101', secondary.id); + await waitFor('the secondary action to reach the application', async () => (await session.status()) === 'Tool activated 1 times'); + await session.click('3102', secondary.id); + await waitFor('the content tool window to close', async () => (await session.windows()).length === 1); + await expect(session.host.request('getWindowRect', { windowId: secondary.id })).rejects.toMatchObject({ code: 'no-such-window' }); + await session.click('1001'); + await waitFor('the main window to remain usable', async () => (await session.status()) === 'Activated 1 times'); + } finally { + await session.dispose(); + } + }); + windowsTest('refuses input to an owner while its popup is open and reports the popup closing', async () => { const session = await FixtureSession.start('popup'); try { diff --git a/packages/agentic/desktop-driver/src/protocol/errors.ts b/packages/agentic/desktop-driver/src/protocol/errors.ts index 1bdcdbf65b..4ca1d9c97a 100644 --- a/packages/agentic/desktop-driver/src/protocol/errors.ts +++ b/packages/agentic/desktop-driver/src/protocol/errors.ts @@ -76,7 +76,7 @@ function convertToWebDriverError(error: unknown): WebDriverError { return new WebDriverError('stale element reference', error.message); } if (error instanceof HostUnsupportedError) { - return new WebDriverError('unsupported operation', error.message); + return new WebDriverError('unsupported operation', error.message, error.data); } return new WebDriverError('unknown error', error instanceof Error ? error.message : String(error)); } @@ -101,8 +101,11 @@ export class HostStaleError extends Error { } export class HostUnsupportedError extends Error { - constructor(message: string) { + readonly data?: Record; + + constructor(message: string, data?: Record) { super(message); this.name = 'HostUnsupportedError'; + this.data = data; } } diff --git a/packages/agentic/desktop-driver/src/server/operations.test.ts b/packages/agentic/desktop-driver/src/server/operations.test.ts index e308562b87..30aa57d084 100644 --- a/packages/agentic/desktop-driver/src/server/operations.test.ts +++ b/packages/agentic/desktop-driver/src/server/operations.test.ts @@ -8,6 +8,7 @@ import type { FakeDesktopHostOptions } from '../hosts/fake/FakeDesktopHost.js'; import { translateNativeError } from '../hosts/native/NativeDesktopHost.js'; import { NativeDriverError } from '../native/NativeDriverError.js'; import type { OperationCatalogEntry } from '../operations/index.js'; +import { attachFailureEffect } from '../protocol/settlement.js'; import { createFakeTree, fakeTreeSize } from '../testing/fakeTree.js'; import type { FakeTreeShape } from '../testing/fakeTree.js'; import { createDesktopDriverTestHarness } from '../testing/protocolHarness.js'; @@ -493,6 +494,39 @@ describe('Desktop Driver agent operations', () => { }); describe('act', () => { + test('keeps a native activation refusal without a receipt or implicit physical fallback', async () => { + await withFixture({ windows: actWindow }, async ({ harness, session }) => { + const click = jest.spyOn(harness.host, 'click').mockImplementationOnce(async () => { + throw translateNativeError( + attachFailureEffect( + new NativeDriverError('unsupported-operation', 'No default action is available.', { + reason: 'activation-unsupported', + }), + 'none', + ), + ); + }); + const before = await session.getInput(); + + await expect(session.act({ action: 'activate', find: { automationId: 'save' } })).rejects.toMatchObject({ + code: 'unsupported operation', + data: { + mutation: rejected(), + nativeCode: 'unsupported-operation', + reason: 'activation-unsupported', + }, + }); + expect(await session.getInput()).toEqual(before); + expect(click.mock.calls.map(([, mode]) => mode)).toEqual(['accessibility']); + expect(harness.host.actions.filter((action) => action.type === 'click')).toEqual([]); + + await expect(session.act({ action: 'click', find: { automationId: 'save' } })).resolves.toMatchObject({ + result: { disposition: 'applied', mechanism: 'physical-pointer' }, + }); + expect(click.mock.calls.map(([, mode]) => mode)).toEqual(['accessibility', 'physical']); + }); + }); + test('clicks, activates, types into, and clears a proven element, and says how each was carried out', async () => { await withFixture({ windows: actWindow }, async ({ harness, session }) => { await expect(session.act({ action: 'click', ...(await proven(session, 'agree')) })).resolves.toMatchObject({ diff --git a/packages/agentic/native-core/references/contributing.md b/packages/agentic/native-core/references/contributing.md index f0e813a66a..9f28d3ddfc 100644 --- a/packages/agentic/native-core/references/contributing.md +++ b/packages/agentic/native-core/references/contributing.md @@ -55,6 +55,11 @@ full catalog before running the selected native tests. Add `--paper` to run the macOS Paper lane; a successful Paper build alone does not establish Paper runtime or input parity. +The legacy Callout `WindowCommands` story exposes +`agentic-storybook-callout-focus` and `agentic-storybook-callout-blur` on its +actual command buttons. Use those identities for unique secondary-window +input proofs; a text child or an arbitrary unnamed button is not a substitute. + Executable cases are declared with `desktopIt` and its `requires` list (see `@fluentui-react-native/storybook-desktop/references/story-tests.md`), not with run-time `skip` guards. Cases that click or type declare `physical-click` and `keyboard`, which diff --git a/packages/agentic/native-core/src/legacy/callout/Callout.stories.tsx b/packages/agentic/native-core/src/legacy/callout/Callout.stories.tsx index 864f43add9..f96de651a8 100644 --- a/packages/agentic/native-core/src/legacy/callout/Callout.stories.tsx +++ b/packages/agentic/native-core/src/legacy/callout/Callout.stories.tsx @@ -76,10 +76,20 @@ const CalloutExample = ({ defaultVisible = false, onDismiss, onShow, showWindowC This content is hosted in a separate native window. {showWindowCommands && ( - calloutRef.current?.focusWindow()} style={styles.command}> + calloutRef.current?.focusWindow()} + style={styles.command} + testID="agentic-storybook-callout-focus" + > Focus window - calloutRef.current?.blurWindow()} style={styles.command}> + calloutRef.current?.blurWindow()} + style={styles.command} + testID="agentic-storybook-callout-blur" + > Blur window From 96b91661e2886aa4a22bd3f08b485e51d11c90d7 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 09:27:21 -0800 Subject: [PATCH 03/15] docs: record W3 portable validation and paused qualification Record the implementation commit, committed changeset validation, clean process audit, and explicit remaining native qualification gates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .../desktop-driver/references/current-status.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index 66c28d843d..dffe430bb8 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -745,7 +745,13 @@ One permitted session check at **2026-11-12T17:21:31Z** reconfirmed remote session 1 as Disconnected, locked, LogonUI running, and without an input desktop. No native or GUI retry followed. -The post-review clean-cache repository aggregate, committed changeset check, -and final native/physical reruns are still pending. Outstanding qualification -stays tracked in [remaining work](remaining-work.md) / #4441; no issue is -closed and no final-helper physical-success claim is made here. +The implementation is committed as +`cd036a2d06ae450f9c26ac0837c4fe140877ffb9`. Its committed +`yarn change:check` **passed**. A post-commit process audit also passed: +`FURN-AUDIT: no FURN desktop processes found`; campaign status reported no +campaign or holder. The branch has not been pushed. + +The post-review clean-cache repository aggregate and final native/physical +reruns are still pending. Outstanding qualification stays tracked in +[remaining work](remaining-work.md) / #4441; no issue is closed and no +final-helper physical-success claim is made here. From 4d39582f2292ff6436dafaba646ee63cb9bdc3cc Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 09:57:36 -0800 Subject: [PATCH 04/15] fix: preserve admitted Windows popups across provider failures Use verified lease/process/thread/class identity and native ownership checks for admitted popups instead of probing their content again. Share one 1000 ms discovery deadline across new candidates, bound UIA waits by its remaining time, and retry candidates left unproved without dropping registered windows. Include admitted popups in publication diagnostics. Add failed-probe and shared-deadline native self-tests, hung-provider fixture regressions, and matching protocol/README/evidence notes. Native execution stays session-gated; the helper-only build and portable checks passed. Part of #4441 Part of #4435 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .../windows-connected-session-refusals.md | 2 + .../desktop-driver/fixtures/win32/README.md | 10 +- .../agentic/desktop-driver/native/PROTOCOL.md | 15 +++ .../desktop-driver/native/windows/README.md | 26 +++-- .../native/windows/src/application.cpp | 50 +++++---- .../native/windows/src/application.h | 3 +- .../native/windows/src/automation.cpp | 32 +++--- .../native/windows/src/automation.h | 3 +- .../native/windows/src/driver.cpp | 4 +- .../native/windows/src/selftest.cpp | 86 ++++++++++++++- .../references/current-status.md | 100 ++++++++++++++++++ .../references/native-helpers.md | 5 +- .../references/remaining-work.md | 6 +- .../windows/win32Fixture.contract.test.ts | 54 ++++++++++ 14 files changed, 347 insertions(+), 49 deletions(-) diff --git a/.changeset/windows-connected-session-refusals.md b/.changeset/windows-connected-session-refusals.md index 3617d4d256..2d8127caff 100644 --- a/.changeset/windows-connected-session-refusals.md +++ b/.changeset/windows-connected-session-refusals.md @@ -5,3 +5,5 @@ Refuse Windows legacy-only semantic activation before dispatch, preserving machine-readable reasons without recording a delivery or implicitly retrying with physical input. Register untitled Windows secondary tool popups only when their exact-process ownership and bounded UIA content are verified, retaining helper-window exclusions. + +Keep admitted popups listable through transient provider failures, share one bounded discovery deadline across new candidates, and include admitted popups in publication diagnostics. diff --git a/packages/agentic/desktop-driver/fixtures/win32/README.md b/packages/agentic/desktop-driver/fixtures/win32/README.md index 5ac1b25d87..c81f202c4f 100644 --- a/packages/agentic/desktop-driver/fixtures/win32/README.md +++ b/packages/agentic/desktop-driver/fixtures/win32/README.md @@ -20,8 +20,8 @@ Remove-Item Env:\FURN_NATIVE_DRIVER_TEST ``` Without `FURN_NATIVE_DRIVER_TEST=1` the suite is skipped. Use `--runInBand`: the scenarios share one desktop and one -build cache, and each starts its own helper process because the mutation quarantine is process-wide. A full run takes -about one minute. The onscreen scenario needs an unlocked interactive desktop for hit testing and capture; every other +build cache, and each starts its own helper process because the mutation quarantine is process-wide. Hang scenarios +include bounded provider-recovery waits. The onscreen scenario needs an unlocked interactive desktop for hit testing and capture; every other scenario places the window off every monitor. `yarn test --runInBand protectedService` runs the protected-service suite, and `yarn test --runInBand operations.contract` runs the agent-operation suite described below. The latter activates the fixture and sends real @@ -78,7 +78,11 @@ The default tree has 18 nodes; `--nodes N` yields about `N + 24` in total. only `ScrollIntoView` are refused before dispatch, without quarantine; an ordinary Invoke remains usable. - **Untitled tool roots:** only visible, positive-size application content owned by the main window is registered. Unowned roots, unrelated owner chains, empty content, hidden roots, zero-size roots, and tooltip roots stay excluded. - Closing the content root retires its identity without affecting the main window. + Publication marks the admitted popup as both listed and tracked; closing it retires its identity without affecting the main window. +- **Popup liveness under a hung provider:** an admitted popup stays listable and its rectangle readable under the same identity. + New-popup discovery remains bounded across multiple candidates, and deferred content appears after the provider recovers. + The helper self-test separately injects failed probes and an exhausted shared deadline to prove that registered popups + bypass UIA and unprobed candidates retry on the next listing. - **Hung provider (`--hang-ms 14000`):** reads and clicks fail as `automation-failed`, windows stay listable, and the helper recovers once the app responds. - **Stuck provider (`--stuck-ms 8000`):** the first click settles as `mutation-indeterminate`, later guarded mutations diff --git a/packages/agentic/desktop-driver/native/PROTOCOL.md b/packages/agentic/desktop-driver/native/PROTOCOL.md index b591683e17..0c78c6b116 100644 --- a/packages/agentic/desktop-driver/native/PROTOCOL.md +++ b/packages/agentic/desktop-driver/native/PROTOCOL.md @@ -253,6 +253,21 @@ events: Queries remain authoritative. Events can invalidate state or wake waits but do not replace a liveness or state query. +For Windows/Win32, an untitled tool popup needs verified native ownership and +UIA content for initial admission to its exact-process lease. Subsequent +`windows` queries retain its opaque identity using native lease/process, +thread/class, visibility, bounds, and ownership checks, without calling UIA +again. A provider timeout or transient content failure is not window closure. +All new popup-content probes share one 1,000 ms discovery deadline per +`windows` request. Unproved or unprobed candidates are omitted and can be +retried by a later request; admitted windows remain listable even if discovery +uses the whole budget. A read deadline does not quarantine mutations, and +cancellation is still honored. Publication diagnostics mark admitted live +popups as both `listed` and `tracked` without another content probe. + +This is internal Windows admission/liveness policy: no request field, response +shape, command, or FDR1 1.0 constant in `protocol.json` changes. + Helpers that advertise the `windowEvents` feature also accept a `windowEvents` command with the target `lease` and return a bounded diagnostic log of recent window notifications and helper commands, and publish a `window-closed` event diff --git a/packages/agentic/desktop-driver/native/windows/README.md b/packages/agentic/desktop-driver/native/windows/README.md index 655dad4ad9..9838725bc8 100644 --- a/packages/agentic/desktop-driver/native/windows/README.md +++ b/packages/agentic/desktop-driver/native/windows/README.md @@ -269,12 +269,26 @@ Global title discovery still excludes untitled tool-window roots. Within an exact-process application lease, `windows` also admits an untitled tool popup when it is visible, has positive native bounds, and its same-process owner chain reaches the verified primary window. Known tooltip and IME helper -classes stay excluded. A bounded UIA content-view probe must verify the exact -native root handle/process and find an exact-process content child; tooltip content and unreadable content do not -qualify. This probe retains no element records or whole-subtree cache and never -quarantines the session. The helper rechecks ownership, process, visibility, -bounds, and the primary/candidate fingerprints after the provider read before -registering the opaque window identity. +classes stay excluded. Initial admission requires a bounded UIA content-view +probe to verify the exact native root handle/process and find an exact-process +content child; tooltip content and unreadable content do not qualify. The +helper rechecks ownership, process, visibility, bounds, and the +primary/candidate fingerprints after the provider read before registering the +opaque window identity. + +An admitted popup remains listable through Win32 liveness checks while its +lease, process, thread, class, visibility, bounds, and owner chain still match. +It is not probed again, so a provider timeout or temporary content re-layout +cannot masquerade as window closure. All new candidates share one 1,000 ms +discovery deadline per `windows` call, including `ElementFromHandle` and content +reads; candidates not proved before that deadline are omitted and retried on a +later listing. Exhausting discovery never removes an already-admitted window. +The probe retains no element records or whole-subtree cache and never +quarantines the session; cancellation still propagates. + +`windowEvents.publication.windows[].listed` includes admitted tool popups +using those same native identity/ownership checks. It does not repeat UIA +discovery or report an admitted live popup as `listed: false, tracked: true`. ### Input authority diff --git a/packages/agentic/desktop-driver/native/windows/src/application.cpp b/packages/agentic/desktop-driver/native/windows/src/application.cpp index d8dbd1072d..b445c6a672 100644 --- a/packages/agentic/desktop-driver/native/windows/src/application.cpp +++ b/packages/agentic/desktop-driver/native/windows/src/application.cpp @@ -51,6 +51,7 @@ constexpr unsigned int kCloseTimeoutMs = 5000; constexpr unsigned int kTerminateWaitMs = 2000; constexpr unsigned int kActivationTimeoutMs = 2000; constexpr unsigned int kWindowResponseTimeoutMs = 2000; +constexpr DWORD kWindowContentDiscoveryBudgetMs = 1000; constexpr UINT kWindowProbeMessageTimeoutMs = 200; constexpr unsigned int kWindowRestoreTimeoutMs = 500; constexpr DWORD kLaunchCleanupGraceMs = 250; @@ -1043,6 +1044,19 @@ bool ApplicationManager::IsWindowClosed(const WindowRecord& record) const { return record.window == nullptr || IsWindow(record.window) == FALSE || windowEvents_.IsClosed(record.id); } +bool ApplicationManager::HasRegisteredWindowIdentity(HWND window, const ApplicationLease& lease) const { + const auto id = windowIdsByHandle_.find(window); + if (id == windowIdsByHandle_.end()) return false; + const auto record = windowsById_.find(id->second); + if (record == windowsById_.end() || record->second.leaseId != lease.id || + record->second.processId != lease.processId || IsWindowClosed(record->second)) return false; + DWORD processId = 0; + GetWindowThreadProcessId(window, &processId); + const WindowFingerprint fingerprint = ReadWindowFingerprint(window); + return processId == lease.processId && record->second.threadId == fingerprint.threadId && + record->second.className == fingerprint.className; +} + void ApplicationManager::RetireWindow(const std::string& windowId) { const auto entry = windowsById_.find(windowId); if (entry != windowsById_.end()) { @@ -1610,13 +1624,14 @@ const ApplicationLease& ApplicationManager::Attach(const json::Value& params, co std::vector ApplicationManager::Windows( const std::string& leaseId, const CancellationToken& token, - const std::function& hasContent) { + const std::function& hasContent) { const auto entry = leases_.find(leaseId); if (entry == leases_.end()) { Fail(kErrorNoSuchLease, "The requested application lease is not held by this helper."); } ApplicationLease& lease = entry->second; token.ThrowIfCancelled(); + const ULONGLONG contentDeadline = GetTickCount64() + kWindowContentDiscoveryBudgetMs; if (!IsProcessAlive(lease.process)) { Fail(kErrorNoSuchLease, "The leased application process has exited."); } @@ -1625,17 +1640,6 @@ std::vector ApplicationManager::Windows( // Closed windows leave the table before the live ones register, so a handle // reused by a later window can never keep its predecessor's identifier. RetireClosedWindows(leaseId); - const auto hasPrimaryIdentity = [&]() { - const auto id = windowIdsByHandle_.find(lease.primaryWindow); - if (id == windowIdsByHandle_.end()) return false; - const auto record = windowsById_.find(id->second); - if (record == windowsById_.end() || record->second.leaseId != leaseId || IsWindowClosed(record->second)) return false; - DWORD processId = 0; - GetWindowThreadProcessId(lease.primaryWindow, &processId); - const WindowFingerprint fingerprint = ReadWindowFingerprint(lease.primaryWindow); - return processId == lease.processId && record->second.threadId == fingerprint.threadId && - record->second.className == fingerprint.className; - }; for (const HWND window : EnumerateAllTopLevelWindows(lease.processId)) { token.ThrowIfCancelled(); DWORD processId = 0; @@ -1643,13 +1647,18 @@ std::vector ApplicationManager::Windows( if (processId != lease.processId || !IsVisibleSizedRoot(window)) continue; const std::wstring title = ReadWindowTitle(window); if ((GetWindowLongPtrW(window, GWL_EXSTYLE) & WS_EX_TOOLWINDOW) != 0 && title.empty()) { - if (!hasContent || !hasPrimaryIdentity() || + if (!HasRegisteredWindowIdentity(lease.primaryWindow, lease) || !IsOwnedContentCandidate(window, lease.primaryWindow, lease.processId)) continue; - const WindowFingerprint before = ReadWindowFingerprint(window); - if (!hasContent(window, lease.processId)) continue; - const WindowFingerprint after = ReadWindowFingerprint(window); - if (!hasPrimaryIdentity() || !IsOwnedContentCandidate(window, lease.primaryWindow, lease.processId) || - before.threadId != after.threadId || before.className != after.className) continue; + // Content proves initial admission, not whether an admitted window closed. + if (!HasRegisteredWindowIdentity(window, lease)) { + if (!hasContent || GetTickCount64() >= contentDeadline) continue; + const WindowFingerprint before = ReadWindowFingerprint(window); + if (!hasContent(window, lease.processId, contentDeadline)) continue; + const WindowFingerprint after = ReadWindowFingerprint(window); + if (!HasRegisteredWindowIdentity(lease.primaryWindow, lease) || + !IsOwnedContentCandidate(window, lease.primaryWindow, lease.processId) || + before.threadId != after.threadId || before.className != after.className) continue; + } } WindowInfo info; info.window = window; @@ -2016,7 +2025,10 @@ json::Value ApplicationManager::PublicationDiagnostic(const ApplicationLease& le const WindowDescriptor descriptor = DescribeWindow(window); json::Value entry = json::Value::Object(); entry.Set("className", json::Value::String(descriptor.className)); - entry.Set("listed", json::Value::Bool(std::find(listed.begin(), listed.end(), window) != listed.end())); + const bool admittedPopup = HasRegisteredWindowIdentity(window, lease) && + HasRegisteredWindowIdentity(lease.primaryWindow, lease) && + IsOwnedContentCandidate(window, lease.primaryWindow, lease.processId); + entry.Set("listed", json::Value::Bool(std::find(listed.begin(), listed.end(), window) != listed.end() || admittedPopup)); entry.Set("owned", json::Value::Bool(descriptor.owned)); if (descriptor.hasTitle) { entry.Set("title", json::Value::String(descriptor.title)); diff --git a/packages/agentic/desktop-driver/native/windows/src/application.h b/packages/agentic/desktop-driver/native/windows/src/application.h index 0584877a49..9ef199fa04 100644 --- a/packages/agentic/desktop-driver/native/windows/src/application.h +++ b/packages/agentic/desktop-driver/native/windows/src/application.h @@ -66,7 +66,7 @@ class ApplicationManager { void Dispose(const CancellationToken& token); std::vector Windows( const std::string& leaseId, const CancellationToken& token, - const std::function& hasContent = {}); + const std::function& hasContent = {}); WindowInfo RequireWindow(const std::string& windowId); const ApplicationLease* FindLease(const std::string& leaseId) const; @@ -126,6 +126,7 @@ class ApplicationManager { void RetireWindow(const std::string& windowId); void RetireClosedWindows(const std::string& leaseId); bool IsWindowClosed(const WindowRecord& record) const; + bool HasRegisteredWindowIdentity(HWND window, const ApplicationLease& lease) const; bool ForegroundBelongsToLease(const ApplicationLease& lease, HWND foreground) const; json::Value PublicationDiagnostic(const ApplicationLease& lease) const; ApplicationLease& CreateLease(std::string ownership, DWORD processId, HANDLE process, HWND primaryWindow, diff --git a/packages/agentic/desktop-driver/native/windows/src/automation.cpp b/packages/agentic/desktop-driver/native/windows/src/automation.cpp index b9e8557002..1d5376b427 100644 --- a/packages/agentic/desktop-driver/native/windows/src/automation.cpp +++ b/packages/agentic/desktop-driver/native/windows/src/automation.cpp @@ -474,46 +474,54 @@ void Automation::Initialize() { automation_ = automation; } -bool Automation::HasWindowContent(HWND window, DWORD processId, const CancellationToken& token) { +bool Automation::HasWindowContent(HWND window, DWORD processId, ULONGLONG expiresAt, const CancellationToken& token) { token.ThrowIfCancelled(); + const ULONGLONG now = GetTickCount64(); + if (now >= expiresAt) return false; + const DWORD timeoutMs = static_cast(std::min(expiresAt - now, kProviderCallTimeoutMs)); const auto automation = automation_; const auto walker = contentWalker_; const auto content = std::make_shared(false); try { - const HRESULT result = BoundedRead( - [automation, walker, window, processId, content]() { + const HRESULT result = RunBoundedProviderCall( + ProviderCall::Read, [automation, walker, window, processId, content, expiresAt]() { + if (GetTickCount64() >= expiresAt) return kUiaTimeoutResult; + // A late read must not start another transaction after the deadline. + const auto checked = [expiresAt](HRESULT result) { + return SUCCEEDED(result) && GetTickCount64() >= expiresAt ? kUiaTimeoutResult : result; + }; winrt::com_ptr root; - HRESULT result = automation->ElementFromHandle(window, root.put()); + HRESULT result = checked(automation->ElementFromHandle(window, root.put())); if (FAILED(result) || !root) return FAILED(result) ? result : E_FAIL; int rootProcessId = 0; - result = root->get_CurrentProcessId(&rootProcessId); + result = checked(root->get_CurrentProcessId(&rootProcessId)); if (FAILED(result)) return result; if (rootProcessId != static_cast(processId)) return S_FALSE; UIA_HWND rootWindow{}; - result = root->get_CurrentNativeWindowHandle(&rootWindow); + result = checked(root->get_CurrentNativeWindowHandle(&rootWindow)); if (FAILED(result)) return result; if (rootWindow != window) return S_FALSE; CONTROLTYPEID rootType = 0; - result = root->get_CurrentControlType(&rootType); + result = checked(root->get_CurrentControlType(&rootType)); if (FAILED(result)) return result; if (rootType == UIA_ToolTipControlTypeId) return S_FALSE; winrt::com_ptr child; - result = walker->GetFirstChildElement(root.get(), child.put()); + result = checked(walker->GetFirstChildElement(root.get(), child.put())); if (FAILED(result) || !child) return FAILED(result) ? result : S_FALSE; int childProcessId = 0; - result = child->get_CurrentProcessId(&childProcessId); + result = checked(child->get_CurrentProcessId(&childProcessId)); if (FAILED(result)) return result; BOOL isContent = FALSE; - result = child->get_CurrentIsContentElement(&isContent); + result = checked(child->get_CurrentIsContentElement(&isContent)); if (FAILED(result)) return result; CONTROLTYPEID type = 0; - result = child->get_CurrentControlType(&type); + result = checked(child->get_CurrentControlType(&type)); if (FAILED(result)) return result; *content = childProcessId == static_cast(processId) && isContent != FALSE && type != UIA_ToolTipControlTypeId; return S_OK; }, - "Reading the owned window content"); + timeoutMs, "Reading the owned window content", kErrorAutomationFailed); token.ThrowIfCancelled(); return SUCCEEDED(result) && *content; } catch (const HelperError&) { diff --git a/packages/agentic/desktop-driver/native/windows/src/automation.h b/packages/agentic/desktop-driver/native/windows/src/automation.h index c74433d3dd..5eb48e401c 100644 --- a/packages/agentic/desktop-driver/native/windows/src/automation.h +++ b/packages/agentic/desktop-driver/native/windows/src/automation.h @@ -81,7 +81,8 @@ class Automation { // A bounded content-view probe for an already identity-checked secondary // window. It retains no element records; unreadable content is not proven. - bool HasWindowContent(HWND window, DWORD processId, const CancellationToken& token); + // expiresAt is the shared windows-call deadline in GetTickCount64 units. + bool HasWindowContent(HWND window, DWORD processId, ULONGLONG expiresAt, const CancellationToken& token); std::vector SnapshotWindowTree(const WindowContext& context, const CancellationToken& token); std::vector Find(const WindowContext& context, const std::string& rootElementId, const Selector& selector, const CancellationToken& token); diff --git a/packages/agentic/desktop-driver/native/windows/src/driver.cpp b/packages/agentic/desktop-driver/native/windows/src/driver.cpp index b5ec2232be..65f1a418b7 100644 --- a/packages/agentic/desktop-driver/native/windows/src/driver.cpp +++ b/packages/agentic/desktop-driver/native/windows/src/driver.cpp @@ -635,7 +635,9 @@ CommandResult Driver::ExecuteCommand(const std::string& command, const json::Val result.result = json::Value::Array(); for (const WindowInfo& info : applications_.Windows( lease->StringField("id"), token, - [&](HWND window, DWORD processId) { return automation_.HasWindowContent(window, processId, token); })) { + [&](HWND window, DWORD processId, ULONGLONG expiresAt) { + return automation_.HasWindowContent(window, processId, expiresAt, token); + })) { result.result.Append(WindowSummary(info)); } return result; diff --git a/packages/agentic/desktop-driver/native/windows/src/selftest.cpp b/packages/agentic/desktop-driver/native/windows/src/selftest.cpp index bea2b6c11f..570f8cad76 100644 --- a/packages/agentic/desktop-driver/native/windows/src/selftest.cpp +++ b/packages/agentic/desktop-driver/native/windows/src/selftest.cpp @@ -965,12 +965,12 @@ class WindowThread { WindowThread(const WindowThread&) = delete; WindowThread& operator=(const WindowThread&) = delete; - HWND Create(const wchar_t* className, const wchar_t* title, WNDPROC procedure = DefWindowProcW) { + HWND Create(const wchar_t* className, const wchar_t* title, WNDPROC procedure = DefWindowProcW, HWND owner = nullptr) { HWND window = nullptr; - Run([&window, className, title, procedure]() { + Run([&window, className, title, procedure, owner]() { RegisterTestWindowClass(className, procedure); window = CreateWindowExW(WS_EX_NOACTIVATE | WS_EX_TOOLWINDOW, className, title, WS_POPUP, -32000, -32000, 64, 64, - nullptr, nullptr, GetModuleHandleW(nullptr), nullptr); + owner, nullptr, GetModuleHandleW(nullptr), nullptr); if (window != nullptr) { ShowWindow(window, SW_SHOWNOACTIVATE); } @@ -1458,6 +1458,85 @@ void TestWindowEventHooks() { "a closed lease has no event log"); } +void TestRegisteredToolWindowLiveness() { + WindowThread windows; + const HWND primary = windows.Create(L"FurnSelfTestToolPrimary", L"Tool primary"); + const HWND popup = windows.Create(L"FurnSelfTestToolPopup", L"", DefWindowProcW, primary); + ApplicationManager applications; + const CancellationToken token; + const std::string leaseId = applications.AdoptWindowForTesting(primary, L"Tool primary").id; + const auto findPopup = [popup](const std::vector& listed) { + const auto found = std::find_if(listed.begin(), listed.end(), + [popup](const WindowInfo& info) { return info.window == popup; }); + return found == listed.end() ? std::string{} : found->id; + }; + std::size_t probes = 0; + const auto initial = applications.Windows(leaseId, token, [&](HWND window, DWORD processId, ULONGLONG) { + probes += 1; + return window == popup && processId == GetCurrentProcessId(); + }); + const std::string popupId = findPopup(initial); + Check(!popupId.empty() && probes == 1, "an owned tool popup is admitted once after content proof"); + if (popupId.empty()) return; + + probes = 0; + const auto unavailable = applications.Windows(leaseId, token, [&](HWND, DWORD, ULONGLONG) { + probes += 1; + return false; + }); + Check(findPopup(unavailable) == popupId && probes == 0, + "a registered tool popup stays listable without repeating a failed content probe"); + + std::vector failed; + const std::string code = FailureCode([&]() { + failed = applications.Windows(leaseId, token, [&](HWND, DWORD, ULONGLONG) -> bool { + probes += 1; + throw HelperError(kErrorAutomationFailed, "Injected content probe failure."); + }); + }); + Check(code == "none" && findPopup(failed) == popupId && probes == 0, + "a registered tool popup never enters a failing provider probe"); + Check(findPopup(applications.Windows(leaseId, token)) == popupId, + "a registered tool popup remains listable when content discovery is unavailable"); + + const json::Value log = applications.WindowEventLog(leaseId); + const json::Value* publication = log.Find("publication"); + const json::Value* published = publication == nullptr ? nullptr : publication->Find("windows"); + const auto containsListedPopup = [](const json::Value& entry) { + return entry.StringField("className") == "FurnSelfTestToolPopup" && + entry.BoolField("listed") && entry.BoolField("tracked"); + }; + Check(published != nullptr && + std::any_of(published->Items().begin(), published->Items().end(), containsListedPopup), + "publication marks an admitted tool popup as both listed and tracked"); + + const HWND deferredFirst = windows.Create(L"FurnSelfTestDeferredTool", L"", DefWindowProcW, primary); + const HWND deferredSecond = windows.Create(L"FurnSelfTestDeferredTool", L"", DefWindowProcW, primary); + probes = 0; + const ULONGLONG started = GetTickCount64(); + const auto bounded = applications.Windows(leaseId, token, [&](HWND, DWORD, ULONGLONG expiresAt) { + probes += 1; + const ULONGLONG now = GetTickCount64(); + if (now < expiresAt) Sleep(static_cast(expiresAt - now + 10)); + return false; + }); + Check(probes == 1 && GetTickCount64() - started < 2500 && bounded.size() == 2 && findPopup(bounded) == popupId, + "one exhausted discovery deadline defers new popups but keeps registered windows listable"); + + probes = 0; + ULONGLONG sharedDeadline = 0; + bool shared = true; + const auto retried = applications.Windows(leaseId, token, [&](HWND window, DWORD, ULONGLONG expiresAt) { + probes += 1; + if (sharedDeadline == 0) sharedDeadline = expiresAt; + shared = shared && sharedDeadline == expiresAt; + return window == deferredFirst || window == deferredSecond; + }); + Check(probes == 2 && shared && retried.size() == 4 && findPopup(retried) == popupId, + "a later listing retries deferred candidates under one fresh shared discovery deadline"); + applications.CloseApplication(leaseId, token); +} + void TestDriverWindowEvents() { WindowThread windows; const HWND window = windows.Create(L"FurnSelfTestDriver", L"Driver window"); @@ -3871,6 +3950,7 @@ int RunSelfTest() { TestLeaseRegistrationFailure(); TestAttachedCloseKeepsQuarantine(); TestWindowEventHooks(); + TestRegisteredToolWindowLiveness(); TestDriverWindowEvents(); TestInputEnvironment(); TestInjectionIntegrity(); diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index dffe430bb8..c26be55d56 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -755,3 +755,103 @@ The post-review clean-cache repository aggregate and final native/physical reruns are still pending. Outstanding qualification stays tracked in [remaining work](remaining-work.md) / #4441; no issue is closed and no final-helper physical-success claim is made here. + +## Windows Fabric and Win32 Paper — popup liveness review fixes — 2026-11-12 UTC + +**Observed source base:** `96b91661e2886aa4a22bd3f08b485e51d11c90d7` +plus the then-uncommitted review fixes below. This section supplements the +earlier Windows evidence; it does not rerun or relabel that evidence. The +earlier section's final helper was the pre-review `b47212dc…` artifact. +No macOS code, ledger section, or qualification changed. + +### Admission, liveness, and publication + +Registered untitled popups now bypass UIA content discovery when their lease, +process, thread/class, primary identity, and owned-content native predicates +still match. A failed content read or temporary re-layout therefore cannot +remove a live, admitted popup from `windows`. New candidates share one +1,000 ms deadline per listing; UIA waits use its remaining time, late reads +stop before another transaction, and unprobed candidates retry later. +Exhausting discovery does not skip registered windows later in the listing. +Publication diagnostics now include admitted live popups as `listed` as well +as `tracked`, without another content probe. + +**Test-first authoring:** the gated fixture regressions and native self-test +were added before production edits. The self-test injects false, throwing, +and unavailable content probes, an exhausted shared deadline, a later retry, +and listed/tracked publication assertions. The fixture checks hung-provider +popup retention/rectangle reads and bounded new-popup discovery/recovery; +its existing admission case also checks publication. Native red/green +execution is **Not run**, not inferred as passed: the desktop remains paused. + +### Compile-only and portable validation + +The helper-only build used the shared W3 lock, no campaign, and no launch. +The following build command is shown with its cache-root argument relative to +the driver package; the recorded invocation resolved that path absolutely: + +```powershell +yarn desktop-driver build-driver --platform windows --cache-root ..\..\..\artifacts\desktop\w3-review\native-cache +``` + +**Passed** x64 release compilation, including the new native self-test. +Lock acquisition was at `2026-11-12T17:45:53Z`, and the wrapper released +successfully with exit 0. No helper, fixture, REX, packaged app, or Metro was +launched. This is compilation, not native runtime qualification. + +- Artifact: `eb91ef8a4164080bcae787aacce3fd5aa5556d8c3d0b22d0cd0c661a50934066` +- Source digest: `89cc81fbb72cb8a79852e5d4d90ce96d8b2982283086236b860289f2664ac557` +- Build fingerprint: `7b38faa5c0eeba1ace7441d613a3f0c126915f78eff58c4a32466825c4e11854` +- Build log: `artifacts\desktop\w3-review\helper-build.log` + +From `packages\agentic\desktop-driver`, with native opt-in unset: + +```powershell +Remove-Item Env:FURN_NATIVE_DRIVER_TEST -ErrorAction SilentlyContinue +yarn test --runInBand win32Fixture nativeDriver +yarn test --runInBand +``` + +**Passed:** focused portable checks, eight tests; 20 native tests **skipped**. +The full portable suite passed 73 suites / 1,201 tests, with eight native +suites / 95 tests **skipped**, not passed. Logs are +`tests-authored-portable.log`, `focused-portable-final.log`, and +`portable-tests.log` beneath `artifacts\desktop\w3-review`. + +From the checkout root, **passed**: + +```powershell +yarn workspace @fluentui-react-native/desktop-driver format +yarn workspace @fluentui-react-native/desktop-driver lint +yarn workspace @fluentui-react-native/desktop-driver build +``` + +The README, protocol, helper/fixture references, and patch changeset describe +the same policy. FDR1 fields, commands, and `native/protocol.json` remain +unchanged. Root `yarn oxfmt --check`, `git diff --check`, and +`yarn markdown-link-check -c .github\markdown-link-check-config.json ` +**passed** for all seven changed Markdown files (66 links). +Committed changeset validation follows this review-fix commit. + +### Remaining session-gated runs + +One permitted check at **2026-11-12T17:52:11Z** observed remote session 1 as +Disconnected, locked, LogonUI running, with no input desktop. No GUI or native +contract was attempted; no reconnect or session move was attempted. + +Still **Not run (session disconnected / W3 desktop grant required)**: + +- Final-helper Fabric main/secondary semantic case. +- Fabric and Paper secondary physical clicks, under `-RequirePhysical`. +- Final native selector below, including the new failed/hung-probe, + shared-deadline/retry, and publication regressions plus the native self-test. + +```powershell +$env:FURN_NATIVE_DRIVER_TEST = '1' +yarn test --runInBand operations.contract win32Fixture.contract nativeDriver +``` + +The clean-cache repository aggregate is **Not run** in this review-fix +segment; W1 holds the buildci-only grant. It remains required before +publication. The native cases remain in [remaining work](remaining-work.md) +and #4441; no new native pass, physical success, or issue closure is claimed. diff --git a/packages/agentic/desktop-driver/references/native-helpers.md b/packages/agentic/desktop-driver/references/native-helpers.md index 2f2a537882..ced392c5d8 100644 --- a/packages/agentic/desktop-driver/references/native-helpers.md +++ b/packages/agentic/desktop-driver/references/native-helpers.md @@ -279,7 +279,10 @@ An application lease can register an untitled, visible, positive-size tool popup only when its exact-process owner chain reaches the verified primary window and a bounded UIA read proves exact-process content. Known IME/tooltip helper classes, unrelated owner chains, hidden or zero-size roots, and -unreadable or empty content stay excluded. This does not broaden global title discovery. +unreadable or empty content stay excluded from initial admission. Once admitted, +a popup stays listable using native identity and ownership rather than another +UIA read. New candidates share one 1,000 ms discovery budget per listing and +are retried later when that budget is exhausted. This does not broaden global title discovery. See the [Windows identity rules](../native/windows/README.md#window-events-and-identity) and [Windows qualification evidence](current-status.md). diff --git a/packages/agentic/desktop-driver/references/remaining-work.md b/packages/agentic/desktop-driver/references/remaining-work.md index b7042f122a..f2814fbe58 100644 --- a/packages/agentic/desktop-driver/references/remaining-work.md +++ b/packages/agentic/desktop-driver/references/remaining-work.md @@ -1094,8 +1094,10 @@ passes the real Paper secondary semantic case. See the separate proof of the legacy Callout's native command outcome. - The final helper reruns `yarn test --runInBand operations.contract win32Fixture.contract nativeDriver` - with `$env:FURN_NATIVE_DRIVER_TEST = '1'`. -- Coordinator review and the clean-cache repository aggregate complete. + with `$env:FURN_NATIVE_DRIVER_TEST = '1'`, including registered-popup + liveness during failed/hung content probes, the shared discovery deadline + and deferred retry, and truthful listed/tracked publication. +- Coordinator review follow-ups and the clean-cache repository aggregate complete. These runs are paused because session 1 disconnected, locked, and lost its input desktop. Resume native/GUI/physical work only with an Active, unlocked diff --git a/packages/agentic/desktop-driver/src/native/windows/win32Fixture.contract.test.ts b/packages/agentic/desktop-driver/src/native/windows/win32Fixture.contract.test.ts index be78d57c9a..a9f4c55bcc 100644 --- a/packages/agentic/desktop-driver/src/native/windows/win32Fixture.contract.test.ts +++ b/packages/agentic/desktop-driver/src/native/windows/win32Fixture.contract.test.ts @@ -32,6 +32,7 @@ type Retention = { privateBytes: number; retainedElementLimit: number; retainedE type WindowEventReport = { events: { descriptor?: { title?: string }; notification: string; windowId?: string }[]; observing: boolean; + publication: { windows: { className: string; listed: boolean; tracked: boolean }[] }; retention: Retention; source: string; }; @@ -267,6 +268,9 @@ describe('Windows helper against a plain Win32 application', () => { return windows.length === 2 ? windows.find((window) => window.id !== session.main.id) : undefined; }); expect(secondary.title).toBe(''); + const tools = (await session.eventReport()).publication.windows.filter((window) => window.className === 'FurnWin32FixtureTool'); + expect(tools.filter((window) => window.listed)).toEqual([expect.objectContaining({ listed: true, tracked: true })]); + expect(tools.filter((window) => !window.tracked).every((window) => !window.listed)).toBe(true); expect(await session.control(secondary.id, '3101')).toMatchObject({ name: 'Tool action', role: 'button' }); await session.click('3101', secondary.id); await waitFor('the secondary action to reach the application', async () => (await session.status()) === 'Tool activated 1 times'); @@ -280,6 +284,56 @@ describe('Windows helper against a plain Win32 application', () => { } }); + windowsTest('keeps an admitted owned popup listable while its provider is hung', async () => { + const session = await FixtureSession.start('tool-hang', { args: ['--tool-windows', '1', '--hang-ms', '22000'] }); + try { + const hang = await session.control(session.main.id, '1007'); + await session.click('1012'); + await waitFor('the tool roots to open', async () => (await session.status()) === 'Opened 7 tool windows'); + const secondary = await waitFor('the owned tool popup to register', async () => + (await session.windows()).find((window) => window.id !== session.main.id), + ); + await session.press(hang); + await delay(300); + + for (let attempt = 0; attempt < 2; attempt += 1) { + const startedAt = Date.now(); + const listed = await session.windows(); + expect(Date.now() - startedAt).toBeLessThan(2_500); + expect(listed.map((window) => window.id)).toEqual(expect.arrayContaining([session.main.id, secondary.id])); + expect(await session.call('getWindowRect', { windowId: secondary.id })).toMatchObject(secondary.rect); + } + } finally { + await session.dispose(); + } + }); + + windowsTest('bounds new-popup discovery while its provider is hung and recovers later', async () => { + const session = await FixtureSession.start('tool-probe-budget', { args: ['--tool-windows', '1', '--hang-ms', '22000'] }); + try { + const hang = await session.control(session.main.id, '1007'); + await session.click('1012'); + await waitFor('the tool roots to open without listing them', async () => (await session.status()) === 'Opened 7 tool windows'); + await session.press(hang); + await delay(300); + + const startedAt = Date.now(); + const listed = await session.windows(); + expect(Date.now() - startedAt).toBeLessThan(2_500); + expect(listed).toContainEqual(session.main); + // Providers that still answer may prove content even while the UI thread is busy. + expect(listed.length).toBeLessThanOrEqual(2); + + await waitFor('the provider to respond again', async () => (await session.status().catch(() => undefined)) === 'Hang finished', { + intervalMs: 250, + timeoutMs: 40_000, + }); + await waitFor('the deferred content popup to register', async () => (await session.windows()).length === 2); + } finally { + await session.dispose(); + } + }); + windowsTest('refuses input to an owner while its popup is open and reports the popup closing', async () => { const session = await FixtureSession.start('popup'); try { From cad25f8c53855c30f2c45f3ce6655f668d09bdc9 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 09:59:07 -0800 Subject: [PATCH 05/15] docs: record W3 review-fix validation Record the review-fix commit, committed patch validation, clean audit, and remaining session-gated native checks without claiming runtime qualification. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .../agentic/desktop-driver/references/current-status.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index c26be55d56..0a6ca144b3 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -831,7 +831,13 @@ the same policy. FDR1 fields, commands, and `native/protocol.json` remain unchanged. Root `yarn oxfmt --check`, `git diff --check`, and `yarn markdown-link-check -c .github\markdown-link-check-config.json ` **passed** for all seven changed Markdown files (66 links). -Committed changeset validation follows this review-fix commit. +The review-fix implementation is committed as +`4d39582f2292ff6436dafaba646ee63cb9bdc3cc`. +Its committed `yarn change:check` **passed**; the log is +`artifacts\desktop\w3-review\change-check.log`. +The process audit passed with +`FURN-AUDIT: no FURN desktop processes found`; no W3 campaign was started, +and shared campaign status reported no campaign or holder. No push was made. ### Remaining session-gated runs From 4208ab36d641bcfe3675836634f18dee2072ef49 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 10:54:49 -0800 Subject: [PATCH 06/15] docs: record W3 default-environment publication validation Record the unchanged-source 178-task aggregate pass, reproduced baseline timing failures, owned-process drain, and remaining native qualification. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .../references/current-status.md | 65 +++++++++++++++++++ .../references/remaining-work.md | 20 +++--- 2 files changed, 77 insertions(+), 8 deletions(-) diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index 0a6ca144b3..029ede9296 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -861,3 +861,68 @@ The clean-cache repository aggregate is **Not run** in this review-fix segment; W1 holds the buildci-only grant. It remains required before publication. The native cases remain in [remaining work](remaining-work.md) and #4441; no new native pass, physical success, or issue closure is claimed. + +## Windows Fabric and Win32 Paper — default-environment publication validation — 2026-11-12 UTC + +**Observed tested source:** `cad25f8c53855c30f2c45f3ce6655f668d09bdc9`. +The coordinator authorized buildci-only use of the shared W3 lock, followed +by draft publication. No app, native contract, helper runtime, or physical +input was launched in this segment. Native opt-in remained unset. TEMP and +concurrency used their defaults; no workaround, guidance, or timeout edits +were made. + +The exact root validation command, inside the W3 machine-wide lock: + +```powershell +cmd.exe /d /c "yarn clean-lage-cache && yarn lage buildci" +``` + +### Failure comparison and unchanged-source retry + +- **Failed, first branch attempt:** lock acquired at `18:33:59Z`, released + at `18:36:20Z`; 136 tasks succeeded and Storybook desktop tests failed. + Four checks failed: the WDIO worker's 7,000 ms deadline, two 10,000 ms + diagnostic timeouts, and the server-integration stop result. +- **Observed cleanup:** Lage exited before its desktop-test descendants. + No independent WDIO command had been started outside the lock. The + descendants finished naturally; their recorded root PID was already gone + when checked. A subsequent process audit exited 0. +- **Failed on clean baseline too:** the same default-environment command + ran on isolated detached `505a2187b0a2451aed7081c9260b343d55742f93`. + It reproduced the WDIO deadline and server-stop readiness race; baseline + diagnostics passed. There were 138 successful tasks and one failed task. + All baseline children drained before the lock released at `18:45:14Z`. + The disposable checkout remained clean and was removed afterward. +- Storybook source/config, desktop-test source/config, scripts, lockfile, + and Lage configuration were byte-unchanged between the baseline and this + branch. **Inferred:** these load-sensitive failures were not caused by + the Windows activation/popup changes; no unrelated source fix was made. +- **Passed, unchanged-source branch retry:** the lock was acquired at + `18:48:10Z`, validation started at `18:48:11Z`, and all **178 tasks passed** + at `18:50:53Z` (zero failed, aborted, pending, or skipped Lage tasks). + Owned descendant drain completed before lock release. This is a task-level + aggregate result, not a claim that opt-in native cases executed. + +The local ignored supervisor `run-buildci-and-drain.ps1` runs the exact +command above and keeps the wrapper alive until owned Node descendants +finish. It does not alter the test environment or send input. Logs are +retained beneath `artifacts\desktop\w3-buildci`: +`buildci-default.log`, `baseline-buildci-default.log`, and +`buildci-default-rerun.log`. + +At `2026-11-12T18:52:16Z`, the post-run audit exited 0: +`FURN-AUDIT: no FURN desktop processes found`. Shared state reported no +campaign or holder. W3 started no campaign; its build-only lock was released. + +### Publication scope and remaining native work + +The final local Fabric semantic case, both final-helper secondary physical +clicks, and final native contracts remain **Not run**. The earlier disconnect +prevented them; after the session became Active/unlocked again, the desktop +was reserved for W1, not a W3 native campaign. No reconnect, session move, +display change, or native retry occurred here. The current review helper +remains compile-only as recorded above. + +The default-environment aggregate requirement is now satisfied. Draft +publication does not close #4441 or replace its outstanding native, 200%, +mixed-DPI, negative-origin, or separate #4491 console-Ctrl+C qualification. diff --git a/packages/agentic/desktop-driver/references/remaining-work.md b/packages/agentic/desktop-driver/references/remaining-work.md index f2814fbe58..008b0b47d1 100644 --- a/packages/agentic/desktop-driver/references/remaining-work.md +++ b/packages/agentic/desktop-driver/references/remaining-work.md @@ -1097,14 +1097,18 @@ passes the real Paper secondary semantic case. See the separate with `$env:FURN_NATIVE_DRIVER_TEST = '1'`, including registered-popup liveness during failed/hung content probes, the shared discovery deadline and deferred retry, and truthful listed/tracked publication. -- Coordinator review follow-ups and the clean-cache repository aggregate complete. - -These runs are paused because session 1 disconnected, locked, and lost its -input desktop. Resume native/GUI/physical work only with an Active, unlocked -session and the serialized campaign; physical work additionally requires -`-RequirePhysical`. No reconnect, session move, or skip-to-pass substitution -is authorized. Earlier eligible physical results remain evidence for their -tested revisions, not final-helper reruns. + +The review follow-ups are implemented, and the default-environment clean-cache +repository aggregate passed all 178 tasks at `cad25f8c5`. Native runs were +paused when session 1 disconnected, locked, and lost its input desktop. +After it became Active/unlocked again, the desktop was reserved for W1; +W3 had only a buildci grant. Resume native/GUI/physical work only with an +Active, unlocked session, an explicit W3 desktop grant, and the serialized +campaign; physical work additionally requires `-RequirePhysical`. No +reconnect, session move, or skip-to-pass substitution is authorized. +Earlier eligible physical results remain evidence for their tested +revisions, not final-helper reruns. Authorized draft publication leaves +these qualification gaps open. ## 9. Qualification and efficiency gates From 28c7690085342a3bc2ee114b2c4c2032d7d050c4 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 11:25:46 -0800 Subject: [PATCH 07/15] fix: distinguish native menu popups in legacy assertion checks Read a live session window tree without activation, preserve unknown-window failures, and keep macOS and non-native behavior unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .changeset/quiet-menu-windows.md | 5 ++ apps/E2E/README.md | 8 ++ apps/E2E/desktop-adapters.test.cjs | 90 ++++++++++++++++++- apps/E2E/src/common/BasePage.ts | 12 +-- apps/E2E/src/common/nativeDesktop.ts | 25 ++++++ .../desktop-driver/references/protocol.md | 8 ++ .../src/client/DesktopDriverClient.ts | 5 ++ .../src/server/createDesktopDriverServer.ts | 13 +++ .../src/server/desktopDriver.test.ts | 57 ++++++++++++ 9 files changed, 217 insertions(+), 6 deletions(-) create mode 100644 .changeset/quiet-menu-windows.md diff --git a/.changeset/quiet-menu-windows.md b/.changeset/quiet-menu-windows.md new file mode 100644 index 0000000000..23a69f8901 --- /dev/null +++ b/.changeset/quiet-menu-windows.md @@ -0,0 +1,5 @@ +--- +'@fluentui-react-native/desktop-driver': patch +--- + +Add a session-scoped, non-activating window tree query so consumers can distinguish legitimate secondary menus from unexpected dialogs without dismissing the menu. diff --git a/apps/E2E/README.md b/apps/E2E/README.md index 686c456e29..31456d95e0 100644 --- a/apps/E2E/README.md +++ b/apps/E2E/README.md @@ -138,6 +138,14 @@ Native setup retains initial window bounds: it does **not** emulate the old maximize hook. Popup lookup remains scoped to the selected window. +On Windows and Win32, the assertion-dialog safety check inspects each secondary +window without selecting or activating it, so it cannot dismiss an open menu. +Only a single accessibility `menu` root without nested dialogs or windows is +recognized as an expected popup. Other secondary windows still fail the check; +unreadable windows fail the test rather than being treated as no assertion. +The native-task-library bypass, Android page check, and other runners' existing +window-count check are unchanged. + WebdriverIO 10 throws a `StrictSelectorError` when a selector matches several elements, where WebdriverIO 9 took the first match. A native tree can expose several elements for one `testID` (a container and its children), and the page diff --git a/apps/E2E/desktop-adapters.test.cjs b/apps/E2E/desktop-adapters.test.cjs index 84f39b1f03..75894876a6 100644 --- a/apps/E2E/desktop-adapters.test.cjs +++ b/apps/E2E/desktop-adapters.test.cjs @@ -1,13 +1,101 @@ const assert = require('node:assert/strict'); -const { afterEach, test } = require('node:test'); +const { afterEach, mock, test } = require('node:test'); const { BasePage } = require('./lib/common/BasePage.js'); const { Keys } = require('./lib/common/consts.js'); const { installLegacyDesktopAttributes, scrollNativeDesktopElement, sendNativeDesktopKeys } = require('./lib/common/nativeDesktop.js'); afterEach(() => { + mock.restoreAll(); delete globalThis.browser; }); +test('Windows assertion checks inspect menu windows without selecting or activating them', async () => { + const reads = []; + globalThis.browser = { + capabilities: { browserName: 'furn-native-desktop', platformName: 'windows', 'furn:endpoint': 'win32' }, + options: { protocol: 'http', hostname: '127.0.0.1', port: 1, path: '/' }, + sessionId: 'session', + getWindowHandles: async () => ['main', 'menu'], + getWindowHandle: async () => 'main', + switchToWindow: () => assert.fail('an assertion check must not change native focus'), + }; + mock.method(globalThis, 'fetch', async (url, init) => { + reads.push(String(url)); + assert.equal(init.method, 'GET'); + return new Response(JSON.stringify({ value: [{ role: 'menu', children: [{ role: 'menuitem', children: [] }] }] })); + }); + assert.equal(await new BasePage().didAssertPopup(), false); + // eslint-disable-next-line @microsoft/sdl/no-insecure-url -- mocked loopback transport; no service is contacted + assert.deepEqual(reads, ['http://127.0.0.1:1/session/session/furn/window/menu/tree']); + browser.getWindowHandles = async () => ['main']; + browser.getWindowHandle = () => assert.fail('one window needs no further inspection'); + assert.equal(await new BasePage().didAssertPopup(), false); + assert.equal(reads.length, 1); +}); + +test('Windows assertion checks fail closed for dialogs, unrecognized roots, and unreadable windows', async () => { + globalThis.browser = { + capabilities: { browserName: 'furn-native-desktop', platformName: 'windows', 'furn:endpoint': 'windows' }, + options: { protocol: 'http', hostname: '127.0.0.1', port: 1, path: '/' }, + sessionId: 'session', + getWindowHandles: async () => ['main', 'other'], + getWindowHandle: async () => 'main', + switchToWindow: () => assert.fail('an assertion check must not change native focus'), + }; + let value; + const reader = mock.method(globalThis, 'fetch', async () => new Response(JSON.stringify({ value }))); + const page = new BasePage(); + for (const roots of [ + [{ role: 'window', name: 'Assertion failed', children: [] }], + [{ role: 'group', children: [] }], + [], + [{ role: 'menu', children: [{ role: 'dialog', children: [] }] }], + [{ role: 'menu', children: [{ role: 'window', children: [] }] }], + [ + { role: 'menu', children: [] }, + { role: 'dialog', children: [] }, + ], + ]) { + value = roots; + assert.equal(await page.didAssertPopup(), true); + } + reader.mock.mockImplementation(async () => { + throw new Error('provider is unavailable'); + }); + await assert.rejects(page.didAssertPopup(), /provider is unavailable/); +}); + +test('Windows assertion checks do not let an open menu hide another unexpected window', async () => { + globalThis.browser = { + capabilities: { browserName: 'furn-native-desktop', platformName: 'windows', 'furn:endpoint': 'win32' }, + options: { protocol: 'http', hostname: '127.0.0.1', port: 1, path: '/' }, + sessionId: 'session', + getWindowHandles: async () => ['main', 'menu', 'assert'], + getWindowHandle: async () => 'main', + }; + mock.method( + globalThis, + 'fetch', + async (url) => + new Response( + JSON.stringify({ + value: [{ role: String(url).endsWith('/menu/tree') ? 'menu' : 'window', children: [] }], + }), + ), + ); + assert.equal(await new BasePage().didAssertPopup(), true); +}); + +test('assertion checks retain non-native and macOS window-count behavior', async () => { + for (const capabilities of [ + { browserName: 'non-native' }, + { browserName: 'furn-native-desktop', platformName: 'macos', 'furn:endpoint': 'macos' }, + ]) { + globalThis.browser = { capabilities, getWindowHandles: async () => ['main', 'other'] }; + assert.equal(await new BasePage().didAssertPopup(), true); + } +}); + test('native checkbox and switch state never falls back to selected', async () => { const page = new BasePage(); const element = { elementId: 'toggle', isSelected: () => assert.fail('checked is not selected') }; diff --git a/apps/E2E/src/common/BasePage.ts b/apps/E2E/src/common/BasePage.ts index 77f7538c5a..f40b2377f2 100644 --- a/apps/E2E/src/common/BasePage.ts +++ b/apps/E2E/src/common/BasePage.ts @@ -12,7 +12,7 @@ import { TESTPAGE_BUTTONS_SCROLLVIEWER, TESTPAGE_CONTENT_SCROLLVIEWER, } from './consts.js'; -import { isNativeDesktop, scrollNativeDesktopElement, sendNativeDesktopKeys } from './nativeDesktop.js'; +import { hasUnexpectedNativeDesktopWindow, isNativeDesktop, scrollNativeDesktopElement, sendNativeDesktopKeys } from './nativeDesktop.js'; import type { PageElement } from './nativeDesktop.js'; const DUMMY_CHAR = ''; @@ -440,9 +440,9 @@ export abstract class BasePage { * WHY/HOW: Unfortunately, some Asserts don't take focus away from the test app. Because of this, the test execution is able * to continue without realizing an assert as popped up (unless the assert is a crashing one). * - * In order to detect an assert, this function gets the number of window handles currently open within the test app. - * Only one instance SHOULD be open at a time (the app itself). If another instance of the app is open, we know an assert dialogue - * has popped up. With this information, we know to fail the test. + * Unexpected secondary windows can indicate a non-focusing assert. Windows native menus are legitimate secondary + * windows, so inspect their accessibility roots without changing focus. Unrecognized windows still fail the check, + * and an unreadable window fails the test rather than silently claiming no assert. * * Additionally, you might think it's logical to simply place this in the afterEach() hook so it's called after every test (rather than duplicating this call in every test). * Unfortunately, afterEach() is designed for setup/teardown - not for determining if a test should fail or not. @@ -456,8 +456,10 @@ export abstract class BasePage { if (PLATFORM === 'android') { return !(await this.isPageLoaded()); } - // If more than 1 instance of the app is open, we know an assert dialogue popped up. const windowHandles = await browser.getWindowHandles(); + if (isNativeDesktop() && browser.capabilities.platformName === 'windows') { + return hasUnexpectedNativeDesktopWindow(windowHandles); + } return windowHandles.length > 1; } diff --git a/apps/E2E/src/common/nativeDesktop.ts b/apps/E2E/src/common/nativeDesktop.ts index 890cdd91a2..3505db9d91 100644 --- a/apps/E2E/src/common/nativeDesktop.ts +++ b/apps/E2E/src/common/nativeDesktop.ts @@ -1,3 +1,4 @@ +import { createDesktopDriverClient, DesktopSessionClient } from '@fluentui-react-native/desktop-driver'; import { Keys } from './consts.js'; /** A page object getter, or the element it has already resolved to. */ @@ -7,6 +8,30 @@ export function isNativeDesktop(): boolean { return browser.capabilities.browserName === 'furn-native-desktop'; } +export async function hasUnexpectedNativeDesktopWindow(handles: readonly string[]): Promise { + if (handles.length <= 1) return false; + const current = await browser.getWindowHandle(); + const url = `${browser.options.protocol}://${browser.options.hostname}:${browser.options.port}${browser.options.path}`; + const session = new DesktopSessionClient( + createDesktopDriverClient({ url }), + browser.sessionId, + Object.fromEntries(Object.entries(browser.capabilities)), + ); + for (const handle of handles) { + if (handle === current) continue; + // Switching windows would activate the target and can dismiss the menu being tested. + const roots = await session.getWindowTree(handle); + if (roots.length !== 1 || roots[0].role !== 'menu') return true; + const pending = [...roots]; + while (pending.length > 0) { + const node = pending.pop()!; + if (node.role === 'dialog' || node.role === 'window') return true; + pending.push(...node.children); + } + } + return false; +} + export async function sendNativeDesktopKeys(element: PageElement, keys: readonly Keys[]): Promise { const target = await element; await browser.elementSendKeys(await target.elementId, ''); diff --git a/packages/agentic/desktop-driver/references/protocol.md b/packages/agentic/desktop-driver/references/protocol.md index 01b62dc7fb..1398b965c9 100644 --- a/packages/agentic/desktop-driver/references/protocol.md +++ b/packages/agentic/desktop-driver/references/protocol.md @@ -84,6 +84,8 @@ Implemented `furn` extensions: - `POST /session/{id}/furn/story/reset`; - `POST /session/{id}/furn/story/args`; - `GET /session/{id}/furn/tree`; +- `GET /session/{id}/furn/window/{window-id}/tree`, reading a currently listed + session window without selecting or activating it (`DesktopSessionClient.getWindowTree`); - `GET /session/{id}/furn/window-events`, when the `windowEvents` feature is advertised in `furn:features`; - `GET /session/{id}/furn/ops` and `POST /session/{id}/furn/ops/{name}`, the @@ -98,6 +100,12 @@ with role, subrole, title, frame, and the matching WindowServer window so transient popups and sheets remain identifiable after they close. The log is evidence for diagnosis only; window and element queries remain authoritative. +The explicit-window tree query refreshes the session's window list and refuses +an unlisted or closed ID with `no such window` before asking for its tree. It +does not change the selected window, native focus, input state, or preview +generation. Like `furn/tree`, it returns a full compact tree rather than a +budgeted agent observation; use agent operations for bounded observations. + Navigation, history, cookies, frames, shadow roots, prompts, printing, arbitrary JavaScript execution, browser CSS values, and new-window creation return `unsupported operation`. diff --git a/packages/agentic/desktop-driver/src/client/DesktopDriverClient.ts b/packages/agentic/desktop-driver/src/client/DesktopDriverClient.ts index f321c6be9a..9f2c3b0cd0 100644 --- a/packages/agentic/desktop-driver/src/client/DesktopDriverClient.ts +++ b/packages/agentic/desktop-driver/src/client/DesktopDriverClient.ts @@ -274,6 +274,11 @@ export class DesktopSessionClient { return this.command('GET', '/furn/tree', undefined, options?.signal); } + /** Reads a live window in this session without selecting it or changing native focus. */ + getWindowTree(windowId: string, options?: DesktopClientCallOptions): Promise { + return this.command('GET', `/furn/window/${encodeURIComponent(windowId)}/tree`, undefined, options?.signal); + } + getWindowEvents(options?: DesktopClientCallOptions): Promise { return this.command('GET', '/furn/window-events', undefined, options?.signal); } diff --git a/packages/agentic/desktop-driver/src/server/createDesktopDriverServer.ts b/packages/agentic/desktop-driver/src/server/createDesktopDriverServer.ts index 26911744e7..1ed52153f0 100644 --- a/packages/agentic/desktop-driver/src/server/createDesktopDriverServer.ts +++ b/packages/agentic/desktop-driver/src/server/createDesktopDriverServer.ts @@ -554,6 +554,19 @@ async function route(context: RouteContext, sessions: SessionManager, targets: T ); return buildDesktopTree(snapshots); } + if (method === 'GET' && command[1] === 'window' && command[2] && command[3] === 'tree' && command.length === 4) { + const windowId = command[2]; + session.windows = await hostCommand(session, 'Reading application windows', (signal) => + session.target.host.windows(session.lease, signal), + ); + if (!session.windows.some(({ id }) => id === windowId)) { + throw new WebDriverError('no such window', `Window "${windowId}" does not exist in this session.`); + } + const snapshots = await hostCommand(session, 'Reading a window accessibility tree', (signal) => + session.target.host.tree(windowId, signal), + ); + return buildDesktopTree(snapshots); + } if (method === 'GET' && command[1] === 'window-events' && command.length === 2) { const windowEvents = session.target.host.windowEvents?.bind(session.target.host); if (!windowEvents) { diff --git a/packages/agentic/desktop-driver/src/server/desktopDriver.test.ts b/packages/agentic/desktop-driver/src/server/desktopDriver.test.ts index 4829fa8478..9fbdc0e26a 100644 --- a/packages/agentic/desktop-driver/src/server/desktopDriver.test.ts +++ b/packages/agentic/desktop-driver/src/server/desktopDriver.test.ts @@ -9,6 +9,63 @@ import { createDesktopDriverTestHarness } from '../testing/protocolHarness.js'; import type { DesktopStoryManifest, StoryOrchestrator, StoryReadyResult } from '../storybook.js'; describe('Desktop Driver W3C remote end', () => { + test('reads a specified live window without activation or changing the selected window', async () => { + const harness = await createDesktopDriverTestHarness({ + windows: [ + { id: 'main', title: 'Main', elements: [] }, + { + id: 'menu', + title: '', + elements: [ + { + id: 'menu-root', + role: 'menu', + scope: 'secondary-window', + windowId: 'menu', + rect: { x: 0, y: 0, width: 100, height: 100 }, + }, + ], + }, + ], + }); + try { + const client = createDesktopDriverClient({ url: harness.server.url }); + const session = await client.newSession({ + alwaysMatch: { platformName: 'windows', 'furn:target': harness.target.id }, + }); + const actions = [...harness.host.actions]; + const tree = await session.getWindowTree('menu'); + expect(tree).toEqual([expect.objectContaining({ role: 'menu' })]); + expect(await session.getWindowHandle()).toBe('main'); + expect(harness.host.actions).toEqual(actions); + await session.delete(); + } finally { + await harness.close(); + } + }); + + test('refuses unlisted and closed windows before trying to inspect their trees', async () => { + const harness = await createDesktopDriverTestHarness(); + try { + const client = createDesktopDriverClient({ url: harness.server.url }); + const session = await client.newSession({ + alwaysMatch: { platformName: 'windows', 'furn:target': harness.target.id }, + }); + const tree = jest.spyOn(harness.host, 'tree'); + const [window] = await session.getWindowHandles(); + for (const handle of ['unlisted', window]) { + if (handle === window) await harness.host.closeWindow(window); + await expect(session.getWindowTree(handle)).rejects.toMatchObject({ + code: 'no such window', + }); + } + expect(tree).not.toHaveBeenCalled(); + await session.delete(); + } finally { + await harness.close(); + } + }); + test.each([ undefined, { supported: false as const, reason: 'Not exposed.' }, From ad8b67c9720a00d7d8df4d27c2ca7436c78d6198 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 11:28:13 -0800 Subject: [PATCH 08/15] docs: record Windows hosted popup compatibility evidence Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .../references/current-status.md | 91 +++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index 029ede9296..cc6175bed7 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -926,3 +926,94 @@ remains compile-only as recorded above. The default-environment aggregate requirement is now satisfied. Draft publication does not close #4441 or replace its outstanding native, 200%, mixed-DPI, negative-origin, or separate #4491 console-Ctrl+C qualification. + +## Windows Fabric and Win32 Paper — hosted popup compatibility follow-up — 2026-11-12 UTC + +**Observed published revision:** `4208ab36d641bcfe3675836634f18dee2072ef49`, +draft PR #4498, workflow run `43284991535` (attempt 1). +**Tested portable correction:** `28c7690085342a3bc2ee114b2c4c2032d7d050c4`. +No native source or FDR1 schema changed in this follow-up. + +### Hosted evidence at the published revision + +- **Passed:** Windows Storybook's opt-in package command, + `yarn workspace @fluentui-react-native/desktop-driver test --runInBand` + with `FURN_NATIVE_DRIVER_TEST=1`: + 79 suites / 1,251 tests; two suites / 45 tests **skipped**. The Windows + fixture, operations, native-driver self-test, and other matching-OS native + suites actually ran. This includes the registered-popup hung-provider, + deferred-discovery, and listed/tracked publication regressions, rather than + merely compiling them. These are hosted fixture results, not the remaining + local real-application or physical-secondary reruns. +- **Passed:** Windows Storybook accessible smoke: 255 executed / 255 passed, + no failures, skips, unsupported cases, or retries. Internal navigation + diagnostics were retained; this count is the final selected-test summary, + not a claim that every diagnostic navigation attempt succeeded. +- **Passed:** Win32 Storybook physical smoke: 288 passed and one + unsupported/quarantined case; no failures or retries. The helper source + digest was `89cc81fbb72cb8a79852e5d4d90ce96d8b2982283086236b860289f2664ac557`, + matching the review helper. +- **Passed:** Windows legacy E2E, JS PR, both Win32 Storybook jobs, publishing, + links, changesets, and CodeQL. Other hosted jobs were still completing + during this follow-up; no whole-workflow pass is claimed. +- **Failed, deterministic:** Win32 legacy E2E executed 201 tests: 191 passed, + ten menu tests failed only at the final “An assert popped up” safety check. + Their menu-open/state checks had passed. The existing guard treated every + second window as an assertion dialog. All 20 retained failure captures + (including suite retries) showed the secondary accessibility root as `menu`. + Correctly registering native menu windows exposed that consumer assumption; + no flake rerun was requested. + +Logs and the bounded downloaded E2E artifact are retained under +`artifacts\desktop\w3-buildci`: `ci-windows-storybook.log`, +`ci-win32-physical.log`, `ci-win32-job.log`, and `ci-win32-e2e`. + +### Test-first correction and boundaries + +`DesktopSessionClient.getWindowTree` now reads a currently listed session +window without selecting or activating it. The server refreshes authoritative +window membership and rejects unknown/closed IDs before tree inspection. +Native focus, selected-window identity, and input state are unchanged. + +The Windows/Win32 legacy guard exempts only a single `menu` root without +nested dialogs or windows. Unrecognized secondary roots still fail; unreadable +windows reject rather than report no assertion. A menu cannot hide another +unexpected window. macOS, non-native runners, Android, and the native-task +library bypass keep their earlier behavior. Diagnostic window events are +not used as authority. The public route is documented in +[the WebDriver contract](protocol.md); it is a full tree query, not a budgeted +agent operation. + +From `apps\E2E`, `yarn test` first produced **two expected failures** for a +legitimate menu and an unreadable secondary. From `packages\agentic\desktop-driver`, +the two focused new routing cases first failed because the read-only route +did not exist. Logs: `e2e-assert-red.log`, `window-tree-red.log`. + +| Exact command | Result | +| ------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------- | +| `yarn test --runInBand desktopDriver.test DesktopDriverClient.test` (desktop-driver) | **Passed:** 52 tests, including non-activation and unknown/closed-window refusal. | +| `yarn test --runInBand` (desktop-driver, native opt-in unset) | **Passed:** 73 suites / 1,203 tests; eight suites / 95 native tests **skipped**. | +| `yarn workspace @fluentui-react-native/e2e-testing test` | **Passed:** all 11 adapter tests, including menus, possible assertions, failed reads, and unchanged other-runner behavior. | +| `yarn workspace @fluentui-react-native/desktop-driver build` | **Passed**. | +| `yarn workspace @fluentui-react-native/e2e-testing build` | **Passed**. | +| `yarn workspace @fluentui-react-native/desktop-driver lint` | **Passed**. | +| `yarn workspace @fluentui-react-native/e2e-testing lint` | **Passed** after marking the mocked loopback URL as test-only. | +| `yarn workspace @fluentui-react-native/desktop-driver format` | **Passed**. | +| `yarn oxfmt --check` | **Passed**. | +| `git diff --check` | **Passed**. | +| `yarn markdown-link-check -c .github\markdown-link-check-config.json ` | **Passed:** E2E README and public protocol, 25 links. | +| `yarn change:check` | **Passed** after committing the implementation and patch changeset `quiet-menu-windows.md`. | + +The new correction is committed locally and awaits coordinator review before +push. Its clean-cache aggregate and affected hosted E2E rerun are **not run +yet**; the earlier 178-task aggregate does not qualify this new source. +W1 holds the live desktop campaign. W3 has not launched an app, native helper, +fixture, WDIO suite, or physical action in this portable correction segment. +The final local Fabric semantic case, both final-helper secondary physical +clicks, and requested local native selector remain **not run**. Hosted +fixture passes do not substitute for those real-app/physical cases. + +The shared process audit returned 2: it listed W5's test tree and the W3 +formatter running concurrently with the audit. Both recorded W3 formatter +PIDs were confirmed exited afterward. No W3 native runtime or campaign is +left running; a globally clean audit is not claimed while peer work is active. From 36efc5a88dc507c626f2d29ac487cfc37e9e4bf3 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 11:33:59 -0800 Subject: [PATCH 09/15] fix: tolerate closing native menus in assertion checks Ignore only no-such-window tree reads and recheck live membership before reporting unexpected roots. Keep provider errors and persistent unexpected windows failing closed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- apps/E2E/README.md | 7 +++- apps/E2E/desktop-adapters.test.cjs | 61 ++++++++++++++++++++++++++++ apps/E2E/src/common/nativeDesktop.ts | 29 +++++++++---- 3 files changed, 87 insertions(+), 10 deletions(-) diff --git a/apps/E2E/README.md b/apps/E2E/README.md index 31456d95e0..94270c34e0 100644 --- a/apps/E2E/README.md +++ b/apps/E2E/README.md @@ -141,8 +141,11 @@ remains scoped to the selected window. On Windows and Win32, the assertion-dialog safety check inspects each secondary window without selecting or activating it, so it cannot dismiss an open menu. Only a single accessibility `menu` root without nested dialogs or windows is -recognized as an expected popup. Other secondary windows still fail the check; -unreadable windows fail the test rather than being treated as no assertion. +recognized as an expected popup. A tree read reporting `no such window` is +treated as a closed popup. Empty or unrecognized roots fail the check only +when a fresh window listing still includes that handle, avoiding false +assertions while a menu closes. Every other tree-read error still fails the +test rather than being treated as no assertion. The native-task-library bypass, Android page check, and other runners' existing window-count check are unchanged. diff --git a/apps/E2E/desktop-adapters.test.cjs b/apps/E2E/desktop-adapters.test.cjs index 75894876a6..0a3f5d50f9 100644 --- a/apps/E2E/desktop-adapters.test.cjs +++ b/apps/E2E/desktop-adapters.test.cjs @@ -86,6 +86,67 @@ test('Windows assertion checks do not let an open menu hide another unexpected w assert.equal(await new BasePage().didAssertPopup(), true); }); +test('Windows assertion checks skip a vanished menu and continue checking other windows', async () => { + let handles = ['main', 'closing-menu']; + globalThis.browser = { + capabilities: { browserName: 'furn-native-desktop', platformName: 'windows', 'furn:endpoint': 'win32' }, + options: { protocol: 'http', hostname: '127.0.0.1', port: 1, path: '/' }, + sessionId: 'session', + getWindowHandles: async () => handles, + getWindowHandle: async () => 'main', + switchToWindow: () => assert.fail('an assertion check must not change native focus'), + }; + mock.method(globalThis, 'fetch', async (url) => + String(url).endsWith('/closing-menu/tree') + ? new Response(JSON.stringify({ value: { error: 'no such window', message: 'The menu closed.' } }), { status: 404 }) + : new Response(JSON.stringify({ value: [{ role: 'window', name: 'Assertion failed', children: [] }] })), + ); + const page = new BasePage(); + assert.equal(await page.didAssertPopup(), false); + handles = ['main', 'closing-menu', 'assert']; + assert.equal(await page.didAssertPopup(), true); +}); + +test('Windows assertion checks recheck an empty or unrecognized root before reporting an assertion', async () => { + let lists = 0; + globalThis.browser = { + capabilities: { browserName: 'furn-native-desktop', platformName: 'windows', 'furn:endpoint': 'win32' }, + options: { protocol: 'http', hostname: '127.0.0.1', port: 1, path: '/' }, + sessionId: 'session', + getWindowHandles: async () => (++lists === 1 ? ['main', 'closing-menu'] : ['main']), + getWindowHandle: async () => 'main', + switchToWindow: () => assert.fail('an assertion check must not change native focus'), + }; + let value; + mock.method(globalThis, 'fetch', async () => new Response(JSON.stringify({ value }))); + const page = new BasePage(); + for (const roots of [[], [{ role: 'group', children: [] }], [{ role: 'menu', children: [{ role: 'dialog', children: [] }] }]]) { + lists = 0; + value = roots; + assert.equal(await page.didAssertPopup(), false); + } +}); + +test('Windows assertion checks do not treat other provider errors as closed windows', async () => { + globalThis.browser = { + capabilities: { browserName: 'furn-native-desktop', platformName: 'windows', 'furn:endpoint': 'win32' }, + options: { protocol: 'http', hostname: '127.0.0.1', port: 1, path: '/' }, + sessionId: 'session', + getWindowHandles: async () => ['main', 'unreadable-menu'], + getWindowHandle: async () => 'main', + }; + let code; + mock.method( + globalThis, + 'fetch', + async () => new Response(JSON.stringify({ value: { error: code, message: 'The provider could not read the tree.' } }), { status: 500 }), + ); + for (const error of ['unknown error', 'stale element reference']) { + code = error; + await assert.rejects(new BasePage().didAssertPopup(), { code: error }); + } +}); + test('assertion checks retain non-native and macOS window-count behavior', async () => { for (const capabilities of [ { browserName: 'non-native' }, diff --git a/apps/E2E/src/common/nativeDesktop.ts b/apps/E2E/src/common/nativeDesktop.ts index 3505db9d91..0ca3a103d5 100644 --- a/apps/E2E/src/common/nativeDesktop.ts +++ b/apps/E2E/src/common/nativeDesktop.ts @@ -1,4 +1,5 @@ -import { createDesktopDriverClient, DesktopSessionClient } from '@fluentui-react-native/desktop-driver'; +import { createDesktopDriverClient, DesktopSessionClient, WebDriverError } from '@fluentui-react-native/desktop-driver'; +import type { DesktopTreeNode } from '@fluentui-react-native/desktop-driver'; import { Keys } from './consts.js'; /** A page object getter, or the element it has already resolved to. */ @@ -20,18 +21,30 @@ export async function hasUnexpectedNativeDesktopWindow(handles: readonly string[ for (const handle of handles) { if (handle === current) continue; // Switching windows would activate the target and can dismiss the menu being tested. - const roots = await session.getWindowTree(handle); - if (roots.length !== 1 || roots[0].role !== 'menu') return true; - const pending = [...roots]; - while (pending.length > 0) { - const node = pending.pop()!; - if (node.role === 'dialog' || node.role === 'window') return true; - pending.push(...node.children); + let roots: DesktopTreeNode[]; + try { + roots = await session.getWindowTree(handle); + } catch (error) { + if (error instanceof WebDriverError && error.code === 'no such window') continue; + throw error; } + // A closing popup can lose its content before its window disappears. + if (!isExpectedMenuTree(roots) && (await browser.getWindowHandles()).includes(handle)) return true; } return false; } +function isExpectedMenuTree(roots: readonly DesktopTreeNode[]): boolean { + if (roots.length !== 1 || roots[0].role !== 'menu') return false; + const pending = [...roots]; + while (pending.length > 0) { + const node = pending.pop()!; + if (node.role === 'dialog' || node.role === 'window') return false; + pending.push(...node.children); + } + return true; +} + export async function sendNativeDesktopKeys(element: PageElement, keys: readonly Keys[]): Promise { const target = await element; await browser.elementSendKeys(await target.elementId, ''); From 3957cd6aec1cf83c8dc62e4d56403fe942505aa4 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 11:35:12 -0800 Subject: [PATCH 10/15] docs: record closing-menu review correction validation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .../references/current-status.md | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index cc6175bed7..06a0c2aed4 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -1017,3 +1017,47 @@ The shared process audit returned 2: it listed W5's test tree and the W3 formatter running concurrently with the audit. Both recorded W3 formatter PIDs were confirmed exited afterward. No W3 native runtime or campaign is left running; a globally clean audit is not claimed while peer work is active. + +## Windows Fabric and Win32 Paper — closing-menu review correction — 2026-11-12 UTC + +**Reviewed base:** `ad8b67c9720a00d7d8df4d27c2ca7436c78d6198`. +**Tested implementation:** `36efc5a88dc507c626f2d29ac487cfc37e9e4bf3`. +The coordinator's review accepted the scoped, non-activating tree query and +conservative menu classification, and identified a closing-popup race in the +legacy E2E adapter. + +The adapter now skips a tree read only when its typed WebDriver error is +`no such window`, then continues checking other handles. Before reporting an +empty, non-menu, or dialog-containing tree as an assertion, it refreshes the +authoritative window list and confirms that handle is still present. +Persistent unexpected windows still fail the check. Transport, provider, +and stale-element errors still reject; no diagnostic event or input fallback +is used. + +**Observed test-first:** `yarn test` in `apps\E2E` initially failed two new +cases: a `no such window` read rejected, and an empty closing-menu tree +reported an assertion. The retained log is +`artifacts\desktop\w3-buildci\popup-race-red.log`. +After the fix, all 14 adapter tests passed, including a vanished menu followed +by a real unexpected window, empty/non-menu roots followed by disappearance, +persistent unexpected roots, and unreadable still-listed windows. + +| Exact command | Result | +| --------------------------------------------------------------------------------------------------------------- | --------------------------------------------- | +| `yarn workspace @fluentui-react-native/e2e-testing build` | **Passed**. | +| `yarn workspace @fluentui-react-native/e2e-testing test` | **Passed:** 14 tests, zero skips or failures. | +| `yarn workspace @fluentui-react-native/e2e-testing lint` | **Passed**. | +| `yarn oxfmt --write apps\E2E\src\common\nativeDesktop.ts apps\E2E\desktop-adapters.test.cjs apps\E2E\README.md` | **Passed**. | +| `yarn oxfmt --check` | **Passed**. | +| `git diff --check` | **Passed**. | +| `yarn markdown-link-check -c .github\markdown-link-check-config.json apps\E2E\README.md` | **Passed:** 19 links. | +| `yarn change:check` | **Passed** after the implementation commit. | + +This correction changes only the private E2E adapter, its focused tests, and +related documentation; no native helper or public protocol changed. The +existing driver patch changeset remains committed. +Per coordinator instruction, the locked clean-cache aggregate, native +contracts, GUI/physical runs, and push are **not run** in this segment. +W1 retains the desktop campaign. W3 awaits its explicit grant for the final +Fabric semantic case, both secondary physical clicks, and final native +contracts/probe tests, then the clean-cache aggregate and publication. From a53451513eb84f5ff62dc7763bd419521c617d5b Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 13:30:10 -0800 Subject: [PATCH 11/15] fix(desktop-driver): tolerate secondary closure during Windows listing Omit only a no-such-window geometry failure whose native handle is already closed. Preserve errors for live handles and unrelated failures. Cover the enumeration-to-serialization race with a deterministic native self-test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .changeset/brave-popup-listings.md | 5 ++ .../agentic/desktop-driver/native/PROTOCOL.md | 6 +++ .../desktop-driver/native/windows/README.md | 6 +++ .../native/windows/src/driver.cpp | 12 ++++- .../native/windows/src/driver.h | 3 ++ .../native/windows/src/selftest.cpp | 47 +++++++++++++++++++ 6 files changed, 78 insertions(+), 1 deletion(-) create mode 100644 .changeset/brave-popup-listings.md diff --git a/.changeset/brave-popup-listings.md b/.changeset/brave-popup-listings.md new file mode 100644 index 0000000000..8ba2e2e657 --- /dev/null +++ b/.changeset/brave-popup-listings.md @@ -0,0 +1,5 @@ +--- +'@fluentui-react-native/desktop-driver': patch +--- + +Keep Windows and Win32 window listings available when a secondary window closes between discovery and native geometry serialization, without hiding failures for live windows. diff --git a/packages/agentic/desktop-driver/native/PROTOCOL.md b/packages/agentic/desktop-driver/native/PROTOCOL.md index 0c78c6b116..0a561fb213 100644 --- a/packages/agentic/desktop-driver/native/PROTOCOL.md +++ b/packages/agentic/desktop-driver/native/PROTOCOL.md @@ -265,6 +265,12 @@ uses the whole budget. A read deadline does not quarantine mutations, and cancellation is still honored. Publication diagnostics mark admitted live popups as both `listed` and `tracked` without another content probe. +If a Windows/Win32 window closes between enumeration and geometry +serialization, `windows` omits it instead of failing the remaining live +listing. This suppresses only `no-such-window` when native liveness confirms +the handle has gone; unrelated errors and failures for still-live handles +propagate. Direct commands for the closed window still fail. + This is internal Windows admission/liveness policy: no request field, response shape, command, or FDR1 1.0 constant in `protocol.json` changes. diff --git a/packages/agentic/desktop-driver/native/windows/README.md b/packages/agentic/desktop-driver/native/windows/README.md index 9838725bc8..1cbd54c4a2 100644 --- a/packages/agentic/desktop-driver/native/windows/README.md +++ b/packages/agentic/desktop-driver/native/windows/README.md @@ -286,6 +286,12 @@ later listing. Exhausting discovery never removes an already-admitted window. The probe retains no element records or whole-subtree cache and never quarantines the session; cancellation still propagates. +If a window closes after enumeration but before its geometry is serialized, +the listing omits it and still returns the other live windows. Only a +`no-such-window` failure confirmed by native handle liveness is absorbed; +failures for live handles and unrelated errors still propagate. Direct +commands for the closed window continue to fail as `no-such-window`. + `windowEvents.publication.windows[].listed` includes admitted tool popups using those same native identity/ownership checks. It does not repeat UIA discovery or report an admitted live popup as `listed: false, tracked: true`. diff --git a/packages/agentic/desktop-driver/native/windows/src/driver.cpp b/packages/agentic/desktop-driver/native/windows/src/driver.cpp index 65f1a418b7..ca7f900d07 100644 --- a/packages/agentic/desktop-driver/native/windows/src/driver.cpp +++ b/packages/agentic/desktop-driver/native/windows/src/driver.cpp @@ -342,6 +342,9 @@ json::Value Driver::ProbeResult(const json::Value& params) { } json::Value Driver::WindowSummary(const WindowInfo& info) { + if (windowSummaryHook_) { + windowSummaryHook_(info.window); + } const WindowMetrics metrics = MeasureWindow(info.window); json::Value value = json::Value::Object(); value.Set("id", json::Value::String(info.id)); @@ -638,7 +641,14 @@ CommandResult Driver::ExecuteCommand(const std::string& command, const json::Val [&](HWND window, DWORD processId, ULONGLONG expiresAt) { return automation_.HasWindowContent(window, processId, expiresAt, token); })) { - result.result.Append(WindowSummary(info)); + try { + result.result.Append(WindowSummary(info)); + } catch (const HelperError& error) { + // A popup can finish closing after enumeration, before its geometry is read. + if (error.code() != kErrorNoSuchWindow || IsWindow(info.window) != FALSE) { + throw; + } + } } return result; } diff --git a/packages/agentic/desktop-driver/native/windows/src/driver.h b/packages/agentic/desktop-driver/native/windows/src/driver.h index 0fe5e7c039..6892546739 100644 --- a/packages/agentic/desktop-driver/native/windows/src/driver.h +++ b/packages/agentic/desktop-driver/native/windows/src/driver.h @@ -84,6 +84,8 @@ class Driver { } // Replaces the hit test that checks where a pointer press or scroll lands. void UseTestHitTester(ClickHitTester hitTest) { hitTester_ = hitTest; } + // Runs after enumeration, immediately before a listed window's geometry is read. + void UseTestWindowSummaryHook(std::function hook) { windowSummaryHook_ = std::move(hook); } private: CommandResult ExecuteCommand(const std::string& command, const json::Value& params, const CancellationToken& token); @@ -166,6 +168,7 @@ class Driver { ForegroundReader foregroundReader_; WindowActivator activator_; ClickHitTester hitTester_{&WindowFromPoint}; + std::function windowSummaryHook_; }; } // namespace furn diff --git a/packages/agentic/desktop-driver/native/windows/src/selftest.cpp b/packages/agentic/desktop-driver/native/windows/src/selftest.cpp index 570f8cad76..0d5a437889 100644 --- a/packages/agentic/desktop-driver/native/windows/src/selftest.cpp +++ b/packages/agentic/desktop-driver/native/windows/src/selftest.cpp @@ -1537,6 +1537,52 @@ void TestRegisteredToolWindowLiveness() { applications.CloseApplication(leaseId, token); } +void TestClosingWindowListing() { + WindowThread windows; + const HWND primary = windows.Create(L"FurnSelfTestClosingListPrimary", L"Listing primary"); + const HWND popup = windows.Create(L"FurnSelfTestClosingListPopup", L"Closing popup", DefWindowProcW, primary); + Driver driver; + const CancellationToken token; + json::Value request = json::Value::Object(); + request.Set("lease", driver.AdoptWindowForTesting(primary, L"Listing primary")); + const CommandResult initial = driver.Execute("windows", request, token); + std::string primaryId; + std::string popupId; + for (const json::Value& info : initial.result.Items()) { + if (info.StringField("title") == "Listing primary") primaryId = info.StringField("id"); + if (info.StringField("title") == "Closing popup") popupId = info.StringField("id"); + } + Check(!primaryId.empty() && !popupId.empty(), "the closing-window test initially lists both native windows"); + if (primaryId.empty() || popupId.empty()) return; + + driver.UseTestWindowSummaryHook([&](HWND window) { + if (window == popup) windows.Destroy(popup); + }); + CommandResult listed; + const std::string code = FailureCode([&]() { listed = driver.Execute("windows", request, token); }); + Check(code == "none" && listed.result.IsArray() && listed.result.Items().size() == 1 && + listed.result.Items().front().StringField("id") == primaryId, + "a secondary closing after enumeration does not fail the live-window listing"); + + driver.UseTestWindowSummaryHook({}); + json::Value closed = json::Value::Object(); + closed.Set("windowId", json::Value::String(popupId)); + Check(FailureCode([&]() { driver.Execute("getWindowRect", closed, token); }) == kErrorNoSuchWindow, + "the omitted closed secondary still rejects direct window commands"); + + driver.UseTestWindowSummaryHook([](HWND) { + throw HelperError(kErrorNoSuchWindow, "Injected geometry failure for a live window."); + }); + Check(FailureCode([&]() { driver.Execute("windows", request, token); }) == kErrorNoSuchWindow, + "a geometry error for a still-live window is not mistaken for closure"); + driver.UseTestWindowSummaryHook([&](HWND window) { + windows.Destroy(window); + throw HelperError(kErrorAutomationFailed, "Injected unrelated listing failure."); + }); + Check(FailureCode([&]() { driver.Execute("windows", request, token); }) == kErrorAutomationFailed, + "closing a window does not suppress an unrelated listing error"); +} + void TestDriverWindowEvents() { WindowThread windows; const HWND window = windows.Create(L"FurnSelfTestDriver", L"Driver window"); @@ -3951,6 +3997,7 @@ int RunSelfTest() { TestAttachedCloseKeepsQuarantine(); TestWindowEventHooks(); TestRegisteredToolWindowLiveness(); + TestClosingWindowListing(); TestDriverWindowEvents(); TestInputEnvironment(); TestInjectionIntegrity(); From 6c626602cf1eedd10425c8a63d794b1e12b1ff95 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 13:35:48 -0800 Subject: [PATCH 12/15] docs(desktop-driver): record Windows final sweep and close-race evidence Keep prior-helper real-app passes separate from the new close-race helper. Preserve the failed sweep, deterministic red/green, six passing fixture replays, and remaining review and qualification gates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .../references/current-status.md | 91 +++++++++++++++++++ .../references/remaining-work.md | 36 +++++--- 2 files changed, 114 insertions(+), 13 deletions(-) diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index 06a0c2aed4..e66a816511 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -1061,3 +1061,94 @@ contracts, GUI/physical runs, and push are **not run** in this segment. W1 retains the desktop campaign. W3 awaits its explicit grant for the final Fabric semantic case, both secondary physical clicks, and final native contracts/probe tests, then the clean-cache aggregate and publication. + +## Windows Fabric and Win32 Paper — final sweep and native close-race correction — 2026-11-12 UTC + +**App qualification revision:** `3957cd6aec1cf83c8dc62e4d56403fe942505aa4`. +**Close-race implementation:** `a53451513eb84f5ff62dc7763bd419521c617d5b`. +The session had returned to Active/unlocked before this campaign. All native +commands below used the W3 campaign and serialized lock with +`-RequirePhysical`; both preflight and postflight reported an eligible +**remote** session with the Default input desktop. No session change was +reported. + +The registered Fabric native app was reused only after checking its install +identity and verifying no native app-source differences from its build +revision. Metro and bundles came from this worktree. Paper used the exact +REX executable with SHA-256 +`3ecd1f58690b029eccf06838dc23411f08b1f3765b99950091f8246aedd74725`. +Apps, Metro, helpers, and fixtures were closed between runs. + +The following results used helper source +`89cc81fbb72cb8a79852e5d4d90ce96d8b2982283086236b860289f2664ac557`, +artifact `eb91ef8a4164080bcae787aacce3fd5aa5556d8c3d0b22d0cd0c661a50934066`. +Run artifacts are under `artifacts\desktop\w3-qualification\runs\`. + +| Endpoint / case | Observed result | Run ID | +| ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | +| Fabric main/secondary semantic action | **Passed:** honest `activation-unsupported` refusal with effect `none`; no fabricated receipt, fallback, or app change. | `98d0ae8f-e84b-43c1-b332-7136dd676f23` | +| Fabric secondary physical click | **Passed:** separately registered window, complete 11-node sample, physical-pointer receipt and public `focus: confirmed`, then retirement. | `537cc479-b3d3-4ef0-93b2-d39c35177653` | +| Paper secondary physical click | **Passed:** exact owned tool window, complete eight-node sample, physical-pointer delivery and retirement. The legacy story has no native-command outcome, so this is not proof of the RN focus command. | `e29ff4aa-62f5-4f53-8a5e-450a5e799466` | + +### Observed failure and test-first correction + +With `$env:FURN_NATIVE_DRIVER_TEST='1'`, the requested package command +`yarn test --runInBand operations.contract win32Fixture.contract nativeDriver` +**failed**: 33 passed, one failed, two macOS-only cases skipped. The tool-content +fixture failed with `no-such-window`, not a session or permission refusal. +Its log remains `artifacts\desktop\w3-buildci\final-native-contracts.log`. +This failed sweep is not claimed as a final native pass. + +A bounded diagnostic replay reproduced the failure on its first fixture. +The command journal isolated it to `windows` immediately after the close +button's successful accessibility click; the destroy event confirmed the +secondary had closed. Native enumeration can return a window that closes +before `WindowSummary` reads its geometry. This is separate from semantic +dispatch and the E2E adapter's closing-tree race. + +The correction omits a geometry failure only when its code is +`no-such-window` **and** native liveness confirms the handle has gone. A +still-live handle's error and every unrelated error propagate. Direct +commands for the omitted window still refuse. No input fallback, +cross-process admission, or wire-schema change was added. + +A deterministic native self-test closes a secondary exactly between +enumeration and geometry serialization. Before the guard, this command +**failed** its new live-window-listing assertion: + +```powershell +yarn test --runInBand nativeDriver --testNamePattern 'handshakes with the Windows helper' +``` + +After the guard, the following command **passed**, two native cases with 26 +unselected cases skipped. It includes the deterministic self-test and the +original real-fixture registration/action/closure scenario: + +```powershell +yarn test --runInBand nativeDriver win32Fixture.contract --testNamePattern 'handshakes with the Windows helper|registers only owned untitled tool roots' +``` + +The self-test also verifies direct closed-window refusal, propagation of an +error for a live handle, and propagation of an unrelated error even when its +window closes. The unchanged diagnostic replay then **passed six isolated +fixture lifecycles**, including close polling and subsequent main-window +actions. Retained logs under `artifacts\desktop\w3-buildci` are +`trace-tool-contract-replay.log`, `closed-window-listing-red.log`, +`closed-window-listing-green.log`, and `trace-tool-contract-green.log`. + +The fixed helper has source +`7119f42fb101452990b7bea13daf685e90a67ad75840fba60968a5efe42f62d1`, +artifact `e0e6fc988fc9214b91e02bd0909c099b203d4f9bfd89c45db8f8c9f9397a05e0`, +and build fingerprint +`afca889e68ccd07b9c1874b6d46b0c2a3a3f635e3e7b09c1b43eca20b9fb6f9c`. +The package `yarn build` and `yarn lint`, root `yarn oxfmt --check`, +`git diff --check`, and Markdown link checks for the native provider README +and protocol **passed**. Patch changeset: `brave-popup-listings.md`. + +The new implementation needs coordinator review before push. The complete +native selector, Fabric semantic case, and both secondary physical clicks +must be rerun against this **new** helper; the earlier app results above do +not qualify its new source digest. The new-source clean-cache aggregate and +hosted CI are **not run** yet. The 200% DPI, mixed-DPI, and negative-origin +gaps remain unchanged. Both campaigns ended, and the final audit reported +`FURN-AUDIT: no FURN desktop processes found`. diff --git a/packages/agentic/desktop-driver/references/remaining-work.md b/packages/agentic/desktop-driver/references/remaining-work.md index 008b0b47d1..2c75f9cdab 100644 --- a/packages/agentic/desktop-driver/references/remaining-work.md +++ b/packages/agentic/desktop-driver/references/remaining-work.md @@ -1085,30 +1085,40 @@ the PR; a hosted capability skip is not completion. ### Windows connected-session follow-up — 2026-11-12 UTC The legacy-only activation refusal and lease-local content-popup registration -corrections have focused native red/green evidence; the final helper also -passes the real Paper secondary semantic case. See the separate +corrections have focused native red/green evidence. At `3957cd6ae`, Fabric +main/secondary semantic refusal and both endpoints' secondary physical +delivery passed in an eligible remote session. The full native sweep then +found a close-during-listing geometry race: 33 cases passed, one failed, +and two macOS-only cases were skipped. The narrow native correction at +`a53451513` has a deterministic red/green self-test, the original fixture +case passing, and six passing isolated fixture replays. See the separate [Windows evidence entry](current-status.md). Keep #4441 open until: -- The final helper reruns Fabric main/secondary semantic activation and both +- The close-race correction receives coordinator review. +- The new helper reruns Fabric main/secondary semantic activation and both endpoints' secondary physical clicks. Physical receipts are not themselves proof of the legacy Callout's native command outcome. -- The final helper reruns +- The new helper reruns `yarn test --runInBand operations.contract win32Fixture.contract nativeDriver` with `$env:FURN_NATIVE_DRIVER_TEST = '1'`, including registered-popup liveness during failed/hung content probes, the shared discovery deadline - and deferred retry, and truthful listed/tracked publication. - -The review follow-ups are implemented, and the default-environment clean-cache -repository aggregate passed all 178 tasks at `cad25f8c5`. Native runs were -paused when session 1 disconnected, locked, and lost its input desktop. -After it became Active/unlocked again, the desktop was reserved for W1; -W3 had only a buildci grant. Resume native/GUI/physical work only with an + and deferred retry, truthful listed/tracked publication, and the + close-after-enumeration self-test. +- The final-source default-environment clean-cache aggregate and affected + hosted Windows/Win32 lanes pass, including the legacy menu assertion + guard and its closing-window correction. + +The earlier default-environment clean-cache repository aggregate passed all +178 tasks at `cad25f8c5`; it does not qualify the later corrections. Native +runs resumed after the disconnected session became Active/unlocked and W3 +received its grant. Both W3 campaigns ended with clean process audits. +Further native/GUI/physical work still requires an Active, unlocked session, an explicit W3 desktop grant, and the serialized campaign; physical work additionally requires `-RequirePhysical`. No reconnect, session move, or skip-to-pass substitution is authorized. Earlier eligible physical results remain evidence for their tested -revisions, not final-helper reruns. Authorized draft publication leaves -these qualification gaps open. +revisions, not new-helper reruns. Authorized draft publication leaves +these qualification gaps and the unrun display-topology cases open. ## 9. Qualification and efficiency gates From 2d987a86f46451c104e01d417c63b4a7080c3f91 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 14:40:21 -0800 Subject: [PATCH 13/15] docs(desktop-driver): record reviewed Windows final qualification Record the 34-test native pass, final Fabric semantic and both secondary physical reruns, default-environment 178-task aggregate, exact helper identities, and clean campaign/drain audits. Retain the distinct historical failure and remaining display-topology limits. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .../references/current-status.md | 78 +++++++++++++++++++ .../references/remaining-work.md | 46 +++++------ 2 files changed, 97 insertions(+), 27 deletions(-) diff --git a/packages/agentic/desktop-driver/references/current-status.md b/packages/agentic/desktop-driver/references/current-status.md index e66a816511..eb1d7a7fc0 100644 --- a/packages/agentic/desktop-driver/references/current-status.md +++ b/packages/agentic/desktop-driver/references/current-status.md @@ -1152,3 +1152,81 @@ not qualify its new source digest. The new-source clean-cache aggregate and hosted CI are **not run** yet. The 200% DPI, mixed-DPI, and negative-origin gaps remain unchanged. Both campaigns ended, and the final audit reported `FURN-AUDIT: no FURN desktop processes found`. + +## Windows Fabric and Win32 Paper — reviewed-helper final qualification — 2026-11-12 UTC + +**Tested revision:** `6c626602cf1eedd10425c8a63d794b1e12b1ff95`, clean worktree +for every real-app run. The coordinator's Opus review of the close-race +correction reported no findings. W3 received a new desktop grant after W2's +recovery verification; the session was **remote**, Active/unlocked, with +the Default input desktop and `physicalEligible: true`. + +All native and real-app commands below ran inside the W3 campaign and +serialized machine lock with `-RequirePhysical`. Every wrapper's before/after +session check stayed eligible. No hosted input skip or semantic action is +counted as physical evidence. + +The source-verified helper cache selected source +`7119f42fb101452990b7bea13daf685e90a67ad75840fba60968a5efe42f62d1`, +artifact `e0e6fc988fc9214b91e02bd0909c099b203d4f9bfd89c45db8f8c9f9397a05e0`, +and build fingerprint +`afca889e68ccd07b9c1874b6d46b0c2a3a3f635e3e7b09c1b43eca20b9fb6f9c`. +The app supervisor used prebuilt-only policy, not an implicit helper build. +The exact native app identity was checked before each run. The registered +Fabric app had no native source differences from its build revision; +Metro and bundles came from this worktree. Paper used the exact REX executable +with SHA-256 +`3ecd1f58690b029eccf06838dc23411f08b1f3765b99950091f8246aedd74725`. +The plain Win32 fixture executable had SHA-256 +`9e082f5a9a50e1a6f93f8d91a0c01f8e880820030a7e9d8b5c1917993fc8bc13`. +Apps, Metro, helpers, and fixtures were cleaned up between runs. + +The native package command below ran from `packages\agentic\desktop-driver` +with `$env:FURN_NATIVE_DRIVER_TEST='1'`. Endpoint supervisor commands ran +from the worktree root; their source and immutable run artifacts remain +under ignored `artifacts\desktop\w3-qualification`. + +| Exact command | Observed result | +| -------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `yarn test --runInBand operations.contract win32Fixture.contract nativeDriver --verbose` | **Passed:** three suites, 34 tests; two macOS-only cases **skipped**. The previously failing tool-content closure case passed. The log is `artifacts\desktop\w3-buildci\final-native-contracts-fixed.log`. | +| `node artifacts\desktop\w3-qualification\run-endpoint.mjs windows --secondary-only --modern-only --activation-diagnostics` | **Passed:** main Pressable/Button and modern secondary refuse honestly with `activation-unsupported`, effect `none`, no delivery receipt, quarantine, physical fallback, or application change. Run `8f6d457c-c0d3-4ab5-9204-6dbf66383dbd`. | +| `node artifacts\desktop\w3-qualification\run-endpoint.mjs windows --secondary-only --modern-only --physical-secondary` | **Passed:** a distinct secondary is registered, its 11-node sample is complete, the click is applied through `physical-pointer`, the public `focus: confirmed` effect is observed, and its handle retires. Run `94c0b321-cebd-4e9b-99e2-3404bbbd80c3`. | +| `node artifacts\desktop\w3-qualification\run-endpoint.mjs win32 --secondary-only --physical-secondary` | **Passed for native delivery:** the exact owned tool secondary is registered, its eight-node sample is complete, a physical-pointer click is applied, and its handle retires. Run `f46d008a-d310-4e6e-98fd-e85941879edd`. The legacy story has no native-command outcome, so this is not proof of the RN focus command. | + +The full native selector executed the review-specific coverage, not just +compiled it: failed-probe retention and shared-deadline/retry self-tests; +the real hung-popup listing and bounded new-popup discovery/recovery cases; +truthful listed/tracked publication; the close-after-enumeration self-test +with live-error and unrelated-error propagation; and both legacy-only +activation refusals. Legitimate Invoke/Toggle and the existing observation, +capture, mutation settlement, containment, cancellation, and retention +contracts also passed. Cancellation completed in 32 ms; retained elements +stayed at 20 after 500 fixture cycles. The earlier failed sweep and +deterministic red/green evidence remain recorded above. + +### Final-source aggregate and hygiene + +After ending the native campaign and obtaining a clean audit, the exact root +command `cmd.exe /d /c "yarn clean-lage-cache && yarn lage buildci"` ran under +the W3 machine lock with **default TEMP and concurrency**, native opt-in +unset, and no policy, timeout, or guidance workaround. It **passed on the +first attempt**: 178 successful tasks, zero failed, aborted, pending, or +skipped tasks. Native tests remain opt-in in this aggregate and are qualified +by the separate 34-test run above. + +The aggregate ran from 22:30:40 to 22:33:27 UTC. The operational supervisor +retained the lock until its owned descendants drained; drain completed +before lock release. Its retained log is +`artifacts\desktop\w3-buildci\final-buildci-fixed.log`. No readiness-flake +rerun was needed. No code changed after the reviewed and qualified source; +this publication update changes only Windows evidence/tracking. + +The campaign ended. Both its post-campaign audit and the post-aggregate audit +reported `FURN-AUDIT: no FURN desktop processes found`. No W3 app, helper, +fixture, Metro, or WDIO process survived. + +The current-source local reruns are complete. Hosted results for the +publication head will be recorded in draft PR #4498, separately from +earlier-head hosted evidence. The 200% DPI, mixed-DPI, and negative-origin +cases remain **not run** for the previously recorded authorization/topology +reasons; console Ctrl+C remains separate work in #4491. diff --git a/packages/agentic/desktop-driver/references/remaining-work.md b/packages/agentic/desktop-driver/references/remaining-work.md index 2c75f9cdab..df849becef 100644 --- a/packages/agentic/desktop-driver/references/remaining-work.md +++ b/packages/agentic/desktop-driver/references/remaining-work.md @@ -1085,33 +1085,25 @@ the PR; a hosted capability skip is not completion. ### Windows connected-session follow-up — 2026-11-12 UTC The legacy-only activation refusal and lease-local content-popup registration -corrections have focused native red/green evidence. At `3957cd6ae`, Fabric -main/secondary semantic refusal and both endpoints' secondary physical -delivery passed in an eligible remote session. The full native sweep then -found a close-during-listing geometry race: 33 cases passed, one failed, -and two macOS-only cases were skipped. The narrow native correction at -`a53451513` has a deterministic red/green self-test, the original fixture -case passing, and six passing isolated fixture replays. See the separate -[Windows evidence entry](current-status.md). Keep #4441 open until: - -- The close-race correction receives coordinator review. -- The new helper reruns Fabric main/secondary semantic activation and both - endpoints' secondary physical clicks. Physical receipts are not themselves - proof of the legacy Callout's native command outcome. -- The new helper reruns - `yarn test --runInBand operations.contract win32Fixture.contract nativeDriver` - with `$env:FURN_NATIVE_DRIVER_TEST = '1'`, including registered-popup - liveness during failed/hung content probes, the shared discovery deadline - and deferred retry, truthful listed/tracked publication, and the - close-after-enumeration self-test. -- The final-source default-environment clean-cache aggregate and affected - hosted Windows/Win32 lanes pass, including the legacy menu assertion - guard and its closing-window correction. - -The earlier default-environment clean-cache repository aggregate passed all -178 tasks at `cad25f8c5`; it does not qualify the later corrections. Native -runs resumed after the disconnected session became Active/unlocked and W3 -received its grant. Both W3 campaigns ended with clean process audits. +corrections and close-during-listing guard have reviewed test-first evidence. +At clean revision `6c626602c`, with the new helper and an eligible remote +session, Fabric main/secondary semantic refusal and both endpoints' +secondary physical delivery passed. The complete native selector passed +34 tests with two macOS-only skips, including failed/hung-probe retention, +the shared discovery deadline and deferred retry, truthful publication, +and the deterministic close-after-enumeration self-test. The +default-environment clean-cache aggregate passed all 178 tasks on its first +attempt. See the separate [Windows evidence entry](current-status.md). + +Keep #4441 open for the unrun 200% DPI, mixed-DPI, and negative-origin cases +and other issue-owned gaps. Native delivery receipts do not prove the +legacy Callout's unreported RN focus-command outcome. Current-head hosted +Windows/Win32 coverage, including the legacy menu assertion guard and its +closing-window correction, is tracked in draft PR #4498; earlier-head CI +does not qualify later code. + +The W3 campaign ended, owned aggregate descendants drained before lock +release, and post-campaign/post-aggregate process audits were clean. Further native/GUI/physical work still requires an Active, unlocked session, an explicit W3 desktop grant, and the serialized campaign; physical work additionally requires `-RequirePhysical`. No From 86be8531e5c42bd33e7c46a400f0d40890cf2784 Mon Sep 17 00:00:00 2001 From: Jason Morse Date: Thu, 12 Nov 2026 14:57:43 -0800 Subject: [PATCH 14/15] docs(desktop-test): anchor Windows connected-session evidence mid-ledger Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be5f341-831b-40d4-9582-337c8ea65709 --- .../desktop-test/references/current-status.md | 96 +++++++++---------- 1 file changed, 48 insertions(+), 48 deletions(-) diff --git a/packages/agentic/desktop-test/references/current-status.md b/packages/agentic/desktop-test/references/current-status.md index 5c9991bf20..e723f7308a 100644 --- a/packages/agentic/desktop-test/references/current-status.md +++ b/packages/agentic/desktop-test/references/current-status.md @@ -705,54 +705,6 @@ by hand only after its owner was proved dead and its resources were proved settled, or after the fenced launch probe's survivor was ended. Nothing reclaims a claim automatically. -## Connected Windows physical-input follow-up - 2026-11-08 UTC - -Source `fec94cec4995c7372b798877c5412ddcb156523b`; Windows 11 Enterprise -build 26300, AMD64, Node 26.13.0. With the console connected, the following -separate, non-agent smokes ran from `apps/storybook` with -`yarn storybook smoke -- --mode stories-and-tests --input physical ---reporter json` and `STORYBOOK_SMOKE_STORY` set to the listed story. -Each had zero failures, skips, unsupported cases, incomplete work, retries, -or flaky cases. The totals include navigation work, not just authored tests. - -| Endpoint | Story and authored assertion | Selected / executed / passed | Physical-click receipts | Run ID | -| -------------- | ----------------------------------------------------------------- | ---------------------------- | ----------------------- | -------------------------------------- | -| Windows Fabric | `components-checkbox--default`: unchecked -> checked -> unchecked | 203 / 203 / 203 | 2 | `ef1ed966-3be3-4cc1-b940-ac0afbb02155` | -| Win32 Paper | `components-checkbox--default`: unchecked -> checked -> unchecked | 177 / 177 / 177 | 2 | `54e6a815-21a4-418c-bfda-848e28549ffe` | -| Windows Fabric | `native-callout--default`: open, light-dismiss, recreate | 203 / 203 / 203 | 3 | `1e521138-f529-45a1-974e-74f90edd42ac` | -| Win32 Paper | `native-callout--default`: open, light-dismiss, recreate | 177 / 177 / 177 | 3 | `44d9dd17-7132-4ebd-8b1e-18fa923ee355` | - -Artifacts are `apps/storybook/artifacts/desktop//run.json`. The -Windows totals contain 202 navigation cases and one authored case; Win32 -contains 176 and one. Popup interaction does not establish F4's separately -registered Storybook secondary-window identity. - -The connected native fixture contract passed 11 tests under the per-user -claim; its retained-element counts were 20 and 23 at cycles 100 and 500. -The separate generic fixture probe passed exact secondary-handle switching -and minimized-capture refusal as described in the driver status reference. -Windows `doctor --permissions` remains Unsupported. - -The real MCP descriptor path independently passed physical checkbox -transitions and target capture on both endpoints. Standalone agent driver -runs `af97fdb3-68de-4ce9-836e-935331528c8e` (Windows) and -`3294d66d-9332-4639-95fe-8d3047f47e46` (Win32) each ended with a -terminal passed lifecycle report and zero selected tests; the manual MCP -operations are not suite selections. The first Windows probe exhausted its -30 safe `bridge-unavailable`/effect-none attempts while the cold Metro bundle -was still building; the later probe passed after the app connected. - -After MCP session close, Windows accepted an exact-app window close. On -Win32 the window closed but its exact REX process remained; it was stopped -only after identity verification. Both deliberately stopped Win32 app runs, -`665e6e85-f0ce-4612-a34d-f140d2bab433` and -`fc71d8fd-ffa4-464e-96ea-616d88d49e0e`, report failed, not graceful cleanup. -The standalone drivers were stopped by a session-local injected SIGTERM, -not physical Ctrl+C. Both owned app/service processes exited, the recorded -ports were free, and the per-user claim and default descriptor were absent. -The separate `serve --agent` interrupt limitation above is fixed but not yet -requalified on Windows. - ## Windows Fabric and Win32 Paper — #4441 connected-session qualification — 2026-11-12 UTC **Observed baseline:** `505a2187b0a2451aed7081c9260b343d55742f93`, @@ -881,3 +833,51 @@ qualification remain not run. MCP costs/budgets are recorded in `@fluentui-react-native/desktop-driver-mcp/references/current-status.md`; Storybook context details are in `@fluentui-react-native/storybook-desktop/references/current-status.md`. + +## Connected Windows physical-input follow-up - 2026-11-08 UTC + +Source `fec94cec4995c7372b798877c5412ddcb156523b`; Windows 11 Enterprise +build 26300, AMD64, Node 26.13.0. With the console connected, the following +separate, non-agent smokes ran from `apps/storybook` with +`yarn storybook smoke -- --mode stories-and-tests --input physical +--reporter json` and `STORYBOOK_SMOKE_STORY` set to the listed story. +Each had zero failures, skips, unsupported cases, incomplete work, retries, +or flaky cases. The totals include navigation work, not just authored tests. + +| Endpoint | Story and authored assertion | Selected / executed / passed | Physical-click receipts | Run ID | +| -------------- | ----------------------------------------------------------------- | ---------------------------- | ----------------------- | -------------------------------------- | +| Windows Fabric | `components-checkbox--default`: unchecked -> checked -> unchecked | 203 / 203 / 203 | 2 | `ef1ed966-3be3-4cc1-b940-ac0afbb02155` | +| Win32 Paper | `components-checkbox--default`: unchecked -> checked -> unchecked | 177 / 177 / 177 | 2 | `54e6a815-21a4-418c-bfda-848e28549ffe` | +| Windows Fabric | `native-callout--default`: open, light-dismiss, recreate | 203 / 203 / 203 | 3 | `1e521138-f529-45a1-974e-74f90edd42ac` | +| Win32 Paper | `native-callout--default`: open, light-dismiss, recreate | 177 / 177 / 177 | 3 | `44d9dd17-7132-4ebd-8b1e-18fa923ee355` | + +Artifacts are `apps/storybook/artifacts/desktop//run.json`. The +Windows totals contain 202 navigation cases and one authored case; Win32 +contains 176 and one. Popup interaction does not establish F4's separately +registered Storybook secondary-window identity. + +The connected native fixture contract passed 11 tests under the per-user +claim; its retained-element counts were 20 and 23 at cycles 100 and 500. +The separate generic fixture probe passed exact secondary-handle switching +and minimized-capture refusal as described in the driver status reference. +Windows `doctor --permissions` remains Unsupported. + +The real MCP descriptor path independently passed physical checkbox +transitions and target capture on both endpoints. Standalone agent driver +runs `af97fdb3-68de-4ce9-836e-935331528c8e` (Windows) and +`3294d66d-9332-4639-95fe-8d3047f47e46` (Win32) each ended with a +terminal passed lifecycle report and zero selected tests; the manual MCP +operations are not suite selections. The first Windows probe exhausted its +30 safe `bridge-unavailable`/effect-none attempts while the cold Metro bundle +was still building; the later probe passed after the app connected. + +After MCP session close, Windows accepted an exact-app window close. On +Win32 the window closed but its exact REX process remained; it was stopped +only after identity verification. Both deliberately stopped Win32 app runs, +`665e6e85-f0ce-4612-a34d-f140d2bab433` and +`fc71d8fd-ffa4-464e-96ea-616d88d49e0e`, report failed, not graceful cleanup. +The standalone drivers were stopped by a session-local injected SIGTERM, +not physical Ctrl+C. Both owned app/service processes exited, the recorded +ports were free, and the per-user claim and default descriptor were absent. +The separate `serve --agent` interrupt limitation above is fixed but not yet +requalified on Windows. From d9a5ffcc0511421717ca6cdfbf9c841834ad1046 Mon Sep 17 00:00:00 2001 From: JasonVMo <35609178+JasonVMo@users.noreply.github.com> Date: Sat, 14 Nov 2026 08:50:51 +0000 Subject: [PATCH 15/15] fix: suppress stack traces in WebDriver errors Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> --- .../desktop-driver/src/operations/envelope.test.ts | 9 +++++++++ packages/agentic/desktop-driver/src/protocol/errors.ts | 2 +- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/packages/agentic/desktop-driver/src/operations/envelope.test.ts b/packages/agentic/desktop-driver/src/operations/envelope.test.ts index 2536038f4d..70fbf9380d 100644 --- a/packages/agentic/desktop-driver/src/operations/envelope.test.ts +++ b/packages/agentic/desktop-driver/src/operations/envelope.test.ts @@ -13,6 +13,15 @@ import { } from './envelope'; import { maxRequestIdLength } from './limits'; +describe('WebDriverError', () => { + test('does not expose stack traces in its serialized response', () => { + const error = new WebDriverError('unknown error', 'Failed.'); + error.stack = 'Error: Failed.\n at /private/source.js:1:1'; + + expect(error.toJSON().stacktrace).toBe(''); + }); +}); + describe('parseOperationRequest', () => { test('defaults a missing body or input to an empty input', () => { expect(parseOperationRequest(undefined)).toEqual({ input: {} }); diff --git a/packages/agentic/desktop-driver/src/protocol/errors.ts b/packages/agentic/desktop-driver/src/protocol/errors.ts index 4ca1d9c97a..f3f5125231 100644 --- a/packages/agentic/desktop-driver/src/protocol/errors.ts +++ b/packages/agentic/desktop-driver/src/protocol/errors.ts @@ -40,7 +40,7 @@ export class WebDriverError extends Error { return { error: this.code, message: this.message, - stacktrace: this.stack ?? '', + stacktrace: '', ...(this.data ? { data: this.data } : {}), }; }