Skip to content

Commit 1bf3da2

Browse files
JasonVMoCopilot
andcommitted
Record permission-enabled macOS qualification evidence and gaps
Preserve the non-AX history, document both renderer smoke failures and the Paper keyboard-clear finding, and add real-descriptor input, lifecycle and MCP measurement evidence without changing product code. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 4088dc5 commit 1bf3da2

4 files changed

Lines changed: 457 additions & 24 deletions

File tree

‎.changeset/tired-knives-make.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,6 @@
22

33
---
44

5-
Record F15 macOS non-Accessibility supporting evidence and the remaining
6-
permission blocks. Documentation only; no package release.
5+
Record F15 macOS supporting evidence, the resolved permission prerequisite,
6+
and bounded native/consumer qualification results and remaining gaps.
7+
Documentation only; no package release.

‎packages/agentic/desktop-driver-mcp/references/current-status.md‎

Lines changed: 152 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,8 @@ tokenizer/version, serialized UTF-8 bytes and characters, text tokens,
7373
instructions overhead, projection/budgets, sample count, cold/warm timing,
7474
success/retry/cancellation/cleanup counts, and approved artifact locator.
7575
Do not claim token reductions from reducing information scope or omitting
76-
discovery. macOS Paper measurements are currently **not recorded here**.
76+
discovery. New macOS Paper samples are recorded in the dated F15
77+
permission-enabled follow-up below; they do not fill earlier provenance gaps.
7778

7879
### Windows Fabric and Win32 Paper - 2026-11-08 UTC
7980

@@ -339,7 +340,7 @@ retirement. No raw image or grant is retained in this reference. Driver
339340
shutdown, app cleanup, and the failed Win32 app-run reports are
340341
recorded in `packages/agentic/desktop-test/references/current-status.md`.
341342

342-
## F15 macOS Paper measurement block — 2026-11-12
343+
## F15 macOS Paper measurement prerequisite — 2026-11-12
343344

344345
Source `505a2187b0a2451aed7081c9260b343d55742f93`; macOS 27.2.1
345346
(26B131), arm64. The stable-signed helper's noninteractive doctor reported
@@ -349,10 +350,156 @@ section "F15 non-AX macOS supporting evidence — 2026-11-12",
349350
retains the exact diagnostic command and signing identity; its local artifact
350351
is `artifacts/desktop/f15-20261112/doctor.json`.
351352

352-
**Blocked:** macOS Paper MCP interaction/text/latency measurements requiring
353-
the app's accessibility tree or physical input need owner permission action.
354-
No Paper Storybook app or MCP session was launched, no measurement command
353+
The initial pass was **blocked**: macOS Paper MCP interaction/text/latency
354+
measurements needed Accessibility/PostEvent. That prerequisite was resolved
355+
later the same day; it is no longer an owner-action block.
356+
In the initial pass no Paper Storybook app or MCP session was launched, no measurement command
355357
was run, and 0 tests were selected/executed (no new pass, failure, or skip
356358
counts). Cleanup is N/A for MCP; the diagnostic helper exited. The non-AX
357359
helper self-test does not qualify the adapter or change historical measurement
358360
claims.
361+
362+
### F15 permission-enabled MCP follow-up — 2026-11-12
363+
364+
**Observed, partial; not macOS requalification.** Source:
365+
`141b88b4b81a1bd9a0472de533681f9274d8cb00`; macOS 27.2.1 (26B131),
366+
arm64. App: Agentic Components Storybook / ReactTestApp, bundle
367+
`com.microsoft.fluentui.agenticstorybook.id915e78595`.
368+
Fabric target: `macos-a6dfeaccb13e643a`; Paper target:
369+
`macos-de43c71d45415315`. Nonprompting doctor reported
370+
Accessibility/PostEvent/ScreenCapture true; no privacy changes were made.
371+
All native/GUI commands used the machine-wide native lock and the stable
372+
`FURN Desktop Driver Development` identity/cache described in the driver
373+
ledger's F15 follow-up.
374+
375+
The local evidence root is `artifacts/desktop/f15-20261112T115411Z/`.
376+
The session-local probe used the public MCP SDK 1.32.2 and shipped
377+
desktop-driver-mcp 0.1.3 stdio CLI with the real private
378+
`agent-connection.json` descriptor. No grant or descriptor contents were
379+
printed or copied to evidence. Tokenizer: `gpt-tokenizer` 4.1.0,
380+
`o200k_base`; compact JSON for discovery, newline-joined MCP text blocks for
381+
operation text. UTF-8 bytes and JavaScript characters were recorded
382+
independently. These bounded samples are not a controlled cold-start
383+
benchmark, reliability window or cross-client qualification.
384+
385+
Fabric command, from the repository root:
386+
`node artifacts/desktop/f15-20261112T115411Z/qualify.mjs fabric lifecycle`.
387+
It starts `yarn storybook driver --macos --agent --reporter json`, then
388+
`yarn storybook run --macos --reporter json` in `apps/storybook`.
389+
Results: **10 passed, 1 failed, 0 skipped manual checks** in
390+
`fabric-lifecycle-qualification.json`. Revocation, dispose-during-open,
391+
ambiguous CREATE and failed DELETE before/after dispatch used the SDK
392+
in-memory transport with the real native service and verified descriptor:
393+
response loss was injected at the public typed-client method boundary,
394+
not with a TCP proxy or by forcing native cleanup failure. Each connection
395+
drained and the target reservation could be reacquired.
396+
397+
The stdio pass observed same-port service replacement, authoritative
398+
invalidation of the old session, fresh verified discovery, `no-session`
399+
refusal of the old mutation, target capture/resource reading, session-close
400+
resource expiry and read cancellation. The first group-SIGTERM harness
401+
attempt retained a descriptor and failed restart; it remains recorded in
402+
`fabric-qualification.json`, with exact-record reconciliation in
403+
`descriptor-reconciliation.json`. It is not a successful ordinary shutdown.
404+
405+
Fabric's physical checkbox click **failed**, not passed: the sequence stopped
406+
before dispatch (`element click intercepted`, `obscured`, effect `none`).
407+
Its 107-token / 438-byte / 1,187.5-ms response measures a refusal.
408+
No successful physical click/check/observe interaction cost is claimed.
409+
The driver's F15 open finding records the reproduction and routing.
410+
411+
| Fabric measurement | Observed result |
412+
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
413+
| Six-tool discovery | 4,693 tokens; 20,287 UTF-8 bytes / characters |
414+
| Server instructions, additional to discovery | 120 tokens; 591 UTF-8 bytes / characters |
415+
| First attach after discovery | 846.4 ms; one sample, excludes service/app startup |
416+
| Default observe, 20 calls per story | Median 21.9–27.8 ms across stories; p90 1,127.7–1,669.0 ms; maximum 1,680.2 ms; 905–948 tokens, 2,778–3,061 bytes per call |
417+
| Idle delta, 20 calls per story | Median 21.5–26.3 ms; p90 1,122.0–1,671.9 ms; maximum 1,688.1 ms; 151–152 tokens, 464–469 bytes per call |
418+
| Read cancellation | Caller aborted after 301.5 ms; subsequent observe passed; no mutation cancellation claim |
419+
420+
The three stories were `components-button--appearance`,
421+
`components-checkbox--constrained-text` and `components-menu--default`.
422+
The default 100-node projection completed in all 60 observations and all
423+
60 deltas; no retries. The slow maxima are retained rather than hidden by
424+
the medians. These samples do not establish equal information content to
425+
historical Windows or other story projections.
426+
427+
#### Paper descriptor, interactions and completeness
428+
429+
From the repository root,
430+
`node artifacts/desktop/f15-20261112T115411Z/qualify.mjs paper`
431+
used `yarn storybook driver --macos --paper --agent --reporter json`
432+
and `yarn storybook run --macos --paper --reporter json`.
433+
It recorded **14 passed, 4 failed, 0 skipped manual assertions**
434+
(`paper-qualification.json`), not selected suite tests. The four failures
435+
were keyboard clear, its dependent semantic-clear precondition, refused
436+
physical Checkbox click and unsupported semantic Checkbox activation.
437+
The latter two sequences were `stopped` despite non-error MCP envelopes;
438+
they were not accepted as interactions.
439+
440+
The same real-descriptor lifetime/fault boundaries described for Fabric
441+
passed on Paper, including same-port invalidation, fresh discovery,
442+
`no-session` old-mutation refusal, read cancellation followed by a usable
443+
session, target PNG resource reading/hash verification and expiry on close.
444+
The exact app PID/start-time/executable survived session and service close.
445+
Paper's channel survived one 22-second idle interval. The fault probes
446+
wrapped public typed-client methods, not native cleanup or TCP failures.
447+
448+
Discovery independently matched Fabric: six tools, **4,693 tokens /
449+
20,287 UTF-8 bytes / characters**, plus 120 tokens / 591 bytes of server
450+
instructions. First attach was 831.6 ms, excluding service/app startup.
451+
All latency values below are client wall time; p90 uses nearest rank.
452+
Each row has 20 calls, zero retries and zero call failures. Defaults used
453+
the 100-node projection, not a full-tree request.
454+
455+
| Paper story / operation | p50 / p90 / max, ms | Text tokens | UTF-8 bytes | Complete projections |
456+
| ------------------------------------------------ | ------------------------ | ----------- | ----------- | -------------------- |
457+
| `components-button--appearance`, observe | 31.8 / 1,142.7 / 1,689.3 | 1,188–1,189 | 3,533–3,537 | 20 / 20 |
458+
| Same story, delta | 31.5 / 1,152.0 / 1,671.9 | 151–152 | 465–467 | 20 / 20 |
459+
| `components-checkbox--constrained-text`, observe | 34.1 / 1,688.0 / 1,698.8 | 1,201–1,202 | 3,750–3,752 | 20 / 20 |
460+
| Same story, delta | 33.9 / 1,696.4 / 1,701.6 | 151–152 | 465–467 | 20 / 20 |
461+
| `components-menu--default`, observe | 34.1 / 1,158.7 / 1,706.5 | 1,313–1,322 | 3,862–3,866 | **0 / 20** |
462+
| Same story, delta | 36.4 / 1,159.7 / 1,710.1 | 151–152 | 468–470 | 20 / 20 |
463+
464+
**Observed:** each Paper Menu observation emitted 100 of 103 matching
465+
nodes, `maxNodes`, three omitted; these are incomplete observations, not
466+
full-information comparison samples. Complete deltas do not repair that
467+
truncation. Fabric's complete projections and these Paper projections are
468+
not equivalent scopes. Window dimensions and display scale were not
469+
retained for this run; the cached stable helper and running app make these
470+
warm operation samples, not controlled cold-start measurements.
471+
Paper read cancellation took 302.2 ms; post-dispatch mutation cancellation
472+
was **Not run**.
473+
474+
The Paper physical-click refusal cost 1,172.4 ms / 106 tokens / 438 bytes;
475+
the provider's **Unsupported** semantic activation cost 743.9 ms /
476+
79 tokens / 329 bytes. Neither is a successful click/check/observe cost.
477+
The independent
478+
`node artifacts/desktop/f15-20261112T115411Z/qualify.mjs paper input`
479+
probe recorded **4 passed, 1 failed, 0 skipped manual checks**:
480+
semantic clear, physical-input delivery, protected args and the following
481+
real stdio MCP interaction passed; Paper keyboard clear still failed after
482+
bounded polling. Exact harnesses and defect reproduction are retained in the
483+
driver and consumer ledgers.
484+
485+
| Successful Paper Input interaction, one sample | ms | Tokens | UTF-8 bytes |
486+
| --------------------------------------------------------------- | ------- | ------ | ----------- |
487+
| `desktop_act` type `F15`, keyboard mechanism, applied/completed | 837.5 | 74 | 261 |
488+
| `desktop_check` value equals `F15`, passed on first read | 30.1 | 96 | 322 |
489+
| Closing default `desktop_observe`, complete, 90 emitted nodes | 1,148.2 | 1,201 | 3,611 |
490+
| Sum of those three calls | 2,015.7 | 1,371 | 4,194 |
491+
492+
This sum excludes discovery, attach, story reset and close. No mutation was
493+
retried; each measured call ran once. It is a successful text-entry/value
494+
interaction, **not physical Checkbox activation** and not directly comparable
495+
to historical toggle costs or projections. Artifact:
496+
`paper-input-qualification.json`.
497+
498+
**Cleanup passed:** `cleanup-audit.json`, 2026-11-12T12:20:26Z, records no
499+
worktree processes, no listeners on 39860/23935/28029, no descriptor and
500+
no per-user workload claim; the native-lock wrapper exited and the lock
501+
was unheld afterward. Reliability, durable TCC continuity, other MCP clients,
502+
native cleanup/survivor faults and the outstanding F3/F4/F5 acceptance
503+
cases listed in the consumer ledger remain **Not run** on this source.
504+
Both renderer smokes failed; the permission prerequisite is resolved but
505+
#4441's macOS acceptance is not met.

0 commit comments

Comments
 (0)