Repository navigation
Description
Current status - 2026-11-16
Authoritative native coverage and reliability promotion remain open. Hosted Storybook lanes exist for macOS, Windows Fabric and Win32 Paper; no physical-input lane is required or has a reliability window yet.
Landed (PR links):
- Desktop testing: declare and select WDIO cases by input lane, platform and scope #4464 declared input/platform/scope lanes and truthful incomplete-lane reporting; fixed the all-skipped success path (closed Infrastructure: Reject all-skipped WDIO runs as completed coverage #4342).
- Desktop testing: migrate stories and split CI input lanes #4466 split the Win32 accessible and physical jobs; the physical job is advisory, outside the aggregate
prneeds. - Desktop testing: enforce session input policy #4468 (F16, Infrastructure: Enforce session input policy and attribute delivered input mechanisms #4467 closed) added the driver-enforced session input policy and delivered-input receipts. Promotion evidence should cite those receipts, not declarations.
- Select desktop stories directly and attribute per-case E2E time #4496 added direct story selection and per-case E2E time attribution.
- fix: qualify Windows activation and Win32 secondary content windows #4498: the hosted PR workflow
43324722221passed all Windows/Win32 jobs on attempt 1 at its publication head. - Fix macOS physical-input postconditions and add an advisory physical CI lane #4500 added the advisory
macOS Storybook PR (physical)job (Fabric,--input physical, no disabled input features), outside the aggregateprneeds. Hosted run43344510349: 343 passed, 3 declared unsupported, 0 failures, retries or flaky cases. One run shows the lane works; it is not a reliability window. The existing macOS Fabric/Paper lanes keep theirphysicalClickmask. Windows/Win32 physical evidence is local, not a managed lane.
Remaining (this issue's gates):
- Owned or managed authoritative interactive runners (macOS physical is hosted and advisory; Windows/Win32 physical are local)
- Required-status enforcement for qualified lanes, and proof that a failing native job blocks merging
- Actual executed cohorts and digests
- Product, app, helper, authority, infrastructure and coverage outcome classification in required gates
- Explicit failure, skip and quarantine metadata; retention and escalation policy
- A consistent helper build and verification policy across the macOS, Windows and Win32 lanes (source-verified cache versus prebuilt-only), documented in
ci-integration.md - Hosted Windows jobs run 0 of the 11 Windows application-ownership consumer cases (no endpoint opt-in, feat(desktop-test): qualify Windows application ownership and containment #4494); decide whether a managed lane should opt in
- Timing-sensitive driver tests (the
boundedCommandSIGKILL/SIGTERM assertion inpackages/agentic/desktop-driver/src/native/boundedCommand.test.ts, and aprivateStoragetimeout) fail intermittently in a plain parallel localyarn lage buildci, reproduced on untouched main (Fix macOS physical-input postconditions and add an advisory physical CI lane #4500, Add neutral macOS application ownership with exact workload identity #4492, Add scoped attach-only runner grants and drain-confirmed desktop handoff #4491); hosted CI passes. A local aggregate is not acceptance evidence until they are deterministic or isolated - The representative sustained reliability window (shared with Infrastructure: Qualify desktop lifecycle failure paths and maintain a support matrix #4451/Infrastructure: Complete Desktop Driver production release qualification #4345; do not double-count)
Green semantic lanes, one-off hosted passes and aggregate checks alone do not satisfy these. Per-run evidence is recorded in pull request descriptions and issue comments; each package's references/current-status.md keeps a status overview, stable topic sections, and only the latest result per native endpoint with a link.
Goal
Qualify and promote the native Storybook pipelines delivered by #4256 into reliable required desktop coverage. Parent: #4275.
Status - 2026-10-01
Open; all three pipelines exist and run native apps. macOS, Windows Fabric, and Win32 Paper explicitly build/verify native helpers, execute the owned stories-and-tests lifecycle, and retain diagnostics.
All three Storybook jobs passed in run 36927750317. The macOS log and retained Windows/Win32 reports each record 29 authored tests: 7 passed, 22 skipped. Current main catalogs contain 168 macOS, 158 Windows, and 152 Win32 stories. Hosted Windows/Win32 disable keyboard, physical click, and wheel; hosted macOS disables physical click.
Current main rules require only the PR status. The aggregate's needs excludes all three Storybook jobs and bundle-size; a green aggregate therefore does not make those checks required.
Remaining scope and acceptance criteria
- Provision authoritative, owned interactive Windows/Win32 and managed macOS lanes with the capabilities their required tests actually need. Locked sessions, denied authority, helper failures, app crashes, and assertion failures have distinguishable results.
- Publish an auditable executed cohort, not just discovered tests or render counts; integrate the regression-parity and accessibility follow-ups.
- Reconcile endpoint-specific catalog identities and the actual WDIO story/test/source digests. Full catalogs intentionally differ.
portablePlanDigestcurrently hashes only legacy static plans (none in the component catalog), so matching that digest does not prove callback equivalence. - Retain versioned machine-readable run, JUnit, event, process-ownership, helper identity, native log, tree, and available screenshot evidence for the active WDIO path. Failure upload must not mask the original error.
- Define bounded retry policy and expiring quarantine records with owner, reason/issue, and expiry. Quarantined/skipped cases do not count toward exit coverage.
- Name runner-capacity, escalation, artifact-retention/redaction/access, and fork-PR safety owners and document their operating policy.
- Define one consistent helper build and verification policy across the macOS, Windows and Win32 lanes (source-verified cache versus prebuilt-only) and document it in
ci-integration.md. - Make the timing-sensitive
boundedCommandandprivateStoragedriver tests deterministic or isolate them, so a plain parallel localyarn lage buildcican serve as acceptance evidence alongside hosted CI. - Measure at least 100 representative executions per endpoint over 14 days, with at least 99% infrastructure success, zero leaked app/helper processes, zero unreleased input, and actionable metadata for every infrastructure failure.
- Prevent all-skipped selected cohorts from being reported as successful completed coverage; preserve truthful mixed pass/skip reports.
- Promote only qualified capabilities through the real required-status path, and prove a relevant failing native job blocks merging.
- Preserve authenticated readiness, exact app/helper leases, loopback-only listeners, source verification, serial physical input, and cleanup/cancellation contracts.
- Document adding/filtering/reproducing native tests and diagnosing failures. Pixel differences remain advisory, not an implicit merge gate.
Evidence and dependencies
- Native PR jobs and aggregate
- Runner authority and promotion guidance
- Executable-test and manifest contract
- Current manifest digest construction
- Delivered milestone: Infrastructure: Add Storybook end-to-end test pipelines #4256. Coordinate Infrastructure: Restore Windows Storybook focus-regression parity #4347 (regression parity), Infrastructure: Complete portable Storybook accessibility coverage and reporting #4343/Infrastructure: Fix native accessibility-state projection gaps on macOS and Win32 #4346 (accessibility), Infrastructure: Complete Desktop Driver production release qualification #4345 (release), Infrastructure: Qualify Windows/Win32 capture fidelity and provenance #4348 (capture), Infrastructure: Restore Storybook stories excluded by native desktop crashes #4344 (catalog), and Infrastructure: Reject all-skipped WDIO runs as completed coverage #4342 (all-skipped reporting; closed via Desktop testing: declare and select WDIO cases by input lane, platform and scope #4464). Share reliability evidence with driver release qualification rather than claiming two independent windows.
Activity
JasonVMo commentedon Nov 16, 2026
Status update. #4500 added an advisory
macOS Storybook PR (physical)job (Fabric,--input physical, no disabled input features), outside the aggregateprneeds. Hosted run 43344510349 passed 343 checks with 3 declared unsupported and 0 failures. That is one run: managed runners, required-status promotion and the 100-run/14-day window remain open here. The existing macOS lanes keep theirphysicalClickmask. #4498's hosted Windows/Win32 jobs passed on attempt 1. Added gaps: one consistent helper build/verification policy across lanes, the hosted Windows ownership consumer opt-in (0 of 11 ran), and the timing-sensitiveboundedCommand/privateStoragetests that fail in a plain parallel local aggregate.