Skip to content

Commit da2b62c

Browse files
JasonVMoCopilot
andauthored
Desktop testing: migrate stories and split CI input lanes (#4466)
* WDIO descriptor and desktopIt helper * Select and account for desktop WDIO cases by input lane Finish part 1 of the input-lane work started in f5a90cd. Cases declare how they run with desktopIt, and a run selects the cases that apply to its input lane, platform and scope and accounts for everything it excludes. desktop-driver - Add pure case selection and click-mode resolution, and move the requirement preflight and click-mode helpers here. - Add the authoring/input leaf export so config-loading code can import scope helpers from source without the authoring index. desktop-test - Report the lane: a lane run is incomplete when it has skipped or undeclared-unsupported results, an empty selection or nothing executed. - Add the Jasmine adapter (desktop-test/jasmine) with desktopIt and desktopDescribe, plus config-level test scopes and --input. storybook-desktop - Add --input accessible|physical to storybook test, driver and smoke. The lane comes from the flag, DESKTOP_INPUT or the attached driver. - Select cases before they run, resolve declared gaps, assess each cohort, and reject direct input calls in declared cases at extraction. - Fail closed when an accessible lane attaches to a driver that does not mask keyboard, physical click and wheel input. native-core and apps/E2E - Migrate the native-core Callout and FocusZone tests and ButtonV1.spec.win.ts to desktopIt. Update the package references and add patch changesets. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Migrate desktop story tests to declared input lanes Replace remaining component story runtime skips with desktopIt requirements, platform scopes and declared gaps. Carry the Win32 preview fixes from #4431 and split accessible and advisory physical Storybook CI lanes without changing aggregate PR requirements. Co-authored-by: Srividya Ramaswamy <sriramas@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 26cc577 commit da2b62c

38 files changed

Lines changed: 1704 additions & 1361 deletions
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@fluentui-react-native/components': patch
3+
---
4+
5+
Declare every component and primitive story test with `desktopIt`, so each case states the input it needs, the endpoints it runs on, and its known gaps instead of skipping at run time, and remove the internal `requireDesktopFocus` story helper.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@fluentui-react-native/desktop-driver': patch
3+
---
4+
5+
Describe the Storybook CI input lanes: the hosted Windows and Win32 jobs run the accessible lane, and an advisory Win32 job runs the physical lane.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@fluentui-react-native/storybook-desktop': patch
3+
---
4+
5+
Close the upstream Storybook HTTP listener and its open connections when the legacy channel service stops, so lingering keep-alive or reconnect sockets cannot keep the channel process alive past its shutdown budget.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@fluentui-react-native/storybook-desktop-runtime': patch
3+
---
4+
5+
Scroll the Win32 story preview inside its viewport and remount the preview for each navigation generation, so story tests start from a fresh, reachable preview.

‎.github/workflows/pr.yml‎

Lines changed: 48 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -445,7 +445,7 @@ jobs:
445445
}
446446
447447
- name: Smoke test Windows Storybook
448-
run: yarn storybook smoke --windows --mode stories-and-tests
448+
run: yarn storybook smoke --windows --mode stories-and-tests --input accessible
449449
working-directory: apps/storybook
450450

451451
- name: Upload Windows Storybook artifacts
@@ -510,7 +510,7 @@ jobs:
510510
if-no-files-found: warn
511511

512512
win32-storybook:
513-
name: Win32 Storybook PR
513+
name: Win32 Storybook PR (accessible)
514514
runs-on: windows-latest
515515
timeout-minutes: 60
516516
steps:
@@ -537,14 +537,58 @@ jobs:
537537
disabled-input-features: keyboard,physicalClick,wheel
538538

539539
- name: Smoke test Win32 Storybook
540-
run: yarn storybook smoke --win32 --mode stories-and-tests
540+
run: yarn storybook smoke --win32 --mode stories-and-tests --input accessible
541541
working-directory: apps/storybook
542542

543543
- name: Upload Win32 Storybook artifacts
544544
if: ${{ always() }}
545545
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
546546
with:
547-
name: Storybook_win32_paper_${{ github.run_id }}_${{ github.run_attempt }}
547+
name: Storybook_win32_paper_accessible_${{ github.run_id }}_${{ github.run_attempt }}
548+
path: |
549+
apps/storybook/artifacts/win32
550+
apps/storybook/artifacts/desktop
551+
${{ steps.desktop-driver.outputs.doctor-path }}
552+
${{ steps.desktop-driver.outputs.cache-root }}/v1/artifacts/**/build.log
553+
${{ steps.desktop-driver.outputs.cache-root }}/v1/diagnostics/**
554+
if-no-files-found: warn
555+
556+
# An advisory pilot outside the aggregate PR check: hosted physical input is not yet qualified (#4349).
557+
win32-storybook-physical:
558+
name: Win32 Storybook PR (physical)
559+
runs-on: windows-latest
560+
timeout-minutes: 60
561+
steps:
562+
- name: Checkout
563+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
564+
565+
- name: Set up toolchain
566+
uses: microsoft/react-native-test-app/.github/actions/setup-toolchain@1c99dd2947a77097bf937701e3dcc90aa109d17e # 5.4.15
567+
with:
568+
platform: windows
569+
node-version: 22
570+
571+
- name: Install dependencies
572+
run: yarn
573+
574+
- name: Build packages
575+
run: yarn build
576+
577+
- name: Set up Win32 Desktop Driver
578+
id: desktop-driver
579+
uses: ./.github/actions/setup-desktop-driver
580+
with:
581+
platform: win32
582+
583+
- name: Smoke test Win32 Storybook
584+
run: yarn storybook smoke --win32 --mode stories-and-tests --input physical
585+
working-directory: apps/storybook
586+
587+
- name: Upload Win32 Storybook artifacts
588+
if: ${{ always() }}
589+
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
590+
with:
591+
name: Storybook_win32_paper_physical_${{ github.run_id }}_${{ github.run_attempt }}
548592
path: |
549593
apps/storybook/artifacts/win32
550594
apps/storybook/artifacts/desktop

‎apps/storybook/README.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,11 @@ yarn storybook smoke --macos --mode stories-and-tests
2323
```
2424

2525
Replace `macos` with `windows` or `win32`. Use `yarn storybook test
26-
--<platform> --list` to discover executable story-test names. Runtime IDs and
26+
--<platform> --list` to discover executable story-test names. Add
27+
`--input accessible` to `test` or `smoke --mode stories-and-tests` to exclude
28+
cases that need physical input, or `--input physical` on an unattended
29+
machine. Accessible runs avoid pointer and keyboard injection, but may still
30+
activate windows or change focus; keep desktop workloads exclusive. Runtime IDs and
2731
listener ports are available through the app's `desktop-driver` command.
2832
See `agent-map.yaml` for the current command contract and paths.
2933

‎apps/storybook/agent-map.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,11 +31,12 @@ services:
3131
executable_tests:
3232
command: yarn storybook test --<platform>
3333
discovery: yarn storybook test --<platform> --list
34-
authoring: top-level wdio callbacks or named callback collections in CSF3 stories
34+
authoring: desktopIt cases with input, platform, and gap descriptors in CSF3 story wdio arrays
3535
config: desktop.config.mts Storybook integration wdio settings
3636
lifecycle: attached-session-per-test-with-isolated-node-worker
3737
context_platform: macos-or-windows-or-win32-target-endpoint
3838
test_filter: --test <name-glob>
39+
input_lane: --input accessible|physical
3940
smoke_mode: stories-and-tests
4041
execution: navigate-and-remount-then-run-tests-grouped-by-story
4142

‎packages/agentic/components/references/tests-and-stories.md‎

Lines changed: 42 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -137,23 +137,41 @@ validate.
137137
Prefer executable WebdriverIO tests under `wdio`, typed with
138138
`WdioStory<StoryObj<typeof Component>>` from
139139
`@fluentui-react-native/storybook-desktop/testing` using `import type`.
140-
The story is the suite (`describe`), and named functions are its cases (`it`):
140+
The story is the suite (`describe`). Declare each case (`it`) with `desktopIt`
141+
from the same entry point, adding a descriptor when the case needs host input
142+
or applies only to some endpoints:
141143

142144
```tsx
143145
export const Default: Story = {
144-
wdio: {
145-
'is enabled': async ({ browser, expect }) => {
146+
wdio: [
147+
desktopIt('is enabled', async ({ browser, expect }) => {
146148
await expect(await browser.$('~story-button')).toBeEnabled();
147-
},
148-
},
149+
}),
150+
desktopIt('moves focus with Tab', { requires: ['physical-click', 'keyboard', 'focus'] }, async ({ browser, expect }) => {
151+
// ...
152+
}),
153+
],
149154
};
150155
```
151156

152157
Each callback receives `browser`, `expect`, `platform`, `desktop`, `signal`,
153158
and `skip`. The typed target `platform` is `macos`, `windows`, or `win32`;
154159
Win32 is distinct from its WebDriver `platformName` of `windows`. Branch
155-
inside callbacks for genuine platform differences and use explicit skip
156-
reasons for unsupported `browser.capabilities['furn:features']`.
160+
inside callbacks for genuine platform differences in one assertion. Do not
161+
call `skip` or feature-check `browser.capabilities['furn:features']` for a
162+
condition the descriptor can state:
163+
164+
- `requires` lists the host capabilities the case uses, and its physical
165+
entries put the case in the physical input lane. Focus and keyboard cases
166+
declare `['physical-click', 'keyboard', 'focus']`. A case that reads only
167+
native state declares nothing and runs in both lanes. Use `click` when either
168+
click transport proves the behavior, `physical-click` for pointer-specific
169+
behavior, and `accessibility-click` for UIA or AX action dispatch.
170+
- `platforms` scopes the case to endpoints, such as `['win32']` or `['!macos']`.
171+
Split a case when endpoints need different transports rather than branching
172+
on the click mode.
173+
- `unsupported` records a known product gap on named endpoints with a reason,
174+
and `quarantine` suspends a flaky case with an issue and an `expires` date.
157175

158176
All catalog tests use WDIO; custom `desktopDriver` plans remain only in runner
159177
compatibility fixtures. Keep names static and callbacks self-contained. Node helpers may be
@@ -162,11 +180,13 @@ story-module bindings. The shared Babel config strips all test functions
162180
before native dependency collection. Named cases get independent previews,
163181
sessions, workers, deadlines, results, and failure evidence.
164182

165-
The single-function `wdio` form remains supported. Run with
166-
`yarn storybook test --<platform>` and optionally `--test <name-glob>`.
167-
`--list` discovers names without executing code. The `stories-and-tests`
183+
Run with `yarn storybook test --<platform>`, optionally with `--test <name-glob>`
184+
and `--input accessible|physical` to select one input lane. `--list` discovers
185+
cases and their resolved input without executing code. The `stories-and-tests`
168186
smoke mode runs these functions alongside remaining legacy plans, grouped by
169-
story. See the executable test contract (`@fluentui-react-native/storybook-desktop/references/story-tests.md`).
187+
story. See the executable test contract
188+
(`@fluentui-react-native/storybook-desktop/references/story-tests.md`) for the
189+
full descriptor, lane, and reporting rules.
170190

171191
The components package's `test:stories` project checks every test-bearing story
172192
without emitting it into the component library. Add new test-bearing files to
@@ -191,9 +211,12 @@ timing.
191211
### Desktop focus test scope
192212

193213
Keep shared requirements executable on Windows Fabric, Office Win32, and macOS.
194-
`requireDesktopFocus` checks native capabilities, not an operating-system
195-
allowlist. A missing capability skips the case with a reason; a platform
196-
difference usually changes one assertion rather than skipping the entire case.
214+
Focus cases declare `requires: ['physical-click', 'keyboard', 'focus']`, which
215+
describes native capabilities, not an operating-system allowlist. The physical
216+
lane runs them, the accessible lane excludes them, and a run without a lane
217+
reports a case whose capability the host lacks as unsupported, with its reason,
218+
instead of executing it. A platform difference
219+
usually changes one assertion rather than excluding the entire case.
197220

198221
| Behavior | Scope and expected result |
199222
| -------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -225,13 +248,15 @@ resolved-style coverage without incidental scene rerenders, plus native visual
225248
evidence for renderer appearance. Likewise, a checked/selected state read after
226249
a physical click does not exercise UIA/AX action dispatch.
227250

228-
Report passed, failed, and skipped counts together. A zero-exit smoke traversal
229-
with platform-skipped focus cases is not cross-platform focus qualification.
251+
Report passed, failed, skipped, unsupported, and excluded counts together. A
252+
zero-exit accessible-lane run excludes every focus case and is not
253+
cross-platform focus qualification.
230254

231255
The macOS Fabric View does not project disabled or selected accessibility state
232256
to AXEnabled/AXSelected (C-A11Y-005 in the
233257
repository evidence reference `accessibility.action-transport-evidence`). Those native
234-
announcement assertions are isolated in explicitly skipped cases; shared
258+
announcement assertions are isolated in cases declaring
259+
`unsupported: { platforms: ['macos'], reason }`; shared
235260
focus, activation, and selection-callback cases still run. Do not substitute
236261
rendered state for an AX result or describe these skips as intended macOS
237262
semantics. Track the native projection gap and revisit the cases when the

‎packages/agentic/components/src/common/desktopFocus.wdio.ts‎

Lines changed: 0 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,5 @@
11
import type { WdioStoryContext } from '@fluentui-react-native/storybook-desktop/testing';
22

3-
export function requireDesktopFocus({ browser, skip }: Pick<WdioStoryContext, 'browser' | 'skip'>): boolean {
4-
const features = browser.capabilities['furn:features'];
5-
if (!features) {
6-
throw new Error('Desktop Driver did not provide feature capabilities.');
7-
}
8-
if (!features.physicalClick || !features.keyboard || !features.focus) {
9-
skip('Native focus qualification requires physical pointer input, keyboard input, and focus observation.');
10-
return false;
11-
}
12-
return true;
13-
}
14-
153
/** Click-to-focus is Windows-specific; ordinary AppKit controls need not take focus on click. */
164
export async function expectWindowsPointerFocus(
175
{ browser, platform }: Pick<WdioStoryContext, 'browser' | 'platform'>,

‎packages/agentic/components/src/components/avatar/avatar.stories.tsx‎

Lines changed: 25 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import type { ReactNode } from 'react';
33
import { StyleSheet, Text, View } from 'react-native';
44

55
import type { Meta, StoryObj } from '@storybook/react-native';
6+
import { desktopIt } from '@fluentui-react-native/storybook-desktop/testing';
67
import type { WdioStory } from '@fluentui-react-native/storybook-desktop/testing';
78

89
import { Avatar } from './avatar';
@@ -108,32 +109,35 @@ export const ActivityRing: Story = {
108109
},
109110
},
110111
},
111-
wdio: {
112-
'AVT-004 exposes the informative identity as a named native image': async ({ browser, expect }) => {
112+
wdio: [
113+
desktopIt('AVT-004 exposes the informative identity as a named native image', async ({ browser, expect }) => {
113114
const avatar = await browser.$('~agentic-storybook-avatar-activity-ring');
114115
await expect(avatar).toExist();
115116
expect(await avatar.getTagName()).toBe('image');
116117
expect(await avatar.getProperty('name')).toBe('Lydia Mitchelson');
117-
},
118-
'toggles the activity ring at every size without crashing or changing avatar bounds': async ({ browser, desktop, expect }) => {
119-
const assert: typeof import('node:assert') = (await import('node:assert')).default;
120-
for (const size of [16, 20, 24, 28, 32, 40, 56, 120]) {
121-
for (const activityRing of [false, true]) {
122-
await desktop.session.updateStoryArgs('components-avatar--activity-ring', { activityRing, size });
123-
await expect(await browser.$('~agentic-storybook-avatar-ring-state')).toHaveText(
124-
`Activity ring ${activityRing ? 'on' : 'off'}, size ${size}`,
125-
);
126-
const avatar = await browser.$('~agentic-storybook-avatar-activity-ring');
127-
await expect(avatar).toExist();
128-
const bounds = await browser.getElementRect(await avatar.elementId);
129-
assert(
130-
Math.abs(bounds.width - size) <= 1 && Math.abs(bounds.height - size) <= 1,
131-
'The activity ring must not resize the avatar.',
132-
);
118+
}),
119+
desktopIt(
120+
'toggles the activity ring at every size without crashing or changing avatar bounds',
121+
async ({ browser, desktop, expect }) => {
122+
const assert: typeof import('node:assert') = (await import('node:assert')).default;
123+
for (const size of [16, 20, 24, 28, 32, 40, 56, 120]) {
124+
for (const activityRing of [false, true]) {
125+
await desktop.session.updateStoryArgs('components-avatar--activity-ring', { activityRing, size });
126+
await expect(await browser.$('~agentic-storybook-avatar-ring-state')).toHaveText(
127+
`Activity ring ${activityRing ? 'on' : 'off'}, size ${size}`,
128+
);
129+
const avatar = await browser.$('~agentic-storybook-avatar-activity-ring');
130+
await expect(avatar).toExist();
131+
const bounds = await browser.getElementRect(await avatar.elementId);
132+
assert(
133+
Math.abs(bounds.width - size) <= 1 && Math.abs(bounds.height - size) <= 1,
134+
'The activity ring must not resize the avatar.',
135+
);
136+
}
133137
}
134-
}
135-
},
136-
},
138+
},
139+
),
140+
],
137141
};
138142

139143
const styles = StyleSheet.create({

0 commit comments

Comments
 (0)