Skip to content

Commit 8574d22

Browse files
JasonVMoCopilot
andauthored
Desktop testing: declare and select WDIO cases by input lane, platform and scope (#4464)
* WDIO descriptor and desktopIt helper * Select and account for desktop WDIO cases by input lane Finish part 1 of the input-lane work started in f5a90cd. Cases declare how they run with desktopIt, and a run selects the cases that apply to its input lane, platform and scope and accounts for everything it excludes. desktop-driver - Add pure case selection and click-mode resolution, and move the requirement preflight and click-mode helpers here. - Add the authoring/input leaf export so config-loading code can import scope helpers from source without the authoring index. desktop-test - Report the lane: a lane run is incomplete when it has skipped or undeclared-unsupported results, an empty selection or nothing executed. - Add the Jasmine adapter (desktop-test/jasmine) with desktopIt and desktopDescribe, plus config-level test scopes and --input. storybook-desktop - Add --input accessible|physical to storybook test, driver and smoke. The lane comes from the flag, DESKTOP_INPUT or the attached driver. - Select cases before they run, resolve declared gaps, assess each cohort, and reject direct input calls in declared cases at extraction. - Fail closed when an accessible lane attaches to a driver that does not mask keyboard, physical click and wheel input. native-core and apps/E2E - Migrate the native-core Callout and FocusZone tests and ButtonV1.spec.win.ts to desktopIt. Update the package references and add patch changesets. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 2e50f64 commit 8574d22

96 files changed

Lines changed: 5583 additions & 565 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.changeset/input-lanes.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
'@fluentui-react-native/desktop-driver': patch
3+
'@fluentui-react-native/desktop-test': patch
4+
'@fluentui-react-native/storybook-desktop': patch
5+
---
6+
7+
Add modality-aware desktop test descriptors: `desktopIt` and `desktopDescribe` declare `requires`, `input`, `platforms`, `scope`, and known gaps for Storybook cases and, through `desktop-test/jasmine`, WebdriverIO Jasmine specs. `--input accessible|physical` (or `DESKTOP_INPUT`) runs one input lane, selects the cases that apply, masks the host for the accessible lane, and reports the lane and what it excluded.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@fluentui-react-native/native-core': patch
3+
---
4+
5+
Declare the Callout and FocusZone desktop tests with `desktopIt` and `requires`, so each runs in the input lane it needs and the Win32 Tab-exit gap is declared instead of skipped at run time.

‎apps/E2E/src/ButtonV1/specs/ButtonV1.spec.win.ts‎

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,17 @@
1+
import { desktopDescribe, desktopIt } from '@fluentui-react-native/desktop-test/jasmine';
2+
13
import { BUTTON_ACCESSIBILITY_LABEL, BUTTON_TEST_COMPONENT_LABEL } from '../../ButtonLegacy/consts';
24
import { BUTTON_A11Y_ROLE, Keys, Attribute } from '../../common/consts';
35
import ButtonV1PageObject from '../pages/ButtonV1PageObject';
46

57
// Before testing begins, allow up to 60 seconds for app to open
6-
describe('ButtonV1 Testing Initialization', () => {
7-
it('Wait for app load', async () => {
8+
desktopDescribe('ButtonV1 Testing Initialization', () => {
9+
desktopIt('Wait for app load', async () => {
810
expect(await ButtonV1PageObject.waitForInitialPageToDisplay()).toBeTrue();
911
});
1012

11-
it('Click and navigate to ButtonV1 test page', async () => {
13+
// Navigation scrolls the page list with the wheel whenever the page button is out of view.
14+
desktopIt('Click and navigate to ButtonV1 test page', { requires: ['click', 'wheel'] }, async () => {
1215
/* Click on component button to navigate to test page */
1316
expect(await ButtonV1PageObject.navigateToPageAndLoadTests()).toBeTrue();
1417

@@ -21,18 +24,19 @@ describe('ButtonV1 Testing Initialization', () => {
2124
});
2225
});
2326

24-
describe('ButtonV1 Accessibility Testing', () => {
27+
// The setup scrolls the test element into view, which can inject native wheel input, so this group is physical.
28+
desktopDescribe('ButtonV1 Accessibility Testing', { requires: ['wheel'] }, () => {
2529
beforeEach(async () => {
2630
await ButtonV1PageObject.scrollToTestElement();
2731
});
2832

29-
it('Validate "accessibilityRole" value defaults to "ControlType.Button".', async () => {
33+
desktopIt('Validate "accessibilityRole" value defaults to "ControlType.Button".', async () => {
3034
await expect(
3135
await ButtonV1PageObject.compareAttribute(ButtonV1PageObject._primaryComponent, Attribute.AccessibilityRole, BUTTON_A11Y_ROLE),
3236
).toBeTruthy();
3337
});
3438

35-
it('Set "accessibilityLabel" prop. Validate "accessibilityLabel" value propagates to "Name" element attribute.', async () => {
39+
desktopIt('Set "accessibilityLabel" prop. Validate "accessibilityLabel" value propagates to "Name" element attribute.', async () => {
3640
await expect(
3741
await ButtonV1PageObject.compareAttribute(
3842
ButtonV1PageObject._primaryComponent,
@@ -42,7 +46,7 @@ describe('ButtonV1 Accessibility Testing', () => {
4246
).toBeTruthy();
4347
});
4448

45-
it('Do NOT set "accessibilityLabel" prop. Validate "Name" element attribute defaults to the button label.', async () => {
49+
desktopIt('Do NOT set "accessibilityLabel" prop. Validate "Name" element attribute defaults to the button label.', async () => {
4650
await expect(
4751
await ButtonV1PageObject.compareAttribute(
4852
ButtonV1PageObject._secondaryComponent,
@@ -53,7 +57,7 @@ describe('ButtonV1 Accessibility Testing', () => {
5357
});
5458
});
5559

56-
describe('ButtonV1 Functional Testing', () => {
60+
desktopDescribe('ButtonV1 Functional Testing', { requires: ['wheel'] }, () => {
5761
/* Scrolls and waits for the Button to be visible on the Test Page */
5862
beforeEach(async () => {
5963
await ButtonV1PageObject.scrollToTestElement();
@@ -63,7 +67,7 @@ describe('ButtonV1 Functional Testing', () => {
6367
await ButtonV1PageObject.resetTest();
6468
});
6569

66-
it('Click primary button. Validate onClick() callback was fired.', async () => {
70+
desktopIt('Click primary button. Validate onClick() callback was fired.', { requires: ['physical-click'] }, async () => {
6771
await ButtonV1PageObject.click(ButtonV1PageObject._primaryComponent);
6872
await expect(
6973
await ButtonV1PageObject.waitForOnClickCallbackToFire(`The primary button failed to fire an onClick callback with a mouse click.`),
@@ -74,7 +78,7 @@ describe('ButtonV1 Functional Testing', () => {
7478
.toBeFalsy();
7579
});
7680

77-
it('Type "Enter" on primary button. Validate onClick() callback was fired.', async () => {
81+
desktopIt('Type "Enter" on primary button. Validate onClick() callback was fired.', { requires: ['keyboard'] }, async () => {
7882
await ButtonV1PageObject.sendKeys(ButtonV1PageObject._primaryComponent, [Keys.ENTER]);
7983
await expect(
8084
await ButtonV1PageObject.waitForOnClickCallbackToFire(
@@ -87,7 +91,7 @@ describe('ButtonV1 Functional Testing', () => {
8791
.toBeFalsy();
8892
});
8993

90-
it('Type "Space" on primary button. Validate onClick() callback was fired.', async () => {
94+
desktopIt('Type "Space" on primary button. Validate onClick() callback was fired.', { requires: ['keyboard'] }, async () => {
9195
await ButtonV1PageObject.sendKeys(ButtonV1PageObject._primaryComponent, [Keys.SPACE]);
9296

9397
await expect(

‎packages/agentic/desktop-driver/README.md‎

Lines changed: 18 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -139,23 +139,24 @@ The [reference index](references/README.md) describes each package-owned guide.
139139

140140
## Public entry points
141141

142-
| Export | Purpose |
143-
| --------------- | ----------------------------------------------------------------------------------------- |
144-
| Package root | Native-helper build/resolution, `DesktopHost`, server, low-level client, and common types |
145-
| `/authoring` | Serializable story plans, selectors, capabilities, and result types |
146-
| `/wdio` | WebdriverIO connection, custom commands, assertions, and story-plan runner |
147-
| `/client` | Low-level typed W3C client |
148-
| `/authority` | Grants, private connection descriptors, and macOS owner-validated storage |
149-
| `/native` | Native-helper build, resolution, and verification without loading the package root |
150-
| `/server` | Embeddable remote end, session manager, and target registry |
151-
| `/agent` | Bounded JSON-safe inspection, action, and evidence API |
152-
| `/operations` | Versioned agent operation catalog, executor, and result types |
153-
| `/artifacts` | Confined, atomic evidence persistence |
154-
| `/runner` | Plan selection, sharding, execution, and result classification |
155-
| `/storybook` | Story manifest and orchestration contracts, without a Storybook dependency |
156-
| `/testing` | Fake host, fake Storybook orchestrator, and protocol harnesses |
157-
| `/cli` | Embeddable Commander command and CLI runner |
158-
| `/cli/reporter` | Shared reporter mode resolution and structured output framing |
142+
| Export | Purpose |
143+
| ------------------ | ----------------------------------------------------------------------------------------- |
144+
| Package root | Native-helper build/resolution, `DesktopHost`, server, low-level client, and common types |
145+
| `/authoring` | Serializable story plans, selectors, capabilities, result types, and case descriptors |
146+
| `/wdio` | WebdriverIO connection, custom commands, assertions, and story-plan runner |
147+
| `/client` | Low-level typed W3C client |
148+
| `/authority` | Grants, private connection descriptors, and macOS owner-validated storage |
149+
| `/native` | Native-helper build, resolution, and verification without loading the package root |
150+
| `/server` | Embeddable remote end, session manager, and target registry |
151+
| `/agent` | Bounded JSON-safe inspection, action, and evidence API |
152+
| `/operations` | Versioned agent operation catalog, executor, and result types |
153+
| `/artifacts` | Confined, atomic evidence persistence |
154+
| `/runner` | Plan selection, sharding, execution, and result classification |
155+
| `/storybook` | Story manifest and orchestration contracts, without a Storybook dependency |
156+
| `/testing` | Fake host, fake Storybook orchestrator, and protocol harnesses |
157+
| `/cli` | Embeddable Commander command and CLI runner |
158+
| `/cli/reporter` | Shared reporter mode resolution and structured output framing |
159+
| `/authoring/input` | Descriptor vocabulary and scope rules with no runtime imports (loads from source) |
159160

160161
The service implements native application semantics rather than pretending to
161162
be a browser. Unsupported browser features return `unsupported operation`

‎packages/agentic/desktop-driver/package.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,11 @@
5353
"import": "./lib/authoring/index.js",
5454
"default": "./src/authoring/index.ts"
5555
},
56+
"./authoring/input": {
57+
"types": "./lib/authoring/input.d.ts",
58+
"import": "./lib/authoring/input.js",
59+
"default": "./src/authoring/input.ts"
60+
},
5661
"./agent": {
5762
"types": "./lib/agent/index.d.ts",
5863
"import": "./lib/agent/index.js",

‎packages/agentic/desktop-driver/references/architecture.md‎

Lines changed: 19 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -120,25 +120,25 @@ handoff are still work in progress; see [current status](current-status.md).
120120

121121
## Module ownership
122122

123-
| Module | Responsibility |
124-
| ------------------ | ------------------------------------------------------------------ |
125-
| `src/protocol` | W3C parsing, actions, capabilities, timeouts, and errors |
126-
| `src/server` | HTTP routing, target registry, sessions, windows, and elements |
127-
| `src/host` | Platform-neutral `DesktopHost` and target contracts |
128-
| `src/hosts/fake` | Deterministic in-memory provider |
129-
| `src/hosts/native` | Long-lived helper process and `DesktopHost` adapter |
130-
| `src/native` | Build, cache, resolution, verification, manifests, and wire types |
131-
| `native/macos` | Swift operating-system implementation |
132-
| `native/windows` | Shared C++ Windows and Win32 implementation |
133-
| `src/client` | Low-level typed W3C client |
134-
| `src/wdio` | Sanctioned WebdriverIO integration and custom commands |
135-
| `src/authoring` | Serializable story plans, selectors, capabilities, and results |
136-
| `src/runner` | Filtering, sharding, execution, and result classification |
137-
| `src/artifacts` | Confined atomic evidence |
138-
| `src/agent` | Bounded JSON-safe operations |
139-
| `src/operations` | Agent operation catalog, executor, references, ledger, and journal |
140-
| `src/testing` | Fake harnesses and Storybook test doubles |
141-
| `src/cli` | Commander parsing, JSON output, and process exit adaptation |
123+
| Module | Responsibility |
124+
| ------------------ | -------------------------------------------------------------------------------- |
125+
| `src/protocol` | W3C parsing, actions, capabilities, timeouts, and errors |
126+
| `src/server` | HTTP routing, target registry, sessions, windows, and elements |
127+
| `src/host` | Platform-neutral `DesktopHost` and target contracts |
128+
| `src/hosts/fake` | Deterministic in-memory provider |
129+
| `src/hosts/native` | Long-lived helper process and `DesktopHost` adapter |
130+
| `src/native` | Build, cache, resolution, verification, manifests, and wire types |
131+
| `native/macos` | Swift operating-system implementation |
132+
| `native/windows` | Shared C++ Windows and Win32 implementation |
133+
| `src/client` | Low-level typed W3C client |
134+
| `src/wdio` | Sanctioned WebdriverIO integration and custom commands |
135+
| `src/authoring` | Serializable story plans, selectors, capabilities, results, and case descriptors |
136+
| `src/runner` | Filtering, sharding, execution, and result classification |
137+
| `src/artifacts` | Confined atomic evidence |
138+
| `src/agent` | Bounded JSON-safe operations |
139+
| `src/operations` | Agent operation catalog, executor, references, ledger, and journal |
140+
| `src/testing` | Fake harnesses and Storybook test doubles |
141+
| `src/cli` | Commander parsing, JSON output, and process exit adaptation |
142142

143143
## Deliberate boundaries
144144

‎packages/agentic/desktop-driver/references/contributing.md‎

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ change-oriented guidance.
2323
| `src/native` | Source build, cache, selection, verification, and native artifact types |
2424
| `native/macos` | Swift implementation and self-test |
2525
| `native/windows` | Shared C++ Windows/Win32 implementation and self-test |
26-
| `src/authoring` | Static JSON plan and result schemas |
26+
| `src/authoring` | Static JSON plan and result schemas, and the case descriptor |
2727
| `src/runner` | Plan selection, capability checks, execution, and classification |
2828
| `src/wdio` | Sanctioned WebdriverIO connection and commands |
2929
| `src/client` | Low-level typed W3C client |
@@ -38,6 +38,39 @@ Do not duplicate native build, cache, or verification logic in Storybook
3838
consumers. Do not move Storybook manifest generation or app lifecycle into this
3939
package.
4040

41+
## Case descriptors and input lanes
42+
43+
`src/authoring/input.ts` and `src/authoring/selection.ts` are the one
44+
implementation of the test case descriptor (`requires`, `input`, `platforms`,
45+
`scope`, and declared gaps). Storybook extraction and its runner, and the
46+
desktop-test Jasmine adapter, all call them; do not reimplement validation,
47+
merging, or a decision in a consumer. Keep both files pure data and functions,
48+
with no Node, React, or WebdriverIO imports, because stories and workers load
49+
them. `input.ts` also has no runtime imports at all (its one import is a type),
50+
and the `./authoring/input` subpath exports it directly: desktop-test's config
51+
loader reads scope domains from it, and a config file is loaded from source
52+
through a CommonJS fallback that cannot resolve this package's `.js`
53+
specifiers, so keep it that way.
54+
55+
- Add a requirement to `desktopRequirements`, to the physical or accessible
56+
requirement set in `input.ts` when it implies an input, and to the
57+
`requirementFeature` map in `selection.ts` for the host feature that backs it,
58+
together. A requirement whose feature a host did not report is unmet.
59+
- `mergeDesktopCaseOptions` only narrows: requirements union, `platforms` and
60+
`scope` intersect, and a nested declaration can never widen or change its
61+
group's input. An overlap that is empty is an error because the case could
62+
never run. A group cannot carry a gap.
63+
- `decideDesktopCase` judges platform, then scope, then the lane, so a case is
64+
excluded for one reason. It resolves a declared gap too, so an expired
65+
quarantine stops a run before anything executes.
66+
- `resolveDesktopClickMode` lets a lane decide the click mode outright. That is
67+
how one `click` requirement is an accessibility click in the accessible lane
68+
and a physical click in the physical lane.
69+
- `accessibleLaneMaskedFeatures` (`keyboard`, `physicalClick`, `wheel`) is what
70+
an accessible lane disables through the native host's `disabledInputFeatures`,
71+
and `unmaskedAccessibleFeatures` fails closed when a host reports any of them
72+
as still available. Change the set together with the hosts that apply it.
73+
4174
## Behavioral rules
4275

4376
- Reserve a target before asynchronous launch or attach.

‎packages/agentic/desktop-driver/references/test-integration.md‎

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -98,15 +98,24 @@ exits, including on assertion failures, timeouts, or crashes. WebdriverIO's
9898
sets `waitforInterval` (100 ms) and `waitforTimeout` (5 s) explicitly; without
9999
an interval, `waitUntil` and `waitFor*` stop after their first unsatisfied poll.
100100

101-
Storybook's preferred top-level `wdio` functions and named function collections
102-
use this pattern with Node's built-in test runner. Each function receives the
103-
target `platform` (`macos`, `windows`, or `win32`), and each named case gets an
104-
independent worker/session. Native feature flags are typed on the WebdriverIO
105-
browser's `furn:features` capability. Button now uses this authoring pattern
106-
instead of serializable plans. Extraction, native-bundle stripping, runner
107-
configuration, and source freshness belong to
108-
`@fluentui-react-native/storybook-desktop`,
109-
not this Storybook-independent package. Static plans below are unchanged.
101+
Storybook's preferred top-level `wdio` export is an array of `desktopIt` cases,
102+
which declare the input and platforms they need. Its older functions and named
103+
function collections use the same pattern with Node's built-in test runner.
104+
Each function receives the target `platform` (`macos`, `windows`, or `win32`),
105+
and each named case gets an independent worker/session. Native feature flags are
106+
typed on the WebdriverIO browser's `furn:features` capability. Button now uses
107+
this authoring pattern instead of serializable plans. Extraction, native-bundle
108+
stripping, runner configuration, and source freshness belong to
109+
`@fluentui-react-native/storybook-desktop`, not this Storybook-independent
110+
package. Static plans below are unchanged.
111+
112+
The descriptor that `desktopIt` accepts (`requires`, `input`, `platforms`,
113+
`scope`, and declared gaps) is defined and decided in `./authoring`, so Storybook
114+
and `desktop-test` share one implementation. A runner that executes one input lane
115+
pins `furn:clickMode` with `resolveDesktopClickMode`: `element.click()` is then an
116+
accessibility click in the accessible lane and a physical click in the physical
117+
lane. `missingDesktopRequirements` judges a case's `requires` against the
118+
session's `furn:features`, and judges a `click` requirement by the pinned mode.
110119

111120
## Author portable story plans
112121

0 commit comments

Comments
 (0)