Skip to content

Commit e3687dc

Browse files
test: Use the deadline cloud test fixtures for xa11y tests. (#487)
Signed-off-by: Karthik Bekal Pattathana <133984042+karthikbekalp@users.noreply.github.com>
1 parent 11eeb9f commit e3687dc

12 files changed

Lines changed: 198 additions & 1324 deletions

requirements-testing.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ numpy >= 2.2.6, < 2.5; python_version < "3.12"
1313
pillow == 12.*
1414
openjd-cli == 0.7.*
1515
fonttools >= 4.0.0
16+
deadline-cloud-test-fixtures >= 0.18.16
1617
# Drives the real submitter dialog via the OS accessibility tree.
1718
# Used by test/integ_xa11y/.
1819
xa11y >= 0.11.0

test/AGENTS.md

Lines changed: 24 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,9 @@ test/
1717
├── integ_xa11y/ # xa11y-driven submitter test, offline mock backend
1818
│ ├── test_cases/ # Self-contained cases: input/ expected/ actual/
1919
│ ├── test_cinema4d.py # Drives the real submitter dialog via xa11y
20-
│ ├── submitter_ui.py # Page-object for driving the dialog from configure.py
20+
│ ├── submitter_ui.py # C4D-specific controls; re-exports shared controls
2121
│ ├── conftest.py # Fixtures, incl. deadline_farm (starts the mock)
2222
│ ├── utils.py # Test utilities
23-
│ ├── mock_aws/ # Hand-rolled offline mock Deadline Cloud backend
2423
│ └── fixtures/ # Test-only sidecar plugin (auto-opens submitter)
2524
└── installer/ # Installer tests
2625
```
@@ -167,12 +166,11 @@ pytest (parent process)
167166
|------|------|
168167
| `test_cinema4d.py` | The test. Orchestrates scene build → launch → UI drive → assertions. Holds the `_CASES` registry. |
169168
| `conftest.py` | Fixtures: locate Cinema 4D, set `C4DPYTHONPATH`, and `deadline_farm` (starts the mock + builds the subprocess env). |
170-
| `submitter_ui.py` | Page-object for driving the dialog (tabs, fields, checkboxes) from a case's `configure.py`. |
171-
| `utils.py` | Helpers: exe resolution, scene build, bundle waiting, golden-bundle + image comparison. |
172-
| `mock_aws/deadline.py` | In-memory mock Deadline backend (rest-json) + HTTP server, with observability (`call_counts`, `request_log`, `unmatched_requests`). |
173-
| `mock_aws/server_process.py` | Runs the mock in a separate process; `RemoteBackend` reads observability over a `GET /__admin__/calls` admin endpoint. |
174-
| `mock_aws/wiring.py` | Builds the temp deadline config + subprocess env overlay pointing C4D at the mock. |
175-
| `mock_aws/fixtures_data.py` | Sanitized real response bodies (fake farm/queue IDs) the mock serves. |
169+
| `submitter_ui.py` | C4D-specific controls plus re-exports of shared controls from `deadline_test_fixtures.xa11y.controls`. |
170+
| `utils.py` | C4D executable/scene/render helpers and C4D's bundle normalization policy. Generic assertions come from `deadline-cloud-test-fixtures`. |
171+
| `deadline_test_fixtures.deadline_mock` | Scenario-driven Deadline REST-JSON mock, out-of-process lifecycle, observability, config, and environment wiring. |
172+
| `deadline_test_fixtures.job_bundle` | Shared case layout, bundle discovery, validation, and structural comparison. |
173+
| `deadline_test_fixtures.images` | Shared render-image comparison. |
176174
| `fixtures/auto_open_submitter/AutoOpenSubmitter.pyp` | Test-only C4D plugin: auto-opens the real submitter and (mock mode) applies the getaddrinfo / `os.startfile` patches. Never shipped. |
177175
| `test_cases/<name>/` | A self-contained case (see *Anatomy of a test case*). |
178176

@@ -224,11 +222,11 @@ $env:DIALOG_DUMP=1; hatch -e integ-xa11y run pytest --no-cov `
224222
test/integ_xa11y/test_cinema4d.py --numprocesses=0 -s -k <case>; $env:DIALOG_DUMP=$null
225223
```
226224

227-
Each line is `<role> "<name>" value="<value>"`. Match on role + name. Gotchas
228-
that the live tree reveals (and which differ by platform) are documented in
229-
`submitter_ui.py` — read them before writing a selector. The reusable page-object
230-
helpers there (`set_priority`, `toggle_checkbox`, `set_spin_button`, …) already
231-
encode the harvested selectors, so prefer them over raw `descendant(...)` calls.
225+
Each line is `<role> "<name>" value="<value>"`. Match on role + name. Shared
226+
cross-platform widget behavior and selector gotchas are documented in
227+
`deadline_test_fixtures.xa11y.controls`; Cinema 4D-specific selectors remain in
228+
`submitter_ui.py`. Prefer the reusable helpers exposed by `submitter_ui.py` over
229+
raw `descendant(...)` calls.
232230

233231
#### Offline mock architecture
234232

@@ -237,20 +235,19 @@ lives in the **sidecar** plugin, not the shipped `DeadlineCloud.pyp`, so the rea
237235
plugin is exercised exactly as a customer would and no test-only code reaches
238236
production.
239237

240-
- **Mock backend** (`mock_aws/deadline.py`): an in-memory Deadline Cloud
241-
simulator serving only the four operations the Export-bundle flow calls with
242-
an empty queue-environment list — `ListFarms`, `GetFarm`, `GetQueue`,
243-
`ListQueueEnvironments` (no `GetQueueEnvironment`, since the env list is
244-
empty). It records `call_counts` / `request_log` / `unmatched_requests` so the
238+
- **Mock backend** (`deadline_test_fixtures.deadline_mock`): an in-memory
239+
Deadline Cloud simulator serving the resource-read operations used by
240+
submitters, with an empty queue-environment list by default. It records
241+
`call_counts` / `request_log` / `unmatched_requests` so the
245242
test can assert exactly which calls reached it and that nothing hit an unmocked
246-
route. Seeded from sanitized real response data in `mock_aws/fixtures_data.py`.
247-
- **Separate process** (`mock_aws/server_process.py`): the mock runs in its own
243+
route. Its default scenario provides stable fake farm and queue resources.
244+
- **Separate process** (`MockDeadlineServerProcess`): the mock runs in its own
248245
process, NOT a thread. xa11y's native `wait_*` calls hold the CPython GIL for
249246
most of their duration, which would starve an in-process server thread and
250247
hang the test for the full 60s timeout. The out-of-process server keeps
251-
serving; the test reads its observability over a `GET /__admin__/calls` admin
252-
endpoint via a `RemoteBackend` proxy.
253-
- **Subprocess wiring** (`mock_aws/wiring.py` + the `deadline_farm` fixture):
248+
serving; the test reads its observability over the package's admin endpoint
249+
via a `RemoteDeadlineBackend` proxy.
250+
- **Subprocess wiring** (`build_mock_environment` + the `deadline_farm` fixture):
254251
a temp `deadline config` names the mock's farm/queue, and the C4D subprocess
255252
env gets `AWS_ENDPOINT_URL_DEADLINE` → mock, dummy AWS creds, telemetry
256253
opt-out (so no STS call fires), an isolated `HOME`, and
@@ -307,7 +304,7 @@ Caveats:
307304
#### Golden-bundle comparison
308305

309306
Direct byte comparison would be too brittle, so before comparing, the helper
310-
(`assert_expected_job_bundle_and_generated_job_bundle_are_equal`) normalizes a
307+
(`BundleNormalization` through the local comparison wrapper) normalizes a
311308
fixed set of moving parts: `PATH_TO_BE_REPLACED` → the local repo prefix;
312309
backslashes → forward slashes (preserving unicode escapes);
313310
`SubmitterIntegrationVersion` (changes every build) → a fixed placeholder; and
@@ -360,8 +357,9 @@ comparison parses both, but commit block YAML (re-serialize with
360357
**New mocked operation:** if a submitter change calls a Deadline operation the
361358
mock doesn't implement, the test fails its `unmatched_requests` assertion (and
362359
the mock logs `404 NO ROUTE`). Add a `@route`-decorated handler in
363-
`mock_aws/deadline.py` and, if it returns resource data, seed
364-
`mock_aws/fixtures_data.py`.
360+
`deadline_test_fixtures.deadline_mock.MockDeadlineBackend`. If it returns
361+
resource data, extend `MockDeadlineScenario` there. Keep DCC-specific launch
362+
behavior in this repository.
365363

366364
## Installer Tests
367365

test/integ_xa11y/conftest.py

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,13 @@
1111

1212
from pathlib import Path
1313

14-
from .mock_aws.server_process import MockServerProcess
15-
from .mock_aws.wiring import build_mock_env, write_deadline_config
14+
from deadline_test_fixtures.deadline_mock import (
15+
MockDeadlineScenario,
16+
MockDeadlineServerProcess,
17+
build_mock_environment,
18+
write_deadline_config,
19+
)
20+
1621
from .utils import c4d_extra_python_paths
1722

1823
# Default install paths per version and platform.
@@ -107,7 +112,7 @@ def deadline_farm(tmp_path):
107112
starve an in-process server thread, so the server needs its own GIL to
108113
keep serving the C4D subprocess. Its observability (call_counts,
109114
request_log, unmatched_requests) is read back over an admin endpoint via
110-
a RemoteBackend proxy,
115+
a RemoteDeadlineBackend proxy,
111116
2. writes a temp deadline config naming the mock's farm/queue,
112117
3. builds the env overlay (endpoint override + dummy creds + telemetry opt-out
113118
+ isolated HOME + config path + mock-mode flag) that the launch env applies
@@ -123,7 +128,7 @@ def deadline_farm(tmp_path):
123128
# 200-600ms. Override via env var to experiment.
124129
delay = float(os.environ.get("MOCK_DEADLINE_RESPONSE_DELAY_S", "0.3"))
125130

126-
server = MockServerProcess(response_delay_s=delay).start()
131+
server = MockDeadlineServerProcess(MockDeadlineScenario(response_delay_s=delay)).start()
127132
# start() has populated these; assert so the types narrow from Optional.
128133
assert server.backend is not None and server.base_url is not None
129134
backend = server.backend
@@ -139,7 +144,7 @@ def deadline_farm(tmp_path):
139144
queue_id=backend.queue_id,
140145
job_history_dir=job_history_dir,
141146
)
142-
env_overlay = build_mock_env(
147+
env_overlay = build_mock_environment(
143148
dict(os.environ),
144149
deadline_endpoint_url=server.base_url,
145150
config_path=config_path,

test/integ_xa11y/mock_aws/__init__.py

Lines changed: 0 additions & 18 deletions
This file was deleted.

test/integ_xa11y/mock_aws/_run_server.py

Lines changed: 0 additions & 36 deletions
This file was deleted.

0 commit comments

Comments
 (0)