You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 86208da
Browse filesBrowse the repository at this point in the historyBrowse files
test: use upstream's per-test timeout, and fix what the full matrix showed
A test times out after 30 seconds, as upstream, and the fixtures count, as
upstream counts them (pyproject.toml). A test marked slow gets three times
that, as test.slow() gives upstream: the ports of the tests that call it
(IF-009, IF-011, FC-001, CC-040, BT-076, BT-077, BT-091, all of
screencast.spec.ts and signals.spec.ts), and the Python-only tests that wait
longer by design (the pause tests, the subprocess and multiprocessing tests).
The wheel installation test gets its own bound. CI and test.sh drop their
--timeout flags.
The timeout method is thread, which ends the process of a test that hangs;
xdist replaces the worker. The signal method can not be used: asyncio's
Handle._run() catches and logs every BaseException but SystemExit and
KeyboardInterrupt, so a timeout that fires while the event loop runs a
callback is lost and the test hangs on (the jobs that ran into the 90-minute
limit). tests/conftest.py cancels the timer before the report of a failure is
built, where it fired inside pytest when a test failed just before its
deadline, and gives the teardown a timer of its own.
Races and failures that the full suites on every bot showed:
- find_free_port() reserves a port with a dual-stack bind, and a listener that
gets 127.0.0.1 but loses ::1 to another process takes a new port instead of
going on with IPv4 only. The IPv6-only server of the socks tests could take a
port that the test server of another worker held on 127.0.0.1, and the
browser, which resolves localhost to ::1 first, reached the wrong server.
- The child of the signal tests closes the browser on SIGHUP and ends when it
has closed, as the Node.js child does. The Python client has no SIGHUP
handler, so the signal ended the child at once, and the driver, left without
a client, killed the browser and could crash on its closed pipe before it
logged the exit; SI-006 failed one time in two on WebKit.
- The child of the signal tests gets a short temporary directory: under
tmp_path, the socket of Chrome's ProcessSingleton in the user data dir was
longer than a Unix socket path allows, and the stable channels crashed at
launch.
- A zone whose coroutine is closed from another context (GeneratorExit) can
not reset its ContextVar token; the reset raised in every job.
- CI runs Linux on ubuntu-22.04, where headless Firefox has WebGL (CA-008,
CA-009); the 24.04 image has none. The async step runs after a failed sync
step too, so that each job reports both suites.
The ports are not changed to pass: AGENTS.md says so now, and tests/transform.md
has the rules of the port of the tests. transform.md, the rules of the port of
the client, moves to playwright/_impl.
Copy file name to clipboardExpand all lines: .claude/skills/playwright-roll/SKILL.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -154,7 +154,7 @@ A few rules of thumb that catch most "actually a PORT" cases:
154
154
155
155
#### PORT
156
156
157
-
Implement the change in `playwright/_impl/<module>.py`, the line-by-line port of `$PW_SRC_DIR/packages/playwright-core/src/client/<module>.ts` (snake_case: `browserContext.ts` is `browser_context.py`). The client change that implements the new API is usually in the same upstream commit as the docs change, or close to it: port it line by line with the rules of [`transform.md`](../../../transform.md), and walk the whole client range as section 14 of that file says (`git -C "$PW_SRC_DIR" log <prev>..<rev> -- packages/playwright-core/src/client packages/isomorphic packages/protocol`). A few reminders:
157
+
Implement the change in `playwright/_impl/<module>.py`, the line-by-line port of `$PW_SRC_DIR/packages/playwright-core/src/client/<module>.ts` (snake_case: `browserContext.ts` is `browser_context.py`). The client change that implements the new API is usually in the same upstream commit as the docs change, or close to it: port it line by line with the rules of [`playwright/_impl/transform.md`](../../../playwright/_impl/transform.md), and walk the whole client range as section 14 of that file says (`git -C "$PW_SRC_DIR" log <prev>..<rev> -- packages/playwright-core/src/client packages/isomorphic packages/protocol`). A few reminders:
Copy file name to clipboardExpand all lines: AGENTS.md
+3-2Lines changed: 3 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ Python bindings for [Playwright](https://playwright.dev). The Python client talk
8
8
9
9
## Layout
10
10
11
-
- `playwright/_impl/` — the client, a line-by-line port of upstream's JS client (`packages/playwright-core/src/client/*.ts`): one module per client file in snake_case (`browserContext.ts` is `browser_context.py`), members in the JS order. `transform.md` (repo root) has the rules of the port; each difference from the JS code has a `# Python:` comment. `isomorphic/`, `protocol/` and `utils/` hold the ports of the parts of `packages/isomorphic`, `packages/protocol` and `packages/utils` that the client uses. The modules whose names start with an underscore are Python only, and the generated API imports most of them by name: `_api_classes.py` (the implementation of each API class), `_async_base.py` / `_sync_base.py` (the base classes of the API objects), `_impl_to_api_mapping.py`, `_errors.py` (the public error classes), `_form_data.py`, `_helper.py`. `_greenlets.py` runs the sync API (see `transform.md`, section 13), `_driver.py` finds the bundled driver, and pytest-playwright uses `_assertions.py` (the soft-assertion scope); the matchers are in `assertions.py`.
11
+
- `playwright/_impl/` — the client, a line-by-line port of upstream's JS client (`packages/playwright-core/src/client/*.ts`): one module per client file in snake_case (`browserContext.ts` is `browser_context.py`), members in the JS order. `playwright/_impl/transform.md` has the rules of the port; each difference from the JS code has a `# Python:` comment. `isomorphic/`, `protocol/` and `utils/` hold the ports of the parts of `packages/isomorphic`, `packages/protocol` and `packages/utils` that the client uses. The modules whose names start with an underscore are Python only, and the generated API imports most of them by name: `_api_classes.py` (the implementation of each API class), `_async_base.py` / `_sync_base.py` (the base classes of the API objects), `_impl_to_api_mapping.py`, `_errors.py` (the public error classes), `_form_data.py`, `_helper.py`. `_greenlets.py` runs the sync API (see `transform.md`, section 13), `_driver.py` finds the bundled driver, and pytest-playwright uses `_assertions.py` (the soft-assertion scope); the matchers are in `assertions.py`.
12
12
-`playwright/async_api/_generated.py`, `playwright/sync_api/_generated.py`, and the public TypedDicts in `playwright/_impl/_api_structures.py` — **auto-generated** from the upstream API docs. Never edit by hand; rerun `./scripts/update_api.sh` after a roll or after changing `scripts/codegen/`.
13
13
-`scripts/codegen/generate_api.js` — the generator. A Node.js script that loads the upstream doc parser (`utils/doclint/api_parser.js`) from `$PW_SRC_DIR`, filters the docs for Python, and writes the whole public API: signatures, types, docstrings, event overloads, the calls into `_impl/`, and the TypedDicts. It does not read `_impl/`.
14
14
-`scripts/codegen/python_rules.js` — the Python facts that the docs do not contain (positional options, struct `total=`, hidden members and options). Policy: the shipped Python API wins; each entry is a candidate for an upstream docs fix, and the generator fails on entries that no longer match anything. Every entry needs a one-line rationale comment above it.
@@ -56,7 +56,8 @@ The tests are a port of the upstream tests in `microsoft/playwright` (`tests/pag
56
56
57
57
- Port each upstream test twice, to `tests/async/` and to `tests/sync/`, in the file that matches the upstream file: `tests/page/page-check.spec.ts` becomes `tests/async/page/test_page_check.py` and `tests/sync/page/test_page_check.py`; `tests/library/chromium/oopif.spec.ts` becomes `tests/async/library/chromium/test_oopif.py`.
58
58
- Name the test `test_<ID>_<title>`: the identifier with `_`, then the `describe` titles and the title in snake_case. `test('EX-181 toHaveCount pass')` in `test.describe('toHaveCount')` becomes `test_EX_181_to_have_count_to_have_count_pass`. A test that a loop declares several times is one function with `@pytest.mark.parametrize`.
59
-
- Port faithfully: the same steps, assertions and skip conditions. A test that fails because of a bug in the Python client stays, marked `@pytest.mark.xfail(reason="python: ...", strict=True)`.
59
+
- Port faithfully: the same steps, assertions and skip conditions. The timeout of a test is 30 seconds, as upstream (`pyproject.toml`); where upstream calls `test.slow()`, mark the port `@pytest.mark.slow`, which triples it. A Python-only test that waits longer than 30 seconds by design gets the same mark. A test that fails because of a bug in the Python client stays, marked `@pytest.mark.xfail(reason="python: ...", strict=True)`.
60
+
-**Never change a port to make it pass.** No skip, condition, mark, wait or assertion that upstream does not have, for any reason: not a browser channel, not a CI image, not an operating system, not flakiness. A port that fails shows a bug in the client, the driver or the environment: fix that, or report it with the evidence. The one allowed mark is the `xfail(reason="python: ...", strict=True)` above, for a bug in the Python client that is not fixed yet. `tests/transform.md` has the rules of the port.
60
61
- A test that can not exist in Python (a JavaScript-only API, Node.js client internals, tools that run in the driver, `@playwright/test` features) goes in `tests/upstream_not_ported.txt`, with the reason.
61
62
-`python scripts/check_test_registry.py` checks all of this. `--missing <code or file glob>` prints the expected names of the tests still to port; `--report` prints the coverage per subsystem.
62
63
- Do not write a new test when an upstream test exists. A Python-only test goes in `tests/async/python/` or `tests/sync/python/`, in the file named after the upstream file of the feature.
0 commit comments