Repository navigation
Commit 0d6dcc8
perf(tests): Use pytest subtests for parametrized tests with heavy setup (#127939)
Some parametrized tests pay for an expensive fixture setup and teardown
on every case, while the case itself runs in milliseconds. This moves
them to pytest 9's built-in `subtests` fixture, so one setup covers all
the cases. Each case still reports on its own through
`subtests.test(msg=..., **kwargs)`, for example `SUBFAILED[count()]
(query_builder='ProfilesQueryBuilder')` or `SUBFAILED[d | b a c]
...::test_deep[single-nochunk]`.
The asserted cases are the same as before. Where cases share state, each
subtest resets it first.
- `tests/sentry/snuba/test_profiles.py`: the `params` fixture created an
org, a team, two projects, and a user for every case, only to build
queries. `test_field_resolution`, `test_aggregate_resolution`,
`test_invalid_field_resolution`, and `test_where_resolution` now loop
over their cases, and both query builders, in subtests. The one xfail
case (`!timestamp:`) is now its own `@pytest.mark.xfail` test, because
marks can't be applied to a single subtest.
- `tests/sentry/spans/test_buffer.py`: `test_basic`, `test_deep`,
`test_deep2`, and `test_parent_in_other_project` run each span
permutation as a subtest. They stay parametrized over the 4 `buffer`
variants (cluster/single × chunking). Each subtest deletes the
`span-buf:*` keys first, the same cleanup the fixture does, so a failed
permutation can't leak state into the next one.
- `tests/sentry/web/test_client_config.py`:
`test_client_config_in_silo_modes` uses `transaction=True`, so every
case paid for a full table flush. The 6 request factories now run as
subtests inside one test per silo mode. Each subtest clears the cache,
and each factory creates its own user and org.
Local timings (pytest-reported, `--reuse-db`, no xdist; the
redis-cluster `buffer` variants can't run in this environment, so
`test_buffer.py` was run with `-k "not cluster"`):
| File | Before | After | Collected items |
| --- | --- | --- | --- |
| `test_profiles.py` | 69.8s | 9.5s | 225 → 59 |
| `test_buffer.py` (non-cluster) | 22.0s | 7.7s | 1287 → 151 |
| ↳ converted tests only | 19.5s | 6.4s | |
| `test_client_config.py` | 39.7s | 22.1s | 32 → 22 |
In the CI backend data, the repeated setup and teardown for these groups
came to roughly 57s for profiles, 55s+ for `test_deep`/`test_deep2`, and
~90s for client config.
Not converted: the small `params` users in `test_profiles.py`
(`test_where_resolution_project_slug`,
`test_order_by_resolution_project_slug`; 6 cases total), the all-skipped
`has`/`!has` tests, and
`test_parent_in_other_project_and_nested_is_segment_span` (3 shallow
permutations). There's little setup to save in those.
CI sharding counts parametrize cases statically, so these tests now
count as fewer items. That's expected.
<!-- junior-request-attribution:start -->
via @joshuarli.
<!-- junior-request-attribution:end -->
<!-- junior-session-footer:start -->
<!-- junior-conversation-id:agent%3A0965bf28c1645fb49bb5b898386b0761 -->
--
[View Junior
Session](https://junior-prod.sentry.dev/conversations/agent%3A0965bf28c1645fb49bb5b898386b0761)
[[Sentry]](https://sentry.sentry.io/explore/conversations/agent%3A0965bf28c1645fb49bb5b898386b0761/?project=4510944073809921)
<!-- junior-session-footer:end -->
---------
Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Joshua Li <14209781+joshuarli@users.noreply.github.com>1 parent ddbc338 commit 0d6dcc8
3 files changed
Lines changed: 562 additions & 745 deletions
0 commit comments