Skip to content

Commit 0d6dcc8

Browse files
perf(tests): Use pytest subtests for parametrized tests with heavy setup (#127939)
Some parametrized tests pay for an expensive fixture setup and teardown on every case, while the case itself runs in milliseconds. This moves them to pytest 9's built-in `subtests` fixture, so one setup covers all the cases. Each case still reports on its own through `subtests.test(msg=..., **kwargs)`, for example `SUBFAILED[count()] (query_builder='ProfilesQueryBuilder')` or `SUBFAILED[d | b a c] ...::test_deep[single-nochunk]`. The asserted cases are the same as before. Where cases share state, each subtest resets it first. - `tests/sentry/snuba/test_profiles.py`: the `params` fixture created an org, a team, two projects, and a user for every case, only to build queries. `test_field_resolution`, `test_aggregate_resolution`, `test_invalid_field_resolution`, and `test_where_resolution` now loop over their cases, and both query builders, in subtests. The one xfail case (`!timestamp:`) is now its own `@pytest.mark.xfail` test, because marks can't be applied to a single subtest. - `tests/sentry/spans/test_buffer.py`: `test_basic`, `test_deep`, `test_deep2`, and `test_parent_in_other_project` run each span permutation as a subtest. They stay parametrized over the 4 `buffer` variants (cluster/single × chunking). Each subtest deletes the `span-buf:*` keys first, the same cleanup the fixture does, so a failed permutation can't leak state into the next one. - `tests/sentry/web/test_client_config.py`: `test_client_config_in_silo_modes` uses `transaction=True`, so every case paid for a full table flush. The 6 request factories now run as subtests inside one test per silo mode. Each subtest clears the cache, and each factory creates its own user and org. Local timings (pytest-reported, `--reuse-db`, no xdist; the redis-cluster `buffer` variants can't run in this environment, so `test_buffer.py` was run with `-k "not cluster"`): | File | Before | After | Collected items | | --- | --- | --- | --- | | `test_profiles.py` | 69.8s | 9.5s | 225 → 59 | | `test_buffer.py` (non-cluster) | 22.0s | 7.7s | 1287 → 151 | | ↳ converted tests only | 19.5s | 6.4s | | | `test_client_config.py` | 39.7s | 22.1s | 32 → 22 | In the CI backend data, the repeated setup and teardown for these groups came to roughly 57s for profiles, 55s+ for `test_deep`/`test_deep2`, and ~90s for client config. Not converted: the small `params` users in `test_profiles.py` (`test_where_resolution_project_slug`, `test_order_by_resolution_project_slug`; 6 cases total), the all-skipped `has`/`!has` tests, and `test_parent_in_other_project_and_nested_is_segment_span` (3 shallow permutations). There's little setup to save in those. CI sharding counts parametrize cases statically, so these tests now count as fewer items. That's expected. <!-- junior-request-attribution:start --> via @joshuarli. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:agent%3A0965bf28c1645fb49bb5b898386b0761 --> -- [View Junior Session](https://junior-prod.sentry.dev/conversations/agent%3A0965bf28c1645fb49bb5b898386b0761) [[Sentry]](https://sentry.sentry.io/explore/conversations/agent%3A0965bf28c1645fb49bb5b898386b0761/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Joshua Li <14209781+joshuarli@users.noreply.github.com>
1 parent ddbc338 commit 0d6dcc8

3 files changed

Lines changed: 562 additions & 745 deletions

File tree

0 commit comments

Comments
 (0)