Repository navigation
ci(experimental): split the long GraalPy shards so no worker outgrows its heap (#2713) - #2720
Merged
Merged
Conversation
… its heap (#2713) On 57a6c36, job 125639681253 lost 8 GraalPy xdist workers, all in the runs of 10-14 minutes (tests/foundation, tests/protocols/internet, tests/protocols/[a-m]*, tests/protocols), 9.8-12 minutes in, at up to 2.65-2.78 GiB RSS; the runs of 2-7 minutes lost none. - SHARDS: split those four by the job's per-module JUnit seconds, so each resulting shard's estimate (JUnit seconds over 3 workers plus the measured ~50 s a run costs) is under 5 minutes. Globs precede their own directory, which takes the remainder. The run over the rest, about 6.7 minutes in that job with no worker lost, is left whole. - test_graalpy_shards_1963: SHARDS follows the step. - workflows.rst: the #1963 figures gain the 57a6c36 measurements and the time-based sizing; the step's comment says how shards are sized. Collecting the tier through util/pytest_shards.py on CPython gives the same 3026 node ids, none twice, for the old and new SHARDS. tests/project passes, each module in its own process.
JarryShaw
added
ci
Owner
Author
|
GOOD TO GO at
|
JarryShaw
removed
the
review: good-to-go
JarryShaw
added a commit
that referenced
this pull request
Nov 3, 2026
… helpers, MH options 6 and 27, the cut SCTP and option-area schemas and dict-built secrets into 1.6.0 (#2449) - Breaking: a make() enumeration field takes a member or an int, its X_default/X_namespace/X_reversed keywords removed with no alias, and names, wide codes and wide option codes are refused (#2692); the schema helpers move to pcapkit.corekit.schema and pcapkit.corekit.fields.byteorder is removed (#2704); MobileNetworkPrefixOption gains prefix_length and keeps an IPv6Address prefix, and TimestampOption's timestamp is None past 9999 (#2709). Folded into the MH data-model entry; old values measured against v1.5.1. - Fixed: the SCTP schema cut inside its padding and a header schema cut inside an option area pack back as captured (#2682), into the SCTP entry; a Decryption Secrets Block rebuilds from its dict as captured (#2707), into the DSB entry. - Folded into unreleased entries: FRAD's Q.922 address refusal and quiet cut rebuild (#2719); a from_data rebuild's checksum status (#2712); a failed ListField fill's ledger (#2706); pad_length's path (#2704); TLS's enumeration fields (#2692); the make() and schema-helper rulings, into the conventions entry (#2692, #2704); the GraalPy shards, into the CI-legs entry (#2720). - 1.5.0's corekit.fields.byteorder role points at corekit.schema.byteorder, where #2704 moved it, so the nitpicky build does not warn on it; 1.5.0's text unchanged. - No entry for the test-only 72b2bd8 (#2715), or the docs-and-test-only 0c07940 (#2718). - CHANGELOG.md regenerated; util/changelog_md.py --check passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
make pylint,make mypy,make isort)make testpasses, and a test case covers the changeWhat is the purpose of your pull request?
ci— workflows or build toolingDescription of your pull request and other information
On
main, the GraalPy experimental leg loses xdist workers: 8 in job 125639681253, and some in every complete run since16ccc4373. Each GraalPy process runs with--vm.Xmx2g --vm.XX:+ExitOnOutOfMemoryError. The workers died 10–12 minutes into the four long shards, at 2.7–2.9 GiB RSS. Shorter shards never lost one.Run unit testsstep,SHARDS: the four long shards are split. Each was sized from the job's JUnit seconds (seconds ÷ 3 workers + 50 s) to run in under 5 minutes. The run over the rest is left whole; it took about 6.7 minutes and lost no worker. The leg comes to about 4750 s against its 6600 s timeout.util/pytest_shards.pygives the same node-id set for the old and newSHARDS, with none collected twice.test_graalpy_shards_1963pins the newSHARDS, and the test: sort the remaining GraalPy-only failures into defects and gates #1963 note inworkflows.rstrecords the measurements. This PR's own GraalPy leg is the timing check.Closes #2713