Skip to content

Commit 2b2314e

Browse files
authored
Merge pull request #355 from PhysShell/claude/p022-s8-memory-metric
fix(perf): the memory number carries the name of what it measures (S8)
2 parents 860fc8c + 157f53f commit 2b2314e

19 files changed

Lines changed: 4107 additions & 51 deletions
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
name: P-022 merge gate
2+
3+
# The merge order for the #263 preregistration, as a check rather than an
4+
# agreement. `mergegate.py` is run against the commit a merge would PRODUCE —
5+
# for a pull_request event that is `github.sha`, the merge ref, not the branch
6+
# head — because a PR number proves someone pressed a button and says nothing
7+
# about what the merged tree contains.
8+
#
9+
# Two jobs, and the gate needs the controls: a gate whose own controls are
10+
# failing must not be the thing that says a merge is safe. A failed controls job
11+
# leaves the gate job unreported, which a required check reads as unsatisfied.
12+
#
13+
# This workflow is one half of the wiring. The other half is a ruleset requiring
14+
# these two checks; without it the job reports and nothing is prevented. The
15+
# names below are the contexts that ruleset must name:
16+
#
17+
# P-022 merge gate controls
18+
# P-022 merge gate
19+
#
20+
# Least privilege as elsewhere in this repository: read-only, and every
21+
# third-party `uses:` pinned to a commit SHA with its version in a comment.
22+
23+
permissions:
24+
contents: read
25+
26+
on:
27+
pull_request:
28+
workflow_dispatch:
29+
30+
jobs:
31+
controls:
32+
name: P-022 merge gate controls
33+
runs-on: ubuntu-latest
34+
steps:
35+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
36+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
37+
with:
38+
python-version: "3.11"
39+
- name: the gate's own controls (fixtures ship the real tools and mutate them)
40+
run: python tests/test_step7_mergegate.py
41+
- name: the wiring's controls (applicability and co-change)
42+
run: python tests/test_step7_mergegate_ci.py
43+
44+
gate:
45+
name: P-022 merge gate
46+
needs: controls
47+
runs-on: ubuntu-latest
48+
steps:
49+
# Full history: the co-change rule compares the gate's own files against
50+
# the base commit, which a shallow clone would not carry.
51+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
52+
with:
53+
fetch-depth: 0
54+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
55+
with:
56+
python-version: "3.11"
57+
- name: the gate, against the commit this merge would produce
58+
env:
59+
# Read through the environment rather than interpolated into the
60+
# script body, so nothing from the event can be read as shell.
61+
MERGE_COMMIT: ${{ github.sha }}
62+
BASE_COMMIT: ${{ github.event.pull_request.base.sha }}
63+
run: |
64+
set -euo pipefail
65+
if [ -n "${BASE_COMMIT}" ]; then
66+
python scripts/step7/mergegate_ci.py --repo . \
67+
--commit "${MERGE_COMMIT}" --base "${BASE_COMMIT}"
68+
else
69+
python scripts/step7/mergegate_ci.py --repo . --commit "${MERGE_COMMIT}"
70+
fi

‎docs/evidence/calibration/p022-263a-design-constants.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"artifact": "p022-263a-calibration-design-constants",
3-
"bound_measurement_harness_digest": "562a7f7232dad2f4c79c6adfe0e1e7e25680b4b6f3444d54824bf0405e3c14b3",
3+
"bound_measurement_harness_digest": "104c384d01bf6060bdec1e7c916053ddb04b97fcbd0b39f8a4fc57b8f139672f",
44
"bound_policy_implementation_digest": "c3068ed7fa880a7083866ead25fe8bf65c87889d242d8af1f7eee01582cd5cbf",
55
"constants": {
66
"G": [

‎docs/evidence/calibration/p022-263a-policy-freeze.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"artifact": "p022-263a-calibration-policy-freeze",
3-
"measurement_harness_digest": "562a7f7232dad2f4c79c6adfe0e1e7e25680b4b6f3444d54824bf0405e3c14b3",
3+
"measurement_harness_digest": "104c384d01bf6060bdec1e7c916053ddb04b97fcbd0b39f8a4fc57b8f139672f",
44
"policy_implementation_digest": "c3068ed7fa880a7083866ead25fe8bf65c87889d242d8af1f7eee01582cd5cbf",
55
"policy_implementation_digest_framing": "sha256 over the source set ordered by the UTF-8 bytes of each repo-relative POSIX path. Each file contributes, with no header and no separator: its path byte length as an 8-byte big-endian unsigned integer, its path's exact UTF-8 bytes, its blob byte length as an 8-byte big-endian unsigned integer, and its exact git blob bytes.",
66
"policy_source_commit": "b4f657a0abdfdfaae199cbc7eee0c47acd8b0057",

‎docs/evidence/calibration/p022-263a-training-preregistration.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@
3030
"anchor_commit": "eedf6d3ed44ecf7bda69fa509dcd23b45960cf76",
3131
"artifact": "p022-263a-calibration-training-preregistration",
3232
"bindings": {
33-
"design_constants_blob_sha1": "d03c8455cbcaf71446f88aeea0957b8e0c56c880",
34-
"measurement_harness_digest": "562a7f7232dad2f4c79c6adfe0e1e7e25680b4b6f3444d54824bf0405e3c14b3",
33+
"design_constants_blob_sha1": "fa02f43bdf795e2f47d8ce781ec3a84f1dee64b5",
34+
"measurement_harness_digest": "104c384d01bf6060bdec1e7c916053ddb04b97fcbd0b39f8a4fc57b8f139672f",
3535
"policy_implementation_digest": "c3068ed7fa880a7083866ead25fe8bf65c87889d242d8af1f7eee01582cd5cbf",
3636
"training_scope_implementation_digest": "614bf9efe6ba2bb10e26a251c10d6c4a8fdaa99985d43ea03d76c6774447d25b",
3737
"training_scope_root": "scripts/training/"

‎docs/notes/p022-263a-instrument.md‎

Lines changed: 40 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -222,19 +222,49 @@ into. **Both halves of every pair are committed**, and run B records run A's
222222
path and sha256, so the verdict can be recomputed from evidence rather than
223223
trusted.
224224

225-
## 9. Peak RSS
225+
## 9. Peak memory
226226

227-
By a named mechanism, recorded, never eyeballed:
227+
By a named mechanism **and a named quantity**, recorded, never eyeballed. The
228+
quantity is not the same on both platforms, and the field says which one it is:
228229

229-
- POSIX: `os.wait4` — the kernel's per-child `ru_maxrss`, for exactly the
230-
process spawned. Chosen over `/usr/bin/time -v` as primary because GNU time is
231-
a package that may simply be absent, and "the tool was missing" is not a
232-
memory measurement. `/usr/bin/time -v` remains the documented fallback.
233-
- Windows: a Job Object, `PeakProcessMemoryUsed` via `QueryInformationJobObject`.
230+
| platform | mechanism | `memory_metric` | what it counts |
231+
|---|---|---|---|
232+
| POSIX | `os.wait4` — the kernel's `ru_maxrss` | `max_process_peak_resident` | peak **resident** set, over the child and the descendants it waited for |
233+
| POSIX fallback | `/usr/bin/time -v` | `max_process_peak_resident` | the same quantity |
234+
| Windows | Job Object `PeakProcessMemoryUsed` via `QueryInformationJobObject` | `max_process_peak_commit` | peak **committed** memory of any process ever associated with the job |
235+
236+
`os.wait4` was chosen over `/usr/bin/time -v` as primary because GNU time is a
237+
package that may simply be absent, and "the tool was missing" is not a memory
238+
measurement.
234239

235-
Where nothing is available the value is `null` **with a reason**, so a silent
236-
absence can never be read as a measured zero. Allocation counts are not yet
237-
captured — recorded as owed on population B's track rather than quietly dropped.
240+
**This used to be one field called `peak_rss_bytes`, and on Windows that was
241+
false.** A witness settles it rather than a manual page: a child that commits
242+
256 MiB and never touches a page is reported as 262.5 MiB through the job
243+
object, while on Linux nothing becomes resident — job and process memory limits
244+
are defined on committed virtual memory. The same witness refuted the other half
245+
of the suspicion: `wait4` on the immediate child does *not* stop at a wrapping
246+
shell. `shell -> heavy child` and `shell -> shell -> heavy child` both returned
247+
the descendant's 260 MiB against 13 MiB for a light control.
248+
249+
So the two strata answer different, honest questions. The numbers are comparable
250+
Rust-vs-Python **within** a platform and **may not be pooled or compared across
251+
platforms**. The resident analogue on Windows is `PeakWorkingSetSize`, and
252+
obtaining it for a tree of processes that come and go needs handle tracking or
253+
polling; this instrument does not sample, and will not perturb the elapsed-time
254+
interval so that two operating systems can pronounce the same noun.
255+
256+
A value produced under any metric kind outside that closed set raises rather
257+
than being recorded, and no alias lets a Windows commit number go on being read
258+
as an RSS number. Where nothing is available the value is `null` **with a
259+
reason**, so a silent absence can never be read as a measured zero. Allocation
260+
counts are not yet captured — recorded as owed on population B's track rather
261+
than quietly dropped.
262+
263+
The repair moved `measurement_harness_digest` from `562a7f7232da` to
264+
`104c384d01bf`. Steps 4, 5 and 6 were re-bound to the new digest in their own
265+
commit; the committed evidence of runs that already happened was **not** rewritten
266+
and still carries the old field name, because it records what the old instrument
267+
produced.
238268

239269
On POSIX the same `wait4` call also carries the child's CPU split, fault counts
240270
and context-switch counts. Those are now kept rather than discarded — see *The

‎pyproject.toml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,17 @@ target-version = "py311"
1515
# UP pyupgrade · C4 comprehensions · RUF ruff-native
1616
select = ["E", "W", "F", "I", "B", "UP", "C4", "RUF"]
1717

18+
[tool.ruff.lint.per-file-ignores]
19+
# scripts/perf_baseline.py is an INSTRUMENT SOURCE: its bytes are hashed into
20+
# the measurement harness digest that the frozen #263 preregistration names
21+
# (104c384d01bf), and steps 4, 5 and 6 are bound to that same value. Rewrapping
22+
# a line there is not a cosmetic edit — it moves the digest and makes the frozen
23+
# contract point at an identity the tree no longer contains, which is exactly
24+
# what scripts/step7/mergegate.py exists to refuse. So the one over-long line
25+
# stays and the linter is told why, rather than the instrument being reformatted
26+
# to please it. Lift this only together with a re-binding of steps 4/5/6.
27+
"scripts/perf_baseline.py" = ["E501"]
28+
1829
[tool.mypy]
1930
# The package is held to --strict. Tests/fuzzers are not (they lean on dynamic
2031
# construction and an AST oracle); ruff still covers them. The evidence tooling

0 commit comments

Comments
 (0)