Skip to content

Commit 00a07d2

Browse files
sourcehawkclaude
andauthored
fix(skills/bounding-heavy-commands): show commands that wait for memory in the check (#91)
* fix(skills/bounding-heavy-commands): give back slots while a command waits for memory The wrapper took its slots, and then it waited until the free memory held its budget. While it waited, it kept the slots but had no scope. So `--check` showed no line for it, and a smaller command that fit in the free memory waited behind slots that nothing used. Now the wrapper gives back its slots while the free memory does not hold its budget. When the memory looks free again, it takes slots through the line and reads the memory again under the memory lock before it starts. So two waiters that see the same free memory cannot both start on it. The wait has the same no-memory exit 75 when no other bounded command runs. While it waits for memory, the wrapper holds a wait file locked in the lock directory. The file names the budget, the slots, the start time, the directory, and the command. It holds no process ID. `--check` prints one `check waiting ... for=memory` line for each wait file, also without the hard cap, and removes a wait file that no process holds. Fixes #80. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(skills/bounding-heavy-commands): explain the check line of a command that waits for memory The wrapper now gives back its slots while it waits for free memory, and `--check` prints a `check waiting ... for=memory` line for each such command. The skill did not say how to read that line. The skill now gives the line format. It says that a waiting command holds no slot and has no place in the line, and that the line means the programs outside the queue hold the memory, not the slots. The report after a long wait, and the rule that a dispatcher gives its subagents, now name the waiting lines too. The lock files now carry no claim that they are empty, because a wait file names its command. In a baseline, agents that read a check with two waiting lines took them as an order in the line ("head of the line", "behind us"). One blamed the only running bounded command for the shortage. With the new text, each agent read the lines correctly and told its dispatcher that programs outside the queue hold the memory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(skills/bounding-heavy-commands): keep a live wait file that takes a stale name A waiter puts its wait file at the same name each time it gives back its slots. A check can lock a stale file at that name, or create an empty one there in the moment after the waiter removed its old file. The waiter's rename can then put its new, held file at the name before the check removes the name, so the check deleted the live file and the waiter dropped out of `--check`. The check now removes the name only when the file at the name is still the file that it locked. Otherwise it skips the name in this pass. Run files need no such change: each unit makes its run file once and never makes it again under the same name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(skills/bounding-heavy-commands): keep the wait age of a command across its waits A waiter wrote a new start time to its wait file each time it gave back its slots. When the memory went away again after the slots came back, the `age` on its `check waiting` line started again at 0, and a command that waited for a long time looked new. The wrapper now keeps the time of its first wait for memory and writes it to each new wait file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(skills/bounding-heavy-commands): say what a deleted wait file does The rule about the lock files said that a deleted lock file makes two commands hold one slot. That is true of a slot file only. A deleted wait file hides a command that waits for memory from `--check`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(skills/bounding-heavy-commands): wait for the wait file in the memory-waiter tests The test helper waited for the "; waiting" line of the large command. The wrapper prints that line before it gives back its slots and writes its wait file, so a check that ran at once could miss the `waiting` line. The helper now also waits until the wait file exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(skills/bounding-heavy-commands): correct two promises of the memory wait's docstring The docstring of `wait_for_memory` said that the call returns at once when the free memory cannot be read. Since the wait gives back its slots, the call first takes them back with `hold`, which can wait in the line. It also said that NoMemory leaves the slots as they are, which a caller can read as "still held". NoMemory can come after the call gave the slots back. The docstring now says that the call returns without a wait for memory when the free memory cannot be read, and that NoMemory can come after the call gave the slots back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(skills/bounding-heavy-commands): keep the slots of a command that waits for memory Giving back the slots while a command waits for memory breaks the fairness of the line. The large waiter loses its turn, and smaller commands that fit can take the memory each time it comes free, so the large one can wait without end. The case that blocked the queue with no bounded command running already ends with exit 75. The wait that is left lasts until a running bounded command frees its memory, and holding the slots there keeps the line fair. A command that waits for memory now keeps its slots and its turn, as before this branch. `wait_for_memory` has the contract and tests that it had on main, plus one `on_wait` callback that runs before each sleep. The wrapper uses it to write its wait file once, and removes the file when the wait ends. So `--check` still prints a `waiting` line, and its `slots` are the slots that the waiter holds. Because the wait file is now written once for each wait, the start time needs no copy and the reader needs no check that the name still holds the file that it locked. The process test now checks that `free-slots` counts the slots of the waiter beside a holder, and the test that a smaller command starts before the waiter is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(skills/bounding-heavy-commands): a waiting command keeps its slots The skill said that a command that waits for memory gives its slots back, leaves the line, and lets smaller commands start before it. The wrapper no longer does that. The skill now says that a `waiting` line is a command that had its turn and keeps its slots while it waits, so it does not lose its turn to a command behind it. It waits for the memory that a running bounded command of a `held` line frees when it ends, because the programs outside the queue hold most of the memory. It starts when that memory is free, or exits 75 when no bounded command runs any more. The report after a long wait and the rule for subagents say that the machine is short of memory, not of slots. With this text, isolated agents that read a check with one `held` line and one `waiting` line each said that the waiting command holds its slots and keeps its turn, and that it starts when the held command ends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(skills/bounding-heavy-commands): a waiter waits for a running command, which has a held line only under the cap The skill said that a waiting command waits for the memory of a running command of a `held` line, and that the waiting commands start when the `held` commands end. Without the hard cap there are no `held` lines, but `--check` still prints the `waiting` lines. The skill now says that the waiter waits for a running bounded command, and that this command has a `held` line where the hard cap is available. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent d0d3f88 commit 00a07d2

3 files changed

Lines changed: 242 additions & 7 deletions

File tree

‎skills/bounding-heavy-commands/SKILL.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -61,10 +61,11 @@ A check prints its lines in a fixed form too:
6161
```
6262
bounded-run: check slots=<n> slot=<n>M free-slots=<n> free=<n>M unused=<n>M cpus=<n> cpus-used=<n> cpus-headroom=<n> cpus-free=<n> line=<free|busy> fits=<n>M
6363
bounded-run: check held unit=<name> budget=<n>M used=<n>M cpus=<n> age=<n>s dir=<directory> command=<command>
64+
bounded-run: check waiting unit=<name> for=memory budget=<n>M slots=<n> age=<n>s dir=<directory> command=<command>
6465
bounded-run: check budget=<n>M slots=<n> headroom=<n>M starts=<yes|no>
6566
```
6667

67-
The first line gives the slots of the queue and their size, the slots that are free now, the free memory, the unused budgets of the running commands, the processors of the machine, the processors that the running bounded commands can use together (`cpus-used`), the processors that the check keeps for the programs outside the queue (`cpus-headroom`, a tenth of `cpus`, rounded up), and the processors that are left for a new bounded command (`cpus-free`: `cpus` less `cpus-headroom` and `cpus-used`, and at least 0). `line=busy` means that a command waits in the line. `fits` is the largest budget whose memory, with its own headroom, is free now. It does not count the slots or the line; `starts` counts all three. Each `held` line is one bounded command that runs now: its budget, the memory it uses, the processors it can use, how long it has run, its working directory, and its command. The held lines, `cpus-used`, and `cpus-free` need the hard cap, so without it there are no held lines and `cpus-used` and `cpus-free` are `-`. Each `budget` line is one `--memory`, in the order given: the slots that it needs, the headroom that the wait keeps free beside that budget, and `starts=yes` when a command of that budget would start now. A `-` means that the check cannot read or count the value at that moment.
68+
The first line gives the slots of the queue and their size, the slots that are free now, the free memory, the unused budgets of the running commands, the processors of the machine, the processors that the running bounded commands can use together (`cpus-used`), the processors that the check keeps for the programs outside the queue (`cpus-headroom`, a tenth of `cpus`, rounded up), and the processors that are left for a new bounded command (`cpus-free`: `cpus` less `cpus-headroom` and `cpus-used`, and at least 0). `line=busy` means that a command waits in the line. `fits` is the largest budget whose memory, with its own headroom, is free now. It does not count the slots or the line; `starts` counts all three. Each `held` line is one bounded command that runs now: its budget, the memory it uses, the processors it can use, how long it has run, its working directory, and its command. The held lines, `cpus-used`, and `cpus-free` need the hard cap, so without it there are no held lines and `cpus-used` and `cpus-free` are `-`. Each `waiting` line is one bounded command that had its turn in the queue and now waits for free memory: its budget, the slots that it holds, how long it has waited, its working directory, and its command. It keeps its slots while it waits, so it does not lose its turn to a command behind it in the line. A `waiting` line means that the machine is short of memory, not of slots: the programs outside the queue hold most of the memory, and the command waits for the memory that a running bounded command frees when it ends. Where the hard cap is available, that command has a `held` line. It starts when that memory is free, or exits 75 when no bounded command runs any more. Each `budget` line is one `--memory`, in the order given: the slots that it needs, the headroom that the wait keeps free beside that budget, and `starts=yes` when a command of that budget would start now. A `-` means that the check cannot read or count the value at that moment.
6869

6970
The exit code of the wrapper is the command's exit code. Exit code 125, with a `bounded-run:` line that names a fault and with no line for the budget and the peak, means that the wrapper itself failed (see §When the wrapper cannot run).
7071

@@ -158,14 +159,14 @@ A command that the project adds later gets its entry when an agent first needs i
158159

159160
## Rules for the wait
160161

161-
A long wait means that the machine is full. It is never a fault to repair: the queue has no state that can go wrong between runs, and an agent that repairs it breaks it. Waiting commands take turns in a line. Only the command at the head of the line tries for slots, so the smaller commands behind it do not take the slots that it waits for. The line has no fixed order, and a head that sees no slot come free for 5 minutes lets the commands behind it try first. So a small command can wait behind a large one, also when a slot is free. Unless an instruction says otherwise (§When an instruction sends a command around the wrapper), the only correct moves are to keep the run in the queue and to tell the user that the machine is full and the command is queued. After a long wait, run the wrapper with `--check`, and in that message name each command of its `held` lines, with its directory and its age, so that the user can see what holds the machine. A subagent tells its dispatcher in its report.
162+
A long wait means that the machine is full. It is never a fault to repair: the queue has no state that can go wrong between runs, and an agent that repairs it breaks it. Waiting commands take turns in a line. Only the command at the head of the line tries for slots, so the smaller commands behind it do not take the slots that it waits for. The line has no fixed order, and a head that sees no slot come free for 5 minutes lets the commands behind it try first. So a small command can wait behind a large one, also when a slot is free. Unless an instruction says otherwise (§When an instruction sends a command around the wrapper), the only correct moves are to keep the run in the queue and to tell the user that the machine is full and the command is queued. After a long wait, run the wrapper with `--check`, and in that message name each command of its `held` and `waiting` lines, with its directory and its age, so that the user can see what holds the machine. When the check has `waiting` lines, also say that the machine is short of memory, not of slots: the programs outside the queue hold most of the memory, and the waiting commands start when a running bounded command ends and frees its memory. A subagent tells its dispatcher in its report.
162163

163164
- **Run a bounded command in the background, with no short timeout around it.** It can wait in the queue for minutes before it starts. Where the harness requires a timeout, use the longest one that it permits. When that timeout ends a run, start the run again through the wrapper, and say in your report that the timeout ended the run and that the code did not fail. A timeout that you calculate from the run time of the command does not count the wait. A timeout that fires during the wait kills a command that never ran, and the result reads like a failure of the code.
164165
- **Plan the time as if the commands run one after the other.** Two bounded commands that start together are safe, because the queue puts them in sequence when the machine cannot hold both. So a parallel start does not always save time, and a promise to the user that depends on it is a guess.
165166
- **Do not lower a budget to start sooner.** A smaller budget does not make the command smaller. It moves the failure inside the command: on Linux as a kill at the hard cap, elsewhere as the swap that the queue exists to prevent. A row at a smaller `--cpus` is not a lower budget: it is a smaller command, measured, and it is the way to start sooner (§The record).
166167
- **Do not run the plain command because the queue is slow.** A slow queue means the machine is full. The plain command is how it freezes. This includes a narrower form of the command: one test file outside the queue is a command that the queue cannot count.
167168
- **Do not set an environment variable whose name starts with `BOUNDED_RUN_`.** The usage text of the wrapper lists variables that change how the queue counts memory, where it keeps its locks, and whether the hard cap applies. They are for a person who tunes the whole machine in a shell profile, and for the tests of the wrapper. A call that sets one does not share the queue with the other sessions, or it runs with no cap. A note in your project memory that recommends one is wrong: delete the note, and tell the user that you did.
168-
- **Do not touch the lock files of the wrapper.** The wrapper keeps them in a directory of its own. Do not look for that directory. Do not delete, move, or edit a file in it, and do not look for the process that holds one of these lock files. The lock files are empty and permanent. They hold no process ID. The kernel holds each lock and releases it at the moment its holder stops, also when the holder crashes, so a stale lock cannot exist. A deleted lock file makes two commands hold one slot. The `held` lines of `--check` are the way to name what holds the queue: do not look further, and never stop a held command, because it belongs to another session or to the user.
169+
- **Do not touch the lock files of the wrapper.** The wrapper keeps them in a directory of its own. Do not look for that directory. Do not delete, move, or edit a file in it, and do not look for the process that holds one of these lock files. The lock files hold no process ID. The kernel holds each lock and releases it at the moment its holder stops, also when the holder crashes, so a stale lock cannot exist. A deleted slot file makes two commands hold one slot, and a deleted wait file hides a command that waits for memory from `--check`. The `held` and `waiting` lines of `--check` are the way to name what holds the queue and what waits for memory: do not look further, and never stop a held or waiting command, because it belongs to another session or to the user.
169170
- **Do not sleep and retry around a collision, not even one time.** A port in use or a tool's lock means two commands wanted one resource. Add the resource to the record as an `--exclusive` name and run through the wrapper. A subagent adds the option to its own command line and names the resource in its report, so that its dispatcher corrects the record. Do not stop the process that holds the resource: it belongs to a different session. When a run with the `--exclusive` name fails on the same collision, the holder did not go through the wrapper or belongs to a different repository: tell the user, or your dispatcher, what holds the resource, and wait for the answer.
170171

171172
### After exit code 75
@@ -192,7 +193,7 @@ A subagent does not see your project memory. It knows the bounded commands only
192193
On a machine on which the wrapper can run, every dispatch prompt for a subagent that will run a heavy command MUST include:
193194

194195
1. The bounded command line for each heavy command, copied from the record, with the `--cpus` and the budget of its default row. When its entry has more than one row, also the `--cpus` and the budget of each row, and its check line with one `--memory` for each row.
195-
2. This rule, in these words: "Run the bounded commands as written. You can make three changes: the `--cpus` and the `--memory` of a row that this prompt gives for the command, a narrower argument at the end of the line, such as one test file, and one more `--exclusive <name>` option after a collision on a port or a lock, which you name in your report. When this prompt gives a command more than one row, run its check line before each run. When `cpus-free` is `-`, use the largest row with `starts=yes`. Otherwise, use the largest row with `starts=yes` whose `--cpus` is not more than `cpus-free`, and when no such row fits, use the smallest row with `starts=yes`. When no row has `starts=yes`, use the default row. Do not run the plain form of a bounded command or of a part of it. For a heavy command that is not in this prompt, use the same wrapper with no `--memory` option. If the file of the wrapper does not exist, stop and report that. Set no environment variable whose name starts with `BOUNDED_RUN_`. A command can wait in the queue for minutes before it starts, so run it in the background and do not put a short timeout around it. A long wait means that the machine is full; it is not a fault to repair. Exit code 75 with a `bounded-run: no-memory` line means that the command did not start, and that the same budget gives 75 again: run the largest row of this prompt whose budget is not more than `fits`, or, when no row fits, report the line. Say in your report how long you waited. After a long wait, run the same wrapper with `--check`, and name in your report each command of its `held` lines, with its directory and its age."
196+
2. This rule, in these words: "Run the bounded commands as written. You can make three changes: the `--cpus` and the `--memory` of a row that this prompt gives for the command, a narrower argument at the end of the line, such as one test file, and one more `--exclusive <name>` option after a collision on a port or a lock, which you name in your report. When this prompt gives a command more than one row, run its check line before each run. When `cpus-free` is `-`, use the largest row with `starts=yes`. Otherwise, use the largest row with `starts=yes` whose `--cpus` is not more than `cpus-free`, and when no such row fits, use the smallest row with `starts=yes`. When no row has `starts=yes`, use the default row. Do not run the plain form of a bounded command or of a part of it. For a heavy command that is not in this prompt, use the same wrapper with no `--memory` option. If the file of the wrapper does not exist, stop and report that. Set no environment variable whose name starts with `BOUNDED_RUN_`. A command can wait in the queue for minutes before it starts, so run it in the background and do not put a short timeout around it. A long wait means that the machine is full; it is not a fault to repair. Exit code 75 with a `bounded-run: no-memory` line means that the command did not start, and that the same budget gives 75 again: run the largest row of this prompt whose budget is not more than `fits`, or, when no row fits, report the line. Say in your report how long you waited. After a long wait, run the same wrapper with `--check`, and name in your report each command of its `held` and `waiting` lines, with its directory and its age. A `waiting` line is a command that holds its slots and waits for memory, because the programs outside the queue hold most of it: say in your report that the machine is short of memory, not of slots."
196197
3. Each command that an instruction exempts from the wrapper, in its plain form, in a list of its own, with the instruction that exempts it. The rule of item 2 does not apply to that list. An instruction of the user that covers one run goes to the one subagent that does that run, and to no other.
197198

198199
Do not dispatch fewer subagents to protect the machine. The queue protects it. Subagents that wait in the queue cost time, not memory. On a machine on which the wrapper cannot run, the next section says how to dispatch.

0 commit comments

Comments
 (0)