forked from procoders/superpowers-v
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathcompound-v-emit-workflow.py
More file actions
9012 lines (8437 loc) · 480 KB
/
Copy pathcompound-v-emit-workflow.py
File metadata and controls
9012 lines (8437 loc) · 480 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
#!/usr/bin/env python3
"""Compound V — Engine C. Turn a `manifest.yaml` into a native Claude Code Workflow
script, and provide the deterministic entry points that script's agents call.
WHAT ENGINE C IS
----------------
Engine C is the primary and default way jobs execute in 3.0 (spec Feature D, ADR
0004). The emitted `.js` is a native Workflow script: one `pipeline()` per
dependency wave, three stages per job — Implement -> Gate -> Record.
The script itself has NO filesystem and NO shell access. Agents it spawns do. So
everything mechanical lives HERE, in Python, and the emitted JS is deliberately
thin: it schedules, it never decides. That is the same reason the scope gate and
the test-contract glob resolution live in Python — a second, weaker
implementation inside a model's shell would diverge from the authority, and a
divergence in an enforcement path silently *passes*.
SUBCOMMANDS
-----------
emit manifest.yaml -> the workflow script (+ per-job test contracts)
gate-receipt the Gate stage's ONE clamped command: run the git-derived scope
gate, compute the PINNED diff digest, run the test floor, and
emit a complete six-field `gate_receipt`.
record the Record stage's ONE clamped command: idempotently persist
results/<job-id>.json + state.json. EVIDENCE ONLY — it writes
nothing into the project checkout.
finalize-wave the serialized end of every wave: run the integration AUTHORITY
over that wave's jobs, then merge and COMMIT the permitted ones.
The only writer into the project checkout.
resume-prepare what /v:resume runs BEFORE relaunching a crashed run: clear the
crashed attempt's baseline pin for every job that has not
integrated, reset those jobs to pending, drop their stale
lane-map entries and archive their superseded receipts. A
relaunch branches a fresh worktree from the CURRENT HEAD, and a
pin from the crashed attempt charges the job with every commit
landed since (proven BLOCKED on 2026-09-03, finding 146).
register-lane the Implement agent's FIRST command: bind its real worktree to
its job id in lane-map.json (which is what makes
hooks/lane-guard.sh able to resolve an acting job at all), and
PIN this job's baseline commit before anything runs.
WHY THE GATE STAGE CANNOT THROW
-------------------------------
`pipeline()` drops an item to `null` and SKIPS ITS REMAINING STAGES when a stage
throws. A throwing Gate would therefore mean: no Record, no state written, no
result file — precisely on the jobs that went wrong. That is the v2.6.4
audit-trail loss reappearing structurally. The emitted Gate stage wraps
everything and returns a verdict for every outcome, including "the gate itself
failed". `null` is FAIL, never pass.
WHAT THIS SCRIPT IS NOT
-----------------------
It is NOT the authority. `scripts/compound-v-integration-gate.py` is: it verifies
or re-derives every receipt before any commit is integrated. A clamp limits what
an agent CAN DO, not what it RETURNS, and a schema proves shape, not execution.
The receipt is defence in depth and an early exit.
"""
from __future__ import annotations
import os
import sys
# Nobody writes bytecode. The scope gate forgives no path by extension (fourth
# review pass, 2026-09-02), so a `__pycache__` entry this process leaves beside a
# script is an out-of-lane write that BLOCKS the job it is plumbing. Set before
# ANY other import — `_import_integration_gate` and `_import_triage_outcomes`
# below both load repo scripts by path, and an import is exactly when a cache
# entry would be written.
sys.dont_write_bytecode = True
def _harden_sys_path():
"""Drop this script's own directory and the cwd from ``sys.path``.
CPython puts the script's directory at ``sys.path[0]``. This script LIVES in
``scripts/``, which is a directory a Compound V job may be given a write lane
over — so a job that writes ``scripts/yaml.py`` gets that file imported, in
this process, by the very `import yaml` the manifest loader runs. The
manifest is the document that declares every job's `write_allowed`, so a
shadowed loader can hand the pipeline a WIDENED lane map and every later
check agrees with it. Same for the cwd (``''``/``'.'``), which is on the path
for ``-c`` and ``-m`` invocations.
Run BEFORE the first non-trivial import, so nothing — stdlib or third-party —
can be resolved out of the tree this pipeline is gating. The sibling repo
scripts this file genuinely needs are loaded by explicit path
(`_load_module_from_path`), never by name, so removing these entries costs
nothing.
"""
# REALPATH, not abspath. CPython puts the RESOLVED script directory on the
# path while `__file__` keeps the symlinked spelling, so on macOS — where
# /var is a symlink to /private/var — the two disagree for every run under a
# temp directory and the entry survived. Caught by this file's own planted-
# `scripts/yaml.py` selftest, which imported the plant and reported the run
# compliant against a lane list of `**`.
def _resolve(path):
return os.path.normcase(os.path.realpath(os.path.abspath(path)))
doomed = {_resolve(os.path.dirname(os.path.abspath(__file__))),
_resolve(os.getcwd())}
kept = []
for entry in sys.path:
try:
resolved = _resolve(entry) if entry else _resolve(os.getcwd())
except Exception: # noqa: BLE001
kept.append(entry)
continue
if resolved in doomed:
continue
kept.append(entry)
sys.path[:] = kept
_harden_sys_path()
import argparse # noqa: E402
import hashlib # noqa: E402
import io # noqa: E402
import datetime # noqa: E402
import json # noqa: E402
import re # noqa: E402
import shutil # noqa: E402
import subprocess # noqa: E402
import tempfile # noqa: E402
import time # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# THERE IS NO DEFAULT REPOSITORY ROOT, and its absence is the point.
#
# Until 3.0.2 this module carried `REPO_DEFAULT = os.path.dirname(HERE)` — the
# repository containing the INSTALLED SCRIPT — and handed it to `record` as the
# `--repo-root` default. Since the emitted Record command passed no `--repo-root`
# at all, a job that edited `README.md` in /work/app had its patch applied into
# /plugins/superpowers-v. A wrong-repository write is the same class as this
# project's 2026-07-13 incident, so the root is now REQUIRED everywhere it
# decides a destination, and its absence FAILS CLOSED rather than picking one.
SCOPE_CHECK_DEFAULT = os.path.join(HERE, "compound-v-scope-check.py")
FASTPATH_DEFAULT = os.path.join(HERE, "compound-v-fastpath-run.py")
INTEGRATION_GATE_DEFAULT = os.path.join(HERE, "compound-v-integration-gate.py")
RESOLVE_MODEL_DEFAULT = os.path.join(HERE, "compound-v-resolve-model.py")
# --------------------------------------------------------------------------- #
# Determinism constraints of the Workflow runtime (verified against the
# installed Claude Code 2.1.238 binary and its own error strings):
#
# "Workflow scripts must be deterministic: Date.now()/Math.random()/new Date()
# are unavailable (breaks resume). Stamp results after the workflow returns,
# or pass timestamps via args."
#
# NOTE, and this is precisely why the check below exists rather than being left
# to the runtime: that static pre-check is applied ONLY to the inline `script`
# input (the guard reads `if (input.script && isNonDeterministic(body))`). A
# workflow launched by `scriptPath` — which is the form Engine C forces, so the
# committed artefact is what ran — SKIPS it and only discovers the problem when
# the global throws mid-run. The generator is the real backstop.
# --------------------------------------------------------------------------- #
FORBIDDEN_PATTERNS = [
(r"\bDate\.now\s*\(", "Date.now()"),
(r"\bMath\.random\s*\(", "Math.random()"),
(r"\bnew\s+Date\s*\(\s*\)", "bare new Date()"),
(r"(?<![\w.$])import\s*\(", "import()"),
]
# The Gate/Record agents are narrowed at spawn. `disallowedTools` is a DENY list
# of tool names, so this enumerates what to remove; Bash must survive, because a
# `bashCommandClamp` whose agent has no Bash "can bind nothing" and the runtime
# REFUSES THE SPAWN outright (verbatim from the binary). StructuredOutput must
# survive too, or schema mode is denied and the spawn is likewise refused.
#
# Honest limit: a denylist cannot cover a tool this build does not have yet. The
# confinement that actually holds is the clamp, which is an ALLOWLIST of command
# forms and is fail-closed ("no clamp rule matches this command" -> deny;
# "permission check crashed" -> deny).
NARROW_DISALLOWED = [
"Read", "Write", "Edit", "MultiEdit", "NotebookEdit", "NotebookRead",
"Glob", "Grep", "WebFetch", "WebSearch", "Task", "Agent", "TodoWrite",
"SlashCommand", "Skill", "Artifact", "ExitPlanMode",
]
# What the IMPLEMENT stage loses. It is not the transport narrowing above — an
# implementer must keep Read/Write/Edit, Glob/Grep and Bash to do the work. It loses
# the tools that either defeat lane attribution or widen the trust boundary:
#
# Task / Agent A nested spawn is not the job. `hooks/lane-guard.sh` resolves a
# write by `agent_id` FIRST (`resolve_job`, :355-372); a nested agent
# carries a different one, and the only fallback is cwd-under-a-
# REGISTERED-WORKTREE — which a `direct`-mode job does not have. So a
# nested agent's writes are logged "job unresolved" and ALLOWED. The
# git-derived gate still sees the bytes afterwards, but attributes
# them to a job that did not write them, which is precisely the
# attribution the whole enforcement chain rests on.
# SlashCommand An implementer running `/v:dispatch` re-enters the pipeline from
# inside one of its own jobs.
# WebFetch / Research is a PRE-FLIGHT phase in this plugin (Trigger 0, the
# WebSearch doc-validator). An implementer holding write access and pulling
# untrusted web content into its own context is the injection surface
# the charter exists for. A job that genuinely needs external material
# gets it the same way every other job does: through a pre-flight, or
# pinned into `read_allowed` and the prompt.
#
# This is a REMOVAL of capability, deliberately. It is stated here rather than
# discovered by whoever wonders why their implementer cannot search.
IMPLEMENT_DISALLOWED = ["Task", "Agent", "SlashCommand", "WebFetch", "WebSearch"]
# What an implementer's shell may run, beyond the three plumbing forms.
#
# Dogfood r2 (2026-09-02, wf_f0505df2-99c) was the first REAL code job Engine C
# dispatched, and it could not do the job: the clamp admitted register-lane and
# two recall reads and nothing else, so the implementer could not run a test, a
# selftest, shellcheck, or `git rm`. Job A wrote "NOTHING IS VERIFIED" in its
# summary and was merged anyway because the Gate ran the floor; job B claimed a
# deletion it had been denied. Every docs-only dogfood before it never needed a
# shell, which is why the clamp survived 3.0.6 to 3.3.7 looking sound.
#
# The narrowing that matters is the one this list still expresses by OMISSION:
# no network (curl, wget, ssh, scp), no privilege (sudo), no scheduler
# (launchctl, crontab, open, osascript), no package installs, and no git that
# commits, pushes, rewrites history, or touches worktrees/remotes — merge-back
# and the wave commit belong to the finalizer. Writes outside the lane are still
# refused by lane-guard where it can parse them and by the git-derived scope gate
# in every case; the clamp is defence in depth, never the authority.
IMPLEMENT_SHELL = [
"Bash(bash:*)", "Bash(sh:*)", "Bash(python3:*)", "Bash(/usr/bin/python3:*)",
"Bash(python:*)", "Bash(node:*)", "Bash(npm:*)", "Bash(npx:*)", "Bash(pytest:*)",
"Bash(make:*)", "Bash(go:*)", "Bash(cargo:*)", "Bash(shellcheck:*)",
"Bash(git status:*)", "Bash(git diff:*)", "Bash(git log:*)", "Bash(git show:*)",
"Bash(git ls-files:*)", "Bash(git grep:*)", "Bash(git rm:*)", "Bash(git mv:*)",
"Bash(git add:*)", "Bash(git rev-parse:*)",
"Bash(ls:*)", "Bash(cat:*)", "Bash(head:*)", "Bash(tail:*)", "Bash(grep:*)",
"Bash(find:*)", "Bash(wc:*)", "Bash(diff:*)", "Bash(sed:*)", "Bash(awk:*)",
"Bash(sort:*)", "Bash(uniq:*)", "Bash(cut:*)", "Bash(tr:*)", "Bash(xargs:*)",
"Bash(pwd:*)", "Bash(echo:*)", "Bash(printf:*)", "Bash(test:*)", "Bash(true:*)",
"Bash(mkdir:*)", "Bash(rm:*)", "Bash(mv:*)", "Bash(cp:*)", "Bash(touch:*)",
"Bash(chmod:*)", "Bash(cd:*)", "Bash(env:*)", "Bash(which:*)", "Bash(date:*)",
]
STAGE_PHASES = ["Implement", "Gate", "Record", "Finalize"]
# Reserve, in tokens, assumed per queued agent when guarding fan-out against the
# native `budget` ceiling. Deliberately a round, declared constant rather than a
# measured figure: we have never measured per-agent spend, and inventing a number
# here would be a fabricated metric. It only has to be large enough that we stop
# scheduling BEFORE `agent()` throws, because a throw skips Record.
BUDGET_RESERVE_PER_AGENT = 50000
# --------------------------------------------------------------------------- #
# small helpers
# --------------------------------------------------------------------------- #
def _run(cmd, cwd=None, env=None, text=True):
"""Run a command; never raise. Returns (rc, out, err)."""
full_env = dict(os.environ)
full_env["PYTHONDONTWRITEBYTECODE"] = "1"
if env:
full_env.update(env)
try:
proc = subprocess.Popen(
cmd, cwd=cwd, env=full_env,
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
)
out, err = proc.communicate()
if text:
return (proc.returncode,
out.decode("utf-8", "replace"),
err.decode("utf-8", "replace"))
return proc.returncode, out, err.decode("utf-8", "replace")
except Exception as exc: # noqa: BLE001 - a gate path must never raise
return 127, ("" if text else b""), str(exc)
def _git(root, args, env=None, text=True):
return _run(["git", "-C", root] + list(args), env=env, text=text)
def _load_yaml(path):
try:
import yaml
except ImportError:
raise SystemExit(
"PyYAML is required. On macOS use /usr/bin/python3, which ships it."
)
with open(path, "r", encoding="utf-8") as fh:
return yaml.safe_load(fh)
def _atomic_write(path, data):
directory = os.path.dirname(os.path.abspath(path))
if directory:
os.makedirs(directory, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=directory or ".", prefix=".cv-tmp-")
try:
with os.fdopen(fd, "w", encoding="utf-8") as fh:
fh.write(data)
os.replace(tmp, path)
tmp = None
finally:
if tmp is not None and os.path.exists(tmp):
os.unlink(tmp)
def _atomic_write_bytes(path, data):
"""`_atomic_write` for a binary artefact — the sealed patch is raw bytes.
A `--binary` diff is not text: it carries literal-byte hunks and paths in
whatever encoding the tree uses, and re-encoding it would change the very
bytes whose sha256 the receipt pins.
"""
directory = os.path.dirname(os.path.abspath(path))
if directory:
os.makedirs(directory, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=directory or ".", prefix=".cv-tmp-")
try:
with os.fdopen(fd, "wb") as fh:
fh.write(data)
os.replace(tmp, path)
tmp = None
finally:
if tmp is not None and os.path.exists(tmp):
os.unlink(tmp)
class _run_dir_lock(object):
"""Hold an exclusive lock across a whole READ-MODIFY-WRITE of a shared file.
`_atomic_write` makes one WRITE atomic. It does nothing whatsoever for the
READ that preceded it, and every shared file in a run dir — lane-map.json,
state.json — is updated read-modify-write by agents that run CONCURRENTLY
within a wave. Two implementers both read the pre-write map, both merge their
own entry into it, and the second write drops the first's.
The entry that goes missing is a LANE. hooks/lane-guard.sh resolves the
acting job from that map; with no entry it resolves nothing, FAILS OPEN, and
silently allows every write that job makes. That is the same hole as the map
having no producer at all — only intermittent, and only under concurrency, so
it presents as a flake rather than as a missing enforcement boundary.
POSIX `flock` is the mechanism. If it cannot be taken, this RAISES rather
than proceeding unlocked: an unserialized merge is exactly the failure being
prevented, and "we could not lock, so we did it anyway" is fail-open.
"""
def __init__(self, run_dir, name="run"):
self.path = os.path.join(run_dir, ".%s.lock" % name)
self.run_dir = run_dir
self._fh = None
def __enter__(self):
import fcntl
os.makedirs(self.run_dir, exist_ok=True)
self._fh = open(self.path, "a+")
fcntl.flock(self._fh.fileno(), fcntl.LOCK_EX)
return self
def __exit__(self, *exc):
if self._fh is not None:
try:
import fcntl
fcntl.flock(self._fh.fileno(), fcntl.LOCK_UN)
finally:
self._fh.close()
self._fh = None
return False
def _read_json(path, default=None):
try:
with open(path, "r", encoding="utf-8") as fh:
return json.load(fh)
except Exception: # noqa: BLE001
return default
def _js_json(obj):
"""JSON safe to paste into a JS source file.
U+2028/U+2029 are legal inside JSON strings but are line terminators to some
JS parsers; escape them rather than find out which parser we got.
"""
text = json.dumps(obj, indent=2, sort_keys=True, ensure_ascii=False)
return text.replace("\u2028", "\\u2028").replace("\u2029", "\\u2029")
def neutralize_in_data(json_text):
"""Escape a forbidden construct's `(` where it appears inside embedded DATA.
A manifest's own prose legitimately talks about these constructs — this
release's task-9 acceptance criterion literally reads "no `Date.now`,
`Math.random`, bare `new Date()` or `import()`" — and that text is carried
into the script as a JSON string inside an agent prompt. It is data, not
code, and it would never execute. But the check that guards this file cannot
tell the difference by looking, and neither can the runtime's own scanner, so
a manifest that documents the rule would be unable to run under it.
Escaping the `(` as `\\u0028` inside the JSON string leaves the DECODED value
byte-identical — the agent reads exactly the prompt the manifest wrote — while
the emitted source carries no literal forbidden construct. Applied ONLY to the
JSON data blobs, never to the template's executable body: real code in the
template must still be refused outright.
"""
def escape_paren(match):
text = match.group(0)
idx = text.rindex("(")
return text[:idx] + "\\u0028" + text[idx + 1:]
for pattern, _name in FORBIDDEN_PATTERNS:
json_text = re.sub(pattern, escape_paren, json_text)
return json_text
def forbidden_hits(script_text):
"""Every forbidden-construct hit in an emitted script. [] means clean."""
hits = []
for pattern, name in FORBIDDEN_PATTERNS:
for match in re.finditer(pattern, script_text):
line = script_text.count("\n", 0, match.start()) + 1
hits.append({"construct": name, "line": line})
return hits
# --------------------------------------------------------------------------- #
# the PINNED diff digest
#
# Recipe taken verbatim from the `diff_digest` property description in
# schemas/job_result.schema.json, which pins it so a producer and the
# verification layer cannot diverge:
#
# `git -C <gate-root> add -A` (which brings untracked files — the half a plain
# `git diff` would miss — into the index), then sha256 over the raw bytes of
# `git -C <gate-root> diff --cached --binary <baseline_commit>`, rendered as
# `sha256:<64-hex>`.
#
# We PREFER to import compound-v-integration-gate.compute_diff_digest, so the
# producer is literally the verifier's own function — the same "one matcher, not
# two" argument the lane guard makes about the glob engine. The local copy is the
# fallback and is behaviourally identical: `add -A` runs against a COPY of the
# index under GIT_INDEX_FILE, so the index CONTENT the diff reads is the same
# while producing a receipt does not mutate the tree being gated.
# --------------------------------------------------------------------------- #
def _load_module_from_path(name, target):
"""Load a repo script by path, FROM SOURCE — never from a cache beside it.
`sys.pycache_prefix` moves both the read and the write of the bytecode cache
to a private directory outside the tree, so a forged
`scripts/__pycache__/<mod>.<tag>.pyc` — an unchecked hash-based one, which
CPython never validates against its source — cannot be executed in this
process (fourth review pass, item 3, 2026-09-02). Returns None on any failure.
IF THE PRIVATE PREFIX CANNOT BE CREATED, NOTHING IS LOADED. The first version
caught the mkdtemp failure and carried on with the DEFAULT cache location —
which is the in-tree `__pycache__` the redirect exists to avoid, so the one
condition an attacker can arrange (a full or unwritable temp dir) turned the
protection off and executed the planted `.pyc` anyway. A protection with a
fallback to the unprotected path is not a protection; this refuses instead,
and the caller degrades to its own local implementation.
"""
prev_prefix = getattr(sys, "pycache_prefix", None)
tmp_pycache = None
try:
import importlib.util
try:
tmp_pycache = tempfile.mkdtemp(prefix="cv-pycache-")
sys.pycache_prefix = tmp_pycache
except Exception: # noqa: BLE001
return None
spec = importlib.util.spec_from_file_location(name, target)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
except Exception: # noqa: BLE001
return None
finally:
try:
sys.pycache_prefix = prev_prefix
except Exception: # noqa: BLE001
pass
if tmp_pycache:
shutil.rmtree(tmp_pycache, ignore_errors=True)
# --------------------------------------------------------------------------- #
# the MANIFEST DIGEST
#
# The manifest declares every job's `write_allowed`. It also lives in the run
# directory, which the pipeline exempts BY NAME from the scope gate so a job's own
# bookkeeping does not read as an out-of-lane write. Put those two facts together
# and a job could widen its own lane map mid-run: every later check would run,
# pass, and prove nothing, because it would be checking against the widened list.
#
# `emit` therefore hashes the manifest at generation time and bakes the digest
# into the workflow script. Gate, Record, Finalize and the integration authority
# all carry it forward and refuse a manifest that no longer hashes to it. The lane
# map that is ENFORCED is provably the lane map that was reviewed.
# --------------------------------------------------------------------------- #
def sha256_file(path):
"""`sha256:<64hex>` of a file's raw bytes, or None when it cannot be read."""
try:
h = hashlib.sha256()
with open(path, "rb") as fh:
for chunk in iter(lambda: fh.read(65536), b""):
h.update(chunk)
return "sha256:" + h.hexdigest()
except (IOError, OSError):
return None
def manifest_digest_fault(manifest_path, expected):
"""A refusal string when `manifest_path` does not hash to `expected`, else None.
An absent `expected` is the documented backward-compatible path — a run
emitted before 3.4.0, or a by-hand invocation. Every emitted command carries
the digest, so the pipeline itself never takes that path.
"""
if not expected:
return None
actual = sha256_file(manifest_path)
if actual == expected:
return None
return (
"manifest %s hashes to %s, not the %s this run was emitted against. The "
"manifest declares every job's write_allowed, so a lane map that changed "
"after emit is refused rather than enforced."
% (manifest_path, actual, expected)
)
# --------------------------------------------------------------------------- #
# the SEALED PATCH artifact
#
# The digest binds a receipt to a tree AT GATE TIME. Nothing used to bind the
# MERGE to that same tree: `merge_back` took a fresh `git diff` of the live
# worktree whenever the finalizer got round to it. Three real consequences, all
# reported by a cross-model review of 3.4.0:
#
# * a worktree reverted to its baseline after the gate merged as "nothing to
# do", was recorded as integrated, and was pruned — the work destroyed;
# * `.pytest_cache/` and friends, written by the test floor that runs AFTER the
# scope check, turned an honest pass into a `contradicted` refusal;
# * any post-gate write to an in-lane file rode into the commit unmeasured.
#
# So the gate SEALS what it approved: `jobs/<id>.patch`, and its sha256 in the
# gate's receipt document. The finalizer applies THAT FILE. Nothing else.
# --------------------------------------------------------------------------- #
def patch_artifact_path(run_dir, job_id):
return os.path.join(run_dir, "jobs", "%s.patch" % job_id)
def build_sealed_patch(root, baseline, paths):
"""(patch bytes, error) — `git diff --cached --binary <baseline> -- <paths>`.
The `git add` runs against a COPY of the index under GIT_INDEX_FILE, exactly
as the digest recipe does, so sealing a patch does not stage anything in the
tree it is sealing. Only the paths the gate APPROVED are added, so a file the
scope gate flagged — or a byproduct a later test writes — cannot be in the
artifact, and therefore cannot be merged.
"""
if not baseline:
return None, "no baseline to seal against"
rc, gitpath, err = _git(root, ["rev-parse", "--git-path", "index"])
if rc != 0:
return None, "cannot locate git index: %s" % (err.strip() or "rc=%d" % rc)
index_path = gitpath.strip()
if not os.path.isabs(index_path):
index_path = os.path.join(root, index_path)
tmpdir = tempfile.mkdtemp(prefix="cv-seal-idx-")
try:
tmp_index = os.path.join(tmpdir, "index")
if os.path.exists(index_path):
shutil.copyfile(index_path, tmp_index)
env = {"GIT_INDEX_FILE": tmp_index}
for path in paths:
if not path:
continue
rc, _o, err = _git(root, ["add", "-A", "--", path], env=env)
if rc != 0:
# A deletion the worker already staged leaves nothing for the
# pathspec to match; it is already in the copied index, so this is
# not a failure. Same reasoning as `_stage_paths`.
rc2, staged, _e = _git(
root, ["diff", "--cached", "--name-only", "--diff-filter=D",
"--", path], env=env)
if not (rc2 == 0 and path in
[l.strip() for l in (staged or "").splitlines()]):
return None, "git add failed for %r while sealing: %s" % (
path, err.strip())
args = ["diff", "--cached", "--binary", baseline, "--"]
args += [p for p in paths if p]
rc, blob, err = _git(root, args, env=env, text=False)
if rc != 0:
return None, "git diff --cached failed while sealing: %s" % err.strip()
return blob, None
finally:
shutil.rmtree(tmpdir, ignore_errors=True)
def sealed_post_image(repo_root, baseline, patch_bytes):
"""({path: blob-oid or None for a deletion}, error) — GIT'S answer, not ours.
The patch is applied to a THROWAWAY index seeded from `baseline` (`git apply
--cached` touches the index only, and GIT_INDEX_FILE keeps it off the real
one), and the resulting blob ids are read back out. That is the exact content
the artifact produces, derived by git from the artifact itself — so the
post-merge proof needs nothing recorded by any party the pipeline constrains.
"""
if not patch_bytes:
return {}, None
tmpdir = tempfile.mkdtemp(prefix="cv-postimg-")
try:
tmp_index = os.path.join(tmpdir, "index")
env = {"GIT_INDEX_FILE": tmp_index}
rc, _o, err = _git(repo_root, ["read-tree", baseline], env=env)
if rc != 0:
return None, "git read-tree %s failed: %s" % (baseline, err.strip())
try:
full_env = dict(os.environ)
full_env["GIT_INDEX_FILE"] = tmp_index
full_env["PYTHONDONTWRITEBYTECODE"] = "1"
proc = subprocess.Popen(
["git", "-C", repo_root, "apply", "--cached", "--binary", "-"],
stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, env=full_env,
)
_o, aerr = proc.communicate(patch_bytes)
if proc.returncode != 0:
return None, (
"the sealed patch does not apply to its own baseline: %s"
% aerr.decode("utf-8", "replace").strip()[:300])
except Exception as exc: # noqa: BLE001
return None, "git apply --cached raised: %s" % exc
rc, names, err = _git(
repo_root, ["diff", "--cached", "--name-only", baseline], env=env)
if rc != 0:
return None, "git diff --cached --name-only failed: %s" % err.strip()
image = {}
for path in [n.strip() for n in (names or "").splitlines() if n.strip()]:
rc2, staged, _e = _git(
repo_root, ["ls-files", "--stage", "--", path], env=env)
oid = None
if rc2 == 0 and staged.strip():
parts = staged.splitlines()[0].split("\t", 1)[0].split()
if len(parts) >= 2:
oid = parts[1]
image[path] = oid
return image, None
finally:
shutil.rmtree(tmpdir, ignore_errors=True)
def head_matches_post_image(repo_root, image):
"""(True, None) iff every path in `image` is in HEAD with that exact blob.
This is the proof that the commit carries the artifact, and it is asked of
git. `state.json` is a cache: it is written by the pipeline, it is exempt by
name from the scope gate, and a worker can therefore set
`merged.integrated: true` on a job that never landed. A cache may say a job is
done; only git may be believed about it.
"""
if image is None:
return False, "no post-image to prove against"
for path, oid in sorted(image.items()):
rc, out, _err = _git(repo_root, ["rev-parse", "--verify", "HEAD:%s" % path])
actual = out.strip() if rc == 0 else None
if oid is None:
if actual:
return False, ("%s is still present in HEAD, but the sealed patch "
"deletes it" % path)
continue
if actual != oid:
return False, ("%s in HEAD is %s, but the sealed patch produces %s"
% (path, actual or "absent", oid))
return True, None
def _import_integration_gate(path=None):
target = path or INTEGRATION_GATE_DEFAULT
if not os.path.exists(target):
return None
return _load_module_from_path("cv_integration_gate", target)
def _compute_diff_digest_local(root, baseline):
rc, gitpath, err = _git(root, ["rev-parse", "--git-path", "index"])
if rc != 0:
return None, "cannot locate git index: %s" % (err.strip() or "rc=%d" % rc)
index_path = gitpath.strip()
if not os.path.isabs(index_path):
index_path = os.path.join(root, index_path)
tmpdir = tempfile.mkdtemp(prefix="cv-emitwf-idx-")
tmp_index = os.path.join(tmpdir, "index")
try:
if os.path.exists(index_path):
shutil.copyfile(index_path, tmp_index)
env = {"GIT_INDEX_FILE": tmp_index}
rc, _, err = _git(root, ["add", "-A"], env=env, text=False)
if rc != 0:
return None, "git add -A failed: %s" % (err.strip() or "rc=%d" % rc)
rc, blob, err = _git(
root, ["diff", "--cached", "--binary", baseline], env=env, text=False
)
if rc != 0:
return None, "git diff --cached failed: %s" % (err.strip() or "rc=%d" % rc)
return "sha256:" + hashlib.sha256(blob).hexdigest(), None
finally:
shutil.rmtree(tmpdir, ignore_errors=True)
# THE RUN DIRECTORY IS THE ONLY DIGEST EXCLUSION, on both sides of the seam.
#
# 3.4.0 development briefly excluded two tracked files by name as well
# (triage-outcomes.jsonl, worker-performance.jsonl), because the pipeline wrote
# them BETWEEN a direct-mode job's Gate and the authority's re-derivation and an
# honest receipt read as `contradicted`. The fourth review pass withdrew that: a
# path excluded from the digest is also a path a worker may rewrite unseen, and
# the pipeline commits triage-outcomes.jsonl by name. The ordering is fixed
# instead — `cmd_finalize_wave` appends the run's `actual` AFTER the authority
# has run over the wave, so nothing the pipeline writes lands inside that window
# and there is nothing left to forgive.
def compute_diff_digest(root, baseline, gate_module=None, exclude_prefixes=None):
"""Both sides of the seam MUST pass the same `exclude_prefixes`, or the gate and
the authority compute different digests over the same tree and every honest
direct-mode receipt reads as `forged` (dogfood 15)."""
module = gate_module if gate_module is not None else _import_integration_gate()
if module is not None and hasattr(module, "compute_diff_digest"):
try:
try:
return module.compute_diff_digest(
root, baseline, exclude_prefixes=exclude_prefixes)
except TypeError:
# An older installed copy without the parameter: fall back rather
# than crash, and accept that it will disagree — loudly, as
# `forged`, which is at least a refusal and not a silent pass.
return module.compute_diff_digest(root, baseline)
except Exception as exc: # noqa: BLE001
return None, "integration-gate digest raised: %s" % exc
return _compute_diff_digest_local(root, baseline)
# --------------------------------------------------------------------------- #
# manifest -> waves
# --------------------------------------------------------------------------- #
def topo_waves(jobs, max_parallel):
"""Dependency waves, each chunked to at most `max_parallel` jobs.
A wave is a BARRIER, and the barrier is load-bearing: it is what preserves
the commit-before-dependent rule. A prerequisite's merge-back only STAGES
(`git apply --index` does not commit), so a dependent worktree created at
HEAD would not contain it. Record commits inside the wave; the next wave's
agents — and therefore the next wave's worktrees — are not spawned until the
whole wave has resolved. Deleting the wave barrier would reintroduce that bug.
`run: serial` jobs get a wave to themselves, in manifest order.
"""
by_id = {}
order = []
for job in jobs:
job_id = job.get("id")
if not job_id:
raise ValueError("every job needs an id")
if job_id in by_id:
raise ValueError("duplicate job id: %s" % job_id)
by_id[job_id] = job
order.append(job_id)
unmet = {}
for job_id in order:
deps = by_id[job_id].get("depends_on") or []
if isinstance(deps, str):
deps = [deps]
missing = [d for d in deps if d not in by_id]
if missing:
raise ValueError(
"job %s depends_on unknown job(s): %s" % (job_id, ", ".join(missing))
)
unmet[job_id] = set(deps)
cap = max(1, int(max_parallel or 1))
done = set()
waves = []
remaining = list(order)
while remaining:
ready = [j for j in remaining if unmet[j] <= done]
if not ready:
raise ValueError(
"dependency cycle among: %s" % ", ".join(sorted(remaining))
)
serial = [j for j in ready if (by_id[j].get("run") or "parallel") == "serial"]
parallel = [j for j in ready if j not in serial]
if serial:
# A serial job runs alone and BEFORE the parallel remainder of its
# level, matching the old dispatcher's "Task 0 serially, then the
# parallel batches".
first = serial[0]
waves.append([first])
done.add(first)
remaining.remove(first)
continue
for start in range(0, len(parallel), cap):
waves.append(parallel[start:start + cap])
for job_id in parallel:
done.add(job_id)
remaining.remove(job_id)
return [[by_id[j] for j in wave] for wave in waves]
# --------------------------------------------------------------------------- #
# agentType — the last native mechanism the audit had open
#
# docs/superpowers/architecture/native-mechanisms.md records `agentType` as the
# one mechanism that exists, covers a need, and was not used: the emitted script
# contained zero occurrences. The need it covers is named in that row — the
# REVIEW GATE. Engine C spawned implement/gate/record and nothing else, so a
# manifest job whose declared `type` is `review` — 3.0's own `task-13-review`,
# "Three-pass Review Gate over the composite", is one — was handed the generic
# IMPLEMENTER prompt: told to write inside a lane and report a summary, with
# none of `agents/spec-reviewer.md`'s three-pass contract reaching it.
#
# So exactly ONE mapping is made, and only where a job's own declared type says
# the work IS that role. The other stages stay anonymous on purpose, and the
# reason is in the JS_TEMPLATE next to them: Gate, Record and Finalize are
# de-tooled single-command transports whose entire safety property is
# `disallowedTools` + `bashCommandClamp`, and every agent under agents/ declares
# no `tools:` restriction at all.
#
# The prefix is READ from the plugin's own manifest rather than assumed. It is
# the install's plugin name, not the checkout's directory name — this very file
# is edited from a git worktree whose directory is a random job id, so deriving
# it from the path would produce a name that resolves to nothing. If the
# manifest or the agent file is missing, no `agentType` is emitted and the job
# stays anonymous: a name that resolves to nothing is worse than no name.
# --------------------------------------------------------------------------- #
AGENT_TYPE_BY_JOB_TYPE = {"review": "spec-reviewer"}
# ...and, from 3.4.0, a DEFAULT for everything else. An implementer used to arrive
# anonymous: the whole of its role was whatever `_implement_prompt` restated inline,
# it inherited the session's own turn budget, and nothing carried the model's own
# guidance on scope, narration or deliverable length. `agents/implementer.md` is that
# role, and arriving as a role is also the ONLY native way to carry a turn cap —
# `maxTurns:` is a field of an agent DEFINITION; the workflow `agent()` options have
# no equivalent (binary 2.1.238: label, phase, schema, model, effort, isolation,
# agentType, plus disallowedTools and bashCommandClamp).
DEFAULT_AGENT_ROLE = "implementer"
# The job types that are REVIEWERS and therefore stay anonymous. Matched
# EXACTLY, never by substring.
#
# The substring form (`any(tok in t for tok in REVIEWER_TOKENS)`) declined every
# type that merely CONTAINED one of those words, so `review_fix` — a job that
# fixes what a review found, and is an implementer in every respect — arrived
# with no role, no turn cap and none of `agents/implementer.md`. This repository
# already fixed exactly that shape once, in `_is_reviewer_job`, where a job
# titled "Writes the thing the reviewer reviews" was classified as a reviewer.
# The lesson did not travel the six hundred lines to here.
#
# `_is_reviewer_job` keeps its looser, word-boundary scan on purpose: it decides
# whether to ESCALATE a model, where over-matching is conservative. This decides
# which ROLE an agent is spawned as, where over-matching silently removes one.
REVIEWER_JOB_TYPES = ("review", "spec_review", "quality_review",
"integration_review")
def agent_role_for(job_type):
"""(role or None, reason or None) — the registered role a job's `type` maps to.
`review` maps to the Review Gate. The other reviewer types decline, WITH a
reason: a decline used to be an indistinguishable `None`, so "this type is a
reviewer and must stay anonymous" and "this lookup found nothing" reached the
emit output as the same silence. Everything else — `review_fix` included — is
an implementer.
"""
t = (job_type or "").strip().lower()
role = AGENT_TYPE_BY_JOB_TYPE.get(t)
if role:
return role, None
if t in REVIEWER_JOB_TYPES:
return None, (
"job type %r is a reviewer, so it stays anonymous: it is not an "
"implementer, and handing it the implementer role would tell a "
"reviewer to write code inside a lane" % t
)
return DEFAULT_AGENT_ROLE, None
def resolve_agent_type(job_type, plugin_dir=None):
"""(agent_type or None, reason). Never guesses a name."""
role, decline = agent_role_for(job_type)
if not role:
return None, decline
root = plugin_dir or os.path.dirname(HERE)
if not os.path.exists(os.path.join(root, "agents", "%s.md" % role)):
return None, "no agents/%s.md under %s" % (role, root)
manifest = os.path.join(root, ".claude-plugin", "plugin.json")
doc = _read_json(manifest, None)
name = (doc or {}).get("name")
if not (isinstance(name, str) and name.strip()):
return None, "plugin manifest %s declares no name" % manifest
return "%s:%s" % (name.strip(), role), None
PLUGIN_ROOT = os.path.dirname(HERE)
def agent_definition(role, root=None):
"""The agent's own definition, for the INLINE FALLBACK.
`agentType` selects a registered agent — and registration is a property of the
session, not of this repository. Dogfood 2026-09-02 (run wf_3b6697df-5e0): the
plugin was updated mid-session, its agents dropped out of the registry, and
every `agent({agentType})` spawn threw `agent type '...' not found` in 26 ms.
The emitted script therefore carries each role's definition verbatim and, on
exactly that error, retries once WITHOUT `agentType`: the definition body as
the prompt's preamble, the frontmatter `model` as `opts.model`, every other
option (schema, disallowedTools, clamp) unchanged. Returns
{"model": str|None, "body": str} or None when the file is absent.
"""
path = os.path.join(root or PLUGIN_ROOT, "agents", "%s.md" % role)
try:
with open(path, "r", encoding="utf-8") as fh:
text = fh.read()
except OSError:
return None
model, max_turns, body = None, None, text
if text.startswith("---"):
parts = text.split("---", 2)
if len(parts) == 3:
fm, body = parts[1], parts[2]
for line in fm.splitlines():
if line.strip().startswith("model:"):
model = line.split(":", 1)[1].strip() or None
# The TURN CAP, read only so the fallback can say what it lost.
# `maxTurns` belongs to the DEFINITION; an inline spawn is not a
# definition, and `agent()` has no option to re-impose it — so on
# that path the cap is gone and the log has to say so rather than
# let a job quietly run uncapped.
elif line.strip().startswith("maxTurns:"):
raw = line.split(":", 1)[1].strip()
try:
max_turns = int(raw)
except ValueError:
max_turns = None
return {"model": model, "max_turns": max_turns, "body": body.strip()}
def _js_parses(script):
"""True when `node --check` accepts the script, or when node is absent (the
check is then skipped, not passed). Twin of compound-v-emit-preflight.py's."""
import shutil, subprocess, tempfile
node = shutil.which("node")
if not node:
return True
with tempfile.NamedTemporaryFile("w", suffix=".mjs", delete=False,
encoding="utf-8") as fh:
# The runtime evaluates a workflow as the BODY of an async function: top-level
# `await` and `return` are legal there and illegal in a bare module, so the
# parse mirrors that wrapping — otherwise `return {` at the end of every
# script reads as a syntax error.
fh.write("(async function () {\n")
fh.write(script.replace("export const meta", "const meta", 1))
fh.write("\n})();\n")
name = fh.name
try:
r = subprocess.run([node, "--check", name], capture_output=True, text=True)
return r.returncode == 0
finally:
os.unlink(name)
def _clamp_rules(job, python_bin, self_path, worker_script_for):
"""The bashCommandClamp for one job's IMPLEMENT agent.
Spec D5.1: a non-`claude` job's clamp MUST admit
`scripts/compound-v-run-<backend>-worker.sh`, or carry no clamp. A clamp that
can bind nothing makes the runtime refuse the spawn — which fails loudly
rather than degrading, but it still means the second family cannot launch.
THE `None` RETURN IS UNREACHABLE FOR A JOB THAT ACTUALLY LAUNCHES, and 3.0.6
described it wrongly. It happens only for an external backend whose worker
script is absent — and `job_entry` REFUSES that job outright a few lines later
("the handoff cannot be materialized"). A `claude` job always carries the
register-lane rule. So no implementer that reaches `agent()` is ever unclamped;
the claim that one could be was a caveat written from reading this function
alone instead of the path around it. `_check("every launched job carries a
clamp", ...)` in the selftest now holds that shut.
Rule syntax is the standard permission rule, validated by the runtime:
`Bash(<command or prefix>)`, tool name case-sensitive, no whitespace padding
inside the parens. An entry that parses to a tool with no rule content is an
"inert clamp entry" and the spawn is refused.
"""
backend = job.get("backend") or "claude"
# `-B` IS PART OF THE ADMITTED FORM, and the rule and the command it admits
# must carry it identically. The scope gate forgives no path by extension