Skip to content

Worktree jobs cannot run their floor: unprovisioned checkout, and the partition forbids any job from provisioning it #15

Description

@khymerao

Follow-on from #12, found on the same seven-job run (ten dispatches over three days). These two are not missed cases - the second is a contradiction between two invariants the system enforces at the same time, so it needs a design decision rather than a patch. Reproduced against clean main (3.5.1, ce4cb19).


3. A job worktree is unprovisioned, and provisioning it is charged to the job

The harness branches each worktree job a fresh checkout. node_modules is gitignored, so the new worktree has none and the JS half of the floor cannot run there. If the agent installs the dependencies, the gate charges every installed file to the job.

Both halves of that are deliberate, and the code says so. compound-v-scope-check.py:

Source 3 is the one the old gate MISSED: --exclude-standard drops gitignored paths, so a worker could write a gitignored file (e.g. dist/, .env, build/) completely undetected. We union it in so any ignored write outside write_allowed is reported as a violation - with NO exceptions. Nothing is forgiven by extension and nothing by name.

and, on the one subtraction that exists:

Worktree mode passes nothing - a fresh worktree add HEAD has no pre-existing untracked.

So in worktree mode preexisting is empty by design, and an install is always an out-of-lane write.

Evidence. Run 2026-09-10-obstacle-memory-grid-3, task-3-producer, worktree wf_f74a1802-318-1: BLOCKED with several thousand violations, all mac-senses/node_modules/** and mcp/node_modules/**.

3a. A related fail-open in the same area - corrected

An earlier revision of this issue claimed task-4-wire passed with a floor that never ran. That was wrong, and I withdraw it: the run record shows status: blocked, tests.exit_code: 1 - its floor did run, went red, and the dogfood-14 rule caught it correctly.

Reading the code around it did surface a real but so-far-unobserved fail-open: _job_result_from reads the floor only through _tests_block_from_floor, which returns None when no check carries a checker, so the floor's own merge_blocked: true is never consulted and the job keeps success. build_review_spec gate 1 honours that same field; Engine C's consumer does not. Never observed in 198 recorded results here or 17 in the reporting project.

That half is self-contained and fixed in #13. It is a latent fail-open found by reading, not a reproduced defect, and it is not the reason the worktree floor could not run - §3 and §4 below are.


4. No job can provision its own worktree, because the partition forbids it

The obvious workaround for #3 is to declare the dependency directories in each job's write_allowed, making the install in-lane. The validator refuses, correctly:

write_allowed overlap: job 'task-3-producer' (mcp/node_modules/**)
and job 'task-5-assembly' (mcp/node_modules/**) can both own the same path

So, simultaneously:

  • every worktree job needs the dependencies present for its floor to run;
  • installing them is a write, and writes must be declared;
  • no two jobs may declare the same path - the partition's central invariant.

Therefore at most one job in a manifest may legally provision, and every other worktree job is left with a floor it cannot run. For any project whose floor needs installed dependencies - most of them - the two invariants cannot both hold.

There is no provisioning mechanism to fall back on. provision|setup_command|install_command|prepare_command returns nothing across compound-v-emit-workflow.py, compound-v-validate-manifest.py and execution-manifest.md.

And the exemption list is deliberately closed, so "just exempt node_modules" contradicts the stated design. RUN_DIR_EXEMPT_BY_NAME is:

a CLOSED LIST - one entry per file class with the reason it cannot be digest-bound [...] Every entry is self-referential, shared, or written by the pipeline AFTER the gate built its list - consequences, not choices.

node_modules is none of those things.

The shipped example has the same shape. examples/manifest.example.yaml declares floor_command: "npm run test:floor" on isolation: worktree jobs - a floor that cannot run in a fresh checkout. Nothing in the plugin says a worktree floor must be self-contained (no hits across skills/, commands/, agents/, README, AGENTS.md, ADRs, TROUBLESHOOTING). The plugin's own dogfood floors are all bash tests/..., python3 ... --selftest or /bin/echo floor-ok - a stdlib project with no install step, which is why this never surfaced here.

And worktree.baseRef: head makes it worse, not better. Without the setting, dependent jobs run direct in the main checkout, where the dependencies exist. Setting it - as #12 correctly recommends - puts more jobs in fresh worktrees and therefore increases exposure to this. The two issues pull in opposite directions, which is worth deciding deliberately.

Suggested resolution

The one route that adds no new exemption class and touches neither the partition nor the validator is the subtraction that already exists:

  1. provision the worktree as part of creating it, before the agent starts;
  2. snapshot untracked/ignored paths after provisioning;
  3. pass that snapshot as preexisting.

That is exactly the semantics direct mode already has - "files this job never created are not attributed to it" - and it makes the current worktree-mode comment ("a fresh worktree add HEAD has no pre-existing untracked") true again rather than an assumption that provisioning breaks. The alternatives (a manifest-level provision_command whose writes are exempt; or an explicit shared-read-write path class distinct from the exclusive lanes) both work but introduce a new concept.

I have not sent a PR for #4 - which of the three you want is an architecture call, not a bugfix, and it is yours to make. Happy to implement whichever you pick.

Acceptance

A project whose floor requires installed dependencies can run a multi-job manifest where every job's floor actually executes, without any job being charged for the dependencies - and with the validator's disjoint-lane invariant untouched.


Not to be done

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions