[Dream Cycle 2026-08-27] evaluation-adapters: darwin entrypoint missing required <repo> positional + flywheel,darwin scan

[ruvnet]

Rotation

DAYINT % 5 == 2 → DEEP=evaluation-adapters, SCAN=flywheel,darwin. No bonus deep-dive (DAYINT % 25 = 2, % 75 = 27). Session commit: 7933c3599abe22df5290f4609d1f93f598feb3de.

Ledger Check

Last 14 rows inspected. zeroMergeStreak=true (dream-machine ledger signals) → biased tonight's candidate to something tiny and easily reviewable. duplicateDirections=[]. Note: PR #38 (dream/2026-08-27-portfolio-cycle, a separate/broader automation) already opened a 2026-08-27 ledger row for a different cohort of repos including a Dream Machine ECT finding (draft PR #35) — orthogonal to tonight's evaluation-adapters surface, no direction overlap, but the same-date LEDGER.md append may need manual reconciliation when both PRs are reviewed (flagged for the human reviewer, not resolved automatically).

Deep Dive: evaluation-adapters

Wired dream.config.json#evaluatorEntrypoints through the existing (unmodified) classifyEntrypointResult classifier (ADR-0002) via a new dream-machine verify-entrypoints [config] command. First real use immediately found a genuine bug in this repo's own config: darwin: "npx @metaharness/darwin evolve --sandbox mock" is missing the tool's required <repo> positional (confirmed via npx @metaharness/darwin --help). The parser then consumes the literal string --sandbox as <repo>, silently creating a --sandbox/ directory inside this repo's working tree and writing .metaharness/variants/* into it — and making the entrypoint non-idempotent (2nd run: blocked, "autonomous or generated child id already exists: g1_v0"). Fixed: darwin: "npx @metaharness/darwin evolve . --sandbox mock". Also added .metaharness/ to .gitignore (the tool's own scratch dir, now correctly located, was previously untracked-but-ungitignored).

Hypothesis (frozen before evaluation)

Given dream.config.json's evaluatorEntrypoints map, when each entry is run through the existing classifier via a new verify-entrypoints command instead of one-by-one manual invocation, then a real, previously undetected defect in a configured entrypoint should surface as blocked/suspicious-silent — without weakening the classifier — and the fix should be independently verifiable via the real bench evaluator with zero regressions.

Evaluation Receipt

Baseline (main @ 7933c35): npm ci && npm run build clean; npx vitest run → 98/98. Candidate: build clean, lint clean; npx vitest run → 104/104 (+6 new tests), 0 regressions. verify-entrypoints before fix: bench: live, darwin: live (1st run) → darwin: blocked (2nd run, real collision). After fix: bench: live, darwin: live on two consecutive runs. Full verbatim transcripts in the committed report.

Darwin Results (bounded, exploratory)

npx @metaharness/darwin evolve . --sandbox mock --generations 2 --children 3 --seed 42 (real invocation, corrected command, run directly as an integration check — not a search over dream-machine's own promotion criteria): winner=baseline, delta=+0.000, 8 sibling variants scored, all ≤ baseline. Confirms the fixed entrypoint completes end-to-end with bounds respected. No lineage promoted; scratch output deleted (gitignored).

Evidence

OBSERVATION (--help usage string) → INFERENCE (missing positional is root cause) → MEASUREMENT (before/after receipts, test counts) → DECISION (config fix, no ADR — parameter/config correction, not architectural per STEP 19) → no REJECTION needed tonight.

Reward-Hack Check

Independent critic pass: no benchmark weakened, no gold data (none exists), no cherry-picking (both failing and passing receipts shown verbatim), no evaluator exploit, no hidden cost (diff: +101/-2 across 5 files), no threshold/safety-boundary touched, no undocumented cache reliance (every command re-run live). Clears reward_hack_clear and critic_clear.

Security Review

Revisits a MUST-ADDRESS-BEFORE-AUTOMATING item the 2026-08-13 report explicitly flagged: auto-feeding evaluatorEntrypoints into the classifier pipes a config-file value into child_process.exec. Addressed by the same reasoning already on record: dream.config.json is repo-committed and PR-reviewed, never runtime-attacker-controlled; verify-entrypoints is operator-invoked only and is not wired into the compiled nightly prompt's automatic execution path (that remains explicitly deferred, same as ADR-0002's own singular command). Filesystem containment: the darwin bug itself was a minor containment concern (writing synthetic variants/*.ts into the checkout); closed by the fix + .gitignore. No credentials touched, no LLM calls, no new network surface. @metaharness/darwin@0.9.3 (19 releases since 2026-06-26) — same active-development/rough-edges pattern as redblue in ADR-0002.

Scan Findings

flywheel: no flywheel entry exists in this repo's evaluatorEntrypoints (only bench, darwin) — by design (ADR-0001 §2.4, optional backend), not a bug. Non-finding, recorded for SCAN completeness.

darwin: the entrypoint bug above, in full.

Competitors

Sakana AI (DGM ~30pp SWE-bench lift, Grade B) · OpenHands (2026 best open-scaffold SWE-bench Verified score, cost-side silent-failure commentary, Grade C) · SWE-agent (public commentary on an undetected 38%→24% harness-drift regression, Grade C) · DSPy/GEPA (metric assumed live by construction, no adapter-liveness layer, Grade C) · AutoGPT lineage (no standard evaluator-adapter trust layer historically, Grade C) · Stop Comparing LLM Agents Without Disclosing the Harness, arXiv:2605.23950 (Grade A, fetched directly tonight) — the general research case for treating an evaluator-adapter's own correctness as first-class. No competitor surveyed ships an equivalent of this repo's classifier.

Gist

LOCAL — no gist-creation tool available this session (GitHub MCP server has no Gist API). Full report committed at docs/dream-cycle/2026-08-27-evaluation-adapters-report.md on this issue's PR branch.

Witness

report_sha256=896eb2d72df5645953fec80a12ddea86f8610afcaf3c958a1d5b080a757b7b7c, session_commit=7933c3599abe22df5290f4609d1f93f598feb3de, witness=6c4b6481acdb0fc121367e38580e968e4c060e6672effc64fecb5a46b6657403. Verification steps in the committed report.

Recommendation

evaluated: accepted. Draft PR forthcoming on dream/2026-08-27-evaluation-adapters. No merge, no self-promotion — human review required.