Skip to content

Latest commit

 

History

History
637 lines (525 loc) · 86.2 KB

File metadata and controls

637 lines (525 loc) · 86.2 KB

Alchemist — Master Tracker

Autonomous C/C++ → verified safe Rust, on a local model (Gemma 4 31B, US-origin).

The one document for the whole climb: ~285 concrete milestones across 8 vertical phases (integration checkpoints — prove the stack on a bigger target) and 6 capability tracks (reusable capabilities, built once, applied everywhere). Each item has a definition-of-done and a difficulty tag. Finish every item and the project is genuinely complete: point it at a C/C++ codebase, walk away, come back to verified safe Rust.

Interactive, checkable version: docs/master-tracker.html (progress persists in-browser).

Progress at a glance (as of 2026-07-12)

Phase / Track Done Status
F · Foundations 12/12 ✅ complete
P0 · Reliability Floor 14/14 ✅ complete (+P0.8a)
P1 · Whole Small/Mid C Library 11/20 items 🔨 EXIT = CONVERGENCE (new-fixes/library → 0), not a lib count. real-library gate 1/5 (jsmn 1/2). parson 0 after 3 runs (fill wall); http-parser cold surfaced 2 new general blockers → generality is the front line. See re-orientation banner.
P1.5 · Generality gate (NEW) 0/5 🔨 the real bottleneck — general API-seq differential engine · coherence linter · per-shape e2e fixtures · decomposition · fill-scaffolding. Gates P2+.
P2 · Scale — Large Single Codebase 0/12 gated on P1.5 (scaling a non-general converter = the grind ×100)
P3 · C++ Frontier 0/8 ⬜ gated on P1.5
P4 · Embedded & Unsafe Boundary 0/8 ⬜ gated on P1.5
P5 · ArduPilot 0/22 ⬜ frontier · gated on P1.5
P6 · Autonomy — Point & Walk Away 0/12 🔨 NOW, not a finale — the generality work IS P6, started 2026-07-14
Track · SEM (semantics) 0/70 ⬜ *
Track · VER (verification) 0/25 🔨 2 active *
Track · MODEL 0/21 🔨 1 active *
Track · IDIOM 0/12 ⬜ *
Track · PERF 0/8 ⬜ *
Track · INFRA 0/20 ⬜ *
TOTAL 34/261 (13%)

* Track vs phase counts overlap and are honest by design. The 6 capability Tracks are the reusable-capability backlog; when a track capability is delivered, it is recorded against the phase item that demanded it (e.g. the struct-carry, type-unifier, e2e-oracle, and decomposition capabilities live as F.7–F.9 and the shape system as P0.8/P0.8a/P0.9 + P1.15), not double-checked in the Track list. So the Track checkboxes read low even though much of their machinery is built and verified — they track remaining capability, phases track delivered capability. The reliability floor (F+P0 = 26 items) is 100% done; the current front is P1.

Where we sit in the field (researched, not guessed)

  • DARPA TRACTOR (2024) — a DoD program to translate all C to idiomatic, safe Rust. Validates the vision at the national-security level and is a funding lane. Our edge: verification-gated, model-writes-every-line, fail-closed.
  • c2rust (Galois/Immunant) — the mature mechanical transpiler, but it emits unsafe, unidiomatic Rust. Turning that into safe idiomatic Rust is the open problem we attack.
  • Laertes (OOPSLA'21) — automated unsafe→safe refactoring; its bounded success is the evidence this is genuinely hard.
  • LLM-translation studies — LLMs translate plausibly but unreliably; correctness demands an oracle. That finding is our thesis: the differential gate, not the model, is the source of truth.
  • Verification ladder — Miri (UB) → Kani (bounded model checking) → Prusti/Creusot/Verus (deductive) → Aeneas (functional translation validation). Climbed in the VER track.

Honest bottom line (REVISED 2026-07-14 after the http-parser cold test): the machinery is built but not yet general — that was the over-claim. A cold run of http-parser (a fresh, different-family library) surfaced 2 new general blockers before it could mint a single vector (&'staticstr lifetime-emit bug that breaks every &'static str/str_lookup return; the pipeline's own C-DLL build failed though the C compiles standalone). The blockers were general (not domain-overfit), which is good — but they existed because shapes/fixes are validated in isolation, not end-to-end on a real library, so every new library is the first real test and surfaces the latent bugs. The true gate to the product is therefore capability GENERALITY (a new unseen library needs ~0 human touches), not climbing to bigger targets. The phase ladder past P1 (scale → C++ → embedded → ArduPilot) is correct in content but must be gated on the generality bar — scaling a non-general converter just scales the per-library grind. "Perfect safe Rust for all of ArduPilot" is impossible as literally stated (hardware code is irreducibly unsafe); the real target is verified safe logic + a thin, audited unsafe shim.

🧭 Re-orientation (2026-07-14): generality is the gate, not target size

The product is "point it at a codebase, walk away, get verified safe Rust." This session's evidence (parson: 3 runs, 0 verified; http-parser: 2 new blockers on a fresh lib) says the bottleneck is generality, not scale. Re-sequencing, without discarding the ladder's content:

  1. P1 exit = CONVERGENCE, not a lib count. The bar is "the Nth unseen library needs ≤1 human touch," measured by new-fixes-per-library → 0 (parson ~8 → http-parser 2 is the right slope; exit at ~0). A lib count alone just measures our patience.
  2. Pull the generality capabilities forward as P1's real content (today buried in the Tracks at "0/70 someday"): the general API-sequence differential engine (collapse the per-family shape zoo into one engine); a coherence linter (catch &'staticstr-class bugs statically in <1s, not a 30-min run); an end-to-end fixture per shape (no shape ships un-exercised on a real function); decomposition for the deep tail; fill-scaffolding for novel structs (the parson wall).
  3. Gate P2+ on the convergence bar — scale/domain phases start only once a new library is ~zero-touch.
  4. P6 (autonomy / "walk away") is NOW, not a finale — the generality work above is P6, started today.

Legend

eng engineering · wire wire existing-but-dead code · research research-hard/open problem · model model-dependent · infra infrastructure · product product/GTM · unsafe irreducible unsafe boundary

Status: [ ] to-do · [~] active · [x] done · [!] blocked


Architecture — how the machine works, and where every item lives

Read this once and the whole tracker becomes navigable: each phase/track item below is a change to one of the components described here. Every item's note names the component and commit; this section says what that component IS and where its code lives. Paths are relative to the repo root (alchemist/…). Line counts are approximate load-bearing sizes, not limits.

0. The one idea

The oracle, not the model, is the source of truth. A local model (Gemma 4 31B, US-origin, on the box at :8086) writes every line of Rust. Nothing it writes is trusted until a byte-exact differential oracle — the compiled C reference, fuzzed through FFI and compared against the Rust — agrees on every input. No oracle ⇒ refuse (fail-closed). We never hand-write translation output; we build, run, and sharpen the converter. A refusal is an honest "we can't prove this yet," never a silent stub.

1. The 6-stage pipeline (pipeline.py orchestrates; cli.py translate is the entry point)

   C source dir
       │
  [1] analyze   analyzer/         parse C, build call graph, detect algorithmic modules
       │                          → parser.py, call_graph.py, module_detector.py, preprocessor.py
  [2] extract   extractor/        per-fn spec: signature, Rust-lifted types, test vectors, category
       │                          → spec_extractor.py, normalizer.py, function_classifier.py, schemas.py
  [3] architect architect/        design the crate/trait/type layout across modules
       │                          → crate_designer.py, trait_extractor.py, type_unifier.py, validator.py
  [4] implement implementer/      TDD fill: skeleton → emit tests → model fills each fn → in-loop verify
       │                          → tdd_generator.py (the heart), skeleton.py, test_generator.py
  [5] verify    verifier/         the 5-gate final differential proof (see §3)
       │                          → auto_config.py, adapter_gen.py, proptest_gen.py, differential_tester.py
  [6] report    reporter/         refusal ledger, metrics, perf, signed receipt
                                   → refusal_ledger.py, metrics.py, perf.py; receipt in verifier/receipt.py

Specs are checkpointed to <subject>/.alchemist/specs/; a run is resumable. solo.py runs a scoped single-/few-function translate for fast iteration; lib_orchestrator.py runs whole libraries.

2. The shape system — the central abstraction (this is what most items touch)

A C function is classified into a shape by its signature. The shape determines how the oracle mints inputs, compares outputs, and what tolerance applies. This is the extensibility surface: adding support for a new kind of function = adding a new shape.

Shape catalog (each is a classify_* + fuzz_* pair in verifier/auto_config.py, plus emitters):

Shape C signature (archetype) Rust lift Oracle
checksum u32 f(const u8*, len[, seed]) fn(&[u8][,seed])->u32 scalar compare, fold boundaries
hash/digest void f(const u8*, len, u8* out) fn(&[u8])->Vec<u8> digest bytes + NIST/FIPS catalog KATs
scalar T f(scalars…) fn(scalars)->T scalar compare
inplace void f(u8* buf, len) fn(&mut[u8]) mutated-buffer compare
buf_transform int f(in,inlen,out,outlen) fn(&[u8])->Vec<u8> out[0..ret]; decoder→roundtrip via _paired_encoder_name
cbuf_out char* f(char*) (text→text, e.g. NMEA) fn(&str)->String result-string compare
cstr_out char* f(char*) (text encoder) fn(&[u8]|&str)->String NUL-terminated string compare
cstr_scalar T f(const char* s, scalars…) fn(&str,…)->T scalar compare
iarray_reduce T f(const int* a, n) fn(&[int])->T scalar reduce
buf_gen u8* f(size n,…) fn(n,…)->Vec<u8> generated-buffer compare
cstr_roundtrip char* f(char*) binary-out decoder fn(&str)->Result<Vec<u8>,E> P1.15: mint via paired C encoder, decode(encode(p))==p
cipher_seq / alloc_seq / hash_seq init+op sharing a struct struct-carried sequence differential (state observer)
scalar_mutator void f(&mut State, scalars) struct-carried state-mutation differential

⚠️ The N-site wiring rule (the single most important thing to know when adding/reading a shape). A shape is not one function — it is wired at 5–7 sites that must all agree, and there are TWO independent vector-dispatch paths that both need the shape or the fill loop silently sees "no test vectors":

  1. classify_<shape>() + fuzz_<shape>_vectors()verifier/auto_config.py
  2. Dispatch Asynthesize_c_vectors() per-fn dispatch — verifier/auto_config.py (the pipeline calls this)
  3. Dispatch B — the TDDGenerator fuzz backfill dispatch — implementer/tdd_generator.py (~line 2416). The generator clears machine-tagged vectors and re-mints via its own dispatch; a shape missing here dies even if Dispatch A produced vectors. (This bit us on P1.15.)
  4. build_diff_config() harness branch — verifier/auto_config.py (feeds the final gate; must be checked before more-general shapes)
  5. proptest block _proptest_<shape>_block() + VALID_CATEGORIESverifier/proptest_gen.py
  6. FFI adapter branch — verifier/adapter_gen.py (emits rust_<fn> and c_<fn> wrappers)
  7. fill-loop test emitter _emit_<shape>_test() + the cargo-test name filter in _test_filters_for_fn()implementer/test_generator.py + tdd_generator.py. (A missing filter ⇒ 0 tests run ⇒ false "no test vectors". Also bit us on P1.15.)

Canonical worked example of all 7 sites: P1.15 cstr_roundtrip (commits 07020fd→c8bff85). Simpler examples: P0.8a cstr_out, P0.8 iarray_reduce/cstr_scalar, P1-baseline buf_gen.

3. The verification gate stack (stage 5; the fail-closed proof)

Emitted by auto_config.build_diff_config()adapter_gen + proptest_gen into a throwaway verify_gen/ crate, run by differential_tester.py:

  • gate 1 cargo check --workspace — it compiles.
  • gate 2 anti-stub scan (implementer/anti_stub.py) — no todo!()/unimplemented!()/trivial-constant returns.
  • gate (structural) no-unsafe proof — the output is safe Rust.
  • gate 3 semantic lints (implementer/semantic_lints.py) — no oracle-evading shortcuts.
  • gate 4 cargo test --workspace — the spec/KAT vectors (C-minted) pass.
  • gate 5 differential proptest — fresh fuzzed inputs, Rust vs C live, byte-exact.

Vector minting & crash isolation (synthesize_c_vectors): the C reference is compiled to a DLL and called via ctypes; because fuzzed inputs hit real C undefined behavior, minting runs in a forked child — a segfault returns 0 vectors (fail-closed) instead of killing the translator. A signed receipt (verifier/receipt.py) seals gates+harnesses+oracle+env with a content-SHA256 (optional HMAC).

4. Type & memory model (how the skeleton compiles cold)

  • struct-carry (verifier/struct_lift.py, wired in pipeline.py) lifts C state structs into Rust so the skeleton compiles before any body is filled. Resolves scalar typedefs (BYTEu8), enum typedefs (jsmntype_ti32), strips #ifdef in struct bodies, and raw-escapes keyword fields (typer#type).
  • type_unifier (architect/type_unifier.py) auto-derives one coherent type model across fractured modules from the analysis (no hand-written registry — F.9).
  • normalizers (extractor/normalizer.py + auto_config lifts) canonicalize specs; solo._load_specs_and_arch re-applies them on every reload so a reloaded spec matches the in-memory one (P0.3).

5. Fill escalation ladder (implementer/tdd_generator.py, cheapest→dearest; won_via records which tier won)

cached (subject-anchored win cache <subject>/.alchemist/wins, P0.4) → deterministic template (init/reset init_templates.py; static-table no-op; free/destroy no-op) → single model fill → multi_sample best-of-N (multi_sample.py) → holistic whole-file fixer (holistic.py, self-terminates after 2 no-op patches, P0.5) → decomposition gcc-proven byte-exact split (structural_decomp.py, F.7). Reference transliteration via reference_probe.py.

6. Reliability & observability layer

  • refusal ledger (reporter/refusal_ledger.py) — the north-star metric: per-fn verified/refused + reason + won_via + telemetry (elapsed_s/llm_calls/output_tokens), plus wins_by_tier rollup. Written to <subject>/.alchemist/refusal_ledger.json.
  • deterministic replayALCHEMIST_DETERMINISTIC=1 forces temp 0 + single-sample everywhere (byte-identical output proven, P0.13).
  • benchmarksbench/leaf/ (26 unseen leaf fns → scorecard, P0.11) and bench/lib/run_libbench.py (whole-lib batch → per-lib + overall refusal, P1.10). Nightly cron runs the real pipeline+model (P0.12).

7. How to trace any tracker item to its code

Every completed item's note carries (a) the component/shape it changed (names map to §1–6 above), (b) the commit SHA, and (c) the box-validation result where model-dependent. To go from an item to code: read the note → find the named module in the map above → git show <sha>. New capability items are almost always "a new shape" (§2, the 7 sites) or "a new gate/lever" (§3/§5).

Component → primary file quick map:

Concern File(s)
Pipeline orchestration pipeline.py, cli.py, solo.py, lib_orchestrator.py
Analyze analyzer/{parser,call_graph,module_detector,preprocessor}.py
Extract / specs extractor/{spec_extractor,normalizer,function_classifier,schemas,fuzz_vectors}.py
Architect / types architect/{crate_designer,trait_extractor,type_unifier,validator,schemas}.py
Shapes & oracle config verifier/auto_config.py (classify/fuzz/synthesize/build_diff_config)
Fill loop (the heart) implementer/tdd_generator.py
Skeleton / tests implementer/{skeleton,test_generator,init_templates}.py
Struct-carry verifier/struct_lift.py
Gate emit verifier/{adapter_gen,proptest_gen,differential_tester,auto_ffi}.py
Gates (stub/lint) implementer/{anti_stub,semantic_lints}.py
Escalation tiers implementer/{multi_sample,holistic,structural_decomp,reference_probe}.py
Report / receipts reporter/{refusal_ledger,metrics,perf}.py, verifier/receipt.py
Benchmarks bench/leaf/, bench/lib/

F · Foundations — already in the ground (DONE)

What exists and is verified today. The platform everything else stands on. Exit: a general, fail-closed, spec-first pipeline with real verified conversions behind it.

Foundations → code (each F item's primary location; see the Architecture §1 map for the full stage breakdown): F.1 pipeline.py + cli.py (stages) · F.2 verifier/auto_config.py + verifier/differential_tester.py (oracle) · F.3 zlib workspace under subjects/zlib*/ + verifier/zlib_config.py · F.4 subjects/sha256/ + standards/catalog.py (CAVP KATs) · F.5 libcrc subject workspace · F.6 implementer/scrubber.py + verifier/auto_ffi.py (static/inline strip + macro neutralize) · F.7 implementer/structural_decomp.py (wired in implementer/tdd_generator.py) · F.8 verifier/e2e_oracle.py (gated in pipeline.py verify) · F.9 architect/type_unifier.py · F.10 implementer/tdd_generator.py (cargo-fix pass, commit 36f1233) · F.11 verifier/auto_config.py + implementer/semantic_lints.py (the three false-refusal fixes) · F.12 Lua subjects under subjects/lua*/.

  • F.1 6-stage spec-first pipeline — analyze → extract → architect → implement → verify → report, checkpointed, model-in-the-loop eng
  • F.2 Fail-closed byte-exact differential oracle — compile C ref, FFI-fuzz vs Rust; no oracle → refuse eng
  • F.3 zlib byte-exact — deflate+inflate+checksums, round-trip byte-exact vs C at levels 0–9 model
  • F.4 SHA-256 CAVP-green — cold → safe Rust, NIST CAVP wired into the gate model
  • F.5 libcrc 9/9 verified workspace — whole lib, zero hand-edits model
  • F.6 Static-internal-fn exposure keystone — strip static/inline + neutralize export macros for oracle linkage eng
  • F.7 structural_decomp wired into refusal escalation — gcc-proven byte-exact split, fail-closed eng
  • F.8 e2e_oracle wired as a fail-closed gate — whole-program observable-behavior differential eng
  • F.9 type_unifier auto-derives from analysis — struct-fracture coherence without a hand-written registry eng
  • F.10 cargo-fix mechanical-repair lever — apply rustc machine-applicable edits before refusing eng
  • F.11 Three verification false-refusals killed — decimal-vs-hex, min(usize,u32), deleted live method eng
  • F.12 Lua leaf functions conquered from real source — luaS_hash + luaO_ceillog2 byte-exact model

P0 · Reliability Floor (✅ COMPLETE — all 14 items + P0.8a)

Make single-function C→Rust on Gemma dependable. The wall we hit this week: correct code, sunk by mechanical trivia and false refusals. Exit (all met): tinychk reliably 4/4 ✅ · unseen leaf fns ≥90% verified (26-subject bench: 24/26 = 92.3%, cold-confirmed) ✅ · refusal ledger live (+ per-fn timing/cost + win-by-tier audit) ✅ · nightly cron runs the real pipeline+model ✅. Also delivered: deterministic-replay mode (byte-identical proven), subject-anchored win cache, spec-reload normalization, two new oracle shapes (iarray_reduce/cstr_scalar), cstr_out final-gate wiring.

  • P0.1 Clean single-fn repro harness — ALCHEMIST_FILL_TRACE=<dir> dumps model Rust + compile error + differential divergence per iteration eng infra (commit 4f49675)
  • P0.2 Root-cause the tinychk differential fails — found via the trace: test_generator._emit_spec_test ordered the KAT call by fuzz-vector dict order (buf, seed) not the signature (seed, buf) → uncompilable test → refused correct code forever. Fixed → tinychk 4/4 OVERALL PASS eng (commit a22d18d)
  • P0.3 Fix verify_gen ↔ spec coupling — the verify stage (and solo) reloaded specs from disk via _load_specs_and_arch, which applied only the generic normalizer, NOT the three auto_config type-lifts the implement stage applies in-memory (byte-buffer→&[u8], C-char-value→i8/u8, digest→fn(&[u8])->Vec<u8>). So a reload saw pre-normalization specs → build_diff_config disagreed with the model's actual (normalized) signature (the P0.8 E0308). Fixed at the root (commit 78d1b32): _load_specs_and_arch now applies the same three (idempotent) normalizers, so every reload is canonical. Validated on box: count_char (reload path) + crc32b (byte-buffer, no regression) both OVERALL PASS. +1 reload regression test. eng
  • P0.4 Win-cache restore determinism — done (commit 3703d4f). The wins cache anchored to workspace_dir.parent/"wins", which for an external --output (translate's normal mode; leaf bench uses a /tmp tempdir) is the SHARED temp ROOT — so every subject wrote to one /tmp/wins keyed only by crate/module/fn. Restore was non-deterministic (depended on where --output sat), subjects could collide, and it silently made the leaf benchmark not cold (prior-run wins short-circuited fills → the reported first-pass was mostly cached restores). Fixed: _wins_cache_path anchors to the source root (subject/.alchemist/wins) via the _source_root the generator already holds, so restore is deterministic regardless of --output and run_leafbench's _clean_state produces a genuinely cold run. +3 tests. Suite 764. The truly-cold re-run confirms the benchmark was NOT inflated: 24/26 OVERALL PASS, first-pass 92.3% — identical to the cached numbers, i.e. the model genuinely first-fills these leaf shapes. eng
  • P0.5 Kill holistic-fixer empty-patch no-ops — done (commit 78bbf9a, implementer/holistic.py). The HolisticFixer loop continued on an empty/all-rejected patch and counted a byte-identical rewrite as a "change", so it could grind every iteration with no progress and no diagnosis. Now: _apply_patch only counts genuine content changes (compares against on-disk); the loop tracks consecutive no-op iterations and bails after 2 with a bail_reason instead of burning the budget; the caller (tdd_generator) surfaces the reason and — since the fixer now self-terminates when stuck — is uncapped from max_iter=1→3 so a productive multi-step fix (error A → then error B) isn't cut off at one pass. +3 unit tests (empty-patch bail, identical-content no-op, real-fix success). Suite 760. eng
  • P0.6 Escalation-ladder audit — done (commit 09a5d51; implementer/tdd_generator.py sets won_via at each win site, reporter/refusal_ledger.py rolls up wins_by_tier). FunctionAttempt.won_via now records which tier PRODUCED each win — cached | template | single | multi_sample | holistic | decomposition (or "" if refused), set at every win site in the fill loop. The refusal ledger rolls it up as wins_by_tier (verified wins per tier, cheap→expensive) and the pipeline prints it, so over any corpus a tier with zero wins is visibly not earning its budget (dead weight or a rare safety net). The tier thresholds ARE the budgets (multi_sample_after/holistic_after/max_iter_per_fn/multi_sample_n, all TDDGenerator ctor knobs). Validated on box: count_char + sum_array both wins by tier: single=1 — i.e. the model first-fills leaf shapes and the escalation tiers are pure safety nets there; harder libraries will light up the higher tiers. +1 test. Suite 765. eng
  • P0.7 Refusal-rate instrumentation — every translate run emits <subject>/.alchemist/refusal_ledger.json (per-fn verified/refused + reason + escalations) and prints the refusal rate infra (reporter/refusal_ledger.py)
  • P0.8 Systematic false-refusal sweep — closed two real gate-coverage gaps (commit 4d26f9f), driven by the benchmark's own planted probes. Added two common leaf shapes end-to-end (classify + TDD vectors + final-gate harness + proptest + FFI adapter): iarray_reduce <scalar> f(const T* a, int n) (sum_array/imax_array) and cstr_scalar <scalar> f(const char* s, ...) (count_char). Two bugs found by RUNNING the converter: (1) a C char value-arg mis-lifts to Rust char (4-byte) — new normalize_char_scalar_params re-lifts to i8/u8 (the byte it is); (2) the verify stage can hold a pre-normalization spec, so build_diff_config emitted any::<char>() while the wrappers were i8 → E0308, now coerced char→i8. All three new subjects OVERALL PASS end-to-end on the box. Corpus grown to 26 with two fresh honest-refusal probes (double scalar, void in-place int array). +11 regression tests. (The broader "audit every one of 257 refusal sites" is deliberately scoped to benchmark-surfaced refusals — probes flush real gaps, speculation doesn't.) eng
  • P0.9 No gate may veto a differentially-proven-correct fn — the P0.11 "verified-function vs green-workspace" gap is FIXED (commit be7897b). Root cause found by running the converter (not the assumed traits-stub): the char* f(char*) cstr_out shape added in P0.8a was wired into the TDD test vectors but NOT into the three sites the FINAL differential gate uses — build_diff_config (→ no harness → config None → refuse), proptest_gen, adapter_gen. Plus a second miss: the extractor labels rot13 "cipher" (it is a substitution cipher) and the cipher/compression early-skip dropped it before the cstr_out branch — the semantic label must not starve a shape with a sound differential. Both fixed + 4 regression tests. Validated end-to-end on the box (Gemma :8086): to_upper / hex_encode / rot13 all FAIL→OVERALL PASS, differential genuinely runs. (Broader anti-stub-lint false-positive sweep folds into P0.8.) eng
  • P0.10 Stale-lock auto-reclamation — already implemented + tested (alchemist/workspace_lock.py reclaims dead-PID locks; test_stale_lock_reclaimed). Session pain was orphaned live processes (SSH-timeout leftovers), not a lock bug — an operational issue, not a code one eng
  • P0.11 Leaf-function benchmark suite — 23 unseen pure C leaf fns (bench/leaf/gen_corpus.py + run_leafbench.pyscorecard.json). First result (box, Gemma :8086): verified 21/23 = 91.3% · first-pass 21/23 = 91.3% · refusal 8.7%. By category: checksum 8/8, scalar 10/10, cstr 3/3, uncovered 0/2 (the 2 planted coverage gaps, honestly refused). Gap it surfaced (now FIXED in P0.9, commit be7897b): initially 21 verified but only 18/23 OVERALL PASS — the 3 single-fn cstr subjects verified byte-exact yet the FINAL differential gate refused them (cstr_out shape unwired past the TDD vectors). After the P0.9 fix a fresh cold run is 21/23 OVERALL PASS (checksum 8/8, scalar 10/10, cstr 3/3, uncovered 0/2). Also fixed a metric bug in the runner: first-pass used (iters or 99) which misscored a perfect iters==0 result — true first-pass is 21/23, not 2/23 infra
  • P0.12 Nightly CI runs the real pipeline+model — done (commit 9627a8d). bench/leaf/nightly.sh is the cron entry point: it checks the local model is reachable (a nightly "passing" against a dead model is a lie), runs the leaf benchmark end-to-end (the model fills every subject), and appends a timestamped scorecard summary to a history log OUTSIDE the repo (survives git reset --hard syncs) + optional ntfy. Installed as a box-local cron (Jesse's call — the model lives on the box): 0 4 * * * bash …/bench/leaf/nightly.sh # alchemist-nightly. Fail-loud on down-model/regression. Closes the standing "CI never runs the model" gap. infra
  • P0.13 Deterministic replay — done (commit e43e59a, implementer/tdd_generator.py_deterministic mode gated on ALCHEMIST_DETERMINISTIC). Found the non-determinism source: even the FIRST fill ran at temp 0.15 (and the multi-sample fan-out at 0.35). New ALCHEMIST_DETERMINISTIC env forces greedy decode (temp 0) at every model call in the fill loop AND collapses the multi-sample fan-out to a single greedy sample; the fuzz seed was already fixed. Empirically proven on box: count_char translated twice with the flag → BYTE-IDENTICAL emitted Rust (diff clean), both OVERALL PASS. Off by default (sampling finds more wins). +2 unit tests. Suite 767. infra
  • P0.14 Per-function timing & cost telemetry — done (commit bf1c1d0; implementer/tdd_generator.py captures the per-fill snapshot, reporter/refusal_ledger.py emits per-fn + rollup). FunctionAttempt now carries elapsed_s / llm_calls / output_tokens, captured per fill via self.llm.stats snapshot deltas around each _fill_in_function (tolerant of a shim without .stats). The refusal ledger emits them per function plus a subject roll-up (total_elapsed_s, total_llm_calls, total_output_tokens, slowest_fn, costliest_fn), and the pipeline prints a one-line spend summary. A cached-win restore reads as elapsed>0 / llm_calls==0 — the cheapest outcome, made visible. Validated on box: crc32b → telemetry: 0.1s, 0 LLM calls (cached restore). +1 rollup test. Suite 761. infra

P1 · Whole Small/Mid C Library — Push-Button (IN PROGRESS)

A never-seen C library → a verified Rust workspace, under 5% refusal, zero human touches. The first genuinely fundable claim. Exit (RE-REDEFINED 2026-07-14 — generality/convergence, not a lib count): ≥5 genuinely-unseen libraries converted FULLY hands-off, each ≥1000 LOC AND structurally hard (a parser/state-machine, or a pointer-linked + heap-allocation data structure — NOT a fat pile of independent leaf functions), each producing a byte-exact verified core + an honest, specific refusal tail (every refusal names what it couldn't prove and why), reproducible + signed-receiptedAND the crucial addition: new-human-fixes-per-library must trend to ~0 (the last 2 of the 5 need ≤1 touch each). Five libs banked via five rounds of hand-fixing proves we can hand-fix five libs; it does NOT prove the product. The convergence metric is the real exit — it is the difference between "we made 5 libraries work" and "the machine works on libraries." See the 2026-07-14 re-orientation banner above. The refusal rate is whatever it honestly is — the deliverable is a trustworthy verified core + a small hand-reviewable refused worklist, which is what c2rust (100% unsafe, unverified) cannot produce. Staged: prove the capability on jsmn (~340 LOC recursive parser — the unit test) first, then scale to the ≥1000-LOC libraries (the integration test). The old "5+ libs · refusal <5%" bar was met on 200-line primitives (the easy 80% of C) and is retired as insufficient for the fundable claim.

🎯 The real fundable claim (what P1 truly closes)

Point Alchemist at a real, messy, unseen C library, walk away, and every function comes back either byte-exact-verified against the original OR honestly, specifically refused with a reason — and the verified core is genuinely trustworthy because nothing shipped without the oracle agreeing. The verified-or-refused GUARANTEE is already always-on (fail-closed). What closes P1 is proving it, at useful COVERAGE, on ≥5 real ≥1000-LOC structurally-hard libraries.

The 3 keystones this requires (the capabilities the small-lib benchmark let us skip):

  1. Whole-program type coherence — ✅ DONE (P1.18, 2026-07-13) — one canonical Rust type per C struct across ALL functions; carry EVERY referenced struct. jsmn exposed it: jsmn_parser came out as both ParserState and Parser, so a caller couldn't share the struct between init and parse. inject_state_shared_types now does a subject-wide canonicalization pass (pick one Rust name per C struct, rewrite every signature to it, emit one shared type per struct). Validated: jsmn's jsmn_parser collapses to a single ParserState across init+parse.
  2. Parser-class oracle — ✅ DONE (P1.19, 2026-07-13) — differentially verify parse(input) → structured output: fuzz inputs (valid/malformed/truncated — real parsers must be safe on malformed), capture the C reference's token array AND signed return/error code, assert the Rust port matches byte-for-byte. New parse_seq shape in auto_config.py; validated model-free on real jsmn (captures {"a":1} → OBJECT/STRING/PRIMITIVE with exact spans, and {→PART / }→INVAL error returns). Unblocks jsmn / parson / http-parser.
  3. Ownership at library scale — ✅ DONE (P1.20, 2026-07-13) — a pointer field to a carried struct is LIFTED to owned safe Rust (T* next → Option<Box<T>> for a single owned child; T* items + a length scalar → Vec<T> for a counted array/DOM) instead of being dropped, plus the transitive closure of reachable node types is emitted, so a linked-list / tree / DOM library gets a coherent compilable skeleton. Validated model-free (inject_state_shared_types on a real self-referential linked list → next: Option<Box<Node>>). Raw scalar/char buffer pointers still drop (rc4/allocators depend on it).

Real-library gate status: 0 / 5 done — but all 3 capability keystones now land, so the pipeline CAN handle the parser + heap-structure classes. jsmn (in-flight unit test) is UNBLOCKED (type-coherent + parser oracle); the model-fills-to-byte-exact end-to-end run is a box run (needs the local model). Candidates: parson (2760, JSON DOM), http-parser (~2500, state machine), a real hash table (pointer structure), heatshrink (stateful stream + ring buffer), tinycbor (codec). What remains is a run campaign, not R&D.

🏃 Run campaign log (started 2026-07-13, on the box + live Gemma)

Lib 1 — jsmn (parser unit test): ✅ VERIFIED 1/2 — the first real "verified core + honest refusal tail" on an unseen parser. jsmn_init verified byte-exact (parser-oracle init observer); jsmn_parse honestly refused (the model can't yet translate the ~200-line INLINED recursive parser — its static helpers are filtered out, so it can't decompose leaf-first). Lib 2 — parson (2760-LOC JSON DOM, opaque-typedef + tagged-union): 🔨 driven through EIGHT skeleton blockers, one run per fix. Started at "won't even compile"; each cold run peeled one general converter bug, all committed+synced: lifetimes → poison macros → file-I/O filter → typedef aliases → return-type/tag coherence → union-drop+alloc-Box → enum-type→i32 → scalar-typedef coherence (50bdfd6). Errors shrank from ~100 (every getter) to single specific type issues. Blocker #8 (JsonValueType ×3 E0425): parson declares enum json_value_type {..} and typedef int JSON_Value_Type; SEPARATELY (not typedef enum {..} T), so the enum-typedef scan missed the alias and the extractor's unresolved Rust type name JsonValueType broke the crate. Fix generalized inject_state_shared_types to resolve BOTH scalar typedefs (collect_scalar_typedefs→base Rust scalar) AND enum typedefs (→i32) via one signature-rewrite; real carried structs (JsonValue) untouched. Validated model-free on a parson typedef replica + 27-test keystone regression. parson9 → blocker #9 (bare Box/undefined-type): cleared the JsonValueType wall and reached implement, but parson-core failed compile with 6 errors: 4× bare Box in fn signatures + 2 extractor-invented undefined error types (SerializationError, ValidationError). Fixed general (bed5d97): no_std modules now use alloc::{boxed::Box,vec::Vec,string::String} so bare collection types resolve in signatures/fields/bodies; and the skeleton emits a pub struct <Name>; placeholder for ANY type a module signature references but never defines (broad scan when the crate has no deps; conservative error-position heuristic otherwise) → the crate always compiles. parson10 (cached specs, bed5d97): parson-core COMPILES ✅ (API ok) — but 0/99 verified, 99 honest refusals, all reason "no verifiable test vectors." parson's entire PUBLIC API is tagged-union DOM ops (json_object_set_string, json_array_append_boolean, json_value_get_type) over a linked JSON tree — none classify into an oracle shape, and its only leaf logic (hex/number/string helpers) is static (filtered). Honest outcome = compiles + 100% fail-closed refusal, zero false claims, but NO verified core → does not yet count toward the ≥5. To give parson a verified core needs a DOM tree-construction differential oracle + union→enum (a dedicated multi-part keystone), OR static-leaf exposure (#160). Decision (Jesse): BUILD parson's DOM oracle now (general to all tagged-union/DOM/AST libs). Part 1 DONE (7c40150) — union flattening: an inline C union field (JSON_Value_Value value;) is lifted to a FLATTENED safe sub-struct (one field per member) instead of dropped — chosen over a Rust enum so the access path v.value.number is identical to C (model writes plain field access, no match/discriminant), fully safe (no union/unsafe). collect_union_typedefs + emit_safe_struct(union_structs=) + PASS-3 closure now emit JsonValue{parent,type:i32,value:JsonValueValue} and JsonValueValue{number:f64,boolean:i32,null:i32,object/array:Option<Box<..>>} (string deferred to Tier-2). 50-test struct-lift/skeleton regression green; JSONValue→JsonValue casing now coheres. Next: Part 2 — the construct-observe oracle shape (pair scalar constructors T* init_X(scalar) with scalar getters scalar get_X(const T*); mint input→observed differential vectors) so null/boolean/number constructors+getters actually classify and verify → parson's first verified core. Then box-validate + bank as lib #2. Lib 3 — http-parser (3024-LOC goto state machine): ⏳ PRE-FLIGHTED + staged. Vendored + model-free-analyzed; 0 unions / 0 file-I/O (dodges parson's worst), 45 gotos, 2 opaque typedefs (covered). Diagnosis showed almost nothing classified → built the missing shape ahead of time.

Campaign has surfaced + fixed ~12 GENERAL converter capabilities (each committed+tested+synced), grouped:

  • Make any real C compile as a coherent Rust skeleton: compile-isolation (a06ff5c, the linchpin — a refused fn no longer sinks its verified siblings); lifetime elision (312d6d8); whole-program type coherence — opaque-typedef aliases keyed, return-only structs emitted, tag/alias unified, enum-type→i32 (faa1154, 31c772d); union-drop + alloc-qualified Box/Vec (1bd5455); no_std-safe rust_body (e7a9483); invalid-escape repair (38943c5); poison-macro/dangling-const filter (5ff2ac3); file-I/O filter (57e4935).
  • New oracle shapes (family-general): parser-class parse_seq + init observer (d1f3358, c2f9e1a); str_lookupconst char* f(enum)→string, a universal C idiom (629d968).
  • Runner autonomy: cache-prune so filter changes take effect without a manual wipe (a1508b6).

The compounding is real: http-parser (a different hard class) needed a shape parson didn't — but building str_lookup once covers http-parser's lookups and parson's _get_type accessors and every library's enum-name idiom. Fix an idiom, not a file. Remaining levers: C-union → Rust-enum keystone (upgrades parson from union-dropped-partial to fuller verification; general to all tagged-union libs); extract static helper fns for leaf-first parser decomposition (unblocks jsmn_parse + parser cores). Then bank the ≥5 real-library splits (parson, http-parser, a hash table, heatshrink/tinycbor).

📏 Distance to P1 complete (estimate, 2026-07-13)

Under the REDEFINED bar (2026-07-13): the infrastructure gates are met; the real-library gate is the whole remaining job.

P1 exit gate (redefined) Status
Signed receipts (tamper-evident) DONEverify_receipt_integrity → True, → False after tamper (P1.14)
Reproducible (fresh-clone → same verified result) DONE — demonstrated on box (P1.14)
Verified-or-refused GUARANTEE (fail-closed) always-on — the architecture; every run
≥5 real ≥1000-LOC structurally-hard libs, hands-off, verified-core + honest refusals 🔴 0 / 5 — the real work. But ALL 3 capability keystones now land: #1 type coherence, #2 parser oracle, #3 ownership-at-scale. The easy-benchmark 4.8% and its 6 primitive passes are RETIRED as insufficient — they're the easy 80% of C.

Honest distance: the infra + guarantee are done, and all 3 hard capability builds now land (type coherence + parser oracle + ownership-at-scale) — the pipeline CAN now produce a coherent, compilable skeleton for the parser and heap-linked-structure classes, which is what blocked every real library. What remains for the fundable claim is the ≥5 real-library conversions — each a box run where the local model fills and the oracle verifies (or honestly refuses). This is a run campaign, not R&D: a multi-week compute campaign on the single box, staged jsmn → real libraries. Capability-complete (can the machine do it?) ≈ 95%; checkbox-complete = P1 11/20. We are NOT at "P1 done" (0/5 real libraries converted), and that's the correct, non-self-deceiving read — but the hard invention is behind us.

The expensive part — building the oracle SHAPES so the pipeline can handle each class of C library — is ~done (decoders, header-only, context-hashes, stateful sequences all landed this session). What remains is mostly a run campaign, not R&D:

  • ~6 library conversions (P1.4–P1.9): "run the converter + triage the occasional new gap." ~0.5–1.5 sessions each ≈ ~6 focused sessions. Risk: a parser-class subject (http-parser) may hit the same type-coherence wall jsmn did (P2.2), +1–2 sessions.
  • Confirmation runs (P1.1 zlib, P1.2 libcrc, P1.3): already-conquered libs, fresh-clone re-prove ≈ ~1 session.
  • Reproducibility + receipts (P1.14): stranger-clone run + doc ≈ ~1 session.
  • jsmn canonical-naming (P2.2), if the parser class is wanted fully clean ≈ ~1–2 sessions.

Bottom line: ~8–10 focused, mostly-unattended run-and-triage sessions — a ~1–2 week compute campaign on the single box, no expected major capability walls (all 3 keystones now land). Capability-complete (can the machine do it?) ≈ 95%; checkbox-complete = P1 11/20. The load-bearing items — the 3 capability keystones — are all behind us; what's left is running the converter on real libraries.

📉 Refusal trajectory (cold 8-lib run_libbench.py)

Run Overall refusal Per-lib
Baseline (start of 2026-07-12) 23.5% 4/8 pass; sha256 & jsmn = 0/0 invisible failures
Post fix-waves 1-2 23.8% base64 0/0→2/2, heap 0/2→2/2; sha256/jsmn still dragged
After P1.15/16/17 + sha256 lint (fcda670) 9.5% base64 2/2, sha256 0/4→3/4, heap 2/2, murmur3 7/7, hashkit 3/3, siphash 1/1, rc4 1/2, jsmn 0/0
✅ After rc4 filter fix (fcda670) 4.8% — UNDER TARGET base64 2/2, sha256 3/4, siphash 1/1, murmur3 7/7, rc4 1/2→2/2, hashkit 3/3, heap 2/2, jsmn 0/0

🎯 4.8% HITS the P1.13 <5% refusal exit criterion (20 verified / 1 refused). The single remaining refusal is sha256_transform — the 64-round compression core, a model-capability frontier (NOT an oracle gap; the whole SHA-256 digest is proven byte-exact via P1.17). rc4 2/2 confirmed the stateful-sequence filter fix. jsmn advanced 0/0 → 0/2 (2026-07-13, commit d0f2d64): the header-only (P1.16) + non-first struct-carry (Token) fixes now make jsmn ingest AND its skeleton compile (struct-carry emits 3 structs; no more "cannot find type Token") — so its 2 functions are now visibly attempted rather than a silent 0/0. The remaining refusal is a distinct, larger gap: jsmn_parse (int f(Parser*, &str, &mut [Token]) -> count) has no oracle shape (parse-string-into-token-array) — the parser-class oracle, shared with P1.8 http-parser, a next capability. Plus the ParserState-vs-Parser inconsistent naming (P2.2 canonical-naming). jsmn is now a visible, precisely-diagnosed target. This session drove the whole-library refusal metric 23.5% → 4.8% purely by closing oracle-shape gaps and gate/accounting false-positives — the model was never the bottleneck. (Note: the P1 exit also needs 5+ libs converted hands-off with published receipts — P1.1–P1.9 — and reproducibility; 4.8% clears the refusal-rate bar.)

Baseline (2026-07-12, bench/lib/run_libbench.py, 8 unseen libs cold): 4/8 OVERALL PASS (siphash, murmur3, rc4, hashkit), overall function-level refusal 23.5%. Batch surfaced TWO problem classes: (a) shape-coverage refusals (base64_decode binary-out, rc4_keystream, heap alloc/free); (b) whole libraries producing 0/0 functions — the dangerous one, invisible to the refusal metric. Both 0/0 cases (sha256, jsmn) were STRUCT-CARRY failures now FIXED: scalar typedefs (BYTE/WORD → u8/u32; commit 9513ed6) + enum typedefs (jsmntype_t → i32) + #ifdef-in-struct-body + Rust-keyword field names (typer#type; commit b0f0227). sha256's SHA256_CTX + jsmn's jsmntok_t/jsmn_parser now carry & compile cold (sha256 went total-failure → normal in-progress fill; the transform is just slow). Then closed a shape-coverage refusal: new buf_gen shape (<byteptr> f(<size> n, ...) -> Vec<u8>, commit a29c24a) — heap:make_buffer now verifies byte-exact on iter 1 (heap 0/2 → 1/2); generalizes to memset/pattern/PRNG fills. +8 regression tests.

Fix wave 2 (2026-07-12, commit 997e546, cold-VALIDATED on box): TWO whole-library unblocks landed + proven end-to-end. (1) free/destroy no-op template (free_noop_template, init_templates.py): a void free_X(ptr) whose C body is only deallocation calls is a no-op in safe Rust (owned Vec/Box drops) — accepted fail-closed (body-confirmed pure-free, else defer to model). heap:free_buffer now verifies → heap 1/2 → 2/2, OVERALL PASS, 0% refusal. (2) undefined-error placeholder (skeleton.py _lib_rs_for): the architect sometimes emits a trait method returning Result<_, SomeError> without defining SomeError, which broke the whole crate's compile → 0/0 functions filled. Skeleton now emits a minimal pub struct SomeError; + Display for any trait-referenced error type not defined/imported. base64 went 0/0 → 1/2: base64_encode verifies byte-exact; base64_decode still refuses (the binary-out decoder below). +7 regression tests, suite 777.

Fix wave 3 (2026-07-12, commits 07020fd→c8bff85, cold-VALIDATED): INVERSE-PAIR ROUNDTRIP ORACLE (P1.15) — base64 0/0 → 1/2 → 2/2 OVERALL PASS. base64_decode (char* f(char*)Result<Vec<u8>,E>) refused with "no verifiable test vectors" (random fuzz strings aren't valid base64). Now the decoder class is unblocked: when a decoder is the inverse of an oracle-able encoder in the same subject, mint valid inputs by running the C ENCODER on random ASCII plaintext and require decode(encode(p)) == p. base64_decode is the first char* binary-out decoder the pipeline verifies byte-exact. Generalizes to inflate/decompress/deserialize. Details in P1.15 below.

Post-fix aggregate (2026-07-12, fresh cold 8-lib batch on all fixes, run_libbench.py): OVERALL REFUSAL 23.8% — ~flat vs the 23.5% baseline, but the flatness is misleading: the wins are real and confirmed at scale (base64 2/2 0% ✅, heap 2/2 0% ✅, murmur3 7/7, hashkit 3/3, siphash 1/1), while the aggregate is dominated by the two hardest libraries, which are DIFFERENT problems than the ones fixed: sha256 0/4 (the 64-round transform is genuinely hard/slow to verify cold — needs a decomposition or longer budget, not a coverage fix) and jsmn 0/0 (header-only + recursive parser — see below). The refusal % is a blunt whole-library aggregate; per-capability the frontier moved (decoder class + header-only libs are new).

sha256 0/4 root-caused + partly fixed (2026-07-12, commit 06166a7): the 0/4 was NOT a timeout — it was a two-stage blocker. (1) Semantic-lint false-positive [FIXED]: lint_sha256 (semantic_lints.py) required the H0[0] constant 0x6a09e667 in EVERY sha256-family function, but it lives only in sha256_init; sha256_transform/update/final were refused on every iteration ("semantic lint rejected iter 1–5"). Scoped the check to skip state-consuming helpers (transform/update/final/compress/block); still enforced on init + monolithic. +2 tests. (2) Remaining blocker [FIXED via P1.17 — DONE]: with the lint clear, all 4 fns hit "no test vectors" — sha256 is a stateful context-hash sequence (init(ctx)update(ctx,data,len)final(ctx,out_digest) over the multi-field SHA256_CTX). The existing hash_seq oracle only handled a SINGLE-SCALAR state with a SCALAR-returning final (FNV). Built the digest-sequence oracle (P1.17, commit add0847) — multi-field ctx + final(ctx, out_digest) — which took sha256 0/4 → 3/4 verified with the whole-sequence digest differential byte-exact GREEN. Lone holdout sha256_transform (64-round core) is a model-capability frontier, not an oracle gap. Unblocks the whole SHA/MD5/SHA-1/HMAC class. See P1.17 below.

jsmn diagnosis + two header-only fixes (2026-07-12, commits dd97d92, 32a3df9): jsmn showed the dangerous 0/0 "invisible failure." Root cause was NOT struct-carry — it was a .c-only assumption in TWO stages: (1) module_detector._group_by_file grouped only .c files into modules, so a header-only library (jsmn, stb-style — whole impl in the .h) produced 0 modules; (2) spec_extractor._extract_module read function bodies only from .c, so even once detected the functions weren't read ("No functions found"). Both fixed (the parser only records definitions, so a .h with a non-empty functions list is a source module; +6 tests). jsmn now passes analyze → extract → architect cleanly (6 fns detected, 2 algorithms after static-helper filtering, architecture validated 0 errors) — the invisible 0/0 dead-end is eliminated. Next jsmn blocker (root-caused, deferred — P2-class): the skeleton fails to compile with cannot find type Token. Precise cause: struct_lift.inject_state_shared_types only carries the params[0] STATE struct (jsmn_parserParserState, keyed on the spec's first-param lift, struct_lift.py:355). jsmntok_t appears as a secondary parameter — the token-array element type (tokens: &mut [Token]) — which struct-carry never emits, and the architect independently renamed it Token, so the reference dangles. The general fix is to carry EVERY struct type referenced in ANY function signature (not just params[0]) under the name the spec/architect uses — a real whole-program-type enhancement (P2). Only THEN does the recursive-descent parser itself have to verify (hardest, research). jsmn is now a visible, root-caused in-progress target, not a silent failure.

Remaining refusal worklist: rc4_keystream — verified via the cipher-sequence gate but the per-fn ledger marks it refused (accounting false-negative; needs sequence-member crediting).

  • P1.1 zlib full workspace green on a FRESH clone (no cached wins) model
  • P1.2 libcrc all-green cold (9/9) from scratch model
  • [~] P1.3 SHA-256 / siphash / base64 / murmur3 cold-green — 3 of 4 fully cold-green in the 8-lib benchmark (2026-07-13): siphash 1/1, base64 2/2, murmur3 7/7, all OVERALL PASS from scratch. sha256 is 3/4 (init/update/final verify + whole-sequence digest differential byte-exact; the lone holdout sha256_transform 64-round core is model-hard). Effectively met bar the one model-hard function. model
  • P1.4 parson (JSON parser) end-to-end model
  • P1.5 tinycbor / a CBOR codec model
  • P1.6 A small container lib (uthash-style) — pointer-heavy research
  • P1.7 A compression codec (heatshrink / miniz) model
  • P1.8 A protocol parser (http-parser subset) — goto-heavy state machine research
  • P1.9 A crypto primitive lib (monocypher subset) — constant-time-sensitive model
  • P1.10 Unattended batch runner — N libs, walk away, collect scorecards — bench/lib/run_libbench.py (commit c235111): cold-runs a curated lib set, aggregates per-lib + overall function-level refusal + a triage worklist infra
  • P1.11 Build-system discovery robustness — already covered by WALL-4 (verifier/build_c_dll.py::build_c_dll + discover_c_build): non-lib-dir exclusion, amalgamation detection, main() filtering, make/cmake prepare_native_build eng
  • P1.12 Dependency-ordered fill at 50–150 fns — covered by P2-I (implementer/tdd_generator.py::_topo_sort_algorithms orders per-module fills leaf-first) eng
  • P1.13 Refusal <5% exit criterion met on an unseen lib — ✅ MET: 4.8% (20 verified / 1 refused) on the cold 8-lib run_libbench.py set (2026-07-13, commit fcda670). Down from a 23.5% baseline; the lone refusal is sha256_transform (a model-capability frontier, not an oracle gap). Achieved purely by closing oracle-shape/gate/accounting gaps this session (P1.15 decoder roundtrip, P1.16 header-only, P1.17 digest-sequence, sha256 lint false-positive, stateful-sequence fill-filter). See the trajectory table above. (The broader P1 exit — 5+ libs converted hands-off with published receipts — is tracked by P1.1–P1.9; this clears the refusal-rate bar.) model
  • P1.17 Digest-sequence oracle — unblocks the CONTEXT-HASH class (SHA-256/SHA-1/SHA-512/MD5/HMAC). DONE + cold-VALIDATED on box: sha256 0/4 → 3/4 verified, and the whole-sequence digest DIFFERENTIAL is byte-exact GREEN (gate 5: sha256_final_digest_matches_c_reference ... ok, 2000 fuzzed inputs 0..512B, Rust init→update→final == compiled C). A decomposed context hash is init(ctx) + update(ctx, data, len) + final(ctx, out_digest) over a MULTI-FIELD context struct, with final writing an N-byte digest into an out-buffer — the existing hash_seq only covered a single-scalar state + scalar-returning final (FNV), so all 4 sha256 fns hit "no test vectors". New hash_digest_seq shape (commits 3f8… → add0847), modeled on cipher_seq's multi-field FFI struct-carry. Architecture / 6 sites: (1) classify_hash_digest_sequence (auto_config.py) — detects the triple(+transform) over a multi-field pointer-free ctx; flexible byte-buffer matchers _is_const_byte_buf/_is_mut_byte_buf accept the raw C array-declarator forms (const BYTE data[], BYTE hash[], BYTE typedef unresolved in sig params); _digest_len_from_specs reads N from final's &mut [u8; N] lift. (2) fuzz_hash_digest_sequence_vectors — drives the compiled-C sequence and authors self-contained rust_body tests: init/update/transform verified by POST-STATE (every ctx field vs C), final by the composed digest. Key design choice: per-function post-state (not one composed test) respects leaf-first fill order — each fn verifies once its deps are filled — and rust_body is self-contained so it needs zero new fill-loop emitter. (3) group passes in synthesize_c_vectors. (4) build_diff_config hash_digest_seq harness on final (emits the repr-C ctx via emit_ffi_struct). (5) adapter_genrust_/c_ wrappers that run the whole sequence and return [u8; N] (C side via FFI). (6) proptest_gen _proptest_hash_digest_seq_block — fuzz the message, compare the N-byte digest. Cross-checked: minted digests == hashlib.sha256 for all vectors. Two cold-iteration bugs fixed: (a) gate 5 first refused "no differential config provided" → added sites 4/5/6; (b) E0308 — the FFI binding declares the byte-buffer params as *const/*mut c_void (BYTE unresolved), so the C wrapper needed as *const _/as *mut _ casts. +6 shape tests, suite 803. Lone holdout sha256_transform (the 64-round compression core) fails its standalone post-state test across 5 iters + holistic — a genuine model-capability frontier (exactly the SHA-256 difficulty F.4/P2.5 needed decomposition levers for), NOT an oracle gap: init/update/final verify independently and the whole-sequence digest is proven byte-exact. eng
  • P1.16 Header-only C library ingestion — a lib whose whole implementation lives in a .h (jsmn, stb-style single-header libs) previously produced 0 modules → 0/0 (silent invisible failure). Fixed the .c-only assumption at both sites it appeared: analyzer/module_detector._group_by_file (a .h with function DEFINITIONS is now a source module; kept in headers too so its struct/typedef info still merges into any sibling .c; self-association guarded to avoid duplicating its own structs) and extractor/spec_extractor._extract_module (reads bodies from .c AND .h/.hpp/.cc/.cpp/...; safe — the parser records only definitions and the module-membership guard keeps us to the module's own functions). jsmn now passes analyze→extract→architect clean (6 fns detected, architecture validated 0 errors; commits dd97d92, 32a3df9; +6 tests). Full jsmn verify is gated on a P2-class architect↔struct-carry type-rename coordination issue (the architect renames jsmntok_t → Token but struct-carry emits the original name) + recursive-parser verification — see the P1 note above. eng
  • P1.15 Inverse-pair roundtrip oracle — unblocks the DECODER class (base64_decode, inflate, decompress, deserialize). DONE + cold-VALIDATED on box: base64 0/0 → 1/2 → 2/2 OVERALL PASS (0% refusal); base64_decode is the first char* binary-out decoder the pipeline verifies byte-exact through all 5 gates. When a char* f(char*) decoder is the declared inverse of an oracle-able encoder in the same subject (_paired_encoder_name table: decode↔encode, decompress↔compress, inflate↔deflate, …), mint valid inputs by running the compiled C ENCODER on random plaintext and require decode(encode(p)) == p — the encoder is the C reference, so it's a sound differential; the lossy C decoder is never called. 7 sites (commits 07020fd→c8bff85): classify_cstr_roundtrip + fuzz_cstr_roundtrip_vectors + vector-synth dispatch + build_diff_config harness (all BEFORE cstr_out) in auto_config.py; cstr_roundtrip adapter (Rust decoder→Vec + uniquely-named c_<decoder>_enc encoder wrapper) in adapter_gen.py; _proptest_cstr_roundtrip_block (mint-via-encoder, assert identity) in proptest_gen.py; _emit_roundtrip_test (.unwrap()-based, no E: PartialEq) + _roundtrip_ fill-loop test filter in tdd_generator/test_generator. Four real bugs found & fixed via cold iteration: (a) tdd_generator's own backfill re-mints vectors and had no roundtrip branch → wired it in; (b) _test_filters_for_fn omitted _roundtrip_ → fill loop ran 0 tests → false "no test vectors"; (c) encoder-failure (NULL/empty output) minted bogus decode("")==<bytes> vectors → skip them; (d) base64's C encoder indexes its table by a SIGNED char → UB on plaintext bytes ≥128 → garbage output a memory-safe decoder can't match → restrict minted plaintext AND the proptest strategy to ASCII (1..127). +12 regression tests (test_cstr_roundtrip_shape.py). eng
  • P1.14 Cold-start reproducibility + signed receipt — ✅ DONE + demonstrated on box (2026-07-13, commit 9971964). A genuine fresh git cloneALCHEMIST_DETERMINISTIC=1 run of base64 → OVERALL PASS 2/2 + a content-SHA256-sealed receipt whose verify_receipt_integrity returns True, and False after tampering (flip overall_passed). Reproducibility is in the RESULT sense: two independent deterministic runs + the fresh-clone run all reproduce the verified PASS. Note (honest): bit-identical Rust is NOT reproducible — greedy decoding on the vLLM GPU isn't bit-deterministic (non-associative float reductions), so base64_encode came out as String+char in one run and Vec<u8>+b'=' in another, both byte-exact-vs-C correct. That is a feature of this architecture: the differential oracle guarantees correctness regardless of which valid implementation the model emits, so a stranger always gets a verified-correct workspace + a tamper-evident receipt. Receipt engine: verifier/receipt.py (content-SHA256 + optional HMAC via ALCHEMIST_RECEIPT_KEY); determinism: ALCHEMIST_DETERMINISTIC (P0.13). infra
  • P1.18 Whole-program type coherence — KEYSTONE #1, ✅ DONE (2026-07-13). One canonical Rust type per C struct across ALL functions, so a caller can init a value in one function and pass it to another. The bug (jsmn): the extractor infers a Rust type per parameter independently, so the SAME C jsmn_parser * surfaced as &mut Parser in jsmn_init and &mut ParserState in jsmn_parse — struct-carry then emitted BOTH Parser and ParserState, and no workspace with init()->Parser feeding parse(&mut ParserState) can ever link → the whole library yields 0 verified functions. The fix (verifier/struct_lift.py::inject_state_shared_types, rewritten): a subject-wide 3-pass canonicalization. PASS 1 walks every module/function and maps each C struct → Counter of the Rust names the extractor gave it, matching each Rust input to its C sig-param BY NAME (positional alignment breaks when the extractor folds length params) via the new _c_struct_of helper (skips single-scalar structs, which correctly unwrap to a bare primitive). Canonical name = most-frequent, tie-break longer-then-alphabetically-later (deterministic; jsmn_parser → ParserState). PASS 2 rewrites every alg.input.rust_type alias → canonical with a word-boundary re.sub. PASS 3 emits exactly ONE SharedType per canonical struct each module references. Safe for existing subjects: a consistently-named struct (rc4/sha256) has canonical == its single existing name, so nothing changes. Validated model-free & deterministically (tests/test_canonical_struct_naming.py, drives the real subjects/jsmn C source with hand-built specs reproducing the split): jsmn's jsmn_parser collapses to a single ParserState across init+parse, jsmntok_t → Token (a non-first/slice-element param) is still carried, and a consistent-naming control is left untouched. This is the prerequisite keystone — no multi-function real library links without it. Remaining jsmn blocker is now keystone #2 (parser-class oracle). eng
  • P1.19 Parser-class oracle — KEYSTONE #2, ✅ DONE + validated model-free on real jsmn (2026-07-13). Differentially verify a parse(input) → structured token output function against the compiled C reference on FUZZED inputs (valid + malformed + truncated). No prior oracle covered this shape: the "output" is a variable-length array of small structs PLUS a signed return code (token count / negative error), not a scalar or byte buffer — so jsmn (and the recursive-descent core of parson / http-parser) could never be verified. New parse_seq shape (verifier/auto_config.py): classify_parse_sequence detects init(S*) + parse(S*, const char* in, <int len>, Tok* out, <uint max>) → int sharing a state struct S where out points at a token struct; fuzz_parse_sequence_vectors drives the compiled C on a curated JSON + malformed + random-ASCII corpus and authors self-contained rust_body tests asserting the Rust parse's RETURN code AND every filled token's fields vs C. Wired into synthesize_c_vectors (attaches to the parse fn; the _body_ fill-filter already covers rust_body vectors, and untagged authored vectors survive the tdd backfill — same path hash_digest_seq uses). Two real subtleties solved: (1) ifdef-gated fields — jsmntok's parent sits inside #ifdef JSMN_PARENT_LINKS, absent from the DEFAULT-macro compiled layout; a naive ctypes overlay would misalign every read. _ifdef_guarded_fields is scoped to the struct body (_extract_struct_body, brace-matched) so the file-wide #ifndef JSMN_HEADER guard around the impl can't mask the real fields — it correctly drops only parent, keeping type/start/end/size. (2) length-folding — the lifted Rust signature folds char*+len → &str/&[u8] and Tok*+num_tokens → &mut [Token]; _parse_rust_arglist reconstructs the call from the spec's actual (possibly folded) inputs and handles both i32 and Result<_,E> returns + keyword-sanitized field access (type → r#type). Validated (tools/jsmn_parse_oracle_check.py + tests/test_parse_sequence_shape.py): classifies jsmn's init+parse; captures {"a":1} → 3 tokens OBJECT(0..7,sz1)/STRING(2..3)/PRIMITIVE(5..6) byte-exact; faithfully captures C's error returns ({→PART −3, }→INVAL −2); emits 64 well-formed differential vectors. jsmn is now UNBLOCKED (type-coherent + parser oracle); the model-fills-jsmn_parse-to-byte-exact end-to-end is a box run (needs the local model). eng
  • P1.20 Ownership at library scale — KEYSTONE #3, ✅ DONE + validated model-free (2026-07-13). A pointer-linked / heap-allocating C data structure (linked list, tree, JSON DOM) could never get a compilable Rust skeleton because struct-carry DROPPED every pointer field — so struct node { struct node *next; } lost its next and no list/tree/parson-class library could link. The fix (verifier/struct_lift.py::emit_safe_struct, rewritten): a pointer field to ANOTHER CARRIED struct is now LIFTED to an owned safe form — T* next → Option<Box<T>> (single owned child: linked-list next, binary-tree left/right; Box breaks the infinite-size self-reference), or Vec<T> when the field reads as a counted array (items/children/entries/… name + a length scalar like count/size/len) — the JSON-array / DOM case. Raw scalar/char buffer pointers (no carried-struct pointee) are still DROPPED (rc4/allocators rely on scalar-offset state not observing them), and the lift is opt-in via a c_to_rust map so existing callers can't regress. inject_state_shared_types (keystone #1's pass) now also emits the TRANSITIVE CLOSURE of structs reachable through owned pointer fields, so every node type an owned link references is defined and the whole structure compiles cold. Validated (tests/test_owned_pointer_lift.py): self-ref nodenext: Option<Box<Node>>; binary tree → two Option<Box<TNode>>; counted array → Vec<Jval>; raw buffer still dropped; opt-in (no map → dropped, unchanged); and inject_state_shared_types over a real self-referential linked-list .c emits Node { val: i32, next: Option<Box<Node>> }. Struct-carry regression (rc4/sha256/jsmntok, all pointer-free) unaffected. With this, all 3 P1 capability keystones land — the pipeline can produce a coherent skeleton for the parser and heap-structure classes; what remains is the real-library run campaign. eng

P2 · Scale — Large Single C Codebase (MID)

A 30k–100k-LOC C project (SQLite core, lwIP) → a verified Rust workspace over a long, resumable, unattended run. Exit: SQLite (or lwIP) core → verified workspace, honest refusals, resumable across days.

  • P2.1 Call-graph partitioning + bottom-up scheduling across 1000s of fns eng
  • P2.2 Cross-module type universe at 1000s of types eng
  • P2.3 Pipeline throughput & memory profiling at scale eng
  • P2.4 Parallel fill fleet with backpressure + transient-error isolation eng
  • P2.5 Persistent run journal (SQLite-backed) infra
  • P2.6 Multi-day resumability + crash recovery eng
  • P2.7 Deterministic large runs infra
  • P2.8 1000-function progress dashboard infra
  • P2.9 Coverage & unverifiable ledger at scale infra
  • P2.10 SQLite core milestone model
  • P2.11 lwIP (network stack) milestone research
  • P2.12 Hybrid C/Rust link-back at scale eng

P3 · C++ Frontier (FAR)

Prove the C++ capability stack (built in SEM) on real targets. ArduPilot is majority C++; the pipeline is C-first today. Exit: a header-only C++ lib + a small C++ project (classes, templates, RAII) → verified idiomatic Rust.

  • P3.1 C++ tree-sitter analyzer + TU model eng
  • P3.2 C++ reference-oracle build + mangling-aware FFI eng
  • P3.3 Header-only C++ library milestone research
  • P3.4 A small class-based C++ project research
  • P3.5 A mid C++ library with templates + STL research
  • P3.6 Method-call differential harness eng
  • P3.7 C++ refusal taxonomy proven (multiple inheritance, heavy TMP flagged) eng
  • P3.8 C++ idiomaticity bar — traits/generics/ownership, not transliterated C++ research

P4 · Embedded & the Unsafe Boundary (FAR)

Where "perfect safe Rust" meets physics. Draw the unsafe boundary in exactly the right place and audit it. Exit: a real driver → safe algorithmic Rust + a thin, audited, clearly-marked unsafe HAL shim, verified at the boundary.

  • P4.1 no_std translation path eng
  • P4.2 Safe-vs-irreducibly-unsafe classifier unsafe
  • P4.3 A real embedded driver end-to-end (IMU/sensor) unsafe
  • P4.4 Boundary verification — safe-core differential + shim contract tests eng
  • P4.5 Unsafe-audit report — every block justified, minimized, reviewed unsafe
  • P4.6 RTOS abstraction shim milestone (ChibiOS/NuttX primitives → traits) unsafe
  • P4.7 Static-allocation / no-heap path proven eng
  • P4.8 Timing-preservation validation unsafe

P5 · ArduPilot (FRONTIER)

The dream target, in flyable slices. Algorithmic layers first (verified), then the boundary, with SITL as the behavioral oracle. Exit: AP_Math / filters / CRC verified byte-exact · a control subset behaviorally matched in SITL · a flyable-in-simulation Rust slice.

  • P5.1 Ingest the ArduPilot build (waf) + board configs eng
  • P5.2 Cross-library dependency graph eng
  • P5.3 Feature / board conditional resolution research
  • P5.4 AP_Math → verified Rust (vectors, matrices, quaternions) model
  • P5.5 Filters & EKF math kernels research
  • P5.6 AP_CRC / checksums (extend ardupilot_crc_verified) eng
  • P5.7 Coordinate / geodesy math model
  • P5.8 AP_HAL utility / logic (non-hardware) eng
  • P5.9 Control math (PID, attitude/position) research
  • P5.10 Embedded Lua scripting engine (ArduPilot embeds Lua — F.12 lands here) model
  • P5.11 MAVLink → Rust research
  • P5.12 Parameter system → Rust eng
  • P5.13 DataFlash / logging format eng
  • P5.14 Sensor drivers (IMU/baro/GPS) — safe logic + audited unsafe shim unsafe
  • P5.15 Bus drivers (I2C/SPI/UART) — audited unsafe unsafe
  • P5.16 Scheduler / real-time loop — timing-preserving unsafe
  • P5.17 SITL as the behavioral oracle — diff flight logs across scenarios research
  • P5.18 Flight-envelope scenario corpus infra
  • P5.19 Incremental fly-in-SITL milestones research
  • P5.20 Full-vehicle subset flyable on Rust research
  • P5.21 HIL / bench validation unsafe
  • P5.22 Safety / certification-mindset audit of the unsafe boundary unsafe

P6 · Autonomy — Point & Walk Away (CONTINUOUS)

alchemist translate <repo> runs unattended for hours→days and returns verified Rust + an honest refusal report. Exit: one command · unattended · resumable · verified workspace + a per-subsystem "% verified / % refused / why" report you can trust.

  • P6.1 Long-run orchestrator (hours→days, resumable, crash-safe) eng
  • P6.2 Refusal queue + auto-escalation eng
  • P6.3 Human-in-the-loop patch UX eng
  • P6.4 Live repo-scale dashboard infra
  • P6.5 Model-routing tiers (cheap→strong by difficulty) infra
  • P6.6 Distributed fill fleet eng
  • P6.7 Global regression vault infra
  • P6.8 Repo-scale signed receipts infra
  • P6.9 One-command UX + docs eng
  • P6.10 Coverage & honesty report (% verified / % refused / why) infra
  • P6.11 Continuous re-verification on upstream drift eng
  • P6.12 Buildable, tested output workspace eng

Track · SEM — C/C++ Semantics Coverage

Every C/C++ construct gets a sound, verified translation strategy — or an honest refusal. The correctness-completeness backbone. Exit: no construct silently mistranslated. Aliasing, unions, and UB respected, not ignored.

Memory & pointers

  • SEM.1 Pointer provenance & aliasing model — C's unrestricted aliasing → Rust's borrow model (the #1 hard problem) research
  • SEM.2 restrict/noalias exploitation → non-overlapping slice APIs research
  • SEM.3 Pointer arithmetic → slice/index or raw-ptr-in-unsafe with bounds proof research
  • SEM.4 Out-of-bounds-by-design (one-past-end, sentinels) → safe iterators/slices research
  • SEM.5 Multiple mutable aliases / shared mutable state → Cell/RefCell/split-borrow/indices research
  • SEM.6 container_of / offsetof / intrusive structures → safe ownership research
  • SEM.7 Tagged / low-bit-stashed pointers → safe encodings research
  • SEM.8 Null-pointer semantics → Option<&T>/Option eng
  • SEM.9 void* generic pointers → generics/enums/Box research

Memory model & UB

  • SEM.10 Uninitialized memory reads → MaybeUninit or refuse eng
  • SEM.11 Type punning via union → safe tagged enum / transmute-with-proof research
  • SEM.12 Type punning via memcpy/reinterpret → from_ne_bytes/to_ne_bytes eng
  • SEM.13 Strict-aliasing violations in real C → detect + preserve behavior research
  • SEM.14 Signed-overflow UB → wrapping/checked/saturating by intent eng
  • SEM.15 Shift/division UB (shift≥width, INT_MIN/-1) → guarded ops eng
  • SEM.16 Integer promotion & implicit conversion → explicit value-preserving casts eng
  • SEM.17 sizeof/offsetof/alignment/#[repr(C)] layout fidelity eng
  • SEM.18 Endianness-dependent code → explicit be/le ops eng
  • SEM.19 Packed structs & unaligned access → #[repr(packed)] + read_unaligned eng
  • SEM.20 Flexible array members → slice-tail / DST research
  • SEM.21 Anonymous structs/unions → nested Rust types eng
  • SEM.22 Bitfields → generated accessors, byte-exact eng

Control flow & functions

  • SEM.23 Computed goto / labels-as-values → state-machine lowering research
  • SEM.24 setjmp/longjmp → structured control flow / Result or refuse research
  • SEM.25 Variadic functions → refuse-or-shim unsafe
  • SEM.26 Function pointers & typedef'd signatures → fn ptr/dyn/enum research
  • SEM.27 Recursion & deep call graphs — stack-safety analysis eng
  • SEM.28 Comma operator / sequence points / eval order preserved eng

Preprocessor

  • SEM.29 Object-like & function-like macros → const/fn/inline, expansion-faithful research
  • SEM.30 Token pasting (##) & stringization (#) research
  • SEM.31 X-macros / macro-generated code → expand-then-translate research
  • SEM.32 Conditional compilation matrix (#if/#ifdef) research
  • SEM.33 Include-graph & translation-unit modeling eng

Qualifiers, atomics, concurrency, platform

  • SEM.34 volatile → read_volatile/write_volatile (MMIO) unsafe
  • SEM.35 _Atomic + memory ordering → std::sync::atomic with correct Ordering research
  • SEM.36 C11 threads/pthreads → std::thread + sync primitives research
  • SEM.37 Thread-local storage → thread_local! eng
  • SEM.38 Data-race detection & safe mapping research
  • SEM.39 libc calls → std/libc mapping eng
  • SEM.40 errno / signals research
  • SEM.41 attribute handling (packed, aligned, noreturn, weak, section) eng
  • SEM.42 Compiler builtins & intrinsics (_builtin*, SIMD) research
  • SEM.43 Inline assembly → refuse + flag unsafe
  • SEM.44 K&R / old-style declarations eng

C++

  • SEM.45 class → struct + impl eng
  • SEM.46 Access control → visibility eng
  • SEM.47 Single inheritance → composition + trait research
  • SEM.48 Multiple inheritance → refuse/flag unsafe
  • SEM.49 Virtual inheritance → refuse/flag unsafe
  • SEM.50 Virtual dispatch → dyn Trait / enum-dispatch research
  • SEM.51 vtable layout fidelity (for FFI) research
  • SEM.52 Abstract base → trait eng
  • SEM.53 Constructors → assoc fns / builders eng
  • SEM.54 Destructors / RAII → Drop eng
  • SEM.55 Operator overloading → std::ops eng
  • SEM.56 Copy/move semantics → Clone/move research
  • SEM.57 References & const& → borrows research
  • SEM.58 Templates → generics + bounds research
  • SEM.59 Template specialization → impls/where research
  • SEM.60 Non-type template params → const generics eng
  • SEM.61 SFINAE / concepts → trait bounds research
  • SEM.62 Template metaprogramming → refuse/flag unsafe
  • SEM.63 STL containers → std eng
  • SEM.64 Iterators → Rust iterators eng
  • SEM.65 Smart pointers → Box/Rc/Arc research
  • SEM.66 Exceptions → Result/panic policy research
  • SEM.67 Namespaces → modules eng
  • SEM.68 Name mangling & overload resolution research
  • SEM.69 Lambdas/closures → Rust closures eng
  • SEM.70 constexpr/consteval → const fn research

Track · VER — Verification & Formal Methods

From differential testing toward equivalence proof. Every function earns a sound gate — or an honest refusal. Exit: zero unverified emissions · each fn tagged with its proof method · cyclic cores verifiable · the oracle itself audited.

  • [~] VER.1 Byte-exact FFI differential (harden/generalize any signature) engcstr_out shape ADDED (P0.8a, commits 189648c+69a3b08): char* f(char*) text transforms now synthesize C-reference vectors + str_exact tests; also fixed the fill-loop filter that dropped _str_/_body_ test schemes. base64: 100%→50% refusal, base64_encode byte-exact verified. Remaining: binary-out char* f(char*) (base64_decode → Result<Vec<u8>>) — needs roundtrip-mint or a byte-return (length-aware) adapter, since the C char* return is NUL-lossy for binary.
  • [~] VER.2 Whole-program e2e observable differential — wired; now exercise widely eng
  • VER.3 Stateful sequence differential (generalize state_mutator) eng
  • VER.4 Coverage-guided fuzz-vector generation eng
  • VER.5 Sanitizer-diff verdict engine — wire sanitizer_diff (C-buggy classification) wire
  • VER.6 Miri UB-freedom gate at scale eng
  • VER.7 Property/roundtrip test synthesis eng
  • VER.8 Metamorphic relations for reference-less code research
  • VER.9 KAT/standards catalog (FIPS/RFC) auto-lookup infra
  • VER.10 Translation validation (bounded equivalence) research
  • VER.11 Kani (bounded model checking) integration research
  • VER.12 Deductive proof (Prusti/Creusot/Verus) research
  • VER.13 Aeneas-style functional translation validation research
  • VER.14 Optimization-invariance equivalence research
  • VER.15 Float/fixed-point differential (tolerance + bit-exact) research
  • VER.16 Concurrency verification (loom) research
  • VER.17 Side-channel / constant-time preservation research
  • VER.18 Oracle provenance & signed receipts (extend signet) infra
  • VER.19 Callback / higher-order-fn differential research
  • VER.20 Unverifiable taxonomy + refusal policy eng
  • VER.21 Verification cost model (rigor scaled to risk) infra
  • VER.22 Mutation-test the ORACLE itself (is the gate sound?) research
  • VER.23 Regression vault — re-verify every conquered fn infra
  • VER.24 Per-fn cross-domain coverage report infra
  • VER.25 "Divergence = a proven C bug" write-up research

Track · MODEL — Model & Generation

Get the most from a local model — and close the self-improvement loop. The model isn't the bottleneck; the harness is. US-origin models only. Exit: high first-pass fill rate via better prompting, retrieval, repair, and a fine-tune on the tool's own verified conversions.

  • MODEL.1 Spec-first chain-of-thought (refine) model
  • MODEL.2 Best-of-N + self-consistency (tune n/temp/selection) model
  • MODEL.3 Verification-in-the-loop generation (feed exact error back) eng
  • MODEL.4 RAG over prior VERIFIED conversions research
  • MODEL.5 Self-training corpus harvest (every verified conversion is a pair) research
  • MODEL.6 Fine-tune Gemma-4-31B on the verified corpus (US-origin) research
  • MODEL.7 Distill a specialized translation model research
  • MODEL.8 Grammar-constrained decoding to valid Rust research
  • [~] MODEL.9 Context management for large fns (32k overflow hit the architect this session) eng
  • MODEL.10 Whole-module / long-context prompting eng
  • MODEL.11 Model eval harness (per-model success on the benchmark) infra
  • MODEL.12 Model routing by difficulty infra
  • MODEL.13 Multi-model ensemble/vote (Gemma, Llama-4) research
  • MODEL.14 Prompt-library versioning + A/B infra
  • MODEL.15 Reference-C + macro injection (harden) eng
  • MODEL.16 Decomposition prompting for hard fns (extend structural_decomp) eng
  • MODEL.17 Mechanical-mistake prompt hardening (pairs with cargo-fix lever) model
  • MODEL.18 Idiomaticity prompting (non-regressive) model
  • MODEL.19 Cost/latency budgeting per fn infra
  • MODEL.20 Local-serving reliability (vLLM :8086, not dead :8090; failover) infra
  • MODEL.21 Determinism controls (temp=0 replay path) infra

Track · IDIOM — Idiomaticity & Quality

The Rust must be genuinely good — idiomatic, clippy-clean, ergonomic — not a transliteration. Verified-preserving, always. Exit: output a Rust engineer would accept in review, correctness never traded for style.

  • IDIOM.1 Transliteration → idiomatic pass (verified-preserving) eng
  • IDIOM.2 Raw index loops → iterators eng
  • IDIOM.3 Sentinel/null returns → Option/Result eng
  • IDIOM.4 Manual buffers → slices/Vec/arrays eng
  • IDIOM.5 Error codes → Result + error enums eng
  • IDIOM.6 clippy-clean output gate eng
  • IDIOM.7 rustfmt-clean output eng
  • IDIOM.8 Idiomatic ownership inference (borrow vs own) research
  • IDIOM.9 Doc-comment carry/generation eng
  • IDIOM.10 Ergonomic public API design research
  • IDIOM.11 Module/crate structure quality eng
  • IDIOM.12 Idiomaticity scored & regression-gated infra

Track · PERF — Performance Parity

The Rust must match or beat the C. A slower translation is a rejected translation. Exit: every conquered subject benchmarked; Rust ≥ C on time/size/allocation, with a regression gate.

  • PERF.1 Perf-parity gate threaded into the receipt (perf.py exists) wire
  • PERF.2 Benchmark suite per subject infra
  • PERF.3 Zero-cost-abstraction verification eng
  • PERF.4 Bounds-check elision where provably safe research
  • PERF.5 Binary-size parity eng
  • PERF.6 Allocation-behavior parity eng
  • PERF.7 SIMD / vectorization preservation research
  • PERF.8 Perf-regression gate across runs infra

Track · INFRA — Infrastructure, Eval & Product

The harness, the metrics, the trust story, and the path to funding. Exit: a public benchmark + leaderboard, reproducible signed receipts, one-command UX, a flagship dogfood demo, and a clear DARPA-TRACTOR/SBIR posture.

  • INFRA.1 The eval harness / leaderboard infra
  • INFRA.2 Public unseen-library benchmark set infra
  • INFRA.3 CI runs the real pipeline+model nightly infra
  • INFRA.4 Reproducible runs (pinned toolchains, seeds, model versions) infra
  • INFRA.5 Product-grade signed conversion receipts infra
  • INFRA.6 One-command UX (alchemist translate <path>) eng
  • INFRA.7 Repo-scale live dashboard infra
  • INFRA.8 Resumable/crash-safe long runs (journal) eng
  • INFRA.9 Distributed fill fleet eng
  • INFRA.10 Cost accounting & budgets infra
  • INFRA.11 Coverage/honesty report (% verified/refused/why) infra
  • INFRA.12 Documentation site + user guide infra
  • INFRA.13 Buildable, tested output workspace eng
  • INFRA.14 Converter supply-chain security infra
  • INFRA.15 DARPA TRACTOR alignment & positioning product
  • INFRA.16 SBIR / dual-use framing (NV013 lineage) product
  • INFRA.17 Flagship dogfood demo (convert a recognizable dependency) eng
  • INFRA.18 Open-source posture & licensing (Apache-2.0) product
  • INFRA.19 Publish the verified C→Rust pair dataset (research asset + moat) research
  • INFRA.20 Continuous re-verification on upstream drift eng

Critical path

F → P0 → P1 → P2 → P3 → P4 → P5 → P6, with tracks SEM · VER · MODEL · IDIOM · PERF · INFRA running alongside continuously. Vertical phases are integration checkpoints; tracks are the capabilities that feed them. The nearest fundable milestone is the P1 exit: an unseen real C library, converted hands-off, verified, under 5% refusal.

Caveats (honest)

  • Physics: "perfect safe Rust for all of ArduPilot" is unachievable as stated — register access, DMA, and ISRs are irreducibly unsafe. The real deliverable is verified safe Rust for the algorithmic layers + a thin, audited, clearly-marked unsafe hardware shim.
  • Research: research-tagged items are open problems, not schedule you can burn down on weekends. The whole ladder rests on the reliability floor (P0) holding.