Phase 3 Rocq parity M3: bridge-aware LLM adapter + reconstruction fallback - #39
Merged
Conversation
…lback Rocq mirror of Lean PR #36 (LLM-assisted Tier-3 reconstruction fallback), plus the load-bearing prerequisite that makes the fallback actually work cross-bridge: the SDK's `Adapter_llm` and `Llm_reconstruct` now produce bridge-appropriate prompts / trace_formats based on `ir.source_system.name`. **Bridge-aware SDK** (`sdk/lib/adapter_llm.ml`, `llm_reconstruct.ml`): - New `rocq_ty` / `rocq_term` / `rocq_paren` parallel to the existing Lean rendering helpers — translate IR's bridge-agnostic type-ref vocabulary (Int / Real / Prop + arrow chains) to Rocq Stdlib syntax (Z / R / Prop with ASCII `->`), and render shell_terms with Rocq's `/\` `\/` `~` connectives, `forall (x : T), body` quantifiers, ASCII Z-scope arithmetic. - New `dialect` record bundles per-bridge state (system_prompt, render_prompt, ty, term, trace_format, cert_format, annotation). `dialect_of_ir` dispatches on `ir.source_system.name`: "rocq" → `rocq_dialect`, everything else → `lean_dialect` (safer LLM-familiar default). - `Adapter_llm.dispatch` and `Llm_reconstruct.translate` both pick the dialect upfront — Rocq home systems get Rocq-flavored prompts asking for Ltac in a fenced ```coq block; minted certs carry `format = trace_format = "rocq-tactic-script"`. - `Verifier.verify` extends its Tier-3 dispatch to recognize both `lean-tactic-script` and `rocq-tactic-script` → both return `Tier3_replay_deferred { trace_format }` (envelope-only; soundness deferred to the home kernel's audit-H1 replay gate). **Closers tightened** (M3.b): Rocq's `close_or_fail` now matches `Tier3_replay_deferred { trace_format = "rocq-tactic-script" }` specifically. A `lean-tactic-script` cert reaching the Rocq closer (misconfigured manifest pointing this bridge at a Lean-flavored LLM endpoint, or future bug) is a clean user error rather than a silent parse failure when Ltac tries to chew on Lean syntax. **Reconstruction fallback** (M3.c, `rocq-bridge/src/pb_rocq_main.ml`): Mirror of Lean's `closeOrFailPrimary` + `closeOrFail` wrap. - The existing `close_or_fail` is renamed `close_or_fail_primary`. - New `replay_reconstructed_script trace_format script` runs the candidate through `Llm_replay.replay_script` (the same audit-H1 gate from M2) and on success emits a `Feedback.msg_info` line naming the source trace format — audit trail visible in build output. - New `try_llm_reconstruct ir cert?` gates on Tier-3 cert with `trace_format ≠ rocq-tactic-script`, calls `Proof_broker.Llm_reconstruct.translate` (direct-linked, no FFI), returns `Some fallback_tac` or `None`. - New `close_or_fail` wraps primary in `Proofview.tclORELSE`; on primary failure invokes `try_llm_reconstruct` lazily (HTTP only fires when primary actually fails) and routes to the fallback if a script came back, else re-raises the primary error. - Audit H1: reconstruction goes through the SAME audit gate as primary LLM-replay (kernel replay + axiom-footprint subset check), so the LLM never widens the trust base. **Test-only tactic** (`g_proof_broker.mlg`): new `llm_reconstruct_test "<format>" "<script>"` drives `replay_reconstructed_script` directly with a string literal — exercises the audit-H1 contract without a live LLM endpoint. Mirror of Lean's evalLlmReconstructTest. **Tests** (Rocq-side mirror of Lean's `llm_reconstruct_test` block): - Positive `pb_llm_reconstruct_axiom_free`: clean `intros; reflexivity` translation closes a Z-equality goal. Allowlisted `[]`. - Negative `assert_fails (llm_reconstruct_test ... "idtac")` — non-closing translation. - Negative `assert_fails (llm_reconstruct_test ... "@@@ ...")` — parse failure. **SDK tests** added bridge-aware coverage: Rocq-flavored `test_prompt_render_rocq` and `test_mock_endpoint_rocq` for `Adapter_llm`; `test_prompt_render_rocq` for `Llm_reconstruct`. Verifies Z translation, Theorem...Proof...Qed scaffold, ```coq fence, and that the Rocq-flavored cert's verify reason carries `trace_format = "rocq-tactic-script"`. Locally verified: SDK + Rocq plugin sources build clean (`dune build sdk rocq-bridge/src` rc=0). Full SDK suite passes including the new Rocq tests. Python schema/gate suite green. JSON allowlist valid. The Rocq theory build path (incl. the new fallback test and Print Assumptions on `pb_llm_reconstruct_axiom_free`) needs CI per the same broken-local-rocq-env caveat from M1/M2. With this, Phase 3 Rocq parity is structurally complete for deliverables #3 and #4. M4 (polish) carries the `fragment_of_logic` consolidation (Phase-4 retro carry) plus a README/delta sync to record the Rocq Phase-3 surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
levineuwirth
added a commit
that referenced
this pull request
May 21, 2026
Documentation-only polish for the Phase-3 Rocq parity work (PRs #37/#38/#39). No code changes. * README §Status gets a new paragraph recording the Rocq Phase-3 surface: HO reifier + HOL closer + opt-in coq-hammer (M1), LLM- replay closer with kernel-Assumptions-API audit gate (M2), bridge-aware SDK + reconstruction fallback (M3). Updates the retrospective list to include phase-3-rocq-parity. * delta.md §2.4 extends the Phase-3-progress block with the M1- through-M4 narrative: SDK reuse pattern, plugin-side wiring effort, and confirmation that the §1.1 architectural payoff ("genuinely cross-system or accidentally Lean-shaped") landed empirically — only LLM prompt rendering needed bridge- awareness, and it consolidated cleanly into a single `dialect` record. "Phase 3 is now structurally complete on both bridges." * RETROSPECTIVES/phase-3-rocq-parity.md (new) captures the lessons in the established phase-{0,4,5} retro format: - Easier-than-expected: SDK reuse, kernel Assumptions API as a clean replacement for `Lean.collectAxioms`, `hauto` as the `aesop` analogue. - Harder-than-expected: cross-bridge drift surface (M1 needed three CI rounds to caught up three Lean-side milestones' worth of stale glue in load_default / adapter_registry / verifier-reason arm / render_path; captured as the `project-rocq-parity-sync-points` project memory), local rocq-core build bug forcing CI-only verification, and the realization that bridge-awareness is an SDK concern (the `dialect` lives in `Adapter_llm`, not in the plugin). - Carried forward: BV/UF reach on Rocq pending Stdlib BitVec library, cross-bridge IR round-trip validation, dune `(using rocq …)` flip when dune ships it, retros for Phases 1/2/3 Lean-side. The `fragment_of_logic` consolidation flagged in RETROSPECTIVES/phase-4.md as a Phase-4 carry had already shipped during an earlier audit pass; the file is now a single shared function in `sdk/lib/smtlib.ml`, with all three adapters (cvc4/cvc5/z3) calling into it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Rocq mirror of Lean PR #36 (LLM-assisted Tier-3 reconstruction fallback) — plus the load-bearing prerequisite that makes the fallback work cross-bridge: the SDK's `Adapter_llm` and `Llm_reconstruct` now produce bridge-appropriate prompts and trace_formats based on `ir.source_system.name`.
Without bridge-awareness, Rocq home systems would receive Lean syntax in LLM responses and the audit_fail path would always fire — fallback exists but never produces parseable scripts. M3 fixes both layers in one PR.
Pieces
M3.a — bridge-aware SDK (`sdk/lib/adapter_llm.ml` + `llm_reconstruct.ml`)
New `rocq_ty` / `rocq_term` / `rocq_paren` parallels to the existing Lean helpers — translate IR's bridge-agnostic vocabulary (Int / Real / Prop + arrow chains) to Rocq Stdlib syntax (Z / R / Prop + ASCII `->`), and render shell_terms with `/\` `\/` `~` connectives, `forall (x : T), body` quantifiers, ASCII Z-scope arithmetic.
New `dialect` record bundles per-bridge state (system_prompt, render_prompt, ty, term, trace_format, cert_format, annotation). `dialect_of_ir` dispatches on `ir.source_system.name`: `"rocq"` → `rocq_dialect`, everything else → `lean_dialect`. Rocq home systems get Rocq-flavored prompts asking for Ltac in a fenced ```coq block; minted certs carry `format = trace_format = "rocq-tactic-script"`.
`Verifier.verify` extends its Tier-3 dispatch to recognize both `lean-tactic-script` and `rocq-tactic-script` → both return `Tier3_replay_deferred { trace_format }` (envelope-only; soundness deferred to the home kernel's audit-H1 replay gate).
M3.b — tightened closers
Rocq's `close_or_fail` now matches `Tier3_replay_deferred { trace_format = "rocq-tactic-script" }` specifically. A `lean-tactic-script` cert reaching the Rocq closer is a clean user error rather than silently trying to parse Lean syntax as Ltac. Lean's `closeOrFail` already gated on `lean-tactic-script`.
M3.c — reconstruction fallback wiring (`rocq-bridge/src/pb_rocq_main.ml`)
Mirror of Lean's `closeOrFailPrimary` + `closeOrFail` wrap from PR #36.
Audit H1: reconstruction goes through the SAME audit gate as primary LLM-replay (kernel replay + axiom-footprint subset check), so the LLM never widens the trust base.
M3.d — test-only tactic + tests
`llm_reconstruct_test "" "<script>"` TACTIC EXTEND drives `replay_reconstructed_script` directly with a string literal — exercises the audit-H1 contract without a live LLM endpoint. Mirror of Lean's `evalLlmReconstructTest`.
Rocq tests in `Test.v`:
SDK tests added Rocq-flavored coverage: `test_prompt_render_rocq` and `test_mock_endpoint_rocq` for `Adapter_llm` verify Z translation, `Theorem ... Proof. ... Qed.` scaffold, ```coq fence, and that the Rocq-flavored cert's verify reason carries `trace_format = "rocq-tactic-script"`. Same for `Llm_reconstruct`.
Verification
Locally: SDK + Rocq plugin sources build clean. SDK suite passes including the new Rocq render/transport tests for both `adapter_llm` and `llm_reconstruct`. Python schema/gate suite green. JSON allowlist valid. Rocq theory build path needs CI per the same broken-local-rocq-env caveat from M1/M2.
Scope
With this, Phase 3 Rocq parity is structurally complete for deliverables #3 and #4. M4 (polish) carries the `fragment_of_logic` consolidation (Phase-4 retro carry) plus a README/delta sync to record the Rocq Phase-3 surface.
🤖 Generated with Claude Code