Skip to content

Phase 3 Rocq parity M3: bridge-aware LLM adapter + reconstruction fallback - #39

Merged
levineuwirth merged 1 commit into
mainfrom
phase-3-rocq-parity-m3
May 21, 2026
Merged

Phase 3 Rocq parity M3: bridge-aware LLM adapter + reconstruction fallback#39
levineuwirth merged 1 commit into
mainfrom
phase-3-rocq-parity-m3

Conversation

@levineuwirth

Copy link
Copy Markdown
Owner

What

Rocq mirror of Lean PR #36 (LLM-assisted Tier-3 reconstruction fallback) — plus the load-bearing prerequisite that makes the fallback work cross-bridge: the SDK's `Adapter_llm` and `Llm_reconstruct` now produce bridge-appropriate prompts and trace_formats based on `ir.source_system.name`.

Without bridge-awareness, Rocq home systems would receive Lean syntax in LLM responses and the audit_fail path would always fire — fallback exists but never produces parseable scripts. M3 fixes both layers in one PR.

Pieces

M3.a — bridge-aware SDK (`sdk/lib/adapter_llm.ml` + `llm_reconstruct.ml`)

New `rocq_ty` / `rocq_term` / `rocq_paren` parallels to the existing Lean helpers — translate IR's bridge-agnostic vocabulary (Int / Real / Prop + arrow chains) to Rocq Stdlib syntax (Z / R / Prop + ASCII `->`), and render shell_terms with `/\` `\/` `~` connectives, `forall (x : T), body` quantifiers, ASCII Z-scope arithmetic.

New `dialect` record bundles per-bridge state (system_prompt, render_prompt, ty, term, trace_format, cert_format, annotation). `dialect_of_ir` dispatches on `ir.source_system.name`: `"rocq"` → `rocq_dialect`, everything else → `lean_dialect`. Rocq home systems get Rocq-flavored prompts asking for Ltac in a fenced ```coq block; minted certs carry `format = trace_format = "rocq-tactic-script"`.

`Verifier.verify` extends its Tier-3 dispatch to recognize both `lean-tactic-script` and `rocq-tactic-script` → both return `Tier3_replay_deferred { trace_format }` (envelope-only; soundness deferred to the home kernel's audit-H1 replay gate).

M3.b — tightened closers

Rocq's `close_or_fail` now matches `Tier3_replay_deferred { trace_format = "rocq-tactic-script" }` specifically. A `lean-tactic-script` cert reaching the Rocq closer is a clean user error rather than silently trying to parse Lean syntax as Ltac. Lean's `closeOrFail` already gated on `lean-tactic-script`.

M3.c — reconstruction fallback wiring (`rocq-bridge/src/pb_rocq_main.ml`)

Mirror of Lean's `closeOrFailPrimary` + `closeOrFail` wrap from PR #36.

  • Existing `close_or_fail` → `close_or_fail_primary`.
  • New `replay_reconstructed_script` routes through `Llm_replay.replay_script` (the audit-H1 gate from M2) and emits a `Feedback.msg_info` audit line on success.
  • New `try_llm_reconstruct` gates on Tier-3 cert with `trace_format ≠ rocq-tactic-script`, calls `Proof_broker.Llm_reconstruct.translate` (direct-linked, no FFI on Rocq), returns `Some fallback_tac` or `None`.
  • New `close_or_fail` wraps primary in `Proofview.tclORELSE`; on primary failure invokes `try_llm_reconstruct` lazily (HTTP only fires when primary actually fails) and routes to the fallback if a script came back, else re-raises the primary error.

Audit H1: reconstruction goes through the SAME audit gate as primary LLM-replay (kernel replay + axiom-footprint subset check), so the LLM never widens the trust base.

M3.d — test-only tactic + tests

`llm_reconstruct_test "" "<script>"` TACTIC EXTEND drives `replay_reconstructed_script` directly with a string literal — exercises the audit-H1 contract without a live LLM endpoint. Mirror of Lean's `evalLlmReconstructTest`.

Rocq tests in `Test.v`:

  • Positive `pb_llm_reconstruct_axiom_free`: clean `intros; reflexivity` translation closes a Z-equality goal. Allowlisted `[]`.
  • Negative `assert_fails (llm_reconstruct_test ... "idtac")` — non-closing.
  • Negative `assert_fails (llm_reconstruct_test ... "@@@ ...")` — parse failure.

SDK tests added Rocq-flavored coverage: `test_prompt_render_rocq` and `test_mock_endpoint_rocq` for `Adapter_llm` verify Z translation, `Theorem ... Proof. ... Qed.` scaffold, ```coq fence, and that the Rocq-flavored cert's verify reason carries `trace_format = "rocq-tactic-script"`. Same for `Llm_reconstruct`.

Verification

Locally: SDK + Rocq plugin sources build clean. SDK suite passes including the new Rocq render/transport tests for both `adapter_llm` and `llm_reconstruct`. Python schema/gate suite green. JSON allowlist valid. Rocq theory build path needs CI per the same broken-local-rocq-env caveat from M1/M2.

Scope

With this, Phase 3 Rocq parity is structurally complete for deliverables #3 and #4. M4 (polish) carries the `fragment_of_logic` consolidation (Phase-4 retro carry) plus a README/delta sync to record the Rocq Phase-3 surface.

🤖 Generated with Claude Code

…lback

Rocq mirror of Lean PR #36 (LLM-assisted Tier-3 reconstruction
fallback), plus the load-bearing prerequisite that makes the
fallback actually work cross-bridge: the SDK's `Adapter_llm` and
`Llm_reconstruct` now produce bridge-appropriate prompts /
trace_formats based on `ir.source_system.name`.

**Bridge-aware SDK** (`sdk/lib/adapter_llm.ml`, `llm_reconstruct.ml`):
- New `rocq_ty` / `rocq_term` / `rocq_paren` parallel to the
  existing Lean rendering helpers — translate IR's bridge-agnostic
  type-ref vocabulary (Int / Real / Prop + arrow chains) to Rocq
  Stdlib syntax (Z / R / Prop with ASCII `->`), and render
  shell_terms with Rocq's `/\` `\/` `~` connectives, `forall (x :
  T), body` quantifiers, ASCII Z-scope arithmetic.
- New `dialect` record bundles per-bridge state (system_prompt,
  render_prompt, ty, term, trace_format, cert_format, annotation).
  `dialect_of_ir` dispatches on `ir.source_system.name`: "rocq" →
  `rocq_dialect`, everything else → `lean_dialect` (safer
  LLM-familiar default).
- `Adapter_llm.dispatch` and `Llm_reconstruct.translate` both pick
  the dialect upfront — Rocq home systems get Rocq-flavored
  prompts asking for Ltac in a fenced ```coq block; minted certs
  carry `format = trace_format = "rocq-tactic-script"`.
- `Verifier.verify` extends its Tier-3 dispatch to recognize both
  `lean-tactic-script` and `rocq-tactic-script` → both return
  `Tier3_replay_deferred { trace_format }` (envelope-only;
  soundness deferred to the home kernel's audit-H1 replay gate).

**Closers tightened** (M3.b): Rocq's `close_or_fail` now matches
`Tier3_replay_deferred { trace_format = "rocq-tactic-script" }`
specifically. A `lean-tactic-script` cert reaching the Rocq closer
(misconfigured manifest pointing this bridge at a Lean-flavored
LLM endpoint, or future bug) is a clean user error rather than a
silent parse failure when Ltac tries to chew on Lean syntax.

**Reconstruction fallback** (M3.c, `rocq-bridge/src/pb_rocq_main.ml`):
Mirror of Lean's `closeOrFailPrimary` + `closeOrFail` wrap.
- The existing `close_or_fail` is renamed `close_or_fail_primary`.
- New `replay_reconstructed_script trace_format script` runs the
  candidate through `Llm_replay.replay_script` (the same audit-H1
  gate from M2) and on success emits a `Feedback.msg_info` line
  naming the source trace format — audit trail visible in build
  output.
- New `try_llm_reconstruct ir cert?` gates on Tier-3 cert with
  `trace_format ≠ rocq-tactic-script`, calls
  `Proof_broker.Llm_reconstruct.translate` (direct-linked, no FFI),
  returns `Some fallback_tac` or `None`.
- New `close_or_fail` wraps primary in `Proofview.tclORELSE`; on
  primary failure invokes `try_llm_reconstruct` lazily (HTTP only
  fires when primary actually fails) and routes to the fallback
  if a script came back, else re-raises the primary error.
- Audit H1: reconstruction goes through the SAME audit gate as
  primary LLM-replay (kernel replay + axiom-footprint subset
  check), so the LLM never widens the trust base.

**Test-only tactic** (`g_proof_broker.mlg`): new `llm_reconstruct_test
"<format>" "<script>"` drives `replay_reconstructed_script`
directly with a string literal — exercises the audit-H1 contract
without a live LLM endpoint. Mirror of Lean's evalLlmReconstructTest.

**Tests** (Rocq-side mirror of Lean's `llm_reconstruct_test` block):
- Positive `pb_llm_reconstruct_axiom_free`: clean `intros;
  reflexivity` translation closes a Z-equality goal. Allowlisted
  `[]`.
- Negative `assert_fails (llm_reconstruct_test ... "idtac")` —
  non-closing translation.
- Negative `assert_fails (llm_reconstruct_test ... "@@@ ...")` —
  parse failure.

**SDK tests** added bridge-aware coverage: Rocq-flavored
`test_prompt_render_rocq` and `test_mock_endpoint_rocq` for
`Adapter_llm`; `test_prompt_render_rocq` for `Llm_reconstruct`.
Verifies Z translation, Theorem...Proof...Qed scaffold, ```coq
fence, and that the Rocq-flavored cert's verify reason carries
`trace_format = "rocq-tactic-script"`.

Locally verified: SDK + Rocq plugin sources build clean (`dune
build sdk rocq-bridge/src` rc=0). Full SDK suite passes including
the new Rocq tests. Python schema/gate suite green. JSON
allowlist valid. The Rocq theory build path (incl. the new
fallback test and Print Assumptions on `pb_llm_reconstruct_axiom_free`)
needs CI per the same broken-local-rocq-env caveat from M1/M2.

With this, Phase 3 Rocq parity is structurally complete for
deliverables #3 and #4. M4 (polish) carries the
`fragment_of_logic` consolidation (Phase-4 retro carry) plus a
README/delta sync to record the Rocq Phase-3 surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@levineuwirth levineuwirth self-assigned this May 21, 2026
@levineuwirth
levineuwirth merged commit 89e7124 into main May 21, 2026
6 checks passed
levineuwirth added a commit that referenced this pull request May 21, 2026
Documentation-only polish for the Phase-3 Rocq parity work
(PRs #37/#38/#39). No code changes.

* README §Status gets a new paragraph recording the Rocq Phase-3
  surface: HO reifier + HOL closer + opt-in coq-hammer (M1), LLM-
  replay closer with kernel-Assumptions-API audit gate (M2),
  bridge-aware SDK + reconstruction fallback (M3). Updates the
  retrospective list to include phase-3-rocq-parity.

* delta.md §2.4 extends the Phase-3-progress block with the M1-
  through-M4 narrative: SDK reuse pattern, plugin-side wiring
  effort, and confirmation that the §1.1 architectural payoff
  ("genuinely cross-system or accidentally Lean-shaped") landed
  empirically — only LLM prompt rendering needed bridge-
  awareness, and it consolidated cleanly into a single `dialect`
  record. "Phase 3 is now structurally complete on both bridges."

* RETROSPECTIVES/phase-3-rocq-parity.md (new) captures the
  lessons in the established phase-{0,4,5} retro format:
  - Easier-than-expected: SDK reuse, kernel Assumptions API as
    a clean replacement for `Lean.collectAxioms`, `hauto` as the
    `aesop` analogue.
  - Harder-than-expected: cross-bridge drift surface (M1 needed
    three CI rounds to caught up three Lean-side milestones'
    worth of stale glue in load_default / adapter_registry /
    verifier-reason arm / render_path; captured as the
    `project-rocq-parity-sync-points` project memory), local
    rocq-core build bug forcing CI-only verification, and the
    realization that bridge-awareness is an SDK concern (the
    `dialect` lives in `Adapter_llm`, not in the plugin).
  - Carried forward: BV/UF reach on Rocq pending Stdlib BitVec
    library, cross-bridge IR round-trip validation, dune
    `(using rocq …)` flip when dune ships it, retros for
    Phases 1/2/3 Lean-side.

The `fragment_of_logic` consolidation flagged in
RETROSPECTIVES/phase-4.md as a Phase-4 carry had already shipped
during an earlier audit pass; the file is now a single shared
function in `sdk/lib/smtlib.ml`, with all three adapters
(cvc4/cvc5/z3) calling into it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@levineuwirth
levineuwirth deleted the phase-3-rocq-parity-m3 branch August 31, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant