Skip to content

feat(gepa): upgrade engine to 0.1.4 - #10209

Merged
isaacbmiller merged 2 commits into
mainfrom
amp/gepa-014-core
Aug 17, 2026
Merged

isaacbmiller merged 2 commits into
mainfrom
amp/gepa-014-core

Conversation

@isaacbmiller

@isaacbmiller isaacbmiller commented Aug 14, 2026 •

Copy link
Copy Markdown
Collaborator

Why this PR exists

DSPy is pinned to GEPA 0.1.1. GEPA 0.1.4 changes the optimization model from one proposal per iteration to an optional multi-proposal pipeline and adds adapter checkpoint state, new acceptance/selection controls, and LM-based reflection-cost accounting.

This PR is intentionally the minimal dependency/compatibility layer. It upgrades GEPA and makes DSPy's existing adapter correct for the new checkpoint and configuration contracts, but does not add candidate-level concurrency. That optimization is isolated in stacked PR #10210. Reflection-cost accounting remains explicitly unsupported rather than introducing a second cost ledger alongside LM history.

Relevant upstream changes: GEPA 0.1.1 → 0.1.4

GEPA 0.1.4 adds:

  • sampling_strategy: one or many (parent, minibatch) proposal tasks per iteration (SameParentSampling, IndependentSampling, PxNSampling)
  • selection_strategy and acceptance_criterion: control which evaluated proposals enter the candidate pool
  • optional adapter.batch_evaluate(items): adapters can batch or parallelize candidate evaluation; evaluate-only adapters use GEPA's ordered sequential fallback
  • optional get_adapter_state() / set_adapter_state(): adapter-private state now participates in checkpoint save/resume
  • max_reflection_cost: reads cumulative cost from the supplied reflection LM
  • tracking lifecycle controls including wandb_attach_existing, mlflow_attach_existing, and tracking_key_prefix

The default SingleMutationSampling path remains the old one-proposal behavior. The multi-proposal design intentionally keeps GEPA's engine sequential and puts batching at the adapter/LM edge. See GEPA 0.1.4 and its parallel-proposals contract.

What changes in DSPy

Upstream capability Behavior in this PR
Multi-proposal sampling/selection Available through gepa_kwargs; DSPy uses GEPA's sequential evaluate() fallback
Adapter checkpoint state DSPy saves/restores the adapter's random.Random state, so resumed proposal sampling follows the uninterrupted run
Reflection-cost budget Non-None max_reflection_cost is rejected clearly because DSPy LMs do not yet expose the cumulative-cost interface GEPA requires; None remains valid and means no cost budget
Tracking controls Documented and passed through gepa_kwargs
Dependency gepa[dspy] == 0.1.1 → 0.1.4

The executable production change is deliberately small: RNG checkpoint hooks and validation of the unsupported cost budget. There is no wrapper, executor, or local GEPA engine patch in this PR.

Behavior and risk

Expected behavior

  • Existing users who keep the default single-proposal strategy retain the prior execution shape.
  • Users can opt into multi-proposal sampling immediately, but candidate evaluations remain sequential until feat(gepa): parallelize candidate evaluation #10210.
  • Checkpoint resume is more reproducible because adapter RNG state is no longer reset.
  • A requested reflection-cost limit now fails at construction instead of being silently ineffective.

Known upstream 0.1.4 limitation

GEPA 0.1.4 incorrectly requests traces while evaluating accepted candidates on the full validation set (gepa-ai/gepa#428). DSPy's traced and ordinary evaluation paths are not equivalent for every runtime failure, so this can produce missing/misaligned validation results and an IndexError in affected runs.

This PR does not monkeypatch GEPA or silently switch evaluation behavior. The upstream trace-aware fix is gepa-ai/gepa#430 and is expected in GEPA 0.1.5. The dependency bump can be reviewed and merged independently; the eventual 0.1.5 pin is a follow-up once published.

Review guide

The functional diff is limited to:

  • pyproject.toml, uv.lock: dependency resolution
  • dspy/teleprompt/gepa/gepa_utils.py: adapter RNG checkpoint state
  • dspy/teleprompt/gepa/gepa.py: passthrough documentation and explicit cost-budget rejection
  • tests/teleprompt/test_gepa.py: RNG round-trip and max_reflection_cost contract

Candidate parallelism is intentionally out of scope here; review it separately in #10210. Reflection-cost accounting remains deferred until it can use DSPy's canonical LM accounting rather than a parallel counter.

Verification

  • uv run pytest tests/teleprompt -q — 81 passed
  • uv run pre-commit run --all-files — passed
  • uv lock --check — passed
  • git diff --check — passed

@greptile-apps

greptile-apps Bot commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

GEPA is upgraded to 0.1.4 with deterministic adapter RNG restoration during checkpoint resume, clear rejection of unsupported reflection-cost budgets, and compatibility for the newly exposed proposal and tracking controls. Checkpoint/resume behavior and focused GEPA tests completed successfully.

Confidence Score: 5/5

No blocking failure remains.

Checkpoint restoration, reflection-cost handling, and GEPA 0.1.4 control forwarding all behaved as intended in executed checks.

T-Rex T-Rex Logs

What T-Rex did

  • Ran the offline GEPA harness against the base and upgrade sources, then executed the focused repository GEPA tests.
  • Observed that the current source created a checkpoint, invoked the adapter save hook twice and the resume hook once, and produced the same next RNG-driven proposal after resume as in the uninterrupted run.
  • Verified that max_reflection_cost=None could be constructed, a non-null reflection cost raised ValueError, and the compiled mode accepted the new proposal and tracking controls; the focused pytest command exited successfully.
  • Before output, checkpoint hooks were unavailable and non-None reflection cost constructed, demonstrating the behavior introduced by the scoped commits.
  • After the run, post-output results showed checkpoint_exists true, save_hook_calls 2, resume_hook_calls 1, deterministic_after_resume true, with cost outcomes None constructed and 1.0 ValueError, and the focused pytest command exited 0.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (5): Last reviewed commit: "fix(gepa): allow an unset reflection cos..." | Re-trigger Greptile

@isaacbmiller
isaacbmiller force-pushed the amp/gepa-014-core branch 2 times, most recently from 0704747 to c865c4d Compare August 17, 2026 19:42
@isaacbmiller
isaacbmiller merged commit aa9d2d0 into main Aug 17, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants