Skip to content

Prepare DSPy for Python 3.15 - #10521

Merged
isaacbmiller merged 10 commits into
stanfordnlp:mainfrom
migurski:raise-python-cap-3.16
Oct 12, 2026
Merged

isaacbmiller merged 10 commits into
stanfordnlp:mainfrom
migurski:raise-python-cap-3.16

Conversation

@migurski

@migurski migurski commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR adds Python 3.15 support to DSPy. With it, pip install dspy and uv add dspy both work on 3.15 from wheels, with no Rust toolchain and no build flags, and LM calls run through the bundled lm15 engine. pydantic 2.14.0 (released 2026-10-08) supplies the cp315 pydantic-core wheels this depends on.

Changes

  1. Raise requires-python from <3.15 to <3.16. DSPy made the same change from <3.14 to <3.15 at 3.1.0 for the 3.14 cycle.
  2. Install litellm and gepa only below 3.15. Both declare requires-python <3.15. pip honours that, so without this change pip install dspy fails on 3.15: it rejects gepa==0.1.4, then falls back to litellm 1.83.7, whose pinned fastuuid and pydantic-core have no cp315 wheels and do not build under PyO3 0.26. uv ignores the cap but still has to build fastuuid from source, which needs Rust and PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1.
    • The lm15 engine serves LM calls without litellm. The litellm-only paths already raise DSPy's require() install hint.
    • dspy.GEPA becomes a stand-in that raises ImportError when gepa is absent, so import dspy still works.
  3. Declare jsonschema. dspy/adapters/types/tool.py imports it, but it only arrived through litellm.
  4. Add 3.15 to the test, extra_test, and build_package matrices as allowed failures (continue-on-error) while 3.15 is a release candidate. allow-prereleases: true lets the leg use the 3.15 rc interpreter. The dev extra keeps litellm and gepa on 3.15, so these legs still test the LiteLLM and GEPA paths. Those legs set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 so fastuuid builds.
  5. Widen two exact-3.14 conditions to 3.14 and later. The dev extra selects litellm[proxy] only below 3.14, and the LiteLLM proxy test fixture skips only on 3.14 exactly.
  6. Update uv.lock. pydantic goes to 2.14.0 and pydantic-core to 2.50.0. The 3.15 resolve takes pyarrow 26.0.0 because pyarrow 25.0.1 has no cp315 wheel.

Verification

On CPython 3.15.0rc3 (linux aarch64), installing the built wheel into a fresh venv with no Rust on PATH and no pyo3 variable:

Installer Install Predict (default engine) dspy.GEPA(...) dspy.Embedder("openai/...")
pip 26.2.1 succeeds works ImportError: gepa is required ImportError: install dspy[litellm]
uv succeeds works ImportError: gepa is required ImportError: install dspy[litellm]

On 3.12, pip still installs litellm and gepa, and dspy.GEPA constructs normally.

Test suites through uv sync with the dev extras:

Python -m 'not extra and not deno' -m 'extra or deno'
3.15.0rc3 2433 passed, 310 skipped, 2 xfailed 244 passed, 2 skipped
3.12 2443 passed, 300 skipped, 2 xfailed 244 passed, 2 skipped

The "Dependencies (latest)" check fails on main as well (test_lm_wraps_litellm_errors_with_metadata), so this PR does not cause it.

Limits on 3.15 outside DSPy

  • LiteLLM features (engine="litellm", Embedder with a model string, the LiteLLM logging helpers, and providers or client settings lm15 does not route) need litellm. pip cannot install litellm on 3.15 because of its requires-python <3.15. uv can, but needs Rust and PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 until fastuuid ships cp315 wheels (Publish CPython 3.15(t) wheels fastuuid/fastuuid#71), and litellm 1.102.0 and later import sre_constants, which Python 3.15 removed. Pin litellm<1.102 on 3.15 for now.
  • GEPA needs gepa, which declares requires-python <3.15. uv can install it on 3.15; pip cannot.

These lift once litellm and gepa support 3.15. DSPy then needs to drop the python_version < '3.15' markers and bump the exact gepa pin.


Prepared with the help of a coding agent (Claude Code); reviewed before submitting.

migurski and others added 4 commits September 28, 2026 09:58
Python 3.15 is being released, but every published dspy declares
requires-python >=3.10,<3.15, so pip/uv refuse to install dspy on
3.15 at resolution time. This raises the upper bound to <3.16 so the
package can be installed and exercised on 3.15.

Metadata-only. It does not itself assert full 3.15 support; it opens
the door to validating and releasing it. Happy to help test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds 3.15 to the test, extra_test and build_package matrices so the
suite and the wheel build are exercised on 3.15. allow-prereleases lets
the 3.15 leg acquire an interpreter before GA; it is a no-op for the
versions that already have stable builds and once 3.15 is released.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 3.15 matrix leg fails at `uv sync`: fastuuid (and other pyo3
extensions) have no cp315 wheel yet, so uv builds them from sdist and
pyo3 refuses to build against a Python newer than its max supported.
PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 is pyo3's documented escape hatch:
build against the stable ABI, run forward-compatibly on 3.15. It is a
no-op on versions with a prebuilt wheel. Drop once deps ship cp315.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the pyo3 abi3 forward-compat env: it cannot make the 3.15 leg
green anyway, because pydantic-core subclasses the native PyTzInfo type
that abi3 forbids. Instead leave the 3.15 leg failing and document why
and what fixes it: the leg goes green with no change here once pydantic
ships a release bundling pydantic-core >= 2.48 (first with cp315 wheels).
Today the newest pydantic (2.13.5) pins pydantic-core 2.46.5, which has
no cp315 wheel, so uv builds it from source and pyo3 cannot compile it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@migurski migurski changed the title build: raise requires-python cap to <3.16 for Python 3.15 support Prepare for Python 3.15: raise requires-python cap, add (currently-red) 3.15 CI legs Sep 28, 2026
@greptile-apps

greptile-apps Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] No outstanding findings block merging.

Summary

The PR expands DSPy’s supported Python range to 3.15, makes incompatible dependencies conditional, updates the lockfile, and adds provisional 3.15 CI coverage. No outstanding findings remain.

Reviews (5) · Last reviewed commit: "fix: keep the GEPA stand-in a Teleprompt..." · Reviewed by Greptile

Comment thread pyproject.toml
@greptile-apps

This comment has been minimized.

migurski and others added 2 commits September 28, 2026 17:58
pyproject raised requires-python to <3.16; regenerate the lock so it
matches (its requires-python and resolution-markers now cover 3.15).
No dependency versions change. Fixes the pyproject/lock mismatch that a
frozen install would reject.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 3.15 legs fail until their native deps ship cp315 wheels, and the
matrix defaults to fail-fast, so a red 3.15 leg cancels the 3.10-3.14
legs and reddens main. Add continue-on-error for the 3.15 matrix value
in test, extra_test and build_package so 3.15 still runs and reports but
neither fails the workflow nor cancels the other legs.

Update the note with the concrete unlock: pydantic 2.14.0 pins a
cp315-capable pydantic-core -- already proven by the published 2.14.0b2
pre-release (pydantic-core 2.49.0, which ships cp315 wheels) -- so once
2.14.0 releases the leg installs from wheels and greens with no change
here. Also name fastuuid as the first build wall.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@migurski

Copy link
Copy Markdown
Contributor Author

Thanks for the review — addressed in the latest push:

  • uv.lock (25c9542): regenerated for the raised requires-python, so it no longer lags pyproject.toml and a frozen sync accepts the <3.16 range.
  • 3.15 fail-fast / red main (6af725b): the 3.15 matrix value is now continue-on-error in test, extra_test and build_package. It still runs and reports, but no longer fails the workflow or, under the default fail-fast, cancels the 3.10–3.14 legs.
  • On “the pydantic-specific explanation is not supported by this run”: with the abi3 forward-compat flag removed, fastuuid is the first build wall — which is what the run shows. pydantic-core is the deeper one: with the flag it fails to compile because it subclasses the native PyTzInfo type, which pyo3's abi3 forbids, so no build flag rescues it. The clean fix is a prebuilt cp315 wheel, and it is imminent — pydantic 2.14.0 pins a cp315-capable pydantic-core, already proven by the published 2.14.0b2 pre-release (pins pydantic-core 2.49.0, which ships cp315 wheels). The workflow note now states this.

@migurski migurski changed the title Prepare for Python 3.15: raise requires-python cap, add (currently-red) 3.15 CI legs Prepare DSPy for Python 3.15 — hold for pydantic 2.14.0 Sep 28, 2026
@migurski

migurski commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

Pydantic 2.14.0 is out, I am updating this PR to reflect it.

pydantic 2.14.0 pins pydantic-core 2.50.0, which ships cp315 wheels.
Bump both in uv.lock, and bump pyarrow to 26.0.0 for the 3.15 resolve
because 25.0.1 has no cp315 wheel.

fastuuid (a litellm dependency) still has no cp315 wheel, and its pyo3
0.26 rejects a Python past 3.14. Set PYO3_USE_ABI3_FORWARD_COMPATIBILITY
on the 3.15 legs of test, extra_test and build_package so it builds
against the stable ABI. pydantic-core no longer needs this path because
it now installs from a wheel.

The dev extra and the LiteLLM proxy test skip matched Python 3.14
exactly, so 3.15 got neither litellm[proxy] nor the skip and 10 proxy
tests errored. Widen both to 3.14 and later.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@migurski migurski changed the title Prepare DSPy for Python 3.15 — hold for pydantic 2.14.0 Prepare DSPy for Python 3.15 Oct 9, 2026
migurski and others added 2 commits October 9, 2026 16:41
litellm (since 1.93.0) and gepa 0.1.4 declare requires-python <3.15.
pip honours that, so `pip install dspy` failed on 3.15: it rejected
gepa, then fell back to litellm 1.83.7, whose pinned fastuuid and
pydantic-core have no cp315 wheels and cannot build under PyO3 0.26.
uv ignores the cap but still had to build fastuuid from source, which
needs Rust plus PYO3_USE_ABI3_FORWARD_COMPATIBILITY.

Mark litellm and gepa `python_version < '3.15'` in the core
dependencies. The bundled lm15 engine serves LM calls without litellm,
and the existing `require()` helper reports an install hint for the
litellm-only paths. Declare jsonschema, which dspy/adapters/types/tool.py
imports but which only arrived through litellm. Replace dspy.GEPA with
a stand-in that raises ImportError when gepa is absent, so `import dspy`
still works. Keep gepa in the dev extra so the 3.15 CI legs still test
GEPA.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Code that wraps dspy.GEPA reads GEPA.__init__ and GEPA.compile at
import time; cmpnd's auto_instrument does. The bare stand-in class had
no compile, so on Python 3.15 without gepa that code raised
AttributeError before the user ever constructed a GEPA. Subclass
Teleprompter so the stand-in has the same interface, and still raise
ImportError on construction.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@isaacbmiller
isaacbmiller merged commit 9f3fcdc into stanfordnlp:main Oct 12, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants