Repository navigation
Prepare DSPy for Python 3.15 - #10521
Merged
Merged
Conversation
Python 3.15 is being released, but every published dspy declares requires-python >=3.10,<3.15, so pip/uv refuse to install dspy on 3.15 at resolution time. This raises the upper bound to <3.16 so the package can be installed and exercised on 3.15. Metadata-only. It does not itself assert full 3.15 support; it opens the door to validating and releasing it. Happy to help test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds 3.15 to the test, extra_test and build_package matrices so the suite and the wheel build are exercised on 3.15. allow-prereleases lets the 3.15 leg acquire an interpreter before GA; it is a no-op for the versions that already have stable builds and once 3.15 is released. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 3.15 matrix leg fails at `uv sync`: fastuuid (and other pyo3 extensions) have no cp315 wheel yet, so uv builds them from sdist and pyo3 refuses to build against a Python newer than its max supported. PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 is pyo3's documented escape hatch: build against the stable ABI, run forward-compatibly on 3.15. It is a no-op on versions with a prebuilt wheel. Drop once deps ship cp315. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the pyo3 abi3 forward-compat env: it cannot make the 3.15 leg green anyway, because pydantic-core subclasses the native PyTzInfo type that abi3 forbids. Instead leave the 3.15 leg failing and document why and what fixes it: the leg goes green with no change here once pydantic ships a release bundling pydantic-core >= 2.48 (first with cp315 wheels). Today the newest pydantic (2.13.5) pins pydantic-core 2.46.5, which has no cp315 wheel, so uv builds it from source and pyo3 cannot compile it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
This comment has been minimized.
This comment has been minimized.
pyproject raised requires-python to <3.16; regenerate the lock so it matches (its requires-python and resolution-markers now cover 3.15). No dependency versions change. Fixes the pyproject/lock mismatch that a frozen install would reject. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 3.15 legs fail until their native deps ship cp315 wheels, and the matrix defaults to fail-fast, so a red 3.15 leg cancels the 3.10-3.14 legs and reddens main. Add continue-on-error for the 3.15 matrix value in test, extra_test and build_package so 3.15 still runs and reports but neither fails the workflow nor cancels the other legs. Update the note with the concrete unlock: pydantic 2.14.0 pins a cp315-capable pydantic-core -- already proven by the published 2.14.0b2 pre-release (pydantic-core 2.49.0, which ships cp315 wheels) -- so once 2.14.0 releases the leg installs from wheels and greens with no change here. Also name fastuuid as the first build wall. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Thanks for the review — addressed in the latest push:
|
Contributor
Author
|
Pydantic 2.14.0 is out, I am updating this PR to reflect it. |
pydantic 2.14.0 pins pydantic-core 2.50.0, which ships cp315 wheels. Bump both in uv.lock, and bump pyarrow to 26.0.0 for the 3.15 resolve because 25.0.1 has no cp315 wheel. fastuuid (a litellm dependency) still has no cp315 wheel, and its pyo3 0.26 rejects a Python past 3.14. Set PYO3_USE_ABI3_FORWARD_COMPATIBILITY on the 3.15 legs of test, extra_test and build_package so it builds against the stable ABI. pydantic-core no longer needs this path because it now installs from a wheel. The dev extra and the LiteLLM proxy test skip matched Python 3.14 exactly, so 3.15 got neither litellm[proxy] nor the skip and 10 proxy tests errored. Widen both to 3.14 and later. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
litellm (since 1.93.0) and gepa 0.1.4 declare requires-python <3.15. pip honours that, so `pip install dspy` failed on 3.15: it rejected gepa, then fell back to litellm 1.83.7, whose pinned fastuuid and pydantic-core have no cp315 wheels and cannot build under PyO3 0.26. uv ignores the cap but still had to build fastuuid from source, which needs Rust plus PYO3_USE_ABI3_FORWARD_COMPATIBILITY. Mark litellm and gepa `python_version < '3.15'` in the core dependencies. The bundled lm15 engine serves LM calls without litellm, and the existing `require()` helper reports an install hint for the litellm-only paths. Declare jsonschema, which dspy/adapters/types/tool.py imports but which only arrived through litellm. Replace dspy.GEPA with a stand-in that raises ImportError when gepa is absent, so `import dspy` still works. Keep gepa in the dev extra so the 3.15 CI legs still test GEPA. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Code that wraps dspy.GEPA reads GEPA.__init__ and GEPA.compile at import time; cmpnd's auto_instrument does. The bare stand-in class had no compile, so on Python 3.15 without gepa that code raised AttributeError before the user ever constructed a GEPA. Subclass Teleprompter so the stand-in has the same interface, and still raise ImportError on construction. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds Python 3.15 support to DSPy. With it,
pip install dspyanduv add dspyboth work on 3.15 from wheels, with no Rust toolchain and no build flags, and LM calls run through the bundled lm15 engine. pydantic 2.14.0 (released 2026-10-08) supplies the cp315 pydantic-core wheels this depends on.Changes
requires-pythonfrom<3.15to<3.16. DSPy made the same change from<3.14to<3.15at 3.1.0 for the 3.14 cycle.requires-python <3.15. pip honours that, so without this changepip install dspyfails on 3.15: it rejectsgepa==0.1.4, then falls back to litellm 1.83.7, whose pinned fastuuid and pydantic-core have no cp315 wheels and do not build under PyO3 0.26. uv ignores the cap but still has to build fastuuid from source, which needs Rust andPYO3_USE_ABI3_FORWARD_COMPATIBILITY=1.require()install hint.dspy.GEPAbecomes a stand-in that raisesImportErrorwhen gepa is absent, soimport dspystill works.jsonschema.dspy/adapters/types/tool.pyimports it, but it only arrived through litellm.test,extra_test, andbuild_packagematrices as allowed failures (continue-on-error) while 3.15 is a release candidate.allow-prereleases: truelets the leg use the 3.15 rc interpreter. The dev extra keeps litellm and gepa on 3.15, so these legs still test the LiteLLM and GEPA paths. Those legs setPYO3_USE_ABI3_FORWARD_COMPATIBILITY=1so fastuuid builds.devextra selectslitellm[proxy]only below 3.14, and the LiteLLM proxy test fixture skips only on 3.14 exactly.uv.lock. pydantic goes to 2.14.0 and pydantic-core to 2.50.0. The 3.15 resolve takes pyarrow 26.0.0 because pyarrow 25.0.1 has no cp315 wheel.Verification
On CPython 3.15.0rc3 (linux aarch64), installing the built wheel into a fresh venv with no Rust on
PATHand no pyo3 variable:Predict(default engine)dspy.GEPA(...)dspy.Embedder("openai/...")ImportError: gepa is requiredImportError: installdspy[litellm]ImportError: gepa is requiredImportError: installdspy[litellm]On 3.12, pip still installs litellm and gepa, and
dspy.GEPAconstructs normally.Test suites through
uv syncwith the dev extras:-m 'not extra and not deno'-m 'extra or deno'The "Dependencies (latest)" check fails on
mainas well (test_lm_wraps_litellm_errors_with_metadata), so this PR does not cause it.Limits on 3.15 outside DSPy
engine="litellm",Embedderwith a model string, the LiteLLM logging helpers, and providers or client settings lm15 does not route) need litellm. pip cannot install litellm on 3.15 because of itsrequires-python <3.15. uv can, but needs Rust andPYO3_USE_ABI3_FORWARD_COMPATIBILITY=1until fastuuid ships cp315 wheels (Publish CPython 3.15(t) wheels fastuuid/fastuuid#71), and litellm 1.102.0 and later importsre_constants, which Python 3.15 removed. Pinlitellm<1.102on 3.15 for now.requires-python <3.15. uv can install it on 3.15; pip cannot.These lift once litellm and gepa support 3.15. DSPy then needs to drop the
python_version < '3.15'markers and bump the exact gepa pin.Prepared with the help of a coding agent (Claude Code); reviewed before submitting.