fix(adapters): serialize non-JSON values one at a time in serialize_for_json

[isaacbmiller]

📝 Changes Description

Since #10555 made InstructionProposer GEPA's default, the reflective dataset keeps raw input and output values and the adapter renders them through serialize_for_json. If any one value can't be serialized by pydantic, for example a numpy.int64 read from a pandas row, serialize_for_json falls back to str() of the whole examples list. When that happens:

• every example renders as a Python repr ('claim_amount': np.int64(1200), 'photo': Image(url=...));
• dspy.Image inputs reach the reflection LM as repr text instead of image parts;
• custom objects lose their __str__.

This PR passes fallback=str to the existing TypeAdapter(...).dump_python(...) call (supported since pydantic 2.11, DSPy's minimum), so only the value pydantic can't serialize becomes str(value). Values that already serialize go through the same call and render unchanged. A top-level unknown value, or a serializer that raises, still falls back to str() of the whole value as before. Other adapters benefit the same way: a container holding a numpy scalar now renders as JSON instead of a repr. Numpy scalars render as JSON strings ("1200"), matching 3.4.0's per-value str().

Testing: test_examples_reach_adapter_as_structured_values now includes a numpy.int64 and an object with a custom __str__ next to a nested dspy.Image. It fails on main (no image part). pytest tests/adapters tests/teleprompt/test_gepa*.py tests/primitives tests/predict → 1131 passed, 267 skipped. Also checked with a pandas + dspy.Image GEPA repro and a live InstructionProposer call on gpt-4o.

AI disclosure: written with Amp (Claude), in a maintainer-directed session. The prompt was to find existing DSPy code that formats values one at a time and reuse it to fix this regression with a minimal diff.

✅ Contributor Checklist

• Pre-Commit checks are passing (locally and remotely)
• Title of your PR / MR corresponds to the required format
• Commit message follows required format {label}(dspy): {message}

⚠️ Warnings

None.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] Safe to merge based on the exercised paths: structured examples and multimodal image content are preserved, with no actionable defects identified.

T-Rex evidence

Requested GEPA test on head

• Ran the requested test directly with the existing virtual environment; it passed, confirming the changed GEPA example flow.

Exact executed serialization validation script

• Executed this harness against the exact base and head serialization functions with identical inputs and tests; its source makes the comparison reproducible.

Serialization and adapter output without nested fallback

• Executed 59 authored checks and 18 pytest cases using the base function; mixed values lost structure and image content, reproducing the behavior being fixed.

Serialization and adapter output with nested fallback

• Executed the identical checks and tests using the head function; all passed, showing preserved nested values and images without ordinary serialization regressions.

Command output from the check

• Ran the requested test directly with the existing virtual environment; it passed, confirming the changed GEPA example flow.

Evidence from the check

• Executed this harness against the exact base and head serialization functions with identical inputs and tests; its source makes the comparison reproducible.

Command output from the check

• Executed 59 authored checks and 18 pytest cases using the base function; mixed values lost structure and image content, reproducing the behavior being fixed.

Command output from the check

• Executed the identical checks and tests using the head function; all passed, showing preserved nested values and images without ordinary serialization regressions.

Summary

serialize_for_json now uses fallback=str so unknown nested values become strings without replacing the whole container.

• Unsupported values become strings without flattening their siblings.

_{Reviews (1) · Last reviewed commit: "fix(adapters): serialize non-JSON values..." · Reviewed by Greptile}