Skip to content

fix(dspy): isolate seeded dataset splitting from global randomness - #10472

Open
emme1t wants to merge 1 commit into
stanfordnlp:mainfrom
emme1t:fix/split-random-state
Open

emme1t wants to merge 1 commit into
stanfordnlp:mainfrom
emme1t:fix/split-random-state

Conversation

@emme1t

@emme1t emme1t commented Sep 22, 2026

Copy link
Copy Markdown

📝 Changes Description

Passing random_state to DataLoader.train_test_split() currently reseeds Python's process-wide generator. This changes subsequent random draws in unrelated code, including when split validation raises an error.

Use a local random.Random(random_state) for explicitly seeded splits. The same seed produces the same split order as before, while the caller's global state is preserved. Calls with random_state=None continue to use and advance the global generator.

Reproduction:

import random
import dspy
from dspy.datasets.dataloader import DataLoader

random.seed(123)
state = random.getstate()
DataLoader().train_test_split([dspy.Example(value=i) for i in range(10)], random_state=42)
assert random.getstate() == state

Validation on Windows / Python 3.13.13:

  • New tests against the original implementation: 3 failed, 3 passed.
  • New tests plus existing Dataset and Example tests: 30 passed, 1 skipped (the existing optional CSV test).
  • Regressions cover seed zero, deterministic split membership/order, unchanged input lists, unseeded global-state behavior, and invalid seeded requests.
  • Ruff and applicable configured pre-commit hooks passed.

✅ Contributor Checklist

  • Pre-Commit checks are passing (locally and remotely): local hooks passed; remote checks are pending.
  • Title of your PR / MR corresponds to the required format.
  • Commit message follows required format {label}(dspy): {message}.

⚠️ Warnings

An explicit random_state now controls the dataset split independently of subsequent global random draws.

@greptile-apps

greptile-apps Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] Safe to merge; no outstanding findings require changes before merging.

Summary

This change updates DataLoader.train_test_split() so an explicitly supplied seed uses an isolated random-number generator rather than reseeding Python’s process-wide random state. It preserves deterministic split membership and ordering for the same seed, retains the existing behavior when no seed is supplied, and adds focused coverage for seed zero, input preservation, invalid split requests, deterministic ordering, and random-state isolation.

No merge-blocking concerns were identified.

Reviews (2) · Last reviewed commit: "fix(dspy): isolate seeded dataset splitt..." · Reviewed by Greptile

@emme1t
emme1t force-pushed the fix/split-random-state branch from d510aaf to 3401e5c Compare October 10, 2026 04:26

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant