Build(deps-dev): bump the python group across 1 directory with 6 updates

[dependabot]

Updates the requirements on datasets, transformers, trl, bitsandbytes, setuptools and setuptools-scm to permit the latest version.
Updates datasets to 5.0.0

Release notes

Sourced from datasets's releases.

5.0.0

Datasets Features

Agent traces

• Parse Agent traces messages for SFT using teich by @​lhoestq in huggingface/datasets#8232

  • Agent traces from claude_code/pi/codex and others can now be loaded with load_dataset
  • Using the teich library (new optional dependency), traces are parsed to messages to enable training on traces using e.g. trl
  • Load the data:

  >>> from datasets import load_dataset
  >>> ds = load_dataset("lhoestq/agent-traces-example", split="train")
  >>> ds[0]["messages"]
  [{'role': 'user', 'content': 'Download a random dataset from Hugging Face, use DuckDB to inspect it, and come back with a short report about it. Be concise and include: dataset name, what files/format you found, row count or rough size if you can determine it,...'
   ...]

  • Train on agent traces:

  trl sft --dataset-name lhoestq/agent-traces-example ...

  • find all the Agent traces datasets on HF here: https://huggingface.co/datasets?format=format:agent-traces&sort=trending

Next-level shuffling in streaming mode

• Use multiple input shards for shuffle buffer by @​lhoestq in huggingface/datasets#8194

  ds = load_dataset(..., streaming=True)
  ds = ds.shuffle(seed=42)
  # or configure local buffer shuffling manually, default is:
  ds = ds.shuffle(seed=42, buffer_size=1000, max_buffer_input_shards=10)

  before👎:

  after✨:

  toy example comparison

  from datasets import IterableDataset
  ds = IterableDataset.from_dict({"i": range(123_456_789)}, num_shards=1024)
  ds = ds.shuffle(seed=42)
  print("Cold start ids:")

... (truncated)

Commits

• 68ac1a9 Release: 5.0.0 (#8239)
• cfe4492 Support composed splits in streaming datasets (#8220)
• fd67320 Keep None as a real null in Json() columns instead of the string "null" (#8231)
• 10cdc81 Fix iterable skip over full Arrow blocks (#8236)
• b7c064d Parse agent traces messages for SFT using teich (#8232)
• 31e92f1 fix: embed_external_files=True for mesh support (#8224)
• d168d5f feat: add TsFile (Apache IoTDB) packaged builder with per-device wide format ...
• 992f3cf fix(map): fix progress bar exceeding total when load_from_cache_file=False (#...
• 8474a91 Fix single lance file form pylance 7.0 (#8225)
• d4284e9 feat: add 3D mesh support and MeshFolder builder (#8055)
• Additional commits viewable in compare view

Updates transformers to 5.14.1

Release notes

Sourced from transformers's releases.

Patch release: v5.14.1

Patch release v5.14.1

This patch solves a few issues which appeared when integrating Inkling model, most notably an issue affecting models using EncoderDecoderCache during assisted generation. It also fixes an issue that could appear during prefill with StaticCache and sdpa without padding for Inkling which uses a position_bias. It contains the following commits:

• Fix sdpa prefill with position_bias (#47359) by @​Cyrilvallez
• Fix assisted decoding for models with EncoderDecoder cache & OlmoHybrid (#47361) by @​Cyrilvallez
• [FP8] Bump kernels version (#47344) by @​vasqu
• Fix deepgemm on multiple devices (#47323) by @​IlyasMoutawwakil

Commits

• a08ace4 Release: v4.14.1
• 1ccf4f1 Fix deepgemm on multiple devices (#47323)
• c98cc10 [FP8] Bump kernels version (#47344)
• e035e45 Fix assisted decoding for models with EncoderDecoder cache & OlmoHybrid (#47361)
• 0eb5e03 Fix sdpa prefill with position_bias (#47359)
• 2fa8c19 v5.14.0
• 6c9318e fix feature extraction
• 8f23330 Small improvement (#47348)
• 76613e6 Add Inkling model (#47347)
• f5a67a4 🚨 Pass tp_plan from lm_heads (#47253)
• Additional commits viewable in compare view

Updates trl to 1.8.0

Release notes

Sourced from trl's releases.

v1.8.0

Features

🎓 KTO is now a stable trainer

After many cycles of KTOTrainer ↔ DPOTrainer alignment work, KTO graduates from trl.experimental.kto to the top-level trl package. Same API as DPO/GRPO/SFT — imports move from experimental, tests move to the main test tree, docs no longer flag it as experimental. The experimental path still works and emits a FutureWarning (removal in v2.0.0).

# Before
from trl.experimental.kto import KTOConfig, KTOTrainer
Now
from trl import KTOConfig, KTOTrainer

Per our telemetry, KTO is the 4th most used trainer in TRL — this graduation was overdue.

by @​albertvillanova in huggingface/trl#6175, huggingface/trl#6287 and huggingface/trl#6345

Environment-owned rewards & multi-environment support

Three interrelated changes make agentic RL training with environments substantially more ergonomic.

Environment-owned reward. If your environment_factory env defines a reserved get_reward() method (no args → float), it's called once per completed rollout and treated as a reward source. reward_funcs becomes optional — no more leaking env state back out to trainer-owned reward funcs.

class WordleEnv:
    def reset(self, **kwargs):
        self._target = sample(words); self._solved = False
def get_reward(self) -> float:       # optional, reserved (not a tool)
    return 1.0 if self._solved else 0.0
def guess(self, word: str) -> str:   # exposed as a tool
self._solved = word == self._target; ...

trainer = GRPOTrainer(
model=model,
train_dataset=dataset,
environment_factory=WordleEnv,       # no reward_funcs needed
)

Multi-environment support. environment_factory now accepts dict[str, factory] in addition to a single callable. Each dataset row selects its environment via an environment column, and only that env's tools are exposed in that row's prompt — so a coding task and a game can train together in one run without leaking each other's tool schemas. Single-callable usage is unchanged.

Same wiring lands in GRPO, AsyncGRPO, DPPO, and GRPO-with-replay-buffer.

Env-owned reward by @​qgallouedec in huggingface/trl#6238; multi-env in huggingface/trl#6001 and huggingface/trl#6002

Entropy regularization for GRPO

... (truncated)

Commits

• 95809b9 Release: v1.8 (#6346)
• 2ee37b8 Docs: treat KTO as a stable trainer (#6345)
• bae8238 Fix incorrect examples in distillation docs (#6334)
• 00d6209 Neuron-friendly chunked_nll via static-shape token packing (#6314)
• 9ffb54e Fix chunked_nll patch hiding VLM kwargs from generate (#6156)
• ce39cbb Raise on quantization_config + already-instantiated model in DPOTrainer/ ...
• 70f95f3 Document that max_steps is required for iterable train datasets (#6333)
• 627a83e Support DatasetDict and IterableDatasetDict as eval_dataset in trainers (#6322)
• 8c75069 Align KTO with DPO: quantization_config trainer argument (#6276)
• cfef01a Environment-owned reward (#6238)
• Additional commits viewable in compare view

Updates bitsandbytes from 0.45.5 to 0.49.2

Release notes

Sourced from bitsandbytes's releases.

0.49.2

Highlights

• The default blocksize of 64 for 4bit quantization is now supported on ROCm. Previously the default was 128, which was a mismatch from the default for other devices.
• ROCm 7.2 build is now included.

What's Changed

• bug: fix 8bitoptim support with fsdp by @​ved1beta in bitsandbytes-foundation/bitsandbytes#1840
• Fix xpu 4bit kernel by @​jiqing-feng in bitsandbytes-foundation/bitsandbytes#1839
• ROCm 7.2 build and doc changes by @​sstamenk in bitsandbytes-foundation/bitsandbytes#1845
• Add CUDA kernel support for 4-bit quantization with blocksize=32 by @​Abdennacer-Badaoui in bitsandbytes-foundation/bitsandbytes#1854
• Add blocksize=64 4-bit quantization support for ROCm CDNA (warp64) GPUs by @​Abdennacer-Badaoui in bitsandbytes-foundation/bitsandbytes#1856
• [Docs Update] QLoRA 4-bit Support on ROCm by @​Abdennacer-Badaoui in bitsandbytes-foundation/bitsandbytes#1857
• [ROCm] Make blocksize=64 default for 4bit by @​matthewdouglas in bitsandbytes-foundation/bitsandbytes#1873
• Handle non-contiguous tensors in quantize/dequantize ops by @​TimDettmers in bitsandbytes-foundation/bitsandbytes#1859
• Fix AdEMAMix scheduler guard and add state_dict round-trip test by @​TimDettmers in bitsandbytes-foundation/bitsandbytes#1861

New Contributors

• @​Abdennacer-Badaoui made their first contribution in bitsandbytes-foundation/bitsandbytes#1854

Full Changelog: bitsandbytes-foundation/bitsandbytes@0.49.1...0.49.2

0.49.1

What's Changed

• Update AMD targets by @​sstamenk in bitsandbytes-foundation/bitsandbytes#1832
• add guard for self.weight.quant_state attribute by @​winglian in bitsandbytes-foundation/bitsandbytes#1837

Full Changelog: bitsandbytes-foundation/bitsandbytes@0.49.0...0.49.1

0.49.0

Highlights

x86-64 CPU Improvements

CPU performance for 4bit is significantly improved on x86-64, with optimized kernel paths for CPUs that have AVX512 or AVX512BF16 support.

AMD ROCm Experimental Wheels

• Experimental support for AMD devices is now included in our PyPI wheels on Linux x86-64.
• We've added additional GPU target devices as outlined in our docs.
• Support for using the default blocksize of 64 for 4bit was added for RDNA GPUs in #1748.

macOS 14+ Wheels

• We're now publishing wheels for macOS 14+!
• The 4bit and 8bit quantization features are supported on MPS by slow implementations. We plan to enable Metal kernels with improved performance in the future.

🚨 Breaking Changes

• Dropped support for Python 3.9.
• Dropped compilation support for Maxwell GPUs in the CUDA backend.

What's Changed

• [ROCm] Update build targets by @​matthewdouglas in bitsandbytes-foundation/bitsandbytes#1788

... (truncated)

Commits

• f0e6ca3 Release 0.49.2
• cace65c Fix AdEMAMix scheduler guard and add state_dict round-trip test (#1861)
• 505a00a Handle non-contiguous tensors in quantize/dequantize ops (#1859)
• c2ae381 [ROCm] Make blocksize=64 default for 4bit (#1873)
• 943e42d Skip typo check on agent markdown files
• b075afc update docs (#1857)
• de89ff7 docs: Add PR review posting guide, agent reference docs, and lint fixes
• a2c92f7 docs: Add human+agent issue triage workflow guide
• 252e6ff docs: Update issue patterns from 43 recently closed issues
• 88c6c71 style: Fix ruff format violation in test_linear4bit.py
• Additional commits viewable in compare view

Updates setuptools from 80.9.0 to 83.0.0

Changelog

Sourced from setuptools's changelog.

v83.0.0

Features

• Require Python 3.10 or later.

Bugfixes

• MANIFEST.in matching (via FileList) is now insensitive to Unicode normalization form. A pattern authored in one form (e.g. NFC, as typically saved by editors) now matches a file whose name is stored on disk in another (e.g. NFD, as produced by macOS APFS/HFS+). Previously an exclude, global-exclude, recursive-exclude, or prune rule could silently fail to drop a non-ASCII-named file from the source distribution, publishing it despite the exclusion -- see GHSA-h35f-9h28-mq5c.

Deprecations and Removals

• pypa/distutils#334

v82.0.1

Bugfixes

• Fix the loading of launcher manifest.xml file. (#5047)
• Replaced deprecated json.__version__ with fixture in tests. (#5186)

Improved Documentation

• Add advice about how to improve predictability when installing sdists. (#5168)

Misc

• #4941, #5157, #5169, #5175

v82.0.0

... (truncated)

Commits

• 6519f72 Bump version: 82.0.1 → 83.0.0
• d1151b1 Merge pull request #5250 from pypa/feature/distutils-d7633fbed
• a2df31e Capture removal of dry_run parameter in changelog.
• 00144dc Moved newsfragment to the release where it occurred.
• a4a5a2b Add news fragment.
• 77470c2 Merge https://github.com/pypa/distutils into feature/distutils-d7633fbed
• 3c43897 Merge pull request #5247 from pypa/copilot/fix-pypy-version-issue
• bb6ea66 Bump PyPy from 3.10 to 3.11 in CI workflow
• a2bc3ac Fix broken intersphinx reference to build's installation docs
• 2d6a739 Use stacked parametrize decorators instead of itertools.product
• Additional commits viewable in compare view

Updates setuptools-scm from 9.2.0 to 10.2.0

Release notes

Sourced from setuptools-scm's releases.

setuptools-scm v10.2.0

Added

• Restore Python 3.8 and 3.9 support, re-enabling use as a build dependency for projects like scikit-build that still support these versions. (#1445)

Miscellaneous

• Move PKG-INFO discovery tests from vcs-versioning to setuptools-scm where the entry points are registered. (#1446)

setuptools-scm v10.1.2

Fixed

• Fix DeprecationWarning leak by threading VcsEnvironment through VersionInferenceConfig and using env.make_reader() in _should_write_to_source. (#1424)

setuptools-scm v10.1.1

Fixed

• Update CI to use PyPy 3.11 as cryptography has no PyPy 3.10 build available (#1421)

setuptools-scm v10.1.0

Added

• Add backward-compatible shims in setuptools_scm.git, setuptools_scm.hg, setuptools_scm.hg_git, and setuptools_scm.scm_workdir so that external code calling get_scm_version(config) or run_describe(config) with an explicit Configuration continues to work. The shim automatically wires _config and VcsEnvironment onto the workdir. (#compat-shims)
• Write scm_version.json and scm_file_list.json into egg-info directories during egg_info, enabling sdist fallback version inference when no VCS is present. Add ScmEggInfoMixin for workdir-based file finding in find_sources(). (#egg-info-metadata)
• Add write_to_source pyproject.toml option to control whether version files are written to the source tree. When unset, a deprecation warning advises setting it explicitly before the default changes in a future major release. The SETUPTOOLS_SCM_WRITE_TO_SOURCE environment variable overrides this setting. (#1301)
• Adopt the workdir-centric pipeline from vcs-versioning: version discovery now follows an explicit env → config → workdir → version chain instead of relying on ambient globals and parse entry points. The egg_info command writes scm_version.json and scm_file_list.json metadata so sdists can infer versions without a VCS checkout. Requires vcs-versioning >= 2.0.0.dev0. (#1378)

Fixed

• Fix worktree file listing test to expect relative paths from the file finder. The test now passes on Linux; Windows remains xfail due to a subprocess limitation with worktree directories. (#620)
• Remove the _warn_on_old_setuptools() check that incorrectly warned when a custom build-backend caused setuptools.__version__ to return the project version instead of setuptools' version. The minimum setuptools version is now enforced via build-system requirements. (#1192)
• Wrap version in setuptools.sic() when normalize = false to prevent setuptools from re-normalizing the version after our hook returns. This preserves CalVer zero-padding (e.g. 2024.01.05) and other non-canonical version strings in dist.metadata.version. (#1354)
• Skip writing non-package version files to build_lib, fixing incorrect inclusion of root-level version files in wheels. (#1364)

Documentation

• Rewrite the GitHub Actions CI/CD example to use a dedicated build job (via build-and-inspect-python-package) and OIDC Trusted Publishers instead of building in publishing jobs with long-lived API tokens. (#1215)

setuptools-scm v10.0.5

Fixed

• Allow dump_version() deprecation warning to be silenced by passing scm_version=None. (#1286)
• Remove [tool.uv.sources] from setuptools-scm/pyproject.toml to fix sdist builds outside the workspace — the workspace root already declares the source mapping for development. (#1330)

setuptools-scm v10.0.4

... (truncated)

Commits

• a705891 Merge pull request #1448 from pypa/release/main
• 630448e Prepare release: setuptools-scm v10.2.0, vcs-versioning v2.2.0
• aaf4950 Merge pull request #1445 from RonnyPfannschmidt/python-legacy
• dd60bf9 Merge pull request #1447 from RonnyPfannschmidt/fix/1446-move-pkginfo-tests
• 535f5ff fix: move PKG-INFO discovery tests to setuptools-scm (#1446)
• 3546a0e feat: restore Python 3.8 and 3.9 support
• 0cde123 Merge pull request #1443 from pypa/release/main
• 51bc391 Prepare release: vcs-versioning v2.1.2
• 310f3c3 Merge pull request #1442 from RonnyPfannschmidt/fix/1439-metadata-workdir-cus...
• b7b70a8 fix: don't re-parse stored tags through tag_regex in MetadataWorkdir (#1439)
• Additional commits viewable in compare view