Introduce compatibility with transformers v5, Sentence Transformers v6 and huggingface_hub v1

[tomaarsen]

Hello!

Heads up, this PR was AI-generated and human-reviewed.

Pull Request overview

• Introduce compatibility with transformers v5, Sentence Transformers v5.4 and v6, huggingface_hub v1 and datasets v4 and v5
• Add Python 3.13 support and rework the CI matrix into explicit dependency presets
• Fix ONNX export with recent torch, onnx and skl2onnx releases

Details

SetFit had fallen behind the rest of the ecosystem. It no longer imported on transformers v5 (default_logdir moved), loading any model from the Hub failed on huggingface_hub v1 (HfHubHTTPError no longer derives from requests, and proxies and resume_download were removed from hf_hub_download), every sentence_transformers.losses, models, training_args and model_card import warns on Sentence Transformers v5.4+, and the CI had been red on every cell since April. This PR keeps support for the old versions (transformers 4.41, Sentence Transformers 3.0, huggingface_hub 0.24, datasets 2.15) while adding the new ones.

The version dependent imports now live in setfit/compat.py, which tries the Sentence Transformers v5.4+ module paths first and falls back to the old ones, and imports default_logdir from transformers.integrations.integration_utils with a fallback. SetFitModel.from_pretrained catches both HfHubHTTPError and EntryNotFoundError, because on huggingface_hub v1 the offline and local_files_only=True misses raise a LocalEntryNotFoundError that no longer inherits from HfHubHTTPError.

On transformers v5, warmup_ratio and logging_dir are gone, so the warmup proportion is now passed as a fractional warmup_steps, and logging_dir is ignored with a warning (v5 reads TENSORBOARD_LOGGING_DIR instead). report_to and seed are now given to the inner SentenceTransformerTrainingArguments at construction, since the integration callbacks are created in Trainer.__init__. Before, report_to was applied afterwards and silently ignored: report_to="none" still attached the CodeCarbon callback on transformers v4, and with the v5 default of "none" all reporting would have disappeared. So report_to is honoured now, which is a small behaviour change. Finally, the integration callbacks of transformers (Trackio, W&B) started reading arguments that the SetFit TrainingArguments does not define, such as args.project, so the SetFit arguments now defer unknown attributes to the transformers arguments of the underlying trainer. Custom callbacks keep receiving the SetFit arguments as before.

datasets v4 returns a lazy Column for dataset["text"], which is converted to a list in Trainer.evaluate and dataset_to_parameters. The ONNX exporter requests the legacy exporter explicitly (torch 2.9 defaults to the dynamo based one, which does not support the opsets used here) and converts the scikit-learn head with a float32 input, matching the float32 embeddings of the body. The double precision path it used to have never actually ran: its type check compared against an onnxconverter_common class that skl2onnx stopped using, and skl2onnx 1.20 no longer ships that dependency at all. openvino.runtime was removed in openvino 2026, so the OpenVINO exporter falls back to the top level API. The model card skips emissions data when the codecarbon tracker cannot report, since recent codecarbon 3.x releases raise for a tracker that was never started, e.g. when saving before training with codecarbon installed.

On the packaging side, evaluate>=0.4.6 is required as older versions cannot download metric scripts with huggingface_hub v1 (HfFolder is gone), the codecarbon range excludes 2.6 (print spam) and 2.7 and 2.8 (a second tracker in the same process, which AbsaTrainer creates, fails with a missing _cloud), hummingbird-ml moved out of the tests extra as it pins onnx<=1.16.1 without Python 3.13 wheels, and spaCy is unpinned on Python 3.10+. Python 3.9 keeps spacy<3.8, numpy<2 and onnx<1.17 through environment markers: the spaCy 3.8 line depends on blis releases without 3.9 wheels, the spaCy 3.7 binaries need numpy 1.x, and the 3.9 wheels of onnx 1.17 to 1.19 crash on Windows once pyarrow.dataset has been imported. These were previously hidden by the spacy<3.7.6 pin and by the openvino and hummingbird extras pinning numpy and onnx.

The CI matrix now runs .[tests] on Python 3.9 to 3.13 on both operating systems, plus two presets as include entries: .[compat_tests] (the minimum pins) on 3.9 and the new .[compat_tests_v4] (transformers<5, sentence-transformers<6, huggingface_hub<1, datasets<5) on 3.11. That is 14 jobs instead of 16. The minimum pins no longer run on 3.12, where pandas 1.5.3 has no wheels (that cell failed at the install step before), and the three presets cover each branch in setfit/compat.py: Sentence Transformers 3.0, 4 to 5.3, and 5.4+, next to transformers v4 versus v5 and huggingface_hub v0 versus v1. Without the v4 preset, that generation was only tested because Python 3.9 happens to resolve to it.

I ran the full suite locally on Windows for each kind of cell:

Cell	Resolution	Result
.[tests], Python 3.13	transformers 5.16.1, sentence-transformers 6.0.1, huggingface_hub 1.30.0, datasets 5.0.1, torch 2.14.0	158 passed, 17 skipped
.[tests], Python 3.9	transformers 4.57.6, sentence-transformers 5.1.2, huggingface_hub 0.36.2, datasets 4.5.0, onnx 1.16.1	158 passed, 17 skipped
.[compat_tests_v4], Python 3.11	transformers 4.57.6, sentence-transformers 5.7.0, huggingface_hub 0.36.2, datasets 4.8.5	158 passed, 17 skipped
.[compat_tests], Python 3.9	transformers 4.41.0, sentence-transformers 3.0.0, huggingface_hub 0.24.0, datasets 2.15.0	158 passed, 17 skipped

The skips are the CUDA only tests, the OpenVINO test and the two Optuna tests (optuna is not part of .[tests]). The suite also passes against the sentence-transformers 6.1 dev checkout with transformers 5.15 and huggingface_hub 1.24, with the Sentence Transformers deprecated import warnings turned into errors.

A few caveats. With transformers v5, warmup_proportion=1.0 results in a single warmup step, as transformers reads a warmup_steps of 1 or more as a step count. The hyperparameter search tests no longer sample the liblinear solver, which scikit-learn 1.9 rejects for multiclass problems, and the hyperparameter optimization guide has the same solver in its search space.

• Tom Aarsen

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Copilot]

🟡 Changes recommended

SetFitModel._from_pretrained can still raise an uncaught TypeError on huggingface_hub v1 when proxies/resume_download are provided, undermining the stated compatibility goal.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates SetFit to remain compatible across major ecosystem changes (transformers v5, Sentence Transformers v6, huggingface_hub v1, datasets v4/v5) while keeping support for older pinned versions, and modernizes CI/testing to cover those compatibility branches.

Changes:

• Added setfit/compat.py to centralize version-dependent imports/paths and updated SetFit internals to use it.
• Updated training argument propagation and callback wiring to work with transformers v5 behavior changes (report_to, warmup handling, logging_dir).
• Refreshed exporters (ONNX/OpenVINO) and expanded tests + CI matrix (Python 3.13, explicit dependency presets).

File summaries

File	Description
tests/test_trainer.py	Switches losses import to compat and adds tests for report_to + warmup behavior.
tests/test_modeling.py	Updates dataset ID and adds coverage for local_files_only loading without SetFit files.
tests/test_model_card.py	Updates dataset ID and adds a model-card generation regression test with codecarbon installed.
tests/test_deprecated_trainer.py	Aligns deprecated trainer tests with compat losses + solver search-space change.
tests/test_deprecated_trainer_distillation.py	Uses compat losses for distillation tests.
tests/exporters/test_openvino.py	Makes OpenVINO test imports conditional to avoid hard dependency.
src/setfit/utils.py	Switches Sentence Transformers losses import to compat.
src/setfit/training_args.py	Adds transformers v5-aware logging_dir behavior and delegates unknown attrs to underlying transformers args.
src/setfit/trainer.py	Routes version-specific ST/transformers pieces through compat; updates callback and warmup/logging propagation; converts dataset columns to lists for datasets v4 behavior.
src/setfit/trainer_distillation.py	Uses compat losses and normalizes dataset column access to lists.
src/setfit/span/modeling.py	Guards docstring slicing against None docstrings.
src/setfit/modeling.py	Updates head base class via compat and adjusts hf_hub_download error handling for hub v1 exceptions.
src/setfit/model_card.py	Prevents codecarbon v3 “not started” reporting errors from breaking model-card generation.
src/setfit/exporters/openvino.py	Adjusts OpenVINO import path for 2026+ while keeping backward compatibility.
src/setfit/exporters/onnx.py	Forces legacy torch.onnx exporter when supported and updates sklearn-head conversion path.
src/setfit/compat.py	New compatibility shim for transformers/ST moved modules and default_logdir.
setup.py	Updates dependency constraints/extras, adds Python 3.13 classifier and python_requires>=3.9.
.github/workflows/tests.yml	Expands CI matrix to Python 3.13 and adds explicit compat presets + HF token for rate limiting.

Review details

Suppressed comments (1)

src/setfit/modeling.py:818

• Same issue as the config download: if proxies or resume_download are passed, huggingface_hub v1 may raise TypeError due to unsupported kwargs, and the error isn't caught here. Retrying without those keys keeps SetFitModel.from_pretrained compatible while still honoring them on older hub versions.

            try:
                model_head_file = hf_hub_download(filename=MODEL_HEAD_NAME, **download_kwargs)
            except (HfHubHTTPError, EntryNotFoundError):
                logger.info(
                    f"{MODEL_HEAD_NAME} not found on HuggingFace Hub, initialising classification head with random weights."
                    " You should TRAIN this model on a downstream task to use it for predictions and inference."
                )
                model_head_file = None

• Files reviewed: 18/18 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.