Repository navigation
Support for datasets v4? #619
Description
Hi there!
I'm looking into migrating https://github.com/mantisai/sieves to datasets v4, but I can't seem to get training with setfit to work with dataset generated with datasets v4 (whereas the same dataset works with sentence-transformers).
Questions:
- Is
setfitindeed not fully compatible withdatasetsv4 (or am I doing something wrong)? - If the answer to the question above is no: is the migration to v4 planned and if so, what's the timeline?
Thanks for the great work!
Activity
tomaarsen commentedon Sep 4, 2026
Hello!
SetFit is compatible with datasets v4 and v5 on
mainvia #647. The remaining incompatibility was thatdataset["column"]returns a lazyColumnsince datasets 4.0, which SetFit passed around as if it were a list. The trainer now converts the columns it consumes, and the suite runs on datasets 4.x and 5.0.1 in CI. This will be part of the upcoming v1.2.0 release, so the sieves migration should be unblocked then.