Skip to content

Commit ea3641d

Browse files
authored
Skip torchtune on Python 3.14 in the examples requirements (pytorch#21832)
## The problem Every Python 3.14 wheel job fails, on all four platforms, before it builds anything. It dies in the pre-build step that installs `requirements-examples.txt`. Here is the chain: ``` requirements-examples.txt -> torchtune declares "pyarrow<21.0.0" -> no pyarrow below 21 publishes a cp314 wheel -> pip downloads pyarrow-20.0.0.tar.gz and compiles it -> the source build needs a prebuilt Arrow C++ library, which is absent -> pip returns non-zero and the whole job dies ``` From a failing log: ``` Collecting pyarrow<21.0.0 (from torchtune ...) Downloading pyarrow-20.0.0.tar.gz ... ERROR: Failed building wheel for pyarrow error: failed-wheel-build-for-install ``` Python 3.10 through 3.13 are fine because pyarrow 20.0.0 ships wheels for those versions, so they download a binary and never compile anything. Only 3.14 falls through to the source build. The first pyarrow release with a cp314 wheel is 22.0.0, which is above torchtune's cap, so there is no version in the allowed range that works on 3.14. ## The fix One environment marker, so torchtune is skipped on Python 3.14: ``` torchtune @ git+https://github.com/pytorch/torchtune.git@3c1872ac... ; python_version < "3.14" ``` The pyarrow cap belongs to torchtune, so it cannot be relaxed from here. Once torchtune is out of the picture on 3.14, the only remaining constraint is `pyarrow>=15.0.0` from `datasets`, and pip picks a cp314 wheel. This is the same approach already used for `coremltools` in `setup.py`, which likewise has no cp314 support. ## What this does and does not fix Being precise, because this change is a prerequisite rather than the whole story. It fixes the pre-build step on all four platforms. With that step passing, the aarch64 Linux and Windows 3.14 jobs run to completion. It does not by itself make the Linux x86_64 and macOS 3.14 jobs green. Once the pre-build step passes, those two go on to run a Core ML case in their wheel smoke test: ``` .ci/scripts/wheel/test_linux.py backend=Backend.CoreMlExportOnly .ci/scripts/wheel/test_macos.py backend=Backend.CoreMlExportAndTest ``` The wheel deliberately omits `coremltools` on 3.14 (`setup.py`, the marker mentioned above), and neither test script has a version guard, so the Core ML case fails with `ModuleNotFoundError: No module named 'coremltools'`. That needs a separate, small change to skip the Core ML case on 3.14, which is a CI test change rather than a requirements change and so is kept out of this commit. A developer install on 3.14 is also worth one note. It previously failed at the pyarrow source build and now succeeds, but `executorch.extension.llm.modules` imports torchtune eagerly, so that one subsystem is unavailable on 3.14. torchtune cannot be installed on 3.14 by any means today, so no version of this change keeps those modules working. ## Testing Verified: - Confirmed against the package index that no pyarrow below 21.0.0 has a cp314 file, and that 22.0.0 is the first release that does. - Reproduced the failure in a real Python 3.14 environment, matching the error chain above. - Confirmed that simply providing cmake is not enough: with cmake working, the source build then fails at `Could not find a package configuration file provided by "Arrow"`. - With this change, installing `requirements-examples.txt` on Python 3.14 succeeds and resolves pyarrow to a cp314 wheel, and `datasets`, `timm`, `torchsr` and `transformers` all import. - On Python 3.13 the same file still installs torchtune with pyarrow 20.0.0 from a wheel, so behaviour below 3.14 is unchanged. Not verified: - A full wheel build or its smoke tests on Python 3.14.
1 parent 813145d commit ea3641d

1 file changed

Lines changed: 4 additions & 1 deletion

File tree

‎requirements-examples.txt‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,8 @@
33
datasets == 3.6.0 # 4.0.0 deprecates trust_remote_code and load scripts. For now pin to 3.6.0
44
timm == 1.0.7
55
torchsr == 1.0.4
6-
torchtune @ git+https://github.com/pytorch/torchtune.git@3c1872ace149f03ef4ffec765d3f0a5fd0399497
6+
# torchtune caps pyarrow below 21, and no pyarrow in that range ships a cp314 wheel,
7+
# so pip falls back to the sdist and its Arrow C++ build fails. Skip it on 3.14 until
8+
# torchtune relaxes the cap; nothing that needs torchtune runs on 3.14 yet.
9+
torchtune @ git+https://github.com/pytorch/torchtune.git@3c1872ace149f03ef4ffec765d3f0a5fd0399497 ; python_version < "3.14"
710
transformers == 5.0.0rc1

0 commit comments

Comments
 (0)