Repository navigation
Commit 3353bd5
authored
Record why scikit-learn is a real dependency, not a mistaken one (pytorch#22269)
## Summary
This started as a removal and became a comment fix, because the removal
was wrong.
The reading was that `scikit-learn>=1.7.1` could never work: coremltools
9.0 accepts scikit-learn
only up to 1.5.1, so `_HAS_SKLEARN` is always False and coremltools
reports its scikit-learn
support disabled on every install.
That is all true, and the conclusion drawn from it was still wrong.
**coremltools does not reach
scikit-learn only through that flag.** `quantization_utils.py` imports
`sklearn.cluster.KMeans` at
the point of use, and that is the *default* clustering path, taken
unless the weight is a single
column of at least ten thousand float16 values. So palettization needs
scikit-learn whatever
`_HAS_SKLEARN` says, and the `CODEBOOK_WEIGHT_ONLY` recipe this repo
exposes fails without it:
```
ModuleNotFoundError: scikit-learn is required for k-means quantization.
```
The version ceiling and the palettization path are simply unrelated. The
ceiling gates
coremltools' converter for scikit-learn's *own* model types, which
nothing here uses.
Palettization never checks the version.
## So what changed
Nothing functional. The dependency stays exactly as it is.
What was wrong was the comment. It said nothing here imports
scikit-learn, which is true but
misleading, and it did not explain why an apparently unsatisfiable pin
is correct. Anyone auditing
dependencies would reach the same wrong conclusion I did, and might act
on it. The comment now
records the real reason, and the test tying this entry to coremltools
says why they cannot be
separated.
## Test plan
Measured with coremltools 9.0 on macOS arm64:
| environment | result |
| --- | --- |
| scikit-learn 1.9.0 present | `_HAS_SKLEARN` False, palettization
works, lut shape `(16, 1)` |
| scikit-learn absent | `ModuleNotFoundError` from
`quantization_utils.py` |
Which confirms both halves: the converter really is disabled, and
palettization is unaffected by
that.
`.ci/scripts/tests/test_coreml_markers.py` passes, 167 tests.1 parent 80f7005 commit 3353bd5
2 files changed
Lines changed: 21 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
221 | 221 | | |
222 | 222 | | |
223 | 223 | | |
224 | | - | |
225 | | - | |
226 | | - | |
227 | | - | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
228 | 230 | | |
229 | 231 | | |
230 | 232 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
696 | 696 | | |
697 | 697 | | |
698 | 698 | | |
699 | | - | |
700 | | - | |
701 | | - | |
702 | | - | |
703 | | - | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
704 | 714 | | |
705 | 715 | | |
706 | 716 | | |
| |||
0 commit comments