Repository navigation
Commit 23355ed
fix(deps): declare httpx, unbreaking partial-install (torch) for every PR (#2547)
## Summary
`partial-install (torch)` has been failing on **every** PR since
2026-09-24 — including PRs whose branches predate the breakage — and
because it is a *collection* error rather than a test failure, it aborts
the entire run:
```
ImportError while importing test module '.../tests/unit/torch/speculative/plugins/test_hf_streaming_dataset.py'
E ModuleNotFoundError: No module named 'httpx'
collected 2243 items / 1 error / 45 skipped
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
```
`unit-pr-required-check` aggregates it, so nothing currently merges on a
fresh run.
## What happened
**No code changed.**
`modelopt/torch/speculative/plugins/hf_streaming_dataset.py` has
imported `httpx` at module scope since #1509 (2026-06-02), and `httpx`
has never appeared in `pyproject.toml`. It arrived only transitively:
`dev-test` → `timm` → `huggingface_hub` → `httpx`.
**huggingface_hub 2.0.0**, published **2026-09-24T12:01:21Z**, replaced
`httpx<1,>=0.23.0` with the separate **`httpx2<3,>=2.0.0`**
distribution. Different package name, so `httpx` stopped being installed
and the chain disappeared.
The boundary is exact — every run *created* before that timestamp
passes, every one after fails:
| PR | run created | result |
|---|---|---|
| #2536 / #2535 | 09-23 22:02 | pass |
| #2500 | 09-23 23:45 | pass — **merged 09-24 20:01 on this stale-green
result** |
| *hub 1.33.0 (still requires httpx)* | *09-24 09:49* | |
| **hub 2.0.0 published** | **09-24 12:01** | ← |
| #2539 | 09-24 16:57 | fail |
| #2544 | 09-24 18:32 | fail |
| #2216 | 09-25 11:58 | fail |
#2500 merging afterwards is not a counterexample: GitHub does not re-run
checks at merge time, so it merged on a result from ~20 hours earlier.
That is also why this went unnoticed.
## The changes
### 1. Declare `httpx` in the `hf` extra
`httpx` is not incidental to streaming — it is the only transport:
- every fetch is HTTP: `POST /v1/completions` to the vLLM serve plus
`GET /meta` and `/desc` against the connector's sidecar, all through
`httpx.Client`;
- there is no non-HTTP path — the base `StreamingDataset._fetch` is an
abstract seam and `EagleVllmStreamingDataset._fetch` is its only
implementation;
- no other HTTP library appears in the module (`requests` / `urllib` /
`aiohttp`: zero hits, and `requests` is not declared either);
- even the retry predicate is built from it: `_TRANSIENT_FETCH_ERRORS =
(httpx.HTTPError, OSError)`.
It belongs in `hf` rather than in the core `dependencies`: the same
module needs `transformers.trainer_pt_utils` at module scope, so one
extra already gates the whole file, and a core install has no use for an
HTTP client. The bound matches the 0.x API the code uses — `httpx` has
no 1.0 release, and 2.x is a different distribution.
This is the part that stops it recurring. `[hf]` currently gets `httpx`
only because `datasets` happens to require it — the same accident with a
different supplier, one release away from repeating.
### 2. Acquire `httpx` in the test through the existing skip guard
The test file already intends to skip where the extra is absent — it has
`pytest.importorskip("transformers")` and a comment explaining why, and
`transformers` is absent in this job too. It broke only because `import
httpx` sat **five lines above** that guard, where a missing module ends
collection instead of skipping one file.
## Verification
- With everything installed: **18 passed**, no behaviour change.
- The import-order property is checked with an AST walk over the
module's top-level statements: no `hf`-extra-only import precedes the
first `importorskip` (which is now line 41).
- A faithful local reproduction was attempted and abandoned honestly:
hiding `httpx` locally also breaks `huggingface_hub` 1.28, which
`modelopt.torch.opt.plugins.huggingface` imports, so the local failure
is not the CI one. CI is the oracle for that half — this PR's own
`partial-install (torch)` run is the check that matters.
## Scope
Two files, five lines of declaration and four of test import order.
Deliberately not folded into any feature PR: it blocks the whole repo,
and burying a repo-wide fix inside unrelated work is how these stay
invisible.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Optional Hugging Face installations now include `httpx`, supporting
features that require HTTP communication without requiring it for all
installations.
* **Tests**
* Hugging Face streaming dataset tests now skip when `httpx` is
unavailable, allowing the remaining test suite to be collected and run
without it.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: h-guo18 <67671475+h-guo18@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent ed7e879 commit 23355ed
3 files changed
Lines changed: 6 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
| 85 | + | |
85 | 86 | | |
86 | 87 | | |
87 | 88 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
34 | 33 | | |
35 | 34 | | |
36 | 35 | | |
37 | 36 | | |
38 | 37 | | |
| 38 | + | |
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments