Skip to content

build(deps): Bump the cargo-candle group in /apps/router/candle-binding with 3 updates - #104

Open
dependabot[bot] wants to merge 1 commit into
devfrom
dependabot/cargo/apps/router/candle-binding/dev/cargo-candle-ca85fe36c2
Open

build(deps): Bump the cargo-candle group in /apps/router/candle-binding with 3 updates#104
dependabot[bot] wants to merge 1 commit into
devfrom
dependabot/cargo/apps/router/candle-binding/dev/cargo-candle-ca85fe36c2

Conversation

@dependabot

@dependabot dependabot Bot commented on behalf of github Aug 3, 2026

Copy link
Copy Markdown
Contributor

Bumps the cargo-candle group in /apps/router/candle-binding with 3 updates: tokenizers, hf-hub and rand.

Updates tokenizers from 0.22.2 to 0.23.1

Release notes

Sourced from tokenizers's releases.

Release v0.23.1

TL;DR

tokenizers 0.23.1 is the first proper stable release in the 0.23 line — 0.23.0 only ever shipped as rc0 because the release pipeline itself was broken (Node side hadn't shipped multi-platform binaries since 2023, Python side was on pyo3 0.27 without free-threaded support). 0.23.1 is the version where everything actually goes out the door together: full Node multi-platform wheels for the first time in years, Python 3.14 (regular and free-threaded 3.14t), full type hints for every Python class, and a stack of measurable perf wins on the BPE / added-vocab hot paths.

There is no functional 0.23.0 published — we tag 0.23.1 directly so users don't accidentally pull a never-shipped version.


🚨 Breaking changes

  • Drop Python 3.9 (#1952) — requires-python = ">=3.10"; 3.9 users stay on 0.22.x.
  • add_tokens normalizes content at insertion (#1995) — re-saved tokenizer.json may differ in the added_tokens block. Existing files load unchanged.
  • Type stubs are precise (#1928, #1997) — methods that returned Any now return real types; mypy --strict may surface previously-hidden errors. Stub layout also moved from tokenizers/<sub>/__init__.pyi to tokenizers/<sub>.pyi. This breaks the surface of some of the processors like RobertaProcessign's __init__ .
  • 3.14t-only: setters/getters return PyResult<T> because of Arc<RwLock<Tokenizer>>; a poisoned lock surfaces as PyException instead of a panic.

⚡ Performance — measured locally on this Mac, not lifted from PRs

Run with cargo bench --bench <name> -- --save-baseline v0_22_2 on v0.22.2, then --baseline v0_22_2 on v0.23.1. Numbers are point-in-time wall clock on a single laptop; relative deltas are what matters, absolute numbers will differ on CI hardware.

Added-vocabulary deserialize — the headline win (#1995, #1999)

bench: improve added_vocab_deserialize to reflect real-world workloads (#2000) is now representative of how transformers actually loads tokenizer.json files. The combined effect of daachorse for the matching automaton plus the normalize-on-insert refactor is enormous on this workload:

benchmark v0.22.2 v0.23.1 change
100k tokens, special, no norm ~410 ms 248 ms −40%
100k tokens, non-special, no norm ~7.1 s 273 ms −96%
100k tokens, special, NFKC ~395 ms 235 ms −40%
100k tokens, non-special, NFKC ~7.4 s 290 ms −96%
400k tokens, special, no norm ~15 s 980 ms −94%

Real-world impact: loading a Llama-3-style tokenizer with a large set of added tokens dropped from "noticeable pause" to "instant".

BPE encode

benchmark v0.22.2 v0.23.1 change
BPE GPT2 encode batch, no cache 530 ms 446 ms −16%
BPE GPT2 encode batch (cached) 690 ms 685 ms noise
BPE GPT2 encode (single) 1.95 s 1.94 s noise
BPE Train (small) 32.6 ms 31.5 ms −3%
BPE Train (big) 1.01 s 988 ms −2%

The BPE per-thread cache PR (#2028) shows much larger wins on highly-parallel workloads (+47–62% at 88+ threads on a server box, per the PR's own measurements on Vera). Single-thread batch numbers above are flat or slightly improved because cache-hit overhead was already low without contention.

Llama-3 encode

... (truncated)

Commits

Updates hf-hub from 0.5.0 to 1.0.0

Changelog

Sourced from hf-hub's changelog.

Releasing hf-hub

This document covers the full release process for the hf-hub crate. If anything here is unclear or out of date, please open a PR.

What gets released

A single tag push releases one artifact:

  • hf-hub Rust crate on crates.io, via .github/workflows/rust-release.yml.

The workflow triggers on tags matching v* (e.g., v1.0.0, v1.0.0-rc.0).

There are no Python components in this repo. The other workspace members are not published:

  • hfrs/ — CLI binary, distributed via cargo install --git.
  • examples/, benches/, integration-tests/ — internal-only, version 0.0.0, never published.

Pre-release checklist

  1. CI is green on main. The Rust workflow must be passing on every platform in the matrix (Ubuntu, Windows, macOS) with both feature configurations ("" and --all-features).
  2. Review the diff since the last release.
    git log --oneline v0.5.0..main
    git diff v0.5.0..main --stat -- hf-hub/
    Pay particular attention to changes under hf-hub/src/ — those are the only changes that actually ship to crates.io.
  3. Identify breaking changes. Anything that changes the public Rust API (types, function signatures, removed re-exports, builder fields) needs to be reflected in the version bump per semver and called out in the release notes.
  4. Run the full pre-release test sweep (see next section).

Pre-release test sweep

Run all of these from the repo root before tagging. They mirror what CI runs, plus a publish dry-run that CI does not currently do.

Format and lint

cargo +nightly fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo clippy --workspace --all-targets --all-features -- -D warnings

Unit tests (hf-hub)

cargo test -p hf-hub
cargo test -p hf-hub --features blocking

Integration tests (integration-tests)

... (truncated)

Commits

Updates rand from 0.9.2 to 0.10.2

Changelog

Sourced from rand's changelog.

[0.10.2] — 2026-07-02

Fixes

  • Fix possible memory safety violation due to deserialization of UniformChar from bad source (#1790)

Changes

  • Document required output order of fn partial_shuffle and apply #[must_use] (#1769)
  • Avoid usage of unsafe in contexts where non-local memory corruption could invalidate contract (#1791)

#1769: rust-random/rand#1769 #1790: rust-random/rand#1790 #1791: rust-random/rand#1791

[0.10.1] — 2026-02-11

This release includes a fix for a soundness bug; see #1763.

Changes

  • Document panic behavior of make_rng and add #[track_caller] (#1761)
  • Deprecate feature log (#1763)

#1761: rust-random/rand#1761 #1763: rust-random/rand#1763

[0.10.0] - 2026-02-08

Changes

  • The dependency on rand_chacha has been replaced with a dependency on chacha20. This changes the implementation behind StdRng, but the output remains the same. There may be some API breakage when using the ChaCha-types directly as these are now the ones in chacha20 instead of rand_chacha (#1642).
  • Rename fns IndexedRandom::choose_multiple -> sample, choose_multiple_array -> sample_array, choose_multiple_weighted -> sample_weighted, struct SliceChooseIter -> IndexedSamples and fns IteratorRandom::choose_multiple -> sample, choose_multiple_fill -> sample_fill (#1632)
  • Use Edition 2024 and MSRV 1.85 (#1653)
  • Let Fill be implemented for element types, not sliceable types (#1652)
  • Fix OsError::raw_os_error on UEFI targets by returning Option<usize> (#1665)
  • Replace fn TryRngCore::read_adapter(..) -> RngReadAdapter with simpler struct RngReader (#1669)
  • Remove fns SeedableRng::from_os_rng, try_from_os_rng (#1674)
  • Remove Clone support for StdRng, ReseedingRng (#1677)
  • Use postcard instead of bincode to test the serde feature (#1693)
  • Avoid excessive allocation in IteratorRandom::sample when amount is much larger than iterator size (#1695)
  • Rename os_rng -> sys_rng, OsRng -> SysRng, OsError -> SysError (#1697)
  • Rename Rng -> RngExt as upstream rand_core has renamed RngCore -> Rng (#1717)

Additions

  • Add fns IndexedRandom::choose_iter, choose_weighted_iter (#1632)
  • Pub export Xoshiro128PlusPlus, Xoshiro256PlusPlus prngs (#1649)
  • Pub export ChaCha8Rng, ChaCha12Rng, ChaCha20Rng behind chacha feature (#1659)
  • Fn rand::make_rng() -> R where R: SeedableRng (#1734)

Removals

  • Removed ReseedingRng (#1722)
  • Removed unused feature "nightly" (#1732)
  • Removed feature small_rng (#1732)

... (truncated)

Commits

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore <dependency name> major version will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
  • @dependabot ignore <dependency name> minor version will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
  • @dependabot ignore <dependency name> will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
  • @dependabot unignore <dependency name> will remove all of the ignore conditions of the specified dependency
  • @dependabot unignore <dependency name> <ignore condition> will remove the ignore condition of the specified dependency and ignore conditions

Bumps the cargo-candle group in /apps/router/candle-binding with 3 updates: [tokenizers](https://github.com/huggingface/tokenizers), [hf-hub](https://github.com/huggingface/hf-hub) and [rand](https://github.com/rust-random/rand).


Updates `tokenizers` from 0.22.2 to 0.23.1
- [Release notes](https://github.com/huggingface/tokenizers/releases)
- [Changelog](https://github.com/huggingface/tokenizers/blob/main/RELEASE.md)
- [Commits](huggingface/tokenizers@v0.22.2...v0.23.1)

Updates `hf-hub` from 0.5.0 to 1.0.0
- [Release notes](https://github.com/huggingface/hf-hub/releases)
- [Changelog](https://github.com/huggingface/hf-hub/blob/main/RELEASE.md)
- [Commits](https://github.com/huggingface/hf-hub/commits)

Updates `rand` from 0.9.2 to 0.10.2
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](rust-random/rand@rand_core-0.9.2...0.10.2)

---
updated-dependencies:
- dependency-name: tokenizers
  dependency-version: 0.23.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: cargo-candle
- dependency-name: hf-hub
  dependency-version: 1.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: cargo-candle
- dependency-name: rand
  dependency-version: 0.10.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: cargo-candle
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot @github

dependabot Bot commented on behalf of github Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Labels

The following labels could not be found: dependencies, rust. Please create them before Dependabot can add them to a pull request.

Please fix the above issues or remove invalid values from dependabot.yml.

@massaindustries

massaindustries commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Validation failed with the production feature set: cargo test --release --no-default-features does not compile. hf-hub 1.0.0 removes the api, Repo, and RepoType API used by this crate, and rand 0.10 removes rand::thread_rng. This needs a source migration and dedicated compatibility tests; it is not safe to merge as a lockfile-only update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant