Skip to content

Commit bac88e7

Browse files
GPU (Metal) cluster_canonicals + Rationer API; drop experimental scaffolding (v0.2.0)
Heterogeneous CPU+GPU exact Ratcliff-Obershelp: the suffix-automaton matching_stats walk runs on a Metal compute kernel (macOS, `gpu` feature), byte-for-byte identical to the CPU path. New `Rationer`/`PreparedRationer`/`Concurrency` handle owns the device + rayon pool once and reuses them. Data-driven routing: GPU only engages for single-group cluster_canonicals (~1.1-1.4x on Apple Silicon); ratio_many and cluster_canonicals_multi measured slower on GPU and stay on CPU (opt-in via DFGPU_* env). Python: Rationer pyclass; macOS wheels ship the GPU path (CI builds --features python,gpu). Housekeeping: graduate gpu/rationer out of new:: to top-level modules; remove dead full_ratio-kernel accessors; consolidate the public API to two tiers (free CPU functions + Rationer). The experimental scaffolding (algorithm trials, research papers, PMC/bench harness, profiling scripts) is kept local and out of git.
1 parent b1299aa commit bac88e7

14 files changed

Lines changed: 3617 additions & 215 deletions

File tree

‎.github/workflows/CI.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,10 @@ jobs:
6565
uses: PyO3/maturin-action@v1
6666
with:
6767
target: ${{ matrix.target }}
68-
args: --release --out dist
68+
# macOS wheels ship the Metal GPU path. CLI `--features` replaces pyproject's
69+
# `features=["python"]`, so list both. `gpu` is macOS-only; degrades to CPU at runtime
70+
# when no Metal device is present.
71+
args: --release --features python,gpu --out dist
6972
sccache: "true"
7073
- uses: actions/upload-artifact@v4
7174
with:

‎Cargo.lock‎

Lines changed: 117 additions & 5 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 34 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "difflib-fast"
3-
version = "0.1.2"
3+
version = "0.2.0"
44
description = "Fast, byte-for-byte exact difflib Ratcliff–Obershelp (gestalt) similarity ratio + single-linkage clustering, via a suffix automaton."
55
keywords = ["difflib", "similarity", "ratcliff-obershelp", "suffix-automaton", "fuzzy"]
66
categories = ["algorithms", "text-processing"]
@@ -21,24 +21,52 @@ rayon = "1"
2121
# Optional Python bindings (the `python` feature, built by maturin). `abi3-py39` → one wheel per
2222
# platform works on CPython 3.9+. The pure-Rust crate has zero Python dependency by default.
2323
pyo3 = { version = "0.28", optional = true, features = ["extension-module", "abi3-py39"] }
24-
# Used ONLY as the bench binary's global allocator (libraries must not set one) — gated so the
25-
# published library never pulls it in. macOS's default malloc madvise churn cost ~25% once parallel.
24+
# Used ONLY as the `bench` binary's global allocator (libraries must not set one) — gated behind the
25+
# `bench` feature so the published library never pulls it in. macOS's default malloc madvise churn
26+
# cost ~25% once parallel, hence the override for benchmarking.
2627
mimalloc = { version = "0.1", default-features = false, optional = true }
28+
# Apple Metal compute bindings — used by the `gpu` feature to run the SAM matching_stats walk on
29+
# the M3 GPU in parallel with the CPU SAM walker (heterogeneous CPU+GPU exact RO). Apple-only;
30+
# the feature does not compile on non-Apple targets, and `cluster_canonicals` falls back to the
31+
# CPU-only path automatically when the GPU device cannot be acquired at runtime.
32+
[target.'cfg(target_os = "macos")'.dependencies]
33+
metal = { version = "0.33", optional = true }
2734

2835
[features]
2936
default = []
3037
python = ["dep:pyo3"]
3138
bench = ["dep:mimalloc"] # enables the `bench` binary's mimalloc global allocator
39+
# Heterogeneous CPU+GPU exact RO — Metal compute shader port of the SAM matching_stats walker.
40+
# macOS-only; the feature wires up automatically on Apple Silicon and is a no-op on other targets.
41+
# The kernel reads the corpus SAMs out of unified shared memory (Apple Silicon's UMA = zero copy
42+
# between CPU and GPU), so there is no host↔device transfer cost. `Rationer` degrades to CPU when
43+
# this is off or no Metal device can be acquired at runtime.
44+
gpu = ["dep:metal"]
45+
# `instrument` adds atomic counters inside `gestalt::longest_in` and its endpos range queries plus
46+
# the RO recursion, plus a `gestalt::instrument::dump()` API that prints histograms of chain-walk
47+
# depth, recursion depth, fmatch distribution, the linear-vs-segtree split, and per-call scan
48+
# lengths. Costs ~10-20% wall when active (relaxed-ordered atomics on the hot path); the
49+
# `cfg(feature)` gate compiles to a no-op in default builds so production stays untouched.
50+
instrument = []
3251

33-
[[bin]]
34-
name = "bench"
35-
required-features = ["bench"]
52+
# `objc`'s `sel!`/`msg_send!` macros (pulled in transitively by the `metal` crate under the `gpu`
53+
# feature) expand to `cfg(feature = "cargo-clippy")` checks — whitelist that cfg so the lint stays
54+
# on for genuinely-unexpected cfgs without flooding gpu builds with dependency-macro noise.
55+
[lints.rust]
56+
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(feature, values("cargo-clippy"))'] }
3657

3758
# Strict clippy: all + pedantic denied (priority -1 so crate-local `#[allow]`s on hot paths override).
3859
[lints.clippy]
3960
all = { level = "deny", priority = -1 }
4061
pedantic = { level = "deny", priority = -1 }
4162

63+
# Gold CPU baseline bench (raw ratio / threshold qualify / cluster). Gated behind `bench` so the
64+
# published library never builds it. The GPU / PMC / cross-impl harness lives outside the tracked
65+
# tree (see perf-local/).
66+
[[bin]]
67+
name = "bench"
68+
required-features = ["bench"]
69+
4270
# Keep symbols + line tables in release so an external sampler (samply) can resolve frames.
4371
[profile.release]
4472
debug = "line-tables-only"

‎README.md‎

Lines changed: 63 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -93,16 +93,56 @@ pairwise ratio.
9393

9494
## API
9595

96+
Two tiers. The **free functions** are the stateless CPU drop-in:
97+
9698
| function | what |
9799
|---|---|
98100
| `ratio(a, b) -> f64` | exact `difflib` ratio (dispatches automaton ⇄ `b2j` per input) |
99101
| `ratio_many(&[(String, String)]) -> Vec<f64>` | exact `ratio` for a batch of pairs, in parallel (rayon) |
100102
| `gestalt::gestalt_ratio(a, b) -> f64` | the suffix-automaton path directly |
101-
| `ratio_reference(a, b) -> f64` | the `b2j` reference port (slow; the test oracle) |
102103
| `cluster_canonicals(&[String], threshold) -> Vec<(Vec<usize>, f64)>` | exact single-linkage clusters + min pairwise ratio |
103-
| `cluster_canonicals_chars(&[Vec<char>], threshold)` | same, over pre-collected `char` vectors |
104104
| `cluster_canonicals_lsh(&[String], threshold, num_perm, band_rows)` | scalable MinHash-LSH variant (candidate-gen + exact verify) for very large corpora |
105105

106+
And `Rationer` is the **stateful handle** that owns long-lived resources (rayon pool, and on
107+
macOS the Metal device) once and reuses them across calls — same exact output, with an optional GPU
108+
path:
109+
110+
```rust
111+
use difflib_fast::Rationer;
112+
113+
let r = Rationer::new(); // builder().build(); default GpuPlusCpu
114+
let clusters = r.cluster_canonicals(&corpus, 0.6); // GPU-accelerated on macOS (see below)
115+
let ratios = r.ratio_many(&pairs); // CPU
116+
```
117+
118+
`Rationer::builder().concurrency(Concurrency::Cpu | Gpu | GpuPlusCpu).threads(n).delta(d).build()`
119+
configures it; `PreparedRationer` (`r.prepare(&strings)`) amortizes SAM-build over many index-pair
120+
queries.
121+
122+
### GPU (macOS + Metal)
123+
124+
Behind the `gpu` cargo feature on Apple Silicon, `Rationer::cluster_canonicals` offloads the
125+
suffix-automaton `matching_stats` walk to a Metal compute kernel — **byte-for-byte identical output**,
126+
**~1.1–1.4×** end-to-end vs the (already fast) CPU path on a single large group:
127+
128+
| corpus | `cluster_canonicals` GPU vs CPU |
129+
|---|---|
130+
| mypy | **1.38×** |
131+
| ha | 1.25× |
132+
| sympy | 1.17× |
133+
| django | 1.14× |
134+
| transformers | 1.08× |
135+
136+
It's a modest, honest win: the GPU only does `matching_stats` (~⅓ of the per-pair cost); the
137+
`longest_in` recursion, filtering and assembly stay on CPU, so Amdahl caps it. `ratio_many` and
138+
`cluster_canonicals_multi` measured *slower* on the GPU at every size tested, so they stay on CPU by
139+
default (the GPU paths remain opt-in via `DFGPU_RATIO_MANY_THRESHOLD` / `DFGPU_MULTI_THRESHOLD`). With
140+
the feature off, on non-macOS, or with no Metal device, every call quietly runs on CPU.
141+
142+
```toml
143+
difflib-fast = { version = "0.1", features = ["gpu"] } # macOS only
144+
```
145+
106146
---
107147

108148
## Correctness
@@ -178,7 +218,7 @@ pick the wheel for you — grab the one for your platform from the
178218

179219
```bash
180220
# macOS Apple Silicon — swap the filename for your platform (see below):
181-
pip install https://github.com/prostomarkeloff/difflib-fast/releases/download/v0.1.2/difflib_fast-0.1.2-cp39-abi3-macosx_11_0_arm64.whl
221+
pip install https://github.com/prostomarkeloff/difflib-fast/releases/download/v0.2.0/difflib_fast-0.2.0-cp39-abi3-macosx_11_0_arm64.whl
182222
```
183223

184224
| platform | wheel suffix |
@@ -236,6 +276,26 @@ The package is **typed** (`py.typed` + `.pyi` stubs — pyright/mypy see the ove
236276
[maturin](https://github.com/PyO3/maturin) (mixed layout: compiled `_difflib_fast` +
237277
`python/difflib_fast/` package); build locally into a venv with `maturin develop --release --features python`.
238278

279+
### GPU from Python (macOS)
280+
281+
The **macOS wheels ship the Metal GPU path**. Use the `Rationer` handle — its `cluster_canonicals`
282+
runs on the GPU when the group is large enough to pay for the dispatch, otherwise CPU; same
283+
byte-for-byte answer either way:
284+
285+
```python
286+
import difflib_fast as df
287+
288+
r = df.Rationer(concurrency="gpu+cpu") # "cpu" | "gpu" | "gpu+cpu" (default)
289+
r.concurrency # "gpu+cpu" if Metal came up, else "cpu"
290+
r.cluster_canonicals(corpus, 0.6) # GPU-accelerated (~1.1–1.4× on Apple Silicon)
291+
r.ratio_many(pairs) # CPU (the GPU offload loses here)
292+
```
293+
294+
On Linux/Windows wheels, or with no Metal device, a `Rationer` transparently runs everything on CPU.
295+
Build a GPU wheel locally on macOS with
296+
`maturin develop --release --features python,gpu` (the CLI `--features` **replaces** the pyproject
297+
default, so list both).
298+
239299
---
240300

241301
## How it works

‎benchmarks.md‎

Lines changed: 32 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,8 @@ per second. Speedups (`Nx`) are **difflib-fast ÷ competitor**.
2222

2323
- **Metric:** exact RO, `autojunk=False`. Every "exact RO" comparison below is **byte-for-byte
2424
verified** against difflib-fast (mismatch counts / max|Δ| reported per section).
25-
- **Harnesses** (all in [`benchmarks/`](benchmarks/), driver: [`run_all.sh`](benchmarks/run_all.sh)):
25+
- **Harnesses** (in [`benchmarks/`](benchmarks/); the `run_all.sh` / `profile.sh` runners are local
26+
perf tooling, not tracked):
2627
- `cargo run --release --features bench --bin bench` — difflib-fast self (raw / threshold / cluster).
2728
- `cargo run --release --example compare` — vs the Rust crates `difflib` + `gestalt_ratio`.
2829
- `bench_vs_cydifflib.py` — vs CyDifflib, single thread.
@@ -55,6 +56,36 @@ code has unusually long functions, and per-pair RO is `O(L·log L)`.)
5556

5657
---
5758

59+
## 1b. GPU (Metal) — heterogeneous `cluster_canonicals`
60+
61+
Behind the `gpu` feature on Apple Silicon, `Rationer::cluster_canonicals` runs the suffix-automaton
62+
`matching_stats` walk on the M3 GPU (Metal compute) while the CPU keeps the filters + `longest_in`
63+
recursion + assembly. **Byte-for-byte identical** to the CPU path (cluster counts equal across all
64+
backends, every run). Measured via the `Rationer` API across its three `Concurrency` backends, one
65+
group, threshold 0.6, N=1500, M3 Pro, best-of-2:
66+
67+
| corpus | CPU | GPU | GPU+CPU | GPU speedup |
68+
|---|---|---|---|---|
69+
| mypy | 2.25 s | 1.62 s | 1.63 s | **1.38×** |
70+
| ha | 2.64 s | 2.10 s | 2.13 s | **1.25×** |
71+
| sympy | 1.57 s | 1.34 s | 1.31 s | **1.17×** |
72+
| django | 1.09 s | 0.95 s | 0.98 s | **1.14×** |
73+
| transformers | 2.69 s | 2.49 s | 2.35 s | **1.08×** |
74+
75+
**Honest read.** The GPU only offloads `matching_stats` — ~⅓ of the per-pair cycle budget; the
76+
`longest_in` chain walk (the dominant cost), the `quick_ratio` filters, the SAM build and the
77+
union-find assembly all stay on CPU. By Amdahl that caps the end-to-end win even before the
78+
fixed per-dispatch overhead, so cluster lands at **1.1–1.4×** and does *not* scale up with N
79+
(at N≈2200 the single dispatch's `(fstate, fmatch)` output buffer grows and goes bandwidth-bound,
80+
dropping back toward ~1.15×). The other two operations measured **slower** on the GPU at every size
81+
tried — `ratio_many` 0.82–0.93× (61k–404k pairs; the CPU intern + prebuilt-SAM path is already
82+
efficient) and `cluster_canonicals_multi` 0.59–0.99× (the find-dup-defs many-small-groups shape never
83+
amortizes the dispatch) — so both stay on CPU by default (GPU opt-in via `DFGPU_RATIO_MANY_THRESHOLD`
84+
/ `DFGPU_MULTI_THRESHOLD`). A widely-quoted "1.5–1.7×" is a *kernel-only* `matching_stats` microbench
85+
(the kernel itself is ~3× CPU throughput); it does not survive end-to-end.
86+
87+
---
88+
5889
## 2. vs CyDifflib (Python) — the fast byte-for-byte Cython difflib
5990

6091
[CyDifflib](https://github.com/rapidfuzz/CyDifflib) (by the RapidFuzz author) is the fastest existing

0 commit comments

Comments
 (0)