Skip to content

Detector optimization campaign (2.21×), one optimization per commit - #6

Open
bouk wants to merge 70 commits into
masterfrom
faster2
Open

Detector optimization campaign (2.21×), one optimization per commit#6
bouk wants to merge 70 commits into
masterfrom
faster2

Conversation

@bouk

@bouk bouk commented Jun 10, 2026

Copy link
Copy Markdown
Member

Detector optimization campaign — 88 → 39.8 ms/image (2.21×)

End-to-end speedup of the AprilTag detector on the 133-image vide_images corpus
(4 threads, default params, tagStandard52h13), output byte-equivalent to baseline at
every commit (4583/4583 detections; coords within 1e-4 px — FMA-contraction noise only).

History is organized as one optimization per commit (70 commits) so each change can
be reviewed and bisected in isolation. PERF_NOTES.md documents the methodology, the
per-stage breakdown, the load-bearing invariants (notably the angle-sort tie order that
no standard sort reproduces), and the measured dead ends.

Where the time went (stage ms, baseline → now)

  • threshold 3.6 → 1.7 — SIMD tile min/max + blur + compare; RLE fused into the
    threshold tasks; buffers cached across frames.
  • unionfind 13.8 → 4.7 — union-find indexed by run (~1.6 MB, not 50 MB); one union
    per adjacent same-value run pair; interleaved-root connect; no per-frame reset.
  • make clusters 27.1 → 14.0 — run-driven, segment-driven emission; chunk-pool point
    storage; per-run rep+gate cache with read-only finds; k-way heap merge with
    parallelized fragment concatenation; flat single-allocation clusters.
  • fit quads 34.6 → ~14 — 8-byte points; u64 sort keys; vectorized key/bbox/dot/
    window-error/filter/maxima loops; 4-way fused merge sort; SoA moments; memoized
    segment fits; per-task scratch reuse.
  • decode+refine 6.5 → ~3.2 — per-task scratch; truncating casts; homography +
    graymodel hoists; vectorized refine_edges sampling.

Build / verification

  • Builds with CMake (-DBUILD_EXAMPLES=ON); AVX2 paths are __AVX2__-guarded with
    scalar fallbacks. benchmark.sh / ab.sh / check.sh are the measurement and
    output-equivalence harnesses.
  • The two largest rewrites (Run-driven gradient clusters …, Shrink struct pt …) are
    single cohesive commits — their parts are co-designed and have no separable compiling
    intermediate; everything else is split to one optimization per commit.

bouk and others added 30 commits June 10, 2026 18:56
@bouk @claude
Add TSV detection/timing output flags to apriltag_demo
3257434 
--save-detections/--save-timing write per-detection and per-stage TSVs
so builds can be compared for identical output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Add hyperfine benchmark script
164eaff 
benchmark.sh wraps hyperfine over vide_images with git-rev-labeled
artifacts; .gitignore covers the TSV and benchmark outputs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Memoize segment fits across the candidate-quad search
dc04beb 
quad_segment_maxima's 4-deep maxima loop re-fit identical segments for
every outer combination; memoize fit_line results per maxima pair.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Hoist the Gaussian low-pass kernel into a per-thread table
98d072c 
The fixed sigma=1/cutoff=0.05 kernel (size 7) was recomputed with exp()
for every cluster; compute it once per thread.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Replace modulo indexing in quad_segment_maxima with wrap adjustments
9c55305 
The error, low-pass, and maxima loops indexed errs[] with % sz; a single
branch adjustment wraps without the integer divide.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Reuse per-task scratch buffers in fit_quad
8a54b42 
fit_quad malloc'd five buffers and compute_lfps calloc'd a sixth for
every cluster; thread one growable per-task scratch struct through
quad_segment_maxima/compute_lfps/do_quad_task instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Ping-pong merge sort: copy points only at the leaves
541bd01 
The old ptsort copied the whole range into temp at every recursion
level. Alternate between two buffers so data is copied only at the
<=5-element leaves. Same splits, leaf networks, and merge tie behavior,
so the sorted order is bit-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Add output-equivalence check script
fe15384 
check.sh diffs --save-detections TSVs against a baseline run with
per-field epsilon gates (id/hamming exact, coords and margin tolerant).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Build with -fno-math-errno
cf1ac9c 
Lets sqrt() and friends compile to bare (vectorizable) instructions
instead of errno-guarded libm calls.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Destroy clusters inside the quad tasks
c88e79c 
Free each cluster in its quad task (parallel, cache-warm) instead of a
serial ~11k-free loop after fit_quads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Run the cluster merge tree on the worker pool
4fd699c 
Each merge-tree level's pairwise merges fan out to the worker pool
instead of running serially on the calling thread.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Size the per-task cluster hash table to a power of two
34ff77b 
Bucket selection becomes a mask instead of an integer divide, and the
table is sized to the slab (well under one entry per 64 slab pixels) so
it stays cache resident instead of spread across 0.2*w*h callocs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Grow the mem_pools array instead of sizing it from nclustermap
20f4bd8 
With the hash table sized independently of the slab, the pool-chunk
array is seeded small and realloc-doubled as chunks fill.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Reuse the bucket index for cluster_hash->hash
62511ad 
The collection loop already iterates buckets in order, so the secondary
key is just the bucket index instead of a re-hash.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Cache the last-hit cluster entry
5a296bd 
Consecutive boundary points almost always extend the same cluster, so
remember the last entry and skip the hash lookup when the id matches.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Defer the pixel's union-find representative lookup
5ae0a9c 
Most pixels are interior to a region and never reach a black/white
boundary neighbor; compute rep0 (and its size gate) lazily on the first
one instead of eagerly for every boundary-candidate pixel.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Inline the union-find set-size lookups in the cluster pass
06e4fb8 
Read uf->size[rep] + 1 directly instead of calling
unionfind_get_set_size in the hot boundary loop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Sort cluster points as packed u64 keys
80812d4 
key64 = monotone(slope) << 32 | ~index. Full-u64 merge comparisons
reproduce the historical merge's take-right-on-tie rule (left-run
elements always carry smaller original indices -> larger complements);
leaf networks compare the slope word only, matching no-swap-on-tie.
Bit-identical ordering, but the sort moves 8-byte keys instead of
12-byte structs and compares without function calls; points are
gathered once at the end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Run-driven gradient clusters with chunk-pool point storage
d1d70af 
Drive cluster construction from row runs: component representatives
resolve once per run (cached) instead of per boundary pixel, 127 spans
skip wholesale, and the same (y, x, neighbor) emission order is kept so
output is bit-identical.

Cluster points accumulate in fixed-size chunks bump-allocated from a
per-task pool instead of doubling zarrays (a frame can produce ~3M
points; growth reallocs were copying tens of MB per frame). Clusters
materialize into exact-size zarrays at collection.

Output verified byte-identical over the 133-image corpus.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Cache threshold buffers across detect calls
716a229 
threshim and the four tile min/max arrays are kept on the detector and
reused instead of allocated and freed every frame; every entry is
rewritten before use.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Heads-only union-find
8eb1235 
Only run heads (plus the lazily-initialized last column) enter the
union-find: the run-driven cluster pass resolves representatives through
run heads, so the per-pixel parent fill (~25MB of writes/frame) was
waste. Unions operate on head ids -- same union graph, same nodes, so
roots, sizes, and output are identical. The serial stitch becomes a
run-pair pass and the per-pixel line code is gone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Batch interior overlap points in gradient clusters
8c0bae8 
While a run and the run below keep overlapping, each pixel emits
exactly its (0,1) and (1,1) points into the same cluster entry, so emit
them in a tight loop without per-pixel pointer or gate checks.

Output verified byte-identical over the 133-image corpus.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Make pt_compare_angle static inline
8fb183e 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Lean inline path for non-wrapping window fit errors
4be5395 
The per-point window error loop computes the small eigenvalue inline
(identical arithmetic) for windows that don't wrap the cluster ends;
only the wrapping windows still call the general fit_line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Branchless key_merge select
b21cf0c 
Merge comparisons are data-dependent coin flips; select with a
conditional move instead of a ~50%-mispredicted branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Replace pairwise cluster merge tree with single-pass k-way heap merge
516bb6e 
Same output: clusters ascend by (hash, id) and split clusters
concatenate their points in task order, exactly as the pairwise tree
produced. Saves a workerpool barrier per level and all intermediate
lists. Detections verified within 1e-4 px of baseline (the build now
uses -march=native, whose FMA contraction shifts coordinates by ~1e-4).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Shrink struct pt to 8 bytes; build sort keys in the slope loop
4802e8d 
The slope was only ever used to order points, and the ordering now runs
on packed u64 keys built directly in fit_quad's angle loop, so the
stored float is dead weight. Dropping it cuts a third off all point
traffic (chunk storage, merges, collection copies, sort gathers).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Sort keys only; lfps reads points through the key indices
c8ae44f 
The sorted point array was only consumed by compute_lfps. Sort just the
key array and have compute_lfps gather points through the key indices,
skipping the per-cluster point permutation and its scratch buffer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
AVX2 paths for the threshold tile min/max and blur
2100301 
__AVX2__-guarded vectorization (scalar fallback kept) for the threshold
stage's 4x4 tile min/max and the 3x3 tile blur. Integer min/max only, so
results are exact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Build shared per-frame run tables once, SIMD RLE scans
20a1044 
Rows were being run-length encoded up to four times per frame in scalar
byte loops (union-find tasks, stitch, and gradient clustering each
re-scanned threshim). A single two-pass parallel build (SIMD count via
change masks, prefix-sum, SIMD fill) now produces packed row-run tables
cached on the detector, and all consumers read those instead of
re-scanning the 6.4MB image.

Output verified within 1e-4 px over the 133-image corpus.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
bouk and others added 24 commits June 10, 2026 19:34
@bouk @claude
Add pgo.sh profile-guided build helper
8134857 
Wraps the two-phase PGO build (bit-identical output).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Initialize run nodes of rows the union-find chunking leaves uncovered
e22e362 
The task chunking can leave the final row outside every task (the
historical code left those pixels lazily initialized and unconnected).
The read-only cluster finds crashed on such nodes' 0xffffffff markers
for image heights that hit this case. Initialize them as size-0
singletons, matching the historical gate behavior exactly.

Fixes test_tag_pose_estimation segfaults; all 8 ctest pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Quickselect for the maxima threshold
7c0434b 
The descending qsort only existed to read one order statistic (the
max_nmaxima-th largest window error); quickselect finds the same value
in linear time without libc comparator calls.

Output verified within 1e-4 px over the 133-image corpus.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Integer index math in the lfps weight pass
c9e1956 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Interleaved root walks and full path compression for run connects
6631b72 
Components and sizes are invariant to the find implementation; the two
root chains are independent, so walking them interleaved hides latency.

Output verified within 1e-4 px over the 133-image corpus.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Fuse two merge levels of the key sort into 4-way merges
36e5b5a 
A virtual-cursor 4-way merge produces exactly the composition of the
two 2-way merges it replaces, so the merge tree's passes over the data
halve while the output stays bit-identical (verified byte-equal
detections). Leaf handling and odd-size levels keep the 2-way code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Vectorize the interior pair batch stores
13e2e8b 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Fuse run-length encoding into the threshold tasks
8b53559 
Each thresholded line (including its right-edge fixup) is RLE'd while
cache hot into per-task buffers whose rows are contiguous, so the
packed table builds with one memcpy per task; the separate count and
fill passes over threshim disappear (build_frame_runs remains for the
deglitch path, which rewrites threshim afterwards). Bottom partial rows
are handled in a serial tail.

Output verified within 1e-4 px; all 8 ctest pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Vectorize the window-error maxima scan
dd9e601 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Gather split-cluster fragments and concatenate once
1bad9ad 
The k-way merge was append-realloc'ing each fragment of a boundary-
crossing cluster into the previous ones (most tall clusters split
across the 40 row slabs, costing ~8ms/frame in realloc copies). Gather
each equal-id group from the heap and concatenate into one exact-size
allocation; unsplit clusters keep the zero-copy pointer move.

Output verified within 1e-4 px over the 133-image corpus.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Fuse the border-orientation dot product into the vectorized key loop
08d458e 
Eight-lane accumulation summed once at the end; the dot only decides
the reversed-border sign, which sits far from zero for any usable
cluster, and the detection set is unchanged on the corpus.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Tune task granularity to 6 tasks per thread
4d7bb51 
Balances per-chunk overhead against load balance across the worker pool.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Pin the A/B harness to one CCD
83a2962 
taskset the four workers onto a single CCD so the small shared
structures don't bounce between L3s when the scheduler spreads them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Pin the official benchmark to one CCD
87dd08d 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Pool cluster_hash entries per task
409ef5c 
One allocation backs all of a task's collection entries instead of
~11k mallocs+frees per frame.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Flat cluster representation
c70f32f 
Clusters are now a single allocation (header + points) instead of a
zarray shell plus separately-malloc'd data: one malloc and one free per
cluster, and merged fragments concatenate into one exact-size block.

Output verified within 1e-4 px; all 8 ctest pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Skip the per-quad copy when decoding a single family
937120c 
Decoding does not modify the quad; the copy only matters when several
families restart from the same fit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Parallelize split-cluster concatenation
e672960 
The k-way heap walk now only records fragments and group boundaries;
the exact-size concatenations fan out to the worker pool into
order-preserving output slots.

Output verified within 1e-4 px; all 8 ctest pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Compute lfps weights in emission order
f99006c 
Pass 1 now walks points sequentially (the cluster's raster-coherent
emission order), keeping its four image-gradient loads cache-local
instead of scattering them around the perimeter in angle order; the
prefix pass gathers the per-point values from the small scratch arrays
through the sorted key indices instead. Identical arithmetic per point.

Output verified within 1e-4 px; all 8 ctest pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Hoist homography loads out of the bit-sampling loop
5f69010 
Same projection expressions; the matd element loads happen once per
quad instead of per bit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Hoist homography loads in the pattern-sampling loop too
d85fd8f 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Keep graymodel coefficients in registers in the bit loop
f74d9cb 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Vectorize the refine_edges sampling loop
73ff7a8 
Four steps per iteration: both bilinear samples, the bounds checks, and
the gradient test run as 4-wide doubles; rejected lanes contribute an
exact 0.0 to the weight accumulators, so only the final reduction order
differs (within 1e-4 px on the corpus). Indices are clamped so masked
lanes still load safely. Scalar tail handles max_steps % 4.

All 8 ctest pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bouk @claude
Document the optimization campaign: methodology, invariants, dead ends
cdb964d 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jun 10, 2026
Closed
Closed
Closed
Closed
Closed
@bouk

bouk commented Jun 11, 2026

Copy link
Copy Markdown
Member Author

https://gist.github.com/bouk/115cde3e7eadcf2b592e0bcfbca21cd1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@bouk