|
| 1 | +# Detector optimization notes (faster branch) |
| 2 | + |
| 3 | +Campaign: 88 ms → 39.8 ms/image detector total (2.21x) on the 133-image |
| 4 | +`vide_images` corpus, 4 threads, default parameters, tagStandard52h13. |
| 5 | +Output is byte-equivalent to baseline acf5e20 at every commit (4583/4583 |
| 6 | +detections, coords within 1e-4 px — FMA contraction noise only). |
| 7 | + |
| 8 | +## Measurement (do not skip this section) |
| 9 | + |
| 10 | +- Machine: AMD EPYC 9454P (8 CCDs × 6 cores), NixOS, shared with a CI |
| 11 | + runner that randomly loads it 20-100x. Absolute timings swing wildly. |
| 12 | +- `./benchmark.sh` — official number (hyperfine wall + per-stage detector |
| 13 | + table from `--save-timing`). Pinned via `taskset -c 0-3`. |
| 14 | +- `./ab.sh <buildA> <buildB> [rounds]` — interleaved A/B with paired |
| 15 | + per-round ratios; the only trustworthy comparison under load. Additive |
| 16 | + noise biases ratios toward 1, so quiet-window numbers are the honest ones. |
| 17 | +- `./check.sh` — epsilon-gated output equivalence vs the baseline corpus |
| 18 | + run (id/hamming exact, coords ≤0.1 px, margin ≤1.0). |
| 19 | +- **Pinning is worth ~28 ms/frame**: unpinned, the idle-machine scheduler |
| 20 | + spreads the 4 workers across CCDs and the shared structures bounce |
| 21 | + between L3s. Exactly 4 CPUs is optimal; 5-6 in the affinity set lets the |
| 22 | + scheduler migrate threads and is *much* worse. Under CI load the numbers |
| 23 | + can look *better* than quiet because load packs the threads onto one CCD. |
| 24 | + |
| 25 | +## What the speedup is made of (stage ms, baseline → now) |
| 26 | + |
| 27 | +- threshold 3.6 → 1.7: SIMD tile min/max + blur + compare; RLE fused into |
| 28 | + the threshold tasks (per-task contiguous buffers, one memcpy each); |
| 29 | + buffers cached across frames. |
| 30 | +- unionfind 13.8 → 4.7: union-find indexed by *run* (~1.6 MB, not 50 MB); |
| 31 | + one union per adjacent same-value run pair; interleaved-root connect |
| 32 | + with full path compression; no per-frame reset (cached). |
| 33 | +- make clusters 27.1 → 14.0: segment-driven emission (entry resolved once |
| 34 | + per run pair, interior pairs as packed u64 / AVX2 stores); per-run |
| 35 | + rep+gate cache with read-only finds; k-way heap merge of per-task lists |
| 36 | + with exact-size fragment-group concatenation (parallelized); pooled |
| 37 | + cluster_hash entries; flat single-allocation clusters {size, pts[]}. |
| 38 | +- fit quads 34.6 → ~14: 8-byte points; u64 sort keys (slope<<32 | ~idx); |
| 39 | + vectorized key/bbox/dot loops; 4-way fused merge sort; SoA moments; |
| 40 | + memoized segment fits; vectorized window-error + 7-tap filter + maxima |
| 41 | + scan; quickselect threshold; per-task scratch reuse; lfps pass 1 in |
| 42 | + emission order (image-locality), pass 2 gathers through key indices. |
| 43 | +- post-fit serial 2.2 → ~0.07: per-task quad accumulation. |
| 44 | +- decode+refine 6.5 → ~3.2: per-task scratch; truncating casts replacing |
| 45 | + modf; homography + graymodel coefficient hoists; refine_edges sampling |
| 46 | + loop vectorized 4-wide (masked lanes contribute exact 0.0). |
| 47 | + |
| 48 | +## Load-bearing invariants (violating these changes detections) |
| 49 | + |
| 50 | +- **Angle-sort tie order.** 87% of clusters contain equal slope keys: the |
| 51 | + ±2^16 quadrant constants crush the dy/dx mantissa. The legacy order is |
| 52 | + produced by hi-word-only leaf networks plus take-right-on-tie merges — |
| 53 | + an inconsistent comparator, so NO standard sort reproduces it; only the |
| 54 | + exact merge-tree structure does. Full-u64 leaves (a consistent order) |
| 55 | + lose 1 detection in 4583 with 0.25 px shifts. |
| 56 | +- The (y, x, conn-order) emission sequence of cluster points, the |
| 57 | + connected_last suppression, and the component-size gates (lazy, |
| 58 | + evaluated against sizes at scan time). |
| 59 | +- Union-find *rep values* may change freely (tree shape is internal); |
| 60 | + components and sizes may not. |
| 61 | +- The union-find task chunking can leave the last image row uncovered for |
| 62 | + some heights; those run nodes must be initialized (see covered_end). |
| 63 | + |
| 64 | +## Measured dead ends (don't re-try without new information) |
| 65 | + |
| 66 | +- LSD radix sort on full-u64 keys: only −2 ms vs the 4-way fused merge, |
| 67 | + and −1 detection. Even ignoring output, sorts are within ~2 ms of the |
| 68 | + committed one. |
| 69 | +- 8-way fused merge: +2.5 ms (7-compare tournament too serial). 4-way is |
| 70 | + the optimal fusion depth; 2-way costs an extra pass. |
| 71 | +- Two-pass emission (count, allocate exact, write direct): +8.5 ms — the |
| 72 | + per-pair sweep machinery dominates, not the point stores. |
| 73 | +- Bit-loop and patterns-loop SIMD in decode: neutral (too few samples per |
| 74 | + quad to amortize lane fold-out). refine_edges SIMD was the win (−1.5). |
| 75 | +- Frame arena for cluster allocations: neutral (glibc tcache already |
| 76 | + covers it). Merged clusters can exceed 256 KB — size slots if revived. |
| 77 | +- Big-cluster-first scheduling in fit_quads: +4 ms (count-based chunking |
| 78 | + concentrates the heavy clusters; hash order already scatters them). |
| 79 | +- `__attribute__((flatten))` on the sort: −3% (I-cache bloat). |
| 80 | +- `-funroll-loops`: −7%. PGO: +7% bit-identical (excluded by decision). |
| 81 | +- Hugepage threshim, SMT-sibling affinity, malloc tunables, 512-bit |
| 82 | + vector width: all neutral or worse. |
| 83 | +- Rem's union-find: incompatible with the size gates (splices merge |
| 84 | + components without visiting roots). |
| 85 | +- No duplicate cluster points exist (measured) — nothing to dedup. |
| 86 | +- No over-cap fragments exist (measured) — nothing to truncate early. |
| 87 | + |
| 88 | +## arm64/NEON port (Apple M3 Pro, vide_images2, 4 threads) |
| 89 | + |
| 90 | +All 14 AVX2 blocks have NEON counterparts (`#elif defined(__ARM_NEON)`), |
| 91 | +output byte-identical to the scalar arm64 build at every commit |
| 92 | +(4583/4583 detections). 29.0 -> 25.0 ms/image detector total (-14%); |
| 93 | +per-commit stage timings in `results.tsv`. |
| 94 | + |
| 95 | +- Measure with `./bench_neon.sh <label> [desc]` (hyperfine wall + |
| 96 | + per-stage table, appends to results.tsv) and `./check_neon.sh` |
| 97 | + (epsilon-gated equivalence vs benchmark_results/dets-baseline.tsv). |
| 98 | + The machine is shared: bench_neon.sh waits for a quiet 1-min load |
| 99 | + window (LOAD_MAX, default 5.0; 3.0 gives trustworthy numbers). |
| 100 | + Runs taken under load inflate *untouched* stages — that's the tell. |
| 101 | +- NEON niceties vs the AVX2 originals: vld4 deinterleaves struct pt |
| 102 | + for free (no blend/permute dance in the key loop); vshrn-narrowed |
| 103 | + nibble masks replace movemask (each byte yields 4 mask bits, so |
| 104 | + ctz>>2 and popcount>>2); vpmin/vpmax pairs collapse the 4x4 tile |
| 105 | + reduction; true u8 shifts drop the AVX2 0x7f masking. |
| 106 | +- The NEON gains are smaller than the x86 ones mostly because 128-bit |
| 107 | + lanes halve the width, and the scalar arm64 baseline was already |
| 108 | + relatively faster than scalar x86 (the M3's OoO core hides more). |
| 109 | +- Measured dead end: widening the 2-wide f64 loops to 4-wide with two |
| 110 | + independent chains (window errors, bbox) is ~2% *worse* on fit |
| 111 | + quads, reproducibly — the OoO engine already overlaps iterations; |
| 112 | + the wider body just adds register pressure. |
| 113 | + |
| 114 | +## Where the remaining time is (quiet, pinned, 39.8 total) |
| 115 | + |
| 116 | +threshold 1.7 · unionfind 4.7 · make clusters 14.0 · fit quads ~14 · |
| 117 | +decode ~3.2 · serial glue ~1.5. Profiles are flat inside the big two |
| 118 | +(cost spread across pair machinery and merge passes; no hotspot). |
| 119 | +3x (29.3 ms) was determined infeasible under the output-equivalence |
| 120 | +constraint on this machine — and relaxing the constraint only buys ~2 ms. |
0 commit comments