Skip to content

Commit b4118fd

Browse files
yaronfcursoragent
andcommitted
docs: record 8h fuzz soak results and corpus coverage
Document the SFV-weighted soak (all PASS, no crashes) and how to replay $GOCACHE corpus for post-soak coverprofiles. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 82aa1e3 commit b4118fd

1 file changed

Lines changed: 41 additions & 2 deletions

File tree

internal-docs/FUZZ.md

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ Package-wide `%` is optional secondary context only. It is diluted by client/han
7575
| `FuzzHMACViaMessage` | ~1.0M | 149 (150) | no |
7676
| `FuzzNewMessage` | ~664k | 280 (286) | no |
7777

78-
Seed coverage (no mutation), rough mean of per-function statement % on the focus files:
78+
Seed coverage (no mutation; `f.Add` + committed `testdata` only), rough mean of per-function statement % on the focus files:
7979

8080
| Profile | signatures | httpparse | fields | digest | message | package |
8181
|---------|------------|-----------|--------|--------|---------|---------|
@@ -85,6 +85,43 @@ Seed coverage (no mutation), rough mean of per-function statement % on the focus
8585

8686
Gaps that seeds now push harder: truncated/malformed SFV, `;tr` / `;bs` / `;sf`, `@query-param`, `content-digest`, trailers, response + associated-request configs.
8787

88+
## Soak (2026-09-05, SFV-weighted ~8h fuzz-time)
89+
90+
Budgets: `FuzzVerifyRequest` 2h, `FuzzVerifyViaMessage` 2h, `FuzzNewMessage` 90m, `FuzzSignAndVerifyHMAC` 75m, `FuzzHMACViaMessage` 75m. Host sleep paused progress overnight; after sleep was disabled, remaining fuzz-time finished on schedule (~22:30 local).
91+
92+
| Target | Result | Execs | New interesting (total) |
93+
|--------|--------|------:|-------------------------|
94+
| `FuzzVerifyRequest` | PASS | 220M | 866 (1121) |
95+
| `FuzzVerifyViaMessage` | PASS | 201M | 826 (1118) |
96+
| `FuzzNewMessage` | PASS | 151M | 315 (612) |
97+
| `FuzzSignAndVerifyHMAC` | PASS | 158M | 471 (676) |
98+
| `FuzzHMACViaMessage` | PASS | 144M | 461 (629) |
99+
100+
**Crashes:** none.
101+
102+
Interesting-input growth continued through the long runs (especially verify / HMAC), with plateaus late in each budget — expected, not a signal to stop early on a short quiet window.
103+
104+
### Post-soak corpus coverage
105+
106+
`go test -fuzz` does not write a coverprofile while mutating. Coverage after a soak is measured by **replaying** the cached corpus:
107+
108+
1. Interesting inputs live under `$GOCACHE/fuzz/github.com/yaronf/httpsign/<Target>/` (not under `testdata/fuzz/` unless copied).
109+
2. Stage those files into `testdata/fuzz/<Target>/` temporarily (hardlinks are fine).
110+
3. `go test -run='^FuzzXxx$' -coverprofile=... .` then exercises the soak corpus as ordinary seeds.
111+
4. Remove the staged files afterward; do **not** bulk-commit the cache dump.
112+
113+
2026-09-05 replay (cache entry counts ≈ soak “total interesting”), mean per-function % on focus files / package total:
114+
115+
| Target | Cache entries | signatures | httpparse | fields | digest | message | package |
116+
|--------|--------------:|-----------:|----------:|-------:|-------:|--------:|--------:|
117+
| `FuzzVerifyRequest` | ~1108 | 26.7% | 67.9% | 19.6% | 10.7% | 65.5% | 21.2% |
118+
| `FuzzVerifyViaMessage` | ~1111 | 21.7% | 67.9% | 19.6% | 10.7% | 70.8% | 20.7% |
119+
| `FuzzSignAndVerifyHMAC` | ~673 | 44.5% | 76.7% | 39.5% | 41.8% | 65.5% | 33.4% |
120+
| `FuzzHMACViaMessage` | ~626 | 42.0% | 76.5% | 39.5% | 41.8% | 70.8% | 34.1% |
121+
| `FuzzNewMessage` | ~604 | 23.5% | 74.7% | 21.3% | 0.0% | 76.5% | 22.2% |
122+
123+
Compared with seed-only baselines, package totals rose a few points (e.g. verify ~18.5%→21.2%, HMAC ~30.8%→33–34%). Most soak “interesting” finds refine edges already near existing coverage rather than opening large new statement regions — still valuable for crash hunting.
124+
88125
## Harness conventions
89126

90127
- **Panic-oriented** (`FuzzVerifyRequest`, `FuzzVerifyViaMessage`): discard expected verify/setup errors; return early on nil/`NewMessage` failure; never `t.Error` on bad signatures.
@@ -93,7 +130,9 @@ Gaps that seeds now push harder: truncated/malformed SFV, `;tr` / `;bs` / `;sf`,
93130

94131
## CI
95132

96-
The `fuzz` job in `.github/workflows/test.yml` runs each target with `-run='^$' -fuzztime=15s` sequentially on Go 1.27. It fails on crash or failing corpus. This is a **smoke**, not a long soak; longer nightly budgets can be added later.
133+
The `fuzz` job in `.github/workflows/test.yml` runs each target with `-run='^$' -fuzztime=15s` sequentially on Go 1.27. It fails on crash or failing corpus. This is a **smoke**, not a long soak.
134+
135+
For occasional long soaks (hours), use the weighted budgets in the soak section above; keep the machine from sleeping so wall clock ≈ fuzz-time. Optional follow-on: a scheduled nightly job with a larger `-fuzztime`.
97136

98137
## Checklist (after library changes that touch parse/sign/verify)
99138

0 commit comments

Comments
 (0)