Skip to content

Commit e20c74b

Browse files
Add over-the-air fieldtest harness to sonance-cli
Single-machine acoustic loopback measurement: play a seeded corpus out the speaker, record it on the mic, decode, and report PER / sync rate / goodput / estimated SNR per payload size. This closes the gap between the simulation-only BER gates and real-hardware behavior. - New `fieldtest` subcommand (--profile/--trials/--sizes/--gap-ms/ --capture-dir/--json), composed from the existing CpalSink/CpalSource, resampling, and WavSink plumbing; no DSP/codec changes. - Pure, hardware-free scoring + JSON builder (xtask build_json style), covered by unit tests including a clean-loopback success and a noise->NoSync integrity check. - docs/field-results.md: reproduction steps, column semantics, honest caveats, and a template for the first real run. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 982276a commit e20c74b

2 files changed

Lines changed: 636 additions & 3 deletions

File tree

docs/field-results.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Over-the-air field results
2+
3+
This is the real-hardware counterpart to the simulated sweeps in
4+
[`docs/architecture.md`](architecture.md). Every BER/room/drift number elsewhere
5+
in the project is measured against the seeded `sonance-sim` channel; this page
6+
records what actually happens when the signal travels through a real speaker,
7+
the air, and a real microphone.
8+
9+
Measurements are produced by the `sonance fieldtest` subcommand (see
10+
[`sonance-cli/src/main.rs`](../sonance-cli/src/main.rs)), which plays a seeded
11+
corpus out the default output device, records it back on the default input
12+
device (single-machine acoustic loopback), decodes the capture, and scores it.
13+
14+
## How to reproduce
15+
16+
```bash
17+
# Build the CLI (cpal backend is on by default for sonance-cli).
18+
cargo build -p sonance-cli --release
19+
20+
# Disable OS voice processing first (AEC / noise-suppression / auto-gain), or
21+
# the loopback will not decode. The tool warns when capture is not raw.
22+
23+
# Run a loopback sweep. Defaults: profile balanced, 10 trials, sizes 64,256,1024.
24+
cargo run -p sonance-cli --release -- fieldtest
25+
26+
# A fuller run, dumping each capture to WAV for inspection and emitting JSON:
27+
cargo run -p sonance-cli --release -- fieldtest \
28+
--profile balanced --trials 20 --sizes 64,256,1024,4096 \
29+
--gap-ms 500 --capture-dir target/captures --json > target/fieldtest.json
30+
```
31+
32+
Outputs:
33+
34+
- a human-readable table (PER, sync rate, goodput, estimated SNR per size),
35+
- `target/fieldtest.json` (or stdout with `--json`), in the same hand-built
36+
JSON shape as the `xtask` sweeps so it drops into the same tooling,
37+
- optional per-trial WAV captures under `--capture-dir`.
38+
39+
## What the columns mean
40+
41+
| Column | Meaning |
42+
|--------|---------|
43+
| `sync %` | Fraction of trials where the chirp matched filter detected a frame (whether or not it then decoded). Separates sync failures from FEC failures. |
44+
| `ok %` | Fraction of trials that decoded byte-exact (1 - PER). |
45+
| `PER` | Packet error rate over the trials. |
46+
| `goodput b/s` | `success_rate * payload_bits / time_on_air`, the same formula as the `xtask` sweeps. |
47+
| `est SNR dB` | A recorded signal-vs-silence RMS ratio. **This is an estimate, not a dialed Es/N0** — the mic never hears pure signal — so it is *not* directly comparable to the simulator's `snr_db`. Use it only as a relative indicator of the run's conditions. |
48+
49+
## Honest caveats
50+
51+
- **Single-machine loopback** measures the speaker -> air -> mic path of *one*
52+
device. It captures the real DSP/sync/FEC behavior over real hardware, but it
53+
is not the same as two independent devices with independent clocks. Two-machine
54+
TX/RX is the planned follow-on.
55+
- **OS voice processing** (echo cancellation, noise suppression, AGC) destroys
56+
the modem signal. The tool warns when `raw_capture()` reports non-raw; results
57+
recorded with processing on are not meaningful.
58+
- **Acoustic coupling** depends heavily on volume, distance, and room. These
59+
are physical conditions the operator sets, not parameters the tool dials.
60+
61+
## Results
62+
63+
> Status: harness landed; first real run pending on hardware with raw capture.
64+
> Fill in the table below from `target/fieldtest.json` after a run. Record the
65+
> exact conditions so the numbers are interpretable and reproducible.
66+
67+
### Run 1 — _pending_
68+
69+
- Date: _TBD_
70+
- Device (speaker + mic): _e.g. MacBook Pro 14" built-in_
71+
- Room / distance / volume: _e.g. quiet office, self-loopback, ~70% volume_
72+
- Raw capture: _yes/no_ (must be yes for valid numbers)
73+
- Profile: `balanced`
74+
- Trials per size: _N_
75+
76+
| Payload | sync % | ok % | PER | goodput b/s | est SNR dB |
77+
|--------:|-------:|-----:|----:|------------:|-----------:|
78+
| 64 B | | | | | |
79+
| 256 B | | | | | |
80+
| 1 KB | | | | | |
81+
82+
Notes: _what worked, what didn't, how it compares to the sim sweep at a
83+
similar estimated SNR._

0 commit comments

Comments
 (0)