|
| 1 | +# Modularity: using sdr_dsp with any SDR |
| 2 | + |
| 3 | +`sdr_dsp` is deliberately split so the signal processing never knows what radio |
| 4 | +produced its samples. This document names the layers, shows where the seam is, and lists |
| 5 | +what remains to be done to drive the library from a radio other than a HackRF. |
| 6 | + |
| 7 | +## The layered model |
| 8 | + |
| 9 | +``` |
| 10 | + [ library ] <-> [ interface ] <-> [ device adapter ] <-> [ SDR hardware ] |
| 11 | + core/ IQSource / examples/ or your radio + its |
| 12 | + pure DSP TXSink (a HAL) code (Adapter pattern) vendor library |
| 13 | +``` |
| 14 | + |
| 15 | +The vocabulary for the middle two boxes: the **interface** is a *hardware |
| 16 | +abstraction layer* (HAL), expressed here as two Python `Protocol`s (structural, |
| 17 | +"duck-typed" contracts). The **device adapter** (a.k.a. driver adapter) is the |
| 18 | +*Adapter design pattern* — a small class that satisfies a protocol by wrapping a |
| 19 | +vendor library. The core depends on the contract, never on a device; that |
| 20 | +dependency inversion is the entire point. |
| 21 | + |
| 22 | +```mermaid |
| 23 | +flowchart LR |
| 24 | + subgraph L["library — core/ (pure numpy/scipy)"] |
| 25 | + DSP["fm_demod · estimate_fm_cfo ·<br/>channelize · measure · modulate ·<br/>framing · sync · features"] |
| 26 | + end |
| 27 | + subgraph I["interface — the HAL (Protocols)"] |
| 28 | + RX["IQSource<br/>sources/base.py"] |
| 29 | + TX["TXSink<br/>sinks/tx_sink.py"] |
| 30 | + end |
| 31 | + subgraph A["device adapter — outside the library"] |
| 32 | + HC["hackrf_capture.py (RX)"] |
| 33 | + HS["hackrf_sink.py (TX)"] |
| 34 | + OTHER["rtlsdr_source.py · usrp_sink.py · …<br/>(you write these)"] |
| 35 | + end |
| 36 | + subgraph H["SDR hardware + vendor library"] |
| 37 | + HW["HackRF / RTL-SDR / USRP / BladeRF …<br/>hackrfpy · pyrtlsdr · SoapySDR · UHD"] |
| 38 | + end |
| 39 | +
|
| 40 | + DSP -- "complex64 arrays in/out" --> RX |
| 41 | + DSP -- "complex64 arrays in/out" --> TX |
| 42 | + RX --> HC --> HW |
| 43 | + RX --> OTHER --> HW |
| 44 | + TX --> HS --> HW |
| 45 | + TX --> OTHER |
| 46 | +
|
| 47 | + %% device-free sources that need no adapter |
| 48 | + FILE["FileSource · ArraySource<br/>(ship in the library)"] --> RX |
| 49 | +``` |
| 50 | + |
| 51 | +## What each layer is, and where it lives |
| 52 | + |
| 53 | +| Layer | What it is | Where it lives | Depends on | |
| 54 | +|-------|-----------|----------------|-----------| |
| 55 | +| library | Pure DSP: arrays in, arrays out. Knows nothing about radios. | `src/sdr_dsp/core/` | numpy, scipy only | |
| 56 | +| interface (HAL) | Two `Protocol`s that define the RX and TX contracts. | `sources/base.py` (`IQSource`), `sinks/tx_sink.py` (`TXSink`) | numpy only | |
| 57 | +| device adapter | A class that satisfies a protocol by wrapping a vendor library. | `examples/` or your own code — **outside** the library | the vendor library | |
| 58 | +| SDR hardware | The radio and its driver stack. | n/a (hardware) | — | |
| 59 | + |
| 60 | +The core's device-independence is a checked fact, not a wish: **nothing under |
| 61 | +`src/` imports a vendor library.** Every `hackrfpy` mention in `src/` is a |
| 62 | +docstring or a comment. Keep it that way (see *Keeping the core device-free*). |
| 63 | + |
| 64 | +## The two contracts |
| 65 | + |
| 66 | +An adapter only has to provide these attributes and methods. Anything that does |
| 67 | +is an `IQSource` / `TXSink` as far as the library is concerned — no base class to |
| 68 | +inherit, no registration. |
| 69 | + |
| 70 | +**Receive — `IQSource`** (`sources/base.py`): |
| 71 | + |
| 72 | +```python |
| 73 | +sample_rate: float # samples per second (Hz) |
| 74 | +center_freq: float # RF center frequency the samples were captured at (Hz) |
| 75 | +def blocks(self) -> Iterator[np.ndarray]: # yield complex64 blocks |
| 76 | + ... |
| 77 | +# A bounded source may also offer read(n); live sources need only blocks(). |
| 78 | +``` |
| 79 | + |
| 80 | +**Transmit — `TXSink`** (`sinks/tx_sink.py`): |
| 81 | + |
| 82 | +```python |
| 83 | +sample_rate: float |
| 84 | +center_freq: float |
| 85 | +def transmit(self, iq: np.ndarray) -> None: # send one complex64 buffer |
| 86 | + ... |
| 87 | +``` |
| 88 | + |
| 89 | +## What ships vs. what you write |
| 90 | + |
| 91 | +Ships in the library (device-free, so the whole pipeline is testable with no |
| 92 | +radio): |
| 93 | + |
| 94 | +- `ArraySource` — wrap an in-memory `complex64` array (tests, synthetic signals). |
| 95 | +- `FileSource` — stream a SigMF recording from disk; satisfies `IQSource`, so |
| 96 | + code written against a file runs unchanged against hardware later. |
| 97 | +- `LoopbackSink` — a `TXSink` that "transmits" into an in-memory buffer, so the |
| 98 | + full TX stack (framing → modulate → sink) can be exercised without a radio. |
| 99 | + |
| 100 | +You write (per radio, once): |
| 101 | + |
| 102 | +- An `IQSource` adapter for receive, and/or a `TXSink` adapter for transmit. |
| 103 | +- The reference adapters to copy are `examples/hackrf_capture.py` (RX) and |
| 104 | + `examples/hackrf_sink.py` (TX). The TXSink docstring names `usrp_sink.py` as |
| 105 | + the intended shape for the next one. |
| 106 | + |
| 107 | +--- |
| 108 | + |
| 109 | +## What needs to be done to support another SDR |
| 110 | + |
| 111 | +Porting the **DSP** to a new radio is small and clean. Porting the **collection |
| 112 | +tooling** is the part with real work in it. The two are separated on purpose. |
| 113 | + |
| 114 | +### 1. Sample-format normalization (belongs in the adapter) |
| 115 | + |
| 116 | +The core only ever sees `complex64`. Real radios do not hand you `complex64` — |
| 117 | +each vendor streams its own on-the-wire format, and converting it is the |
| 118 | +adapter's job, not the core's. This is the single most common thing a new |
| 119 | +adapter must get right. |
| 120 | + |
| 121 | +| Radio (vendor lib) | Native sample format | Normalize to `complex64` by | |
| 122 | +|--------------------|----------------------|------------------------------| |
| 123 | +| HackRF (hackrfpy) | `ci8` (interleaved int8 I/Q) | `(i + 1j*q) / 128.0` | |
| 124 | +| RTL-SDR (pyrtlsdr) | `cu8` (interleaved uint8, offset 127.5) | `((i - 127.5) + 1j*(q - 127.5)) / 127.5` | |
| 125 | +| USRP (UHD) | `cf32` / `sc16` | cast to `complex64` (already float) / scale int16 by 32768 | |
| 126 | +| SoapySDR devices | `CF32` or `CS16` (device-dependent) | as above, per the stream format you request | |
| 127 | + |
| 128 | +The library already does exactly this for HackRF `ci8` in `io/sigmf.py` |
| 129 | +(`load_iq`) and reuses it in `FileSource` — that is the template. An adapter's |
| 130 | +`blocks()` should apply the equivalent normalization for its device and yield |
| 131 | +`complex64`. **Do the scaling once, in the adapter; never push a device format |
| 132 | +into the core.** |
| 133 | + |
| 134 | +A good adapter also, in `blocks()`: |
| 135 | + |
| 136 | +- yields `np.ascontiguousarray(..., dtype=np.complex64)` (some vendor buffers are |
| 137 | + views or non-contiguous); |
| 138 | +- fills `sample_rate` and `center_freq` from what it actually tuned/requested, |
| 139 | + not from a constant, so downstream measurements (e.g. `estimate_fm_cfo`) are |
| 140 | + correct; |
| 141 | +- handles partial/short reads at the stream tail without raising. |
| 142 | + |
| 143 | +### 2. The collection tooling is HackRF-shaped (a real gap, not a bug) |
| 144 | + |
| 145 | +The DSP is device-agnostic; the *gain-search and capture-quality* tooling is |
| 146 | +not, and this is worth stating plainly so nobody assumes a free port: |
| 147 | + |
| 148 | +- `sources/probe.py` expects a handle `h` with hackrfpy's `capture(...)` |
| 149 | + signature and routes captures through a file to dodge the Windows pipe |
| 150 | + corruption (see the module docstring). |
| 151 | +- `core/gain_search.py` (`search_gain`) assumes the HackRF's three-knob gain |
| 152 | + model: `lna` / `vga` / `amp`, with the specific "SNR improves with amp+LNA, |
| 153 | + not VGA" strategy. That is right for the HackRF and wrong in the specifics for |
| 154 | + most other radios. |
| 155 | +- `tools/preflight_collection.py` builds on both. |
| 156 | + |
| 157 | +Porting a *receiver + demod* path to another SDR needs none of this — write an |
| 158 | +`IQSource` and you are done. Porting *auto-gain search* needs a per-device gain |
| 159 | +model. The clean way to do it, when the need arises, is to lift the gain |
| 160 | +abstraction into its own small protocol (e.g. a `GainModel` with the device's |
| 161 | +knobs and a `set(...)` method) so `search_gain` can drive any radio, rather than |
| 162 | +special-casing hackrfpy. Until then, treat auto-gain as HackRF-only. |
| 163 | + |
| 164 | +### 3. Keeping the core device-free (the invariant) |
| 165 | + |
| 166 | +The whole design rests on `src/` importing no vendor library. That is easy to |
| 167 | +break accidentally (one convenient `import` in a core module). Consider a guard |
| 168 | +test that walks `src/sdr_dsp/` and asserts no module imports `hackrfpy` (or any |
| 169 | +known vendor lib) — it is a one-file test that permanently protects the seam. |
| 170 | +Adapters live in `examples/`, which is exempt. |
| 171 | + |
| 172 | +### 4. A minimal adapter template |
| 173 | + |
| 174 | +Receive (satisfies `IQSource`): |
| 175 | + |
| 176 | +```python |
| 177 | +import numpy as np |
| 178 | + |
| 179 | +class RtlSdrSource: # structural IQSource; no base class needed |
| 180 | + def __init__(self, dev, sample_rate, center_freq, block_size=65536): |
| 181 | + self._dev = dev # an opened pyrtlsdr device |
| 182 | + self.sample_rate = float(sample_rate) |
| 183 | + self.center_freq = float(center_freq) |
| 184 | + self.block_size = int(block_size) |
| 185 | + |
| 186 | + def blocks(self): |
| 187 | + for raw in self._dev.read_bytes_stream(self.block_size): # cu8 |
| 188 | + i = raw[0::2].astype(np.float32); q = raw[1::2].astype(np.float32) |
| 189 | + yield ((i - 127.5) + 1j * (q - 127.5)) / 127.5 # -> complex64 |
| 190 | +``` |
| 191 | + |
| 192 | +Transmit (satisfies `TXSink`): |
| 193 | + |
| 194 | +```python |
| 195 | +class RtlSdrHasNoTx: ... # RTL-SDR is RX-only; TX example uses HackRF |
| 196 | + |
| 197 | +class MyTxSink: |
| 198 | + def __init__(self, dev, sample_rate, center_freq): |
| 199 | + self._dev = dev |
| 200 | + self.sample_rate = float(sample_rate) |
| 201 | + self.center_freq = float(center_freq) |
| 202 | + |
| 203 | + def transmit(self, iq: np.ndarray) -> None: |
| 204 | + self._dev.write(to_device_format(np.ascontiguousarray(iq, np.complex64))) |
| 205 | +``` |
| 206 | + |
| 207 | +Everything downstream — `fm_demod`, `estimate_fm_cfo`, `channelize`, the demod |
| 208 | +family, framing, `measure` — runs against these unchanged. |
| 209 | + |
| 210 | +### Porting checklist |
| 211 | + |
| 212 | +- [ ] Write an `IQSource` adapter; normalize the device format to `complex64` in |
| 213 | + `blocks()` (§1). |
| 214 | +- [ ] Set `sample_rate` / `center_freq` from what was actually tuned. |
| 215 | +- [ ] (If transmitting) write a `TXSink` adapter; prove the stack against |
| 216 | + `LoopbackSink` first, then swap in the device. |
| 217 | +- [ ] Validate against a file first: run the pipeline through `FileSource` on a |
| 218 | + saved capture, then against the live adapter — output should match. |
| 219 | +- [ ] Leave `core/` untouched. If you found yourself editing it, the seam moved |
| 220 | + to the wrong place. |
| 221 | +- [ ] Only if you need auto-gain: add a per-device gain model (§2). Otherwise set |
| 222 | + gains manually in the adapter. |
| 223 | + |
| 224 | +--- |
| 225 | + |
| 226 | +*This is a personal research library; the abstractions above are real and |
| 227 | +enforced today, but the collection-tooling generalization in §2 is future work, |
| 228 | +not a shipped feature. See `sources/base.py`, `sinks/tx_sink.py`, and |
| 229 | +`examples/hackrf_capture.py` for the authoritative reference.* |
0 commit comments