Skip to content

Commit 7b255f0

Browse files
committed
modularity setup notes
1 parent 1d7da55 commit 7b255f0

1 file changed

Lines changed: 229 additions & 0 deletions

File tree

‎docs/MODULARITY.md‎

Lines changed: 229 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,229 @@
1+
# Modularity: using sdr_dsp with any SDR
2+
3+
`sdr_dsp` is deliberately split so the signal processing never knows what radio
4+
produced its samples. This document names the layers, shows where the seam is, and lists
5+
what remains to be done to drive the library from a radio other than a HackRF.
6+
7+
## The layered model
8+
9+
```
10+
[ library ] <-> [ interface ] <-> [ device adapter ] <-> [ SDR hardware ]
11+
core/ IQSource / examples/ or your radio + its
12+
pure DSP TXSink (a HAL) code (Adapter pattern) vendor library
13+
```
14+
15+
The vocabulary for the middle two boxes: the **interface** is a *hardware
16+
abstraction layer* (HAL), expressed here as two Python `Protocol`s (structural,
17+
"duck-typed" contracts). The **device adapter** (a.k.a. driver adapter) is the
18+
*Adapter design pattern* — a small class that satisfies a protocol by wrapping a
19+
vendor library. The core depends on the contract, never on a device; that
20+
dependency inversion is the entire point.
21+
22+
```mermaid
23+
flowchart LR
24+
subgraph L["library — core/ (pure numpy/scipy)"]
25+
DSP["fm_demod · estimate_fm_cfo ·<br/>channelize · measure · modulate ·<br/>framing · sync · features"]
26+
end
27+
subgraph I["interface — the HAL (Protocols)"]
28+
RX["IQSource<br/>sources/base.py"]
29+
TX["TXSink<br/>sinks/tx_sink.py"]
30+
end
31+
subgraph A["device adapter — outside the library"]
32+
HC["hackrf_capture.py (RX)"]
33+
HS["hackrf_sink.py (TX)"]
34+
OTHER["rtlsdr_source.py · usrp_sink.py · …<br/>(you write these)"]
35+
end
36+
subgraph H["SDR hardware + vendor library"]
37+
HW["HackRF / RTL-SDR / USRP / BladeRF …<br/>hackrfpy · pyrtlsdr · SoapySDR · UHD"]
38+
end
39+
40+
DSP -- "complex64 arrays in/out" --> RX
41+
DSP -- "complex64 arrays in/out" --> TX
42+
RX --> HC --> HW
43+
RX --> OTHER --> HW
44+
TX --> HS --> HW
45+
TX --> OTHER
46+
47+
%% device-free sources that need no adapter
48+
FILE["FileSource · ArraySource<br/>(ship in the library)"] --> RX
49+
```
50+
51+
## What each layer is, and where it lives
52+
53+
| Layer | What it is | Where it lives | Depends on |
54+
|-------|-----------|----------------|-----------|
55+
| library | Pure DSP: arrays in, arrays out. Knows nothing about radios. | `src/sdr_dsp/core/` | numpy, scipy only |
56+
| interface (HAL) | Two `Protocol`s that define the RX and TX contracts. | `sources/base.py` (`IQSource`), `sinks/tx_sink.py` (`TXSink`) | numpy only |
57+
| device adapter | A class that satisfies a protocol by wrapping a vendor library. | `examples/` or your own code — **outside** the library | the vendor library |
58+
| SDR hardware | The radio and its driver stack. | n/a (hardware) | — |
59+
60+
The core's device-independence is a checked fact, not a wish: **nothing under
61+
`src/` imports a vendor library.** Every `hackrfpy` mention in `src/` is a
62+
docstring or a comment. Keep it that way (see *Keeping the core device-free*).
63+
64+
## The two contracts
65+
66+
An adapter only has to provide these attributes and methods. Anything that does
67+
is an `IQSource` / `TXSink` as far as the library is concerned — no base class to
68+
inherit, no registration.
69+
70+
**Receive — `IQSource`** (`sources/base.py`):
71+
72+
```python
73+
sample_rate: float # samples per second (Hz)
74+
center_freq: float # RF center frequency the samples were captured at (Hz)
75+
def blocks(self) -> Iterator[np.ndarray]: # yield complex64 blocks
76+
...
77+
# A bounded source may also offer read(n); live sources need only blocks().
78+
```
79+
80+
**Transmit — `TXSink`** (`sinks/tx_sink.py`):
81+
82+
```python
83+
sample_rate: float
84+
center_freq: float
85+
def transmit(self, iq: np.ndarray) -> None: # send one complex64 buffer
86+
...
87+
```
88+
89+
## What ships vs. what you write
90+
91+
Ships in the library (device-free, so the whole pipeline is testable with no
92+
radio):
93+
94+
- `ArraySource` — wrap an in-memory `complex64` array (tests, synthetic signals).
95+
- `FileSource` — stream a SigMF recording from disk; satisfies `IQSource`, so
96+
code written against a file runs unchanged against hardware later.
97+
- `LoopbackSink` — a `TXSink` that "transmits" into an in-memory buffer, so the
98+
full TX stack (framing → modulate → sink) can be exercised without a radio.
99+
100+
You write (per radio, once):
101+
102+
- An `IQSource` adapter for receive, and/or a `TXSink` adapter for transmit.
103+
- The reference adapters to copy are `examples/hackrf_capture.py` (RX) and
104+
`examples/hackrf_sink.py` (TX). The TXSink docstring names `usrp_sink.py` as
105+
the intended shape for the next one.
106+
107+
---
108+
109+
## What needs to be done to support another SDR
110+
111+
Porting the **DSP** to a new radio is small and clean. Porting the **collection
112+
tooling** is the part with real work in it. The two are separated on purpose.
113+
114+
### 1. Sample-format normalization (belongs in the adapter)
115+
116+
The core only ever sees `complex64`. Real radios do not hand you `complex64` —
117+
each vendor streams its own on-the-wire format, and converting it is the
118+
adapter's job, not the core's. This is the single most common thing a new
119+
adapter must get right.
120+
121+
| Radio (vendor lib) | Native sample format | Normalize to `complex64` by |
122+
|--------------------|----------------------|------------------------------|
123+
| HackRF (hackrfpy) | `ci8` (interleaved int8 I/Q) | `(i + 1j*q) / 128.0` |
124+
| RTL-SDR (pyrtlsdr) | `cu8` (interleaved uint8, offset 127.5) | `((i - 127.5) + 1j*(q - 127.5)) / 127.5` |
125+
| USRP (UHD) | `cf32` / `sc16` | cast to `complex64` (already float) / scale int16 by 32768 |
126+
| SoapySDR devices | `CF32` or `CS16` (device-dependent) | as above, per the stream format you request |
127+
128+
The library already does exactly this for HackRF `ci8` in `io/sigmf.py`
129+
(`load_iq`) and reuses it in `FileSource` — that is the template. An adapter's
130+
`blocks()` should apply the equivalent normalization for its device and yield
131+
`complex64`. **Do the scaling once, in the adapter; never push a device format
132+
into the core.**
133+
134+
A good adapter also, in `blocks()`:
135+
136+
- yields `np.ascontiguousarray(..., dtype=np.complex64)` (some vendor buffers are
137+
views or non-contiguous);
138+
- fills `sample_rate` and `center_freq` from what it actually tuned/requested,
139+
not from a constant, so downstream measurements (e.g. `estimate_fm_cfo`) are
140+
correct;
141+
- handles partial/short reads at the stream tail without raising.
142+
143+
### 2. The collection tooling is HackRF-shaped (a real gap, not a bug)
144+
145+
The DSP is device-agnostic; the *gain-search and capture-quality* tooling is
146+
not, and this is worth stating plainly so nobody assumes a free port:
147+
148+
- `sources/probe.py` expects a handle `h` with hackrfpy's `capture(...)`
149+
signature and routes captures through a file to dodge the Windows pipe
150+
corruption (see the module docstring).
151+
- `core/gain_search.py` (`search_gain`) assumes the HackRF's three-knob gain
152+
model: `lna` / `vga` / `amp`, with the specific "SNR improves with amp+LNA,
153+
not VGA" strategy. That is right for the HackRF and wrong in the specifics for
154+
most other radios.
155+
- `tools/preflight_collection.py` builds on both.
156+
157+
Porting a *receiver + demod* path to another SDR needs none of this — write an
158+
`IQSource` and you are done. Porting *auto-gain search* needs a per-device gain
159+
model. The clean way to do it, when the need arises, is to lift the gain
160+
abstraction into its own small protocol (e.g. a `GainModel` with the device's
161+
knobs and a `set(...)` method) so `search_gain` can drive any radio, rather than
162+
special-casing hackrfpy. Until then, treat auto-gain as HackRF-only.
163+
164+
### 3. Keeping the core device-free (the invariant)
165+
166+
The whole design rests on `src/` importing no vendor library. That is easy to
167+
break accidentally (one convenient `import` in a core module). Consider a guard
168+
test that walks `src/sdr_dsp/` and asserts no module imports `hackrfpy` (or any
169+
known vendor lib) — it is a one-file test that permanently protects the seam.
170+
Adapters live in `examples/`, which is exempt.
171+
172+
### 4. A minimal adapter template
173+
174+
Receive (satisfies `IQSource`):
175+
176+
```python
177+
import numpy as np
178+
179+
class RtlSdrSource: # structural IQSource; no base class needed
180+
def __init__(self, dev, sample_rate, center_freq, block_size=65536):
181+
self._dev = dev # an opened pyrtlsdr device
182+
self.sample_rate = float(sample_rate)
183+
self.center_freq = float(center_freq)
184+
self.block_size = int(block_size)
185+
186+
def blocks(self):
187+
for raw in self._dev.read_bytes_stream(self.block_size): # cu8
188+
i = raw[0::2].astype(np.float32); q = raw[1::2].astype(np.float32)
189+
yield ((i - 127.5) + 1j * (q - 127.5)) / 127.5 # -> complex64
190+
```
191+
192+
Transmit (satisfies `TXSink`):
193+
194+
```python
195+
class RtlSdrHasNoTx: ... # RTL-SDR is RX-only; TX example uses HackRF
196+
197+
class MyTxSink:
198+
def __init__(self, dev, sample_rate, center_freq):
199+
self._dev = dev
200+
self.sample_rate = float(sample_rate)
201+
self.center_freq = float(center_freq)
202+
203+
def transmit(self, iq: np.ndarray) -> None:
204+
self._dev.write(to_device_format(np.ascontiguousarray(iq, np.complex64)))
205+
```
206+
207+
Everything downstream — `fm_demod`, `estimate_fm_cfo`, `channelize`, the demod
208+
family, framing, `measure` — runs against these unchanged.
209+
210+
### Porting checklist
211+
212+
- [ ] Write an `IQSource` adapter; normalize the device format to `complex64` in
213+
`blocks()` (§1).
214+
- [ ] Set `sample_rate` / `center_freq` from what was actually tuned.
215+
- [ ] (If transmitting) write a `TXSink` adapter; prove the stack against
216+
`LoopbackSink` first, then swap in the device.
217+
- [ ] Validate against a file first: run the pipeline through `FileSource` on a
218+
saved capture, then against the live adapter — output should match.
219+
- [ ] Leave `core/` untouched. If you found yourself editing it, the seam moved
220+
to the wrong place.
221+
- [ ] Only if you need auto-gain: add a per-device gain model (§2). Otherwise set
222+
gains manually in the adapter.
223+
224+
---
225+
226+
*This is a personal research library; the abstractions above are real and
227+
enforced today, but the collection-tooling generalization in §2 is future work,
228+
not a shipped feature. See `sources/base.py`, `sinks/tx_sink.py`, and
229+
`examples/hackrf_capture.py` for the authoritative reference.*

0 commit comments

Comments
 (0)