|
| 1 | +# Contributing to ESP EyeBond Collector |
| 2 | + |
| 3 | +Thanks for your interest in improving the firmware. This document covers the architecture, the protocol contract it implements, and the validation workflow. |
| 4 | + |
| 5 | +If you're a user looking to install or troubleshoot the bridge, the [README](README.md) is the right place to start. |
| 6 | + |
| 7 | +--- |
| 8 | + |
| 9 | +## Project Philosophy |
| 10 | + |
| 11 | +The firmware is split into two strictly separated layers: |
| 12 | + |
| 13 | +``` |
| 14 | +platform-independent protocol core (plain C++) → thin ESPHome glue (Arduino) |
| 15 | +``` |
| 16 | + |
| 17 | +The most important rule: |
| 18 | + |
| 19 | +> **All protocol logic lives in the core; the glue does I/O only.** |
| 20 | +
|
| 21 | +The core (`namespace eybond`) includes **nothing** from Arduino or ESPHome — this is enforced by the host build. It is event-driven ("sans-IO"): the platform feeds it events (UDP datagram, TCP data, UART bytes, time ticks) and the core returns actions through an `Actions` interface. That is why it compiles with plain `g++` and is fully covered by host tests, and why a second platform (e.g. an ESP-IDF socket layer) would only need a new glue file, not protocol changes. |
| 22 | + |
| 23 | +--- |
| 24 | + |
| 25 | +## Architecture |
| 26 | + |
| 27 | +``` |
| 28 | +components/eybond_collector/ # flat layout: ESPHome does not copy a component's subdirectories |
| 29 | +├── frame.{h,cpp} # core: 8-byte header >HHHBB, frame assembly |
| 30 | +├── stream_splitter.{h,cpp} # core: TCP stream parsing — AT lines vs binary frames |
| 31 | +├── at_handler.{h,cpp} # core: AT dictionary (DTUPN/WFSS/FWVER/...), byte-exact replies |
| 32 | +├── discovery.{h,cpp} # core: set>server=IP:PORT; → rsp>server=2; + PN from MAC |
| 33 | +├── profile.h # core: virtual collector identity |
| 34 | +├── core.{h,cpp} # core: orchestrator — heartbeat, FC dispatch, UART bridge, reconnect |
| 35 | +├── eybond_collector.{h,cpp} # ESPHome glue (Arduino: WiFiUDP/WiFiClient + uart) |
| 36 | +├── baud_select.h # ESPHome glue: runtime baud-rate select entity |
| 37 | +├── __init__.py # ESPHome config schema |
| 38 | +└── select.py # ESPHome select platform schema |
| 39 | +``` |
| 40 | + |
| 41 | +--- |
| 42 | + |
| 43 | +## Protocol Contract |
| 44 | + |
| 45 | +The behavioral specification is the integration's own fake collector: |
| 46 | +`ha-eybond-local/tests/helpers/fake_collector.py` and `.local/tools/fake_collector_lib.py`. The firmware is a port of that logic. When those files change, re-run `cross_check.py`. |
| 47 | + |
| 48 | +- **Discovery**: UDP `58899`, `set>server=IP:PORT;` → reply `rsp>server=2;` + reverse TCP to Home Assistant. |
| 49 | +- **Heartbeat**: FC=1 on connect, then periodic (default 60 s); payload is the PN truncated to 14 bytes. A server FC=1 is answered with its own `tid`. |
| 50 | +- **FC=2** (query collector): the core answers parameters it knows from its own state (PN, firmware, current endpoint, baud); the glue answers the rest (hardware, IP, SSID, RSSI, Wi-Fi scan list). Unknown parameters → `01 <param>` (refused), like the factory fake. |
| 51 | +- **FC=3** (set collector): writes are *staged* and committed together. Param 41/43 stage the target Wi-Fi SSID/password; param 21 stages the server endpoint; param 29 (`system_operation`) **commits everything staged** — endpoint and/or Wi-Fi — and is refused only when nothing is staged. Param 34 (baud) applies immediately. Everything else is refused `01 <param>`. The committed endpoint is persisted to NVS (so the bridge reconnects to the same HA across reboots / can be moved to another HA); incoming discovery still overrides it live. `AT+CLDSRVHOST1=` applies an endpoint immediately (no staging) and retargets the link. |
| 52 | +- **FC=4** (forward): payload → UART as-is; the inverter reply (end detected by a `response_gap` of silence) → frame with the same `tid`/`devcode`/`devaddr`. **Inverter silence → no reply at all** — the integration distinguishes a timeout from a zero-filled answer, so this is load-bearing. |
| 53 | +- **AT lines** (`AT+...\n`) are interleaved with binary frames on the same TCP stream; they are told apart by the 3-byte `AT+` prefix. |
| 54 | +- Inverter requests are serialized: one in flight, with a `command_spacing` pause (factory ~850 ms) between commands. |
| 55 | + |
| 56 | +### Identity |
| 57 | + |
| 58 | +The PN is synthetic: `V00` + the 15-digit decimal MAC → the PN18 format `^[A-Z]\d{17}$` the integration accepts. Override with `pn:` (the schema validator accepts synthetic values only: a letter + 13/17 digits). `devcode` defaults to `0x0000`, which the integration handles through its `unknown_0x0000` profile. |
| 59 | + |
| 60 | +### AT+VDTU capability probe |
| 61 | + |
| 62 | +The firmware answers `AT+VDTU?` with a capability string: |
| 63 | + |
| 64 | +``` |
| 65 | +AT+VDTU:esp-collector,<semver>;features=local_only,no_cloud,wifi_params,endpoint_write;uart=<baud,data,stop,parity>;spacing_ms=<n>;queue=<n> |
| 66 | +``` |
| 67 | + |
| 68 | +EyeBond Local probes this on link-up. The `esp-collector,` prefix identifies a virtual bridge so cloud-only flows can be hidden and an honest device name/version can be shown. Factory collectors do not return the prefix. For every other command the bridge behavior remains compatible with the factory collector path. When the capability string is unset (core without glue), the reply is an empty value, like the factory reference for unknown commands. |
| 69 | + |
| 70 | +--- |
| 71 | + |
| 72 | +## Validation Pipeline |
| 73 | + |
| 74 | +Four levels, fastest first. The bench cycle runs them in order. |
| 75 | + |
| 76 | +```bash |
| 77 | +# 1. Core host unit tests (plain g++, no hardware) |
| 78 | +cd host_tests && make test |
| 79 | + |
| 80 | +# 2. Byte-for-byte cross-check against the integration's own builders |
| 81 | +../../.venv/bin/python3 cross_check.py |
| 82 | + |
| 83 | +# 3. Linux end-to-end: the real core (host_sim: POSIX sockets + a PTY in place of |
| 84 | +# the UART) against the integration's protocol code — discovery, heartbeat, AT, |
| 85 | +# the FC4 bridge, and timeouts |
| 86 | +../../.venv/bin/python3 e2e_sim.py |
| 87 | + |
| 88 | +# host_sim can also run standalone — the real integration in Home Assistant |
| 89 | +# (or .local/tools) can talk to it, with a fake inverter on the PTY: |
| 90 | +make build/host_sim && ./build/host_sim --udp-port 58899 |
| 91 | + |
| 92 | +# Build the firmware (esphome lives in the project-local .venv) |
| 93 | +.venv/bin/esphome compile examples/esp8266-d1-mini.yaml # ESP8266 release preset |
| 94 | +.venv/bin/esphome compile examples/esp32-devkit.yaml # ESP32 release preset |
| 95 | + |
| 96 | +# 4. FULL autonomous bench cycle (unit → cross-check → compile → OTA → hardware E2E) |
| 97 | +./bench_cycle.sh [device-address] |
| 98 | +``` |
| 99 | + |
| 100 | +### Hardware-in-the-loop bench (no inverter required) |
| 101 | + |
| 102 | +An ESP8266 connected to the host by USB only. The same USB bridge that flashes the board exposes UART0, so the host plays the inverter: `host_tests/fake_pi30_serial.py` answers PI30 commands on `/dev/ttyUSB0` (`2400 8N1`) with the values from the integration's PI30 driver tests. `host_tests/bench_e2e.py` plays the Home Assistant side (discovery, reverse TCP, AT, FC4, collector parameters, baud switching) using the integration's own builders. The bench config is `examples/bench-esp8266.yaml` (synthetic PN hard-coded so no MAC-derived identity reaches logs; faster timings for quick iteration). |
| 103 | + |
| 104 | +`host_tests/bench_probe.py` runs the integration's real `Pi30Driver.async_probe()` against the live device — the same detection step the Home Assistant config flow performs. |
| 105 | + |
| 106 | +Bench gotchas worth knowing: |
| 107 | + |
| 108 | +- The first flash is over USB; every later flash is OTA, leaving the USB port free for the fake inverter. |
| 109 | +- Opening the serial port pulses DTR/RTS and reboots the board, so `bench_e2e` retries discovery until Wi-Fi is back (~10–15 s). |
| 110 | +- The ESP8266 boot ROM prints at 74880 baud, which looks like garbage at 2400 and can prepend itself to the first PI30 request — the fake inverter searches for a CRC-valid suffix. |
| 111 | +- If a live Home Assistant also owns the bench device, its periodic discovery can steal the bridge mid-test (last redirect wins); test steps re-acquire the link and retry. |
| 112 | + |
| 113 | +--- |
| 114 | + |
| 115 | +## Safety Invariants |
| 116 | + |
| 117 | +These are inherited from the parent project and must not be weakened: |
| 118 | + |
| 119 | +- **One master per bus.** While the bridge is active, no other Modbus/serial master may poll the inverter in parallel. |
| 120 | +- **The bridge never writes to the inverter on its own** — it only forwards what the integration sent. |
| 121 | +- **No real identifiers in tracked files** — no real PNs/SNs/passwords/SSIDs in code, tests, or example configs. Synthetic only (`V00000200000000001`-style). Secrets live in `secrets.yaml`, which is git-ignored. |
| 122 | +- **Honest timeouts** — never synthesize a reply on the inverter's behalf. |
| 123 | + |
| 124 | +--- |
| 125 | + |
| 126 | +## Current Limitations |
| 127 | + |
| 128 | +- Arduino framework only — ESP8266, ESP32-arduino, and **BK72xx/RTL87xx via LibreTiny** |
| 129 | + (the same glue compiles on all three; LibreTiny defines `USE_ARDUINO`). ESP-IDF would be |
| 130 | + a separate adapter (the core is platform-independent; only a socket glue layer is needed). |
| 131 | +- **LibreTiny (BK72xx) caveats:** its `UARTComponent` has no `load_settings()`, so runtime |
| 132 | + baud/format change (select / `AT+UART=` / FC=3 param 34) is unavailable — baud/framing are |
| 133 | + fixed by YAML there. Software serial is chip-dependent, so the inverter must be on a |
| 134 | + hardware UART. `WiFiClient::setNoDelay` and `IPAddress::toString` are absent and are guarded |
| 135 | + out (`#ifndef USE_LIBRETINY` / manual octet formatting). |
| 136 | +- `AT+SYST` returns an empty value (no time source); attach a `time:` component if you want a real timestamp. |
| 137 | +- `WiFiClient::connect` is blocking (a rare event, but it can briefly stall the loop on connect). |
| 138 | +- The SMG/Modbus path is implemented through the same transparent bridge as PI30 and passes simulation, but has not yet been verified on real SMG hardware. |
| 139 | + |
| 140 | +--- |
| 141 | + |
| 142 | +## Roadmap |
| 143 | + |
| 144 | +- **Integration-side support:** EyeBond Local detects the `esp-collector,` VDTU prefix, |
| 145 | + gates cloud-only flows, forces the collector operation mode to "HA only" for a bridge, |
| 146 | + and shows an honest device name. |
| 147 | +- **ESP32 + ESP-IDF adapter** — a socket glue layer over `CollectorCore::Actions`. |
| 148 | +- **BLE provisioning:** ESP8266 has no BLE radio. ESP32 builds can expose the bridge's BLE Wi-Fi provisioning path when `esp32_ble_server:` and `ble_provisioning: true` are enabled; keep an eye on firmware image size because BLE + Wi-Fi can exceed small default partitions. |
0 commit comments