|
| 1 | +# Modbus timeouts |
| 2 | + |
| 3 | +How register I/O deadlines, pacing, and retries work in sunsynk and the Home Assistant add-on. |
| 4 | +Background on the v1.0.0 stack change is [modbus-connection.md](modbus-connection.md). User-facing |
| 5 | +tuning tips are in [fault-finding](../www/docs/guide/fault-finding.md#c-reducing-timeouts) and |
| 6 | +[multi-options](../www/docs/reference/multi-options.md). |
| 7 | + |
| 8 | +Reported regressions after the v1.0.0 move from pymodbus to tmodbus / modbus-connection are tracked |
| 9 | +in [#672](https://github.com/kellerza/sunsynk/issues/672). |
| 10 | + |
| 11 | +## Shared behaviour (all transports) |
| 12 | + |
| 13 | +### Add-on `TIMEOUT` |
| 14 | + |
| 15 | +The **`TIMEOUT`** option (default **10** seconds) is the deadline for: |
| 16 | + |
| 17 | +- opening the link (connect), and |
| 18 | +- each individual register read or write attempt. |
| 19 | + |
| 20 | +It is **not** a cap on an entire sensor batch or poll cycle. `Sunsynk.read_sensors()` may issue many |
| 21 | +FC03 groups in one call; each group gets its own deadline. |
| 22 | + |
| 23 | +Before v1.0.0 edge fixes, the library also wrapped batches in `asyncio.timeout(2 × TIMEOUT)`. That |
| 24 | +double layer is **removed** — only the modbus-connection / Solarman client deadline applies now. |
| 25 | + |
| 26 | +Raising `TIMEOUT` makes the client wait **longer for a missing reply**. It does **not** insert a |
| 27 | +pause **between** successful requests. For that, see [message spacing](#message-spacing) below. |
| 28 | + |
| 29 | +### Add-on retries and stale inverters |
| 30 | + |
| 31 | +Above the driver, `AInverter` adds: |
| 32 | + |
| 33 | +| Layer | Behaviour | |
| 34 | +| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | |
| 35 | +| `read_sensors_retry()` | Up to **3** attempts per scheduled read, **0.2s** apart; then retries **one sensor at a time** if the batch failed | |
| 36 | +| Stale quiet | After **`STALE_INVERTER_AFTER_SECONDS`** without a successful read, skip Modbus for **`STALE_INVERTER_SKIP_SECONDS`**, then probe identity | |
| 37 | +| Timeout counter | `Sunsynk.timeouts` increments on timeout; exposed as the **RS485 timeout** MQTT sensor | |
| 38 | + |
| 39 | +These apply to Modbus (serial / TCP) and Solarman alike. |
| 40 | + |
| 41 | +### Message spacing |
| 42 | + |
| 43 | +For Modbus connections opened via [`open_connection()`](../src/sunsynk/connection.py), the library |
| 44 | +sets **`message_spacing = 0.05`** (50ms) by default. modbus-connection waits that long after the |
| 45 | +**previous request finished** before starting the next one on the same link (shared across all |
| 46 | +`for_unit()` handles on that port). |
| 47 | + |
| 48 | +Pass `message_spacing=0` to `open_connection()` to disable. There is no add-on option for this yet; |
| 49 | +the default is fixed in code (`DEFAULT_MESSAGE_SPACING`). |
| 50 | + |
| 51 | +--- |
| 52 | + |
| 53 | +## Serial |
| 54 | + |
| 55 | +**PORT examples:** `/dev/ttyUSB0`, `/dev/serial/by-id/usb-FTDI_…` |
| 56 | + |
| 57 | +Direct RS485 through a USB adapter uses **tmodbus** RTU via |
| 58 | +[modbus-connection](https://home-assistant-libs.github.io/modbus-connection/) (`ModbusConnection` |
| 59 | + |
| 60 | +- `ModbusSerialParams`). |
| 61 | + |
| 62 | +### Why v1.0.0 regressed on serial |
| 63 | + |
| 64 | +Pre-1.0.0 **pymodbus** (`PySunsynk`) was relatively forgiving on direct serial: extra asyncio |
| 65 | +overhead, per-group `asyncio.timeout`, and behaviour that often masked bus timing issues. |
| 66 | + |
| 67 | +v1.0.0 **tmodbus** is stricter: |
| 68 | + |
| 69 | +1. **No turnaround gap by default** — Modbus RTU requires a silent interval (3.5 character times) |
| 70 | + before a new frame. tmodbus enforces that gap from the **send** time (`_last_frame_ended_at` is |
| 71 | + updated when the request is written). By the time the inverter replies, that interval has already |
| 72 | + elapsed, so the **next poll can go out immediately** after the response is parsed. |
| 73 | +2. **Slow slaves** — Deye inverters and USB-FTDI adapters often need tens of milliseconds after a |
| 74 | + reply before they accept another master request. Polling back-to-back produces intermittent |
| 75 | + timeouts (~every 10s with default `TIMEOUT`, matching field reports in #672). |
| 76 | +3. **FTDI latency** — FT232R and similar chips buffer serial data (default latency timer ~16ms). |
| 77 | + tmodbus documents that true inter-frame silence detection is unreliable on USB serial; it sizes |
| 78 | + frames by content instead. Late bytes can still land in the buffer after a timeout. |
| 79 | +4. **No retry on timeout** — modbus-connection configures tmodbus with `response_retry_strategy` |
| 80 | + that does **not** retry bare `TimeoutError`. One missed frame surfaces as an error immediately. |
| 81 | +5. **Buffer not flushed on timeout** — on RTU timeout, tmodbus leaves unread bytes in the protocol |
| 82 | + buffer. The next request can then see garbage (`Received data with no pending requests`, |
| 83 | + `Cannot frame response with unsupported function code 0x79`) until the link recovers or the |
| 84 | + add-on is restarted. |
| 85 | + |
| 86 | +Increasing **`TIMEOUT`** alone does not fix (1)–(3); it only waits longer before declaring failure. |
| 87 | + |
| 88 | +### Fix: 50ms message spacing |
| 89 | + |
| 90 | +`sunsynk.connection.open_connection()` now defaults to **50ms** `message_spacing` for all Modbus |
| 91 | +ports (including serial). That gap is enforced **after each completed request** by |
| 92 | +modbus-connection’s `Pacer`, giving the inverter and adapter time to turn the bus around. |
| 93 | + |
| 94 | +This matches common RS485 guidance (roughly **50–100ms** between commands on slow devices) and is |
| 95 | +the main code-side mitigation for #672 on direct serial. |
| 96 | + |
| 97 | +### What still helps on serial |
| 98 | + |
| 99 | +- Lower **`READ_SENSORS_BATCH_SIZE`** (e.g. 8) — smaller FC03 reads, less time on the wire per |
| 100 | + request. |
| 101 | +- **`READ_ALLOW_GAP`** — batch nearby registers; fewer total requests. |
| 102 | +- **mbusd** — run mbusd on the serial port and point the add-on at `tcp://host:502` (see |
| 103 | + [fault-finding](../www/docs/guide/fault-finding.md#direct-serial)). |
| 104 | +- **One client per port** — do not share the serial device with mbusd, Node-RED, or a second add-on |
| 105 | + instance. |
| 106 | +- **Cabling / termination** — see fault-finding; unrelated to the v1.0.0 stack but often confused |
| 107 | + with it. |
| 108 | + |
| 109 | +### Code path |
| 110 | + |
| 111 | +```text |
| 112 | +AInverter.read_sensors() |
| 113 | + → Sunsynk.read_sensors() # groups sensors, one FC03 per group |
| 114 | + → unit.read_holding_registers() |
| 115 | + → modbus-connection TmodbusUnit # connect + Pacer (50ms) + tmodbus RTU |
| 116 | +``` |
| 117 | + |
| 118 | +Implementation: [`connection.py`](../src/sunsynk/connection.py), |
| 119 | +[`sunsynk.py`](../src/sunsynk/sunsynk.py), |
| 120 | +[`driver.py`](../src/ha_addon_sunsynk_multi/driver.py) (`_shared_modbus_connection`). |
| 121 | + |
| 122 | +--- |
| 123 | + |
| 124 | +## TCP |
| 125 | + |
| 126 | +**PORT examples:** `tcp://192.168.1.50:502`, `serial-tcp://gateway:502`, `udp://host:502` |
| 127 | + |
| 128 | +Modbus TCP, RTU-over-TCP, and UDP use the same **tmodbus** / modbus-connection stack as serial. The |
| 129 | +add-on passes **`TIMEOUT`** and the default **50ms** `message_spacing` into `open_connection()`. |
| 130 | + |
| 131 | +### `tcp://` (Modbus TCP / MBAP) |
| 132 | + |
| 133 | +Typical setups: |
| 134 | + |
| 135 | +- **mbusd** or another gateway exposing Modbus TCP on the LAN. |
| 136 | +- **RS485→Ethernet** devices (Waveshare, USR, etc.) speaking Modbus TCP toward the host. |
| 137 | + |
| 138 | +Requests are serialized on **one `ModbusConnection` per PORT** string. Multiple inverters on the |
| 139 | +same gateway share that connection; modbus-connection’s lock and pacer ensure only one transaction |
| 140 | +is in flight at a time. |
| 141 | + |
| 142 | +Timeouts here are usually **gateway or multi-master** issues rather than RTU turnaround: |
| 143 | + |
| 144 | +- **`Received unexpected response with Transaction ID`** — another client is talking to the same |
| 145 | + gateway, or the gateway returned a late MBAP frame for an earlier request (#672, multi-inverter |
| 146 | + setups). |
| 147 | +- **`Inverter … stale`** — no successful read within `STALE_INVERTER_AFTER_SECONDS`; often follows |
| 148 | + repeated timeouts, not necessarily total loss of connectivity (external Modbus tools may still |
| 149 | + work). |
| 150 | + |
| 151 | +Raising `TIMEOUT` can help on **slow gateways** or long RTU legs behind TCP. The **50ms spacing** |
| 152 | +still applies between requests on the shared connection (including when two `MODBUS_ID`s share one |
| 153 | +`tcp://` PORT). |
| 154 | + |
| 155 | +### `serial-tcp://` (RTU-over-TCP) |
| 156 | + |
| 157 | +Same as TCP for the socket side, but the framer is **RTU**. The gateway translates RTU frames on the |
| 158 | +RS485 side. Spacing and turnaround constraints from [Serial](#serial) apply |
| 159 | +**on the bus behind the gateway**; the 50ms default helps gateways that forward requests immediately |
| 160 | +without pacing. |
| 161 | + |
| 162 | +If the gateway buffers poorly under rapid polling, prefer **`tcp://`** (native Modbus TCP) when the |
| 163 | +device supports it, or reduce batch size / read frequency. |
| 164 | + |
| 165 | +### `udp://` |
| 166 | + |
| 167 | +Modbus UDP (socket framing) via tmodbus. Same timeout and message-spacing defaults. RTU-over-UDP |
| 168 | +(`serial-udp://`) is **not** supported by the tmodbus backend. |
| 169 | + |
| 170 | +### Code path |
| 171 | + |
| 172 | +Same as serial; only `url_to_params()` differs ([`connection.py`](../src/sunsynk/connection.py)). |
| 173 | + |
| 174 | +--- |
| 175 | + |
| 176 | +## Solarman |
| 177 | + |
| 178 | +**PORT example:** `solarman://192.168.1.182:8899` plus **`DONGLE_SERIAL_NUMBER`** |
| 179 | + |
| 180 | +Solarman does **not** use modbus-connection or tmodbus. It uses |
| 181 | +[`SolarmanUnit`](../src/sunsynk/solarman.py) over **PySolarmanV5Async** (proprietary V5 tunnel). |
| 182 | + |
| 183 | +### Timeouts |
| 184 | + |
| 185 | +- **`TIMEOUT`** maps to **`socket_timeout`** on the Solarman client (connect and reads). |
| 186 | +- **`TimeoutError`** on read: log, **disconnect**, re-raise — **no** retry inside `SolarmanUnit` |
| 187 | + (unlike other exceptions). |
| 188 | +- Other read errors: up to **`RETRY_ATTEMPTS` (5)** with **2s** sleep and disconnect between tries. |
| 189 | + |
| 190 | +There is **no** `message_spacing` knob on Solarman; pacing is whatever the dongle and |
| 191 | +PySolarmanV5Async implement, plus the add-on’s normal poll schedules. |
| 192 | + |
| 193 | +### Tuning |
| 194 | + |
| 195 | +- Use **`solarman://`** explicitly (not `tcp://` with a dongle serial — that remaps at startup). |
| 196 | +- Reduce read frequency in **Schedules**; Solarman links are slower than wired RS485. |
| 197 | +- Do not point a serial `PORT` at a Solarman dongle IP; that is a different transport (#672 reports |
| 198 | + conflating the two). |
| 199 | + |
| 200 | +### Code path |
| 201 | + |
| 202 | +```text |
| 203 | +create_sunsynk() → SolarmanUnit → PySolarmanV5Async.read_holding_registers() |
| 204 | +``` |
| 205 | + |
| 206 | +No shared `ModbusConnection`; one Solarman client per inverter port entry. |
| 207 | + |
| 208 | +--- |
| 209 | + |
| 210 | +## Quick reference |
| 211 | + |
| 212 | +| Topic | Serial / TCP | Solarman | |
| 213 | +| ------------------------------ | ------------------------------------- | ------------------------------------ | |
| 214 | +| Library | modbus-connection + tmodbus | PySolarmanV5Async | |
| 215 | +| Add-on `TIMEOUT` | Per connect / FC03 / FC16 attempt | `socket_timeout` | |
| 216 | +| Default gap after each reply | **50ms** (`message_spacing`) | — | |
| 217 | +| Driver retry on `TimeoutError` | No (immediate error) | No (disconnect) | |
| 218 | +| Add-on batch retry | Yes (`read_sensors_retry`) | Yes (same) | |
| 219 | +| Multi-inverter same PORT | One connection, `for_unit(MODBUS_ID)` | Not supported (one dongle per entry) | |
| 220 | + |
| 221 | +### Add-on options that affect this |
| 222 | + |
| 223 | +User-facing keys from [multi-options](../www/docs/reference/multi-options.md) (and |
| 224 | +[schedules](../www/docs/reference/schedules.md)). There is **no** add-on option for |
| 225 | +`message_spacing`; it is fixed at 50ms in `open_connection()`. |
| 226 | + |
| 227 | +| Option | Role | |
| 228 | +| ------------------------------- | --------------------------------------------------------------------------------------------- | |
| 229 | +| `TIMEOUT` | Deadline in **seconds** for connect and each register read/write (default **10**) | |
| 230 | +| `READ_SENSORS_BATCH_SIZE` | Max registers per FC03 group (smaller → more requests, less time per request) | |
| 231 | +| `READ_ALLOW_GAP` | Extra registers allowed between addresses so groups merge (fewer requests) | |
| 232 | +| `STALE_INVERTER_AFTER_SECONDS` | Grace after last good read before the inverter is marked stale | |
| 233 | +| `STALE_INVERTER_SKIP_SECONDS` | Quiet period with no polling while stale, then an identity probe | |
| 234 | + |
| 235 | +`DRIVER` is obsolete and ignored (startup fails if it is still set). mbusd has its own `TIMEOUT` if |
| 236 | +you use that add-on as a serial-to-TCP gateway. The mbusd timeout should always be lower than the |
| 237 | +addon timeout. |
| 238 | + |
| 239 | +## Related changes (edge, post-1.0.0) |
| 240 | + |
| 241 | +- [modbus-connection#213](https://github.com/home-assistant-libs/modbus-connection/pull/213) — |
| 242 | + serial `timeout` passed through to tmodbus (fixes `TIMEOUT` being ignored on RTU; does not add |
| 243 | + spacing). |
| 244 | +- **50ms `message_spacing`** in `open_connection()` — addresses back-to-back polling on serial / |
| 245 | + RTU-backed links ([#672](https://github.com/kellerza/sunsynk/issues/672)). |
| 246 | +- Removal of nested **`asyncio.timeout`** wrappers — avoids double cancellation; see edge CHANGELOG. |
0 commit comments