Merge pull request #24 from azebro/dev

Add configurable reporting interval and live reporting features

Author: azebro

.devcontainer/devcontainer-lock.json

@@ -0,0 +1,9 @@
+{
+  "features": {
+    "ghcr.io/devcontainers/features/github-cli:1": {
+      "version": "1.1.0",
+      "resolved": "ghcr.io/devcontainers/features/github-cli@sha256:d22f50b70ed75339b4eed1ba9ecde3a1791f90e88d37936517e3bace0bbad671",
+      "integrity": "sha256:d22f50b70ed75339b4eed1ba9ecde3a1791f90e88d37936517e3bace0bbad671"
+    }
+  }
+}

CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## [Unreleased]
+
+### Added
+
+- **Configurable reporting mode** — New "Live reporting" toggle (default on) under **Configure → Change reporting settings**. When disabled, readings are cached and delivered as a time-weighted average every *reporting interval* (5–300 s, default 30), reducing CPU and recorder load on installations with many panels. Energy totals are always integrated from raw readings, so only the instantaneous values are averaged.
+
+### Changed
+
+- Reporting interval default changed from 5 to 30 seconds; allowed range tightened from 1–300 to 5–300 seconds.
+- Config entry schema bumped to version 5 (adds `live_reporting`, default `True`, via automatic migration).
+- The reporting interval is now collected on its own follow-up screen that appears only when live reporting is turned off, so the interval is never shown (effectively greyed out) while live reporting is enabled.
+
+### Fixed
+
+- Live reporting no longer buffers readings internally, preventing unbounded memory growth over time in the default mode.
+- Live reporting now publishes an immutable copy of node data to Home Assistant, avoiding rare races where consumers (sensors, diagnostics) could observe the data changing mid-read.
+- Averaged interval mode now flushes buffered readings on schedule during quiet periods (e.g. at dusk), instead of waiting for the next reading or a reconnect.
+- The version 5 migration clamps any pre-existing reporting interval below 5 seconds up to the new minimum, so the reporting settings form stays submittable after upgrade.
+
 ## [1.2.0] - 2026-03-07
 
 ### Changed

README.md

@@ -15,6 +15,7 @@ Please note that this integration requires RS485 to TCP converter that needs to
 ## Features
 
 - **Real-time streaming** — Push-based data delivery with sub-second latency (no polling).
+- **Configurable reporting** — Choose **live reporting** (default; every reading delivered as it arrives) or a **time-weighted averaged interval** that caches readings and pushes a single representative value every few seconds to reduce CPU and recorder load on large installations.
 - **Per-optimizer sensors** — 12 sensor entities per Tigo TS4 module: performance,
   power, voltage in/out, current in/out, temperature, DC-DC duty cycle, RSSI,
   daily energy, total energy, and readings today.
@@ -120,10 +121,22 @@ Check your Home Assistant logs after the gateway has been running for a while (u
 Go to **Settings → Devices & Services → PyTap → Configure** to:
 
 - **Change connection** — Update the gateway IP address and port.
+- **Change reporting settings** — Toggle live reporting or set the averaged reporting interval (see [Reporting modes](#reporting-modes)).
 - **Add** new optimizer modules (if the barcode was previously seen on the bus, it resolves instantly).
 - **Remove** modules you no longer want to track.
 - **Save and close** to apply changes.
 
+### Reporting modes
+
+PyTap can deliver readings to Home Assistant in two ways, selectable under **Configure → Change reporting settings**:
+
+- **Live reporting (default)** — Each reading is pushed to Home Assistant as soon as it arrives from the gateway, giving sub-second latency. Best for most installations.
+- **Averaged interval** — When live reporting is turned off, readings are cached and a **time-weighted average** is pushed once per *reporting interval* (5–300 seconds, default 30). Weighting each reading by how long it remained the latest value yields a representative average even when readings arrive at uneven intervals. This reduces event-loop, CPU, and recorder/database load on installations with many panels.
+
+> **Setting the interval:** The reporting interval only applies in averaged mode, so it is hidden while **Live reporting** is enabled. To change it, open **Change reporting settings**, untick **Live reporting**, and click **Submit** — a follow-up **Reporting Interval** screen then lets you enter the value. Leaving **Live reporting** ticked skips that screen entirely.
+
+Energy totals (daily and lifetime) are always integrated from the raw readings, so the reporting mode only affects the instantaneous values (power, voltage, current, temperature) shown on dashboards — never the accumulated energy figures.
+
 ### Diagnostics Download
 
 PyTap diagnostics are available from the integration page:
@@ -157,7 +170,7 @@ PyTap Coordinator
         └── Sensor entities update in real time
 ```
 
-PyTap uses a background listener thread that streams data from the gateway, parses protocol frames, and dispatches events to the Home Assistant event loop. Only events matching your configured barcodes create or update sensor entities.
+PyTap uses a background listener thread that streams data from the gateway, parses protocol frames, and dispatches events to the Home Assistant event loop. Only events matching your configured barcodes create or update sensor entities. In the default **live reporting** mode each update is forwarded immediately; in **averaged interval** mode updates are buffered and flushed as a time-weighted average every few seconds (see [Reporting modes](#reporting-modes)).
 
 On startup, PyTap restores persisted coordinator state (barcode mappings, energy accumulators, and last node snapshots). Sensor entities also use Home Assistant restore fallback if needed, so a restart during low/no production does not force entities into unavailable when prior data exists.
 

custom_components/pytap/__init__.py

@@ -20,10 +20,14 @@
     CONF_MODULE_PEAK_POWER,
     CONF_MODULE_STRING,
     CONF_MODULES,
+    CONF_WRITE_INTERVAL,
     DEFAULT_LIVE_REPORTING,
     DEFAULT_PEAK_POWER,
     DEFAULT_STRING_NAME,
+    DEFAULT_WRITE_INTERVAL,
     DOMAIN,
+    MAX_WRITE_INTERVAL,
+    MIN_WRITE_INTERVAL,
 )
 from .coordinator import PyTapDataUpdateCoordinator
 
@@ -162,6 +166,18 @@ async def async_migrate_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
         new_data = {**entry.data}
         if CONF_LIVE_REPORTING not in new_data:
             new_data[CONF_LIVE_REPORTING] = DEFAULT_LIVE_REPORTING
+        # The v5 reporting form constrains write_interval to
+        # [MIN_WRITE_INTERVAL, MAX_WRITE_INTERVAL].  Earlier versions allowed
+        # smaller values, so clamp any stale value to keep the options form
+        # submittable after upgrade.
+        if CONF_WRITE_INTERVAL in new_data:
+            try:
+                interval = int(new_data[CONF_WRITE_INTERVAL])
+            except (TypeError, ValueError):
+                interval = DEFAULT_WRITE_INTERVAL
+            new_data[CONF_WRITE_INTERVAL] = max(
+                MIN_WRITE_INTERVAL, min(MAX_WRITE_INTERVAL, interval)
+            )
         hass.config_entries.async_update_entry(entry, data=new_data, version=5)
 
     return True

custom_components/pytap/config_flow.py

@@ -40,6 +40,8 @@
     DEFAULT_PORT,
     DEFAULT_WRITE_INTERVAL,
     DOMAIN,
+    MAX_WRITE_INTERVAL,
+    MIN_WRITE_INTERVAL,
 )
 
 _LOGGER = logging.getLogger(__name__)
@@ -322,48 +324,73 @@ async def async_step_change_connection(
     async def async_step_change_reporting(
         self, user_input: dict[str, Any] | None = None
     ) -> ConfigFlowResult:
-        """Allow the user to change reporting settings.
+        """Toggle live reporting.
 
-        Presents a 'Live reporting' checkbox (default ON) that delivers
-        readings as they arrive.  When unchecked, a write_interval field
-        appears and readings are cached, then pushed as a time-weighted
-        average every ``write_interval`` seconds.
+        Live reporting is presented on its own step with just the checkbox.
+        Home Assistant's static forms cannot grey out a field reactively, so
+        the reporting interval is collected on a separate ``set_interval``
+        step that is only reached when live reporting is turned off. When it
+        is left on, the interval is irrelevant and never shown.
         """
         if user_input is not None:
-            new_data = {**self._config_entry.data}
             live = user_input.get(CONF_LIVE_REPORTING, DEFAULT_LIVE_REPORTING)
-            new_data[CONF_LIVE_REPORTING] = live
             if live:
-                # Live mode — clear any stored interval
+                # Live mode — clear any stored interval and finish.
+                new_data = {**self._config_entry.data}
+                new_data[CONF_LIVE_REPORTING] = True
                 new_data.pop(CONF_WRITE_INTERVAL, None)
-            else:
-                new_data[CONF_WRITE_INTERVAL] = user_input.get(
-                    CONF_WRITE_INTERVAL, DEFAULT_WRITE_INTERVAL
+                self.hass.config_entries.async_update_entry(
+                    self._config_entry, data=new_data
                 )
+                return await self.async_step_init()
+            # Interval mode — collect the interval on the next step.
+            return await self.async_step_set_interval()
+
+        current_live = self._config_entry.data.get(
+            CONF_LIVE_REPORTING, DEFAULT_LIVE_REPORTING
+        )
+        schema = vol.Schema(
+            {
+                vol.Optional(CONF_LIVE_REPORTING, default=current_live): bool,
+            }
+        )
+
+        return self.async_show_form(
+            step_id="change_reporting",
+            data_schema=schema,
+        )
+
+    async def async_step_set_interval(
+        self, user_input: dict[str, Any] | None = None
+    ) -> ConfigFlowResult:
+        """Collect the reporting interval (only reached when live reporting is off)."""
+        if user_input is not None:
+            new_data = {**self._config_entry.data}
+            new_data[CONF_LIVE_REPORTING] = False
+            new_data[CONF_WRITE_INTERVAL] = user_input.get(
+                CONF_WRITE_INTERVAL, DEFAULT_WRITE_INTERVAL
+            )
             self.hass.config_entries.async_update_entry(
                 self._config_entry, data=new_data
             )
             return await self.async_step_init()
 
-        current_live = self._config_entry.data.get(
-            CONF_LIVE_REPORTING, DEFAULT_LIVE_REPORTING
-        )
         current_write_interval = self._config_entry.data.get(
             CONF_WRITE_INTERVAL, DEFAULT_WRITE_INTERVAL
         )
         schema = vol.Schema(
             {
-                vol.Optional(
-                    CONF_LIVE_REPORTING, default=current_live
-                ): bool,
                 vol.Optional(
                     CONF_WRITE_INTERVAL, default=current_write_interval
-                ): vol.All(vol.Coerce(int), vol.Range(min=5, max=300)),
+                ): vol.All(
+                    vol.Coerce(int),
+                    vol.Range(min=MIN_WRITE_INTERVAL, max=MAX_WRITE_INTERVAL),
+                ),
             }
         )
 
         return self.async_show_form(
-            step_id="change_reporting",
+            step_id="set_interval",
             data_schema=schema,
         )
 

custom_components/pytap/const.py

@@ -23,6 +23,8 @@
 DEFAULT_SCAN_INTERVAL = 30
 DEFAULT_LIVE_REPORTING = True
 DEFAULT_WRITE_INTERVAL = 30
+MIN_WRITE_INTERVAL = 5
+MAX_WRITE_INTERVAL = 300
 RECONNECT_TIMEOUT = 60
 RECONNECT_DELAY = 5
 RECONNECT_RETRIES = 0

custom_components/pytap/coordinator.py

@@ -293,7 +293,9 @@ def _perform_midnight_reset(self) -> None:
         """Reset all daily accumulators and push updated data to sensors."""
         self._midnight_reset_unsub = None
         today = dt_util.now().date().isoformat()
-        _LOGGER.info("Midnight daily reset — resetting daily accumulators for %s", today)
+        _LOGGER.info(
+            "Midnight daily reset — resetting daily accumulators for %s", today
+        )
 
         data_changed = False
         for barcode, acc in self._energy_state.items():
@@ -359,57 +361,27 @@ def _listen(self) -> None:
 
                         if self._live_reporting:
                             # Live mode — push every change immediately
-                            if self._ha_update_pending:
-                                self.data["counters"] = parser.counters
-                                self.hass.loop.call_soon_threadsafe(
-                                    self.async_set_updated_data,
-                                    dict(self.data),
-                                )
-                                self._ha_update_pending = False
+                            self._push_live_update(parser)
                         else:
                             # Interval mode — buffer readings and push
                             # time-weighted averages once per write_interval.
-                            now = time.monotonic()
-                            if self._ha_update_pending and (
-                                now - self._last_ha_update
-                                >= self._write_interval
-                            ):
-                                self.data["counters"] = parser.counters
-                                snapshot = self._build_averaged_snapshot(
-                                    now
-                                )
-                                per_node_counts = {
-                                    barcode: len(readings)
-                                    for barcode, readings
-                                    in self._reading_buffers.items()
-                                    if readings
-                                    and barcode in snapshot["nodes"]
-                                }
-                                _LOGGER.debug(
-                                    "HA update: %d node(s) — %s",
-                                    len(per_node_counts),
-                                    ", ".join(
-                                        f"{snapshot['nodes'][b].get('name', b)}: "
-                                        f"{n} reading(s)"
-                                        for b, n in per_node_counts.items()
-                                    ),
-                                )
-                                self._reading_buffers.clear()
-                                self.hass.loop.call_soon_threadsafe(
-                                    self.async_set_updated_data,
-                                    snapshot,
-                                )
-                                self._last_ha_update = now
-                                self._ha_update_pending = False
-                    elif (
-                        RECONNECT_TIMEOUT > 0
-                        and (time.monotonic() - last_data_time) > RECONNECT_TIMEOUT
-                    ):
-                        _LOGGER.warning(
-                            "No data from gateway for %ds, reconnecting",
-                            RECONNECT_TIMEOUT,
-                        )
-                        break
+                            self._maybe_flush_interval(parser, time.monotonic())
+                    else:
+                        # No data this read cycle (socket read timeout).
+                        now = time.monotonic()
+                        if not self._live_reporting:
+                            # Flush buffered readings on schedule so sensors
+                            # keep updating during production lulls.
+                            self._maybe_flush_interval(parser, now)
+                        if (
+                            RECONNECT_TIMEOUT > 0
+                            and (now - last_data_time) > RECONNECT_TIMEOUT
+                        ):
+                            _LOGGER.warning(
+                                "No data from gateway for %ds, reconnecting",
+                                RECONNECT_TIMEOUT,
+                            )
+                            break
 
             except Exception as err:
                 if self._stop_event.is_set():
@@ -429,7 +401,7 @@ def _listen(self) -> None:
                     else:
                         self.hass.loop.call_soon_threadsafe(
                             self.async_set_updated_data,
-                            dict(self.data),
+                            {**self.data, "nodes": dict(self.data["nodes"])},
                         )
                     self._ha_update_pending = False
                 with self._source_lock:
@@ -460,6 +432,58 @@ def _listen(self) -> None:
                     return
                 time.sleep(0.1)
 
+    def _push_live_update(self, parser: Any) -> None:
+        """Push an immediate snapshot to HA (live reporting mode).
+
+        Publishes an *immutable* copy of the ``nodes`` mapping so the listener
+        thread can keep mutating ``self.data["nodes"]`` without HA-side
+        consumers (sensors, diagnostics) ever iterating a dict that changes
+        size mid-read.  Runs in the executor thread.
+        """
+        if not self._ha_update_pending:
+            return
+        self.data["counters"] = parser.counters
+        self.hass.loop.call_soon_threadsafe(
+            self.async_set_updated_data,
+            {**self.data, "nodes": dict(self.data["nodes"])},
+        )
+        self._ha_update_pending = False
+
+    def _maybe_flush_interval(self, parser: Any, now: float) -> None:
+        """Flush buffered readings as a time-weighted average (interval mode).
+
+        Pushes a snapshot once the write interval has elapsed since the last
+        update.  Called both when data arrives and during idle read cycles so
+        buffered readings are not stranded when the stream goes quiet.  Runs
+        in the executor thread.
+        """
+        if not self._ha_update_pending or (
+            now - self._last_ha_update < self._write_interval
+        ):
+            return
+        self.data["counters"] = parser.counters
+        snapshot = self._build_averaged_snapshot(now)
+        per_node_counts = {
+            barcode: len(readings)
+            for barcode, readings in self._reading_buffers.items()
+            if readings and barcode in snapshot["nodes"]
+        }
+        _LOGGER.debug(
+            "HA update: %d node(s) — %s",
+            len(per_node_counts),
+            ", ".join(
+                f"{snapshot['nodes'][b].get('name', b)}: {n} reading(s)"
+                for b, n in per_node_counts.items()
+            ),
+        )
+        self._reading_buffers.clear()
+        self.hass.loop.call_soon_threadsafe(
+            self.async_set_updated_data,
+            snapshot,
+        )
+        self._last_ha_update = now
+        self._ha_update_pending = False
+
     def _init_mappings_from_parser(self, parser: Any) -> None:
         """Pre-populate barcode/node mappings from the parser's persistent state.
 
@@ -643,12 +667,19 @@ def _handle_power_report(self, event: PowerReportEvent) -> bool:
             "daily_reset_date": acc.daily_reset_date,
             "last_update": now.isoformat(),
         }
-        self._reading_buffers.setdefault(barcode, []).append(
-            (
-                time.monotonic(),
-                {field: self.data["nodes"][barcode][field] for field in _AVERAGED_FIELDS},
+        # Only buffer readings in interval/averaged mode.  In live mode the
+        # raw values are pushed immediately, so buffering would grow without
+        # bound and never be consumed.
+        if not self._live_reporting:
+            self._reading_buffers.setdefault(barcode, []).append(
+                (
+                    time.monotonic(),
+                    {
+                        field: self.data["nodes"][barcode][field]
+                        for field in _AVERAGED_FIELDS
+                    },
+                )
             )
-        )
         self._schedule_save()
         return True