Repository navigation
fix(gateway): send each hub command once and wait for its ack - #5231
Merged
Merged
Conversation
The generic write_and_poll loop re-calls the gateway's select/number/switch handlers every 2 s until the read-back matches, and each call published the command again. A slow dongle write therefore queued up to ten Modbus writes on the hub, and PredBat reported writes as failed that the hub was still working through. The hub already publishes an ack per command on predbat/devices/<id>/ack/<command_id>. Subscribe to it and track control writes per entity: once an ack has been matched on this connection, an identical command is published once per 30 s window (while waiting for the ack, after an ok ack, and after a refusal), a different value goes out immediately, and an unanswered command is re-sent once before acks stop being relied on. Ok acks update cached slot times and enable switches from the applied values; refusals are logged with the hub's reason; a "replay" rejection re-sends under a new id. Without the ack subscription, or before any ack is seen, behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
… command ids Review follow-ups: - keep the recent ids of a tracked value, so an ack for an earlier send (before suppression is on) confirms it instead of being ignored; - a refusal after an ok from another unit is still reported and kept, and only the first ok ack updates the cached slot times; - seed the command id counter from the clock so ids do not restart at PBAT1 after a restart (the hub's replay check, and a late ack for the previous process's id); - time the ack window with a monotonic clock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
Review follow-ups: - record ok/refused per id: an ok for any send of a value means it was applied, while within one id (several units) a refusal outranks an ok; - a failed publish drops only its own id, so earlier sends stay matchable; - derive the slot entities from the entity's own suffix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
Review follow-ups: - only mark the cache as updated when an ack actually carried usable values, so a staged/recorded first ok does not block a later plain one; - a refusal for the id whose ok updated the cache restores the last telemetry instead of leaving the optimistic value in place; - every refusal restarts the cooldown window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
…refusal Review follow-ups: - a fresh attempt for an already-answered value keeps its earlier ids, so a late ack from another unit is still logged and acted on; a failed publish only removes its own id; - every refusal for an id is logged and restarts the window, not just the first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
… on publish errors Review follow-ups: - remember every id whose ok updated the cache, across fresh attempts, so a late refusal from another unit still restores telemetry; - a publish error no longer removes an id whose ack already arrived while the publish awaited the broker's confirmation; - a replay rejection is logged as "refused by hub: replay" (and still re-sent under a new id). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
Review follow-ups: - a fresh attempt keeps earlier ids' outcomes (pruned with the ids), so a refusal from an earlier attempt still outranks that id's late ok; - a replay arriving after the command was answered is treated as another unit's refusal: logged and restarts the window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
…s the entry A late replay for an earlier id (e.g. from another unit after a fresh attempt) is handled as a refusal instead of discarding the current attempt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
…back on publish errors Review follow-ups: - a replay for the newest unanswered id drops the entry unless another send in the same attempt already applied the value, in which case only that id is forgotten (acks from earlier attempts do not count); - a failed publish restores the id history as it was, including an id pruned to make room. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
…, time replays Review follow-ups: - prune the per-attempt id list with the kept ids; - roll back a failed publish only if no ack of any kind arrived meanwhile (an ack counter, not an outcome comparison); - a replay of the newest id after another send applied the value still restarts the refusal window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
A failing publish then never loses an earlier id, even when the history is full and acks for other ids arrive while it awaits the broker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BaJjLpuMEtQYfvmTRcFJf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PredBat's generic
write_and_poll_*loop checks the read-back every 2 s and, until it matches, calls the gateway'sselect_event/number_event/switch_eventagain, up to 10 times.GatewayMQTTpublished the command on every call. When the inverter's dongle takes several seconds to accept a Modbus write, each re-send queues another full write on the hub. The first write usually lands, but only after PredBat's ~20 s window. By then PredBat has already loggedInverter 0 write to <x> failed, and the hub is still working through the duplicates.The hub already publishes an ack for every command on
predbat/devices/<id>/ack/<command_id>, withok, the refusal reason, and the values it applied. PredBat never subscribed to it, so it could not tell "slow but succeeding" from "refused".Solution (gateway component only; the shared
inverter.pyloop is unchanged)predbat/devices/<id>/ack/+alongsidestatusandonline._COMMAND_ACK_WINDOW). This covers waiting for the ack, after an ok ack while telemetry catches up, and after a refusal. The generic loop keeps polling in the meantime, so each write normally sends the command once.applied, so the read-back matches straight away. Staged or recorded EMS slot endpoints and number entities still wait for telemetry.Warn: GatewayMQTT: <command> for <serial> refused by hub: <error>and does not re-send that value within the window. The write is still reported as failed, but the hub is no longer hammered.replayrejection: the next attempt goes out immediately under a new id. The command id counter is now seeded from the clock, so ids no longer restart atPBAT1after a restart. A restart was the main way to hit the hub's replay check, or to match a late ack meant for the previous process.Compatibility
ack/+.Testing
TestCommandAckcases inapps/predbat/tests/test_gateway.py:python3 ../apps/predbat/unit_test.py -t gatewayand--quickpass. pre-commit (ruff, black, cspell) passes.🤖 Generated with Claude Code