Skip to content

fix(utils): keep the newest of same-minute changes in minute_data - #5473

Open
piomar123 wants to merge 1 commit into
springfall2008:mainfrom
piomar123:fix/minute-data-same-minute-newest
Open

piomar123 wants to merge 1 commit into
springfall2008:mainfrom
piomar123:fix/minute-data-same-minute-newest

Conversation

@piomar123

Copy link
Copy Markdown

Problem

minute_data carries the newest history value forward to "now" (the gap fill from the last sample) using newest_state, picked with if minutes < newest_age. Since timestamps are truncated to the minute, the last two changes of a sensor that fall in the same minute have the same age, and the strict < keeps the older one. It is then used for every minute up to now, until the sensor changes again.

Seen live: a heat pump room sensor (1C resolution) flickers 22/23 every few seconds overnight and settled on 23 (22 at 02:15:46, 23 at 02:15:48). Predheat's internal_temperature read 22 for the next 10 hours, so it planned heating all day. The transition minute itself held 23, so the series even disagreed with itself.

Fix

<=, so the last of the tied items wins. Every backwards=True source passes history oldest first (HA API client sorts it, the DB query is ORDER BY datetime, GE Cloud data is time ordered), and minute_data_state already got the same change in #2896.

Impact review

  • newest_state is only used by the "fill from last sample until now" step, i.e. calls without to_key. Rate, carbon and tariff calls (to_key) are untouched.
  • Backwards history (load, import/export, SoC, battery power, Predheat inputs, cost/carbon today, load_ml, yesterday adjustment): changes only when the two newest items share a minute, and then the value carried to now is the newest one, matching the transition minute.
  • Incrementing sensors: no change in output. clean_incrementing_reverse already ignored the small backwards step the stale value caused (checked, including a same-minute reset).
  • Forward data without to_key (load forecasts, Solcast, Predheat weather): the tie needs two items with the same timestamp, so in practice no change.

Tests

Tests 27-29 in test_minute_data: same-minute changes with and without smoothing (fail before, pass after), and a same-minute counter update that must stay unchanged. ./run_all --quick passes, and pre-commit is clean.

🤖 Generated with Claude Code

minute_data carries the newest history value forward to "now" with
`minutes < newest_age`. Timestamps are truncated to the minute, so when the
last two changes of a sensor fall in the same minute both have the same age
and the strict comparison keeps the older one. It is then held until the
next change: a room sensor flickering 22/23 that settled on 23 was read as
22 for 10 hours (Predheat internal_temperature), while the transition
minute itself already held 23.

History is oldest first (HA API, Predbat's DB, GE Cloud), so `<=` makes the
last of the tied items win, matching minute_data_state, which got the same
fix in springfall2008#2896. Incrementing sensors are unaffected because
clean_incrementing_reverse already ignores the small backwards step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@piomar123
piomar123 marked this pull request as ready for review October 10, 2026 17:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant