Refactor minute_data to utils and fix some issues with energy dashboard by springfall2008 · Pull Request #2826 · springfall2008/batpred

springfall2008 · 2025-10-25T20:34:09Z

No description provided.

- Fix class info on GECloud/fox to allow energy flow in HA

Copilot

Pull Request Overview

This PR refactors the minute_data family of functions from the fetch.py module into a new utils.py module, making them reusable across the codebase. Additionally, it fixes issues with energy dashboard sensors by adding proper state_class attributes.

Moved minute_data, minute_data_state, history_attribute_to_minute_data, and clean_incrementing_reverse functions to utils.py
Updated all call sites across multiple files to import and use these functions from utils
Added state_class: "total_increasing" to energy sensors in gecloud.py and fox.py for proper Home Assistant energy dashboard integration

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
apps/predbat/utils.py	Added four utility functions (`minute_data`, `minute_data_state`, `history_attribute_to_minute_data`, `clean_incrementing_reverse`) moved from fetch.py, and imported `MAX_INCREMENT` from config
apps/predbat/fetch.py	Removed the four functions now in utils.py and updated imports to use them from utils
apps/predbat/unit_test.py	Updated test calls to use `minute_data` from utils instead of as a method on `my_predbat`
apps/predbat/solcast.py	Updated imports and function calls to use `minute_data` and `history_attribute_to_minute_data` from utils
apps/predbat/predheat.py	Updated imports, removed `self.minute_data` assignment, and updated function calls to use `minute_data` from utils
apps/predbat/output.py	Updated imports and function calls to use `minute_data` from utils
apps/predbat/octopus.py	Updated imports and function calls to use `minute_data` from utils
apps/predbat/inverter.py	Updated imports and function calls to use `minute_data` and `minute_data_state` from utils
apps/predbat/gecloud.py	Added `state_class: "total_increasing"` to energy sensor definitions
apps/predbat/futurerate.py	Updated imports, removed `self.minute_data` assignment, and updated function calls to use `minute_data` from utils
apps/predbat/fox.py	Added device_class and state_class attributes to energy-related sensors
apps/predbat/predbat.py	Updated version number to v8.27.0

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-10-25T20:35:17Z

+    Get historical data for state (e.g. predbat status)
+    """
+    mdata = {}
+    last_state = "unknown"


Variable last_state is initialized twice with different values (first as 'unknown' string on line 19, then as integer 0 on line 21). Remove the redundant initialization on line 19.

Suggested change

last_state = "unknown"

Copilot · 2025-10-25T20:35:17Z

+        rindex = 60 * 24 * days - minute - 1
+        state = mdata.get(rindex, state)
+        mdata[rindex] = state
+        minute += 1


The minute variable is incremented but never used. This line appears to be dead code since the loop already advances via the range iterator. Remove this line.

Suggested change

minute += 1

Copilot · 2025-10-25T20:35:17Z

+        try:
+            timestamp_key = str2time(key)
+            oldest_date = min(oldest_date, timestamp_key)
+        except (ValueError, TypeError) as e:


Exception variable e is captured but never used. Remove the exception variable: except (ValueError, TypeError):

Suggested change

except (ValueError, TypeError) as e:

except (ValueError, TypeError):

Copilot · 2025-10-25T20:35:17Z

+    adata = {}
+    io_adjusted = None
+    newest_state = 0
+    prev_state = 0


[nitpick] The variables adata and io_adjusted have unclear initialization logic. If adjust_key is not provided, io_adjusted remains None but adata is always created as an empty dict. Consider initializing both based on whether they'll be used, or document why adata is always created.

Suggested change

adata = {}

io_adjusted = None

newest_state = 0

prev_state = 0

# adata and io_adjusted are only initialized if adjust_key is provided

adata = None

io_adjusted = None

newest_state = 0

Copilot · 2025-10-25T20:35:18Z

                    "friendly_name": f"Fox {sn} {name}",
                }
+                # Set device and state class
+                if units in ['kWh', 'kW', 'W', 'W']:


The condition contains duplicate 'W' values in the list. This appears to be a typo. Based on the comment 'Set device and state class' and the attribute being set to 'energy', this should likely check for energy-related units only: if units in ['kWh', 'Wh']:

Suggested change

if units in ['kWh', 'kW', 'W', 'W']:

if units in ['kWh', 'Wh']:

…factor1

Copilot

Pull Request Overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-10-25T20:45:30Z

+    mdata = {}
+    last_state = "unknown"
+    newest_state = "unknown"
+    newest_age = 999999


The variable newest_state is initialized to "unknown" (string) on line 21, but later on line 64 it's assigned a state value which could be a string. However, on line 66 it's used to initialize the state variable for the forward fill loop. The initial value should remain consistent with the expected state type rather than switching between numeric initialization (as seen in the original code with newest_state = 0) and string initialization.

Copilot · 2025-10-25T20:45:30Z

+        history.append({"last_updated": key, "state": value})
+    max_age = now_utc - oldest_date
+    max_days = max(max_age.days, 1)
+    mdata, ignore_io_adjusted = minute_data(history, max_days, now_utc, "state", "last_updated", backwards=backwards, smoothing=False, scale=1.0, clean_increment=False, required_unit=None)


The variable ignore_io_adjusted suggests this return value is intentionally unused. Consider using _ as the variable name to make this intention explicit: mdata, _ = minute_data(...)

Suggested change

mdata, ignore_io_adjusted = minute_data(history, max_days, now_utc, "state", "last_updated", backwards=backwards, smoothing=False, scale=1.0, clean_increment=False, required_unit=None)

mdata, _ = minute_data(history, max_days, now_utc, "state", "last_updated", backwards=backwards, smoothing=False, scale=1.0, clean_increment=False, required_unit=None)

Copilot · 2025-10-25T20:45:30Z

        if data:
-            self.temperatures = self.minute_data(data, self.forecast_days, now_utc, "temperature", "datetime", backwards=False, smoothing=True, prev_last_updated_time=now_utc, last_state=self.external_temperature[0])
+            self.temperatures, ignore_io_adjusted = minute_data(
+                data, self.forecast_days, now_utc, "temperature", "datetime", backwards=False, smoothing=True, prev_last_updated_time=now_utc, last_state=self.external_temperature[0], max_increment=MAX_INCREMENT


Missing import for MAX_INCREMENT constant. The constant is used here but there's no corresponding import statement added to the file's imports section.

Copilot · 2025-10-25T20:45:30Z

            if history:
-                data_points = self.minute_data(history[0], self.max_days_previous, now_utc, "state", "last_updated", backwards=True, smoothing=smoothing, scale=scaling / total_count, clean_increment=incrementing, accumulate=data_points)
+                data_points, ignore_io_adjusted = minute_data(
+                    history[0], self.max_days_previous, now_utc, "state", "last_updated", backwards=True, smoothing=smoothing, scale=scaling / total_count, clean_increment=incrementing, accumulate=data_points, max_increment=MAX_INCREMENT


Missing import for MAX_INCREMENT constant. The constant is used here but there's no corresponding import statement added to the file's imports section.

It's defined at the top of the file

…factor1

… are fetched (springfall2008#5286) fetch_octopus_rates() replaced self.io_adjusted on every call, so the export and gas fetches that follow the import fetch reset it to {} - every BottlecapDave IOG user with metric_octopus_export or metric_octopus_gas set lost all is_intelligent_adjusted markers. Regression from springfall2008#2826, which dropped the old "only when adjust_key is set" guard. Among other things this left dynamic_load_car_strip_feed_rates() with nothing to strip, so a car cancelled by springfall2008#5229 still left its dispatch priced cheap and the house battery was planned into it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

springfall2008 added 2 commits October 25, 2025 21:30

- Move minute_data into a function

787ae13

- Fix class info on GECloud/fox to allow energy flow in HA

Missing library

cf632eb

springfall2008 requested a review from Copilot October 25, 2025 20:34

Copilot AI reviewed Oct 25, 2025

View reviewed changes

pre-commit-ci-lite Bot and others added 5 commits October 25, 2025 20:35

[pre-commit.ci lite] apply automatic fixes

ecdd175

Fixes

3bfc975

Merge branch 'refactor1' of github.com:springfall2008/batpred into re…

6a59569

…factor1

[pre-commit.ci lite] apply automatic fixes

631fb38

Update custom-dictionary-workspace.txt

73e7656

springfall2008 requested a review from Copilot October 25, 2025 20:44

Copilot AI reviewed Oct 25, 2025

View reviewed changes

pre-commit-ci-lite Bot and others added 7 commits October 25, 2025 20:45

[pre-commit.ci lite] apply automatic fixes

4be3966

Remove unused import

af0b263

Merge branch 'refactor1' of github.com:springfall2008/batpred into re…

72c75ae

…factor1

Tidy

a8afb02

[pre-commit.ci lite] apply automatic fixes

1d6ad34

State class total

876ad79

Merge branch 'refactor1' of github.com:springfall2008/batpred into re…

f1d737a

…factor1

springfall2008 merged commit da12ad4 into main Oct 25, 2025
1 check passed

springfall2008 deleted the refactor1 branch October 25, 2025 21:05

This was referenced Sep 28, 2026

ESS battery still being planned to charge and charging at peak rate after implementing #5229 #5286

Closed

fix(octopus): keep Intelligent dispatch markers when export/gas rates are fetched (#5286) #5290

Merged

Navigation Menu

Repository navigation

Refactor minute_data to utils and fix some issues with energy dashboard - #2826

Refactor minute_data to utils and fix some issues with energy dashboard#2826
springfall2008 merged 14 commits into
mainfrom
refactor1

springfall2008 commented Oct 25, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

Copilot AI Oct 25, 2025

Uh oh!

springfall2008 Oct 25, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	except (ValueError, TypeError) as e:
	except (ValueError, TypeError):

	if units in ['kWh', 'kW', 'W', 'W']:
	if units in ['kWh', 'Wh']:

	mdata, ignore_io_adjusted = minute_data(history, max_days, now_utc, "state", "last_updated", backwards=backwards, smoothing=False, scale=1.0, clean_increment=False, required_unit=None)
	mdata, _ = minute_data(history, max_days, now_utc, "state", "last_updated", backwards=backwards, smoothing=False, scale=1.0, clean_increment=False, required_unit=None)

Conversation

springfall2008 commented Oct 25, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

springfall2008 Oct 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants