Don't trust Intelligent slots the car won't need (consider_full) (#5403)

* feat(octopus): don't trust Intelligent slots the car won't need (#4482)

With octopus_intelligent_consider_full on, load_octopus_slots() already ends
each car's slots where it reaches its limit, so the car is modelled not to
draw in the rest - but every dispatch minute still kept its cheap rate, and
the house battery planned around slots Octopus only bills cheap when the car
charges in them.

octopus_surplus_minutes() works out the future dispatch minutes no car still
needs (rounded out to whole half hours, shared across cars, outside the fixed
23:30-05:30 window), and both rate paths now withhold them the way they
already do for a cancelled car: rate_add_io_slots() doesn't stamp them, and
dynamic_load_car_strip_feed_rates() removes a discount the rate feed already
delivered. Elapsed minutes keep their rate so today's cost stays true.
Behaviour is unchanged with consider_full off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(octopus): a cancelled car's slots still count as needed by the car-need gate

dynamic_load_car_check() cancels a car by zeroing its future slots and keeping
the kWh in kwh_cancelled. The car-need gate read that zero as "the car won't
need it", so with octopus_intelligent_consider_full on it stripped the cheap
rate from the rest of the half hour #5229 deliberately keeps for a car seen
charging in it. Cancellation decides a cancelled car's rate itself; the gate
now only answers whether a slot is beyond the car's need.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(octopus): keep the current half hour out of the car-need gate

The surplus started at minutes_now, which usually falls mid-period, so the rest
of the current half hour lost its low rate once the car was full - even when
the car charged earlier in it, which makes Octopus bill the whole period cheap.
Surplus now starts at the next half-hour boundary. Clamping back to the start
of the current period instead would have marked elapsed minutes as surplus too,
and rate_add_io_slots() applies the skip to past minutes, misreporting today's
cost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(octopus): decide the car-need gate per car before sharing (#5403 review)

Copilot's review of the gate found four ways it could misprice a dispatch:

- The surplus started at the next half-hour boundary computed as a ceiling,
  which is the current boundary when the floored clock sits on one (a fetch at
  14:02 reads 14:00), so the current half hour could still lose its rate. It now
  starts at the strictly next boundary.
- A cancelled car's kwh_cancelled counted as needed for the whole slot, so it
  kept a full car's surplus minutes cheap though neither car would charge then.
  A car's need now counts only before its own cheap rate ends
  (dynamic_load_car_strip_from()), which still keeps the half hour GH#5316
  keeps for a car seen charging in it.
- A car not modelled from its Octopus slots was simply skipped, so a full car's
  overlapping surplus stripped its discount too. Its dispatches are now kept,
  up to its own strip_from.
- An untrimmed dispatch in progress (octopus_intelligent_slot isn't trimmed to
  now, unlike the Octopus API component's) is capped from its original start,
  so the capped slot can end before the car finishes. The running dispatch is
  now left alone; dynamic load cancels it in real time once the car stops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(octopus): judge a running dispatch from now, and replan when the surplus changes (#5403 review)

Trefor's review of #5403:

1. A dispatch in progress was exempt from the gate for its whole length, so
   one long dispatch kept all its surplus cheap from the moment it started,
   while the same charge chunked into half-hour dispatches didn't. With
   consider_full on, load_octopus_slots() now caps a running dispatch's need
   from its remaining span and energy (as trim_started_dispatch() already does
   for the Octopus API component, but octopus_intelligent_slot doesn't), after
   the slot-cap chunking, which still counts it from its real start. The
   exemption is gone.

2. A change in the surplus didn't force a replan - the rates can move without
   the slots moving, as the car's SoC changes how many half hours it needs.
   fetch_sensor_data() now replans when it changes, guarded by save like
   dynamic_load_car_check(). Only the part of the previous surplus still ahead
   is compared, so time passing alone isn't a change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Author: chalfontchubby

apps/predbat/fetch.py

@@ -1188,6 +1188,7 @@ def fetch_sensor_data(self, save=True):
         # Dynamic load: cancel the Octopus Intelligent slots of a car that is in one but not charging -
         # before the rates are built, so a cancelled dispatch never gets its cheap rate
         dynamic_load_car_changed = self.dynamic_load_car_check(save=save)
+        octopus_surplus_changed = False
 
         if "rates_export_octopus_url" in self.args:
             # Fixed URL for rate export
@@ -1241,6 +1242,12 @@ def fetch_sensor_data(self, save=True):
         if import_rates:
             self.rate_scan(import_rates, print=False)
             self.rate_import_base, self.rate_min_base, self.rate_max_base = self.rate_base_min_max(import_rates)
+            previous_surplus = self.octopus_surplus
+            self.octopus_surplus = self.octopus_surplus_minutes()
+            # The rates can move without the slots moving (the car's SoC changes how many half hours it needs)
+            if save and self.octopus_surplus_changed(previous_surplus):
+                self.log("Octopus Intelligent: dispatch minutes no car needs changed to {}".format(len(self.octopus_surplus)))
+                octopus_surplus_changed = True
             import_rates = self.dynamic_load_car_strip_feed_rates(import_rates)
             import_rates, self.rate_import_replicated = self.rate_replicate(import_rates, self.io_adjusted, is_import=True)
             self.rate_import_no_io = import_rates.copy()
@@ -1374,7 +1381,7 @@ def fetch_sensor_data(self, save=True):
         else:
             self.load_inday_adjustment = 1.0
 
-        force_replan = dynamic_load_car_changed
+        force_replan = dynamic_load_car_changed or octopus_surplus_changed
         # Compare on the change-detection signature, not the raw slots, so the per-cycle re-clocking
         # of an in-progress dispatch (start advanced to now, energy scaled to remaining time) does not
         # force a replan every cycle while a slot is active - only genuine slot changes do

apps/predbat/octopus.py

@@ -3742,6 +3742,14 @@ def load_octopus_slots(self, car_n, octopus_slots, octopus_intelligent_consider_
                 # Emission is future-only, so dispatch_billed_off_peak()'s completed-dispatch exemption
                 # (#4946) can never apply here - a plain location test is equivalent and is kept as-is.
                 if (end_minutes > start_minutes) and (end_minutes > self.minutes_now) and (not location or location == "AT_HOME"):
+                    if octopus_intelligent_consider_full and start_minutes < self.minutes_now:
+                        # A dispatch in progress: only its remaining span and energy are still to come, so cap the
+                        # car's need from now, not from the dispatch's original start. The Octopus API component
+                        # already trims a running dispatch (trim_started_dispatch()); octopus_intelligent_slot doesn't.
+                        # Done after the slot-cap chunking above, which counts the dispatch from its real start.
+                        kwh = kwh * (end_minutes - self.minutes_now) / (end_minutes - start_minutes)
+                        kwh_original = kwh
+                        start_minutes = self.minutes_now
                     kwh_expected = kwh * self.car_charging_loss
                     if octopus_intelligent_consider_full:
                         kwh_expected = max(min(kwh_expected, limit - car_soc), 0)
@@ -3854,6 +3862,9 @@ def rate_add_io_slots(self, car_n, rates, octopus_slots):
                         if strip_from is not None and minute >= strip_from:
                             continue  # Car is not charging, its dispatch is not trusted as cheap
 
+                        if minute in self.octopus_surplus:
+                            continue  # No car needs this dispatch minute, see octopus_surplus_minutes()
+
                         if minute in saved_slots:
                             continue  # Already applied a low rate slot to this minute, skip
                         else:
@@ -3915,6 +3926,74 @@ def rate_add_io_slots(self, car_n, rates, octopus_slots):
 
         return rates
 
+    def in_iog_fixed_window(self, minute):
+        """
+        Whether minute falls in the fixed 23:30-05:30 Intelligent Go window, cheap by tariff rather than by
+        dispatch, so no dispatch decision ever changes its rate.
+        """
+        window = OCTOPUS_NIGHT_RATE_WINDOWS["iog"]
+        window_start = window["start"][0] * 60 + window["start"][1]
+        window_end = window["end"][0] * 60 + window["end"][1]
+        minute_of_day = minute % (24 * 60)
+        return minute_of_day >= window_start or minute_of_day < window_end
+
+    def octopus_surplus_from(self):
+        """
+        The first minute the car-need gate can withhold: the half hour after the current one, as the car may
+        already have charged in the current one, making the whole period cheap.
+        """
+        return (self.minutes_now // 30 + 1) * 30
+
+    def octopus_surplus_changed(self, previous):
+        """
+        Whether this cycle's surplus differs from the previous cycle's, so the plan must be recomputed. Only the
+        part of the previous surplus still ahead is compared, so time passing alone isn't a change.
+        """
+        start = self.octopus_surplus_from()
+        return {minute for minute in previous if minute >= start} != self.octopus_surplus
+
+    def octopus_surplus_minutes(self):
+        """
+        Future Octopus Intelligent dispatch minutes no car can use, when octopus_intelligent_consider_full is on.
+
+        load_octopus_slots() already ends each car's slots where the car reaches its limit, so a dispatch
+        minute no car still needs is one no car will draw in, and Octopus only bills a dispatch cheap when
+        the car charges in it (#4482). Its cheap rate is withheld from the house battery's plan the same way
+        a cancelled car's is (see dynamic_load_car_strip_feed_rates()).
+
+        Decided per car, then shared, as the rates are shared: a minute stays cheap if any car can still use
+        it before the point that car's own cheap rate ends (dynamic_load_car_strip_from()). A car can use
+        - its needed slots - with kWh left, or kWh kept in kwh_cancelled for a cancelled car. A dispatch in
+          progress is capped from now (see load_octopus_slots()), so this holds mid-dispatch too;
+        - every dispatch, for a car not modelled from its Octopus slots this cycle (e.g. ignored while unplugged).
+
+        Rounded out to whole 30 minute rate periods, from the half hour after the current one (the car may have
+        charged earlier in it, making the whole period cheap) and outside the fixed window.
+        """
+        if not self.octopus_intelligent_consider_full:
+            return set()
+        dispatch_minutes = set()
+        usable_minutes = set()
+        next_period = self.octopus_surplus_from()
+        for car_n in range(min(self.num_cars, len(self.octopus_slots))):
+            car_slots = self.car_charging_slots[car_n] if car_n < len(self.car_charging_slots) else []
+            modelled = any(slot.get("octopus", False) for slot in car_slots)
+            strip_from = self.dynamic_load_car_strip_from(car_n)
+            usable_until = strip_from if strip_from is not None else float("inf")
+            for slot in self.octopus_slots[car_n]:
+                start_minutes, end_minutes, _, _, _ = self.decode_octopus_slot(car_n, slot, raw=True, boundaries_only=True)
+                if start_minutes == end_minutes:
+                    continue
+                start_minutes, end_minutes = round_out_to_period(start_minutes, end_minutes)
+                dispatch_minutes.update(range(max(start_minutes, next_period), end_minutes))
+                if not modelled:
+                    usable_minutes.update(range(start_minutes, min(end_minutes, usable_until)))
+            for slot in car_slots:
+                if slot.get("kwh", 0) > 0 or slot.get("kwh_cancelled", 0) > 0:
+                    start_minutes, end_minutes = round_out_to_period(slot["start"], slot["end"])
+                    usable_minutes.update(range(start_minutes, min(end_minutes, usable_until)))
+        return {minute for minute in dispatch_minutes - usable_minutes if not self.in_iog_fixed_window(minute)}
+
     def dynamic_load_car_strip_feed_rates(self, rates):
         """
         Remove an Octopus Intelligent discount the rate feed itself delivered for a dispatch of a car that
@@ -3932,7 +4011,7 @@ def dynamic_load_car_strip_feed_rates(self, rates):
         (see dynamic_load_car_strip_from()), whatever another cancelled car's dispatch covers.
         """
         cancelled_cars = [car_n for car_n in range(self.num_cars) if self.dynamic_load_car_effective.get(car_n, False)]
-        if not cancelled_cars or not self.io_adjusted:
+        if (not cancelled_cars and not self.octopus_surplus) or not self.io_adjusted:
             self.dynamic_load_car_stripped = 0
             return rates
 
@@ -3954,23 +4033,22 @@ def dynamic_load_car_strip_feed_rates(self, rates):
                     trusted_minutes.update(range(start_minutes, min(end_minutes, strip_from)))
                     cancelled_minutes.update(range(max(start_minutes, strip_from), end_minutes))
 
-        window = OCTOPUS_NIGHT_RATE_WINDOWS["iog"]
-        window_start = window["start"][0] * 60 + window["start"][1]
-        window_end = window["end"][0] * 60 + window["end"][1]
         stripped = 0
-        for minute in sorted(cancelled_minutes - trusted_minutes):
+        stripped_surplus = 0
+        for minute in sorted((cancelled_minutes - trusted_minutes) | self.octopus_surplus):
             if not self.io_adjusted.get(minute, False):
                 continue
-            minute_of_day = minute % (24 * 60)
-            if minute_of_day >= window_start or minute_of_day < window_end:
+            if self.in_iog_fixed_window(minute):
                 continue
             rates[minute] = self.rate_max_base
             del self.io_adjusted[minute]
             stripped += 1
+            if minute in self.octopus_surplus:
+                stripped_surplus += 1
         # The feed is re-read every cycle, so the same minutes are stripped again each time - log only
         # when that changes, not every 5 minutes for the length of a cancellation
         if stripped and stripped != self.dynamic_load_car_stripped:
-            self.log("Octopus Intelligent: removed the dispatch rate from {} minutes of cars {} which are not charging".format(stripped, cancelled_cars))
+            self.log("Octopus Intelligent: removed the dispatch rate from {} minutes - {} of cars {} which are not charging, {} no car needs".format(stripped, stripped - stripped_surplus, cancelled_cars, stripped_surplus))
         self.dynamic_load_car_stripped = stripped
         return rates
 

apps/predbat/predbat.py

@@ -560,6 +560,7 @@ def reset(self):
         self.dynamic_load_car_sensors = {}
         self.dynamic_load_car_effective = {}
         self.dynamic_load_car_stripped = 0
+        self.octopus_surplus = set()
         self.battery_capacity_nominal = False
         self.battery_scaling_auto = False
         self.releases = {}