Add configurable whole-site import diagnostics (no enforcement)

[arturonaredo]

Status — 14 October 2026: Code from #5486, #5498 and #5505 is installed together at my home; household control is operating, but physical EV commissioning remains pending. See the consolidated report and use cases.

v9.4.0 update — 14 October 2026: this branch now includes the official release 9bbe3f75b8275781dd7418237444fb9ab6154b18 through a fast-forward branch update with a merge commit; existing published history is preserved. Current head: 60a7b3a3ca41121cc09f4f63c3b9146afd010a0d. The three PRs remain independently owned. Their combined production code matches the v9.4.0 integration installed and verified at home at 15:30 UTC; the publication additionally corrects two optional-backend test runners. HA and the Predbat web UI reported v9.4.0, a fresh plan was generated without a runtime error, and protected settings were retained. Sustained physical EV delivery, deadline completion and real-tariff midnight savings remain unverified.

Current-head CI: run 38594150205: completed / action_required. Upstream execution awaits maintainer approval. Local validation is not an upstream CI pass.

Scope: diagnostic foundation, not enforcement

Related to #2449 and #1199. This PR does not fix or close either issue.
It proposes a configurable whole-site threshold and truthful diagnostics as a first
step towards a joint household/battery/EV import constraint. It is now installed at my home alongside #5498 and #5505; the dated report above separates observed household operation from pending EV commissioning.

Implemented

• Literal scalar import_limit in W, default 0 (disabled), read once per site.
  It is not summed or minimised across inverters. Reloads can raise, lower or remove it.
  Invalid values are rejected rather than silently converted to disabled.
• Saved Python predictions measure raw step-average site import before graph rounding:
  peak W, excess kWh, minutes over, first violation and checked forecast interval.
  Modelled EV load outside the house CT is included once, assuming a shared grid connection.
• The final best plan publishes predbat.best_import_limit with
  disabled / unavailable / within_limit / exceeded states and enforced: false.
  The scope is the entire forward forecast, not elapsed daily energy or only end_record.
• No clipping of grid demand, invented battery discharge, soft cost penalty, changed
  EV power, actuator writes or kernel dispatch changes. Optimisation trials keep the
  C++ kernel; saving runs already use Python. Stale diagnostics are cleared on later trials.
• English documentation describes the current contract, its limits and a staged
  proposal for actual constraint-aware optimisation.

Why not a large monetary penalty?

A finite penalty does not establish that all feasible candidates outrank all violating
ones. Nor is it valid to shift excess import onto a battery that is already at reserve
and then clamp its SoC without accounting for the missing energy. This draft only
observes the native energy balance; it does not pretend either shortcut is enforcement.

Current v9.4.0 validation

• Exact standalone head 60a7b3a3ca41121cc09f4f63c3b9146afd010a0d: 346/346 registered test groups passed, including slow suites and a freshly built/loaded native kernel. No optional-backend skips.
• Final local combination: 371/371 groups, no skips, with PREDBAT_JOINT_TEMPLATE_REQUIRED=1 and a fresh native kernel; tree efe99e7dc749a91d346dd68c9cd2a9637bf03d09. The shared-file resolutions preserve the independent contributions.
• Repository pre-commit hooks pass. Indexed source and Git trees were unchanged across each validation run.
• The installed production code also passed 25 isolated Linux groups plus kernel parity. The final publication differs from that deployment only in the two test runners, not runtime code.
• GitNexus impact and staged/release-relative change analyses were run. Broad planner paths remain high/critical impact; dynamic test-registry calls are not fully resolved by the graph.
• Source ownership was checked by subtracting the diagnostics and template contributions from the installed integration and recovering the joint-planning tree, with explicit shared-hunk resolutions.
• Local software checks and the home version/plan observation are not physical EV charging acceptance.

Original standalone verification

Independently rerun locally in an isolated Python 3.12 environment (macOS):

• Focused import_limit module: 12 checks passed, including the reserved-battery
  fixed-EV counterexample, metered/outside-CT EV accounting, raw precision, real cache
  hits, actual native-kernel dispatch and the real Plan.run_prediction HA publication path.
• Full --quick suite with PREDBAT_KERNEL_REQUIRED=1: passed, including kernel
  parity; four explicitly slow tests were skipped. This is not a claim of full slow-suite
  or Linux/hardware validation.
• Repository pre-commit hooks on all changed files: passed. git diff --check: passed.
• MkDocs build and rendered new page: passed, with the same three unrelated warnings
  as an unchanged upstream snapshot; no new warnings.
• GitNexus upstream impact and final staged/default-branch change analysis reviewed.
  The prediction/configuration hooks have broad callers, so the checks explicitly verify
  unchanged return tuples, energy accounting, costs, native dispatch and saved curves.

Example: a synthetic 32.1536 kWh battery already at its 10% reserve, a 1.2 kW house
and a fixed 6.77 kW EV over one hour still import 7.97 kWh, cycle the battery by
0 kWh, and report 1.97 kWh above a 6000 W threshold. The diagnostic does not
hide that demand or claim to make the plan feasible.

cd coverage
PREDBAT_KERNEL_REQUIRED=1 python ../apps/predbat/unit_test.py --test import_limit > import-limit-tests.log 2>&1
PREDBAT_KERNEL_REQUIRED=1 python ../apps/predbat/unit_test.py --quick > quick-tests.log 2>&1

Commands assume the repository test dependencies and native kernel are already available.

At the original validation checkpoint, upstream CI was not yet verified: the
code-quality run
reported action_required, with no jobs listed. The local results above
are not being presented as a successful upstream CI run.

Explicitly not implemented

• Feasibility-first candidate ordering, joint EV slot rescheduling and automatic
  battery preparation under the site threshold.
• Grid-only battery-charge curtailment which preserves PV charging and matches
  executable inverter rates/modes, including battery-first ECO behaviour.
• Reliable EV cable detection, executor pause/restart hysteresis, departure-target
  guarantees, multi-inverter dispatch allocation or real-time electrical protection.
• Rolling demand-tariff averages, per-phase limits or a hardware peak-shaving driver.

within_limit refers only to this forecast's step averages. exceeded does not mean
every possible alternative is infeasible. Existing inverter definitions, including
local SK configurations, remain untouched; no production configuration or private
Home Assistant dump is included.

Feedback requested: whether this diagnostics-first split and the scalar setting/output
names are a useful foundation before extending the optimiser, kernel result contract
and native EV planning together.

[arturonaredo]

v9.4.0 publication — 14 October 2026

v9.4.0 update — 14 October 2026: this branch now includes the official release 9bbe3f75b8275781dd7418237444fb9ab6154b18 through a fast-forward branch update with a merge commit; existing published history is preserved. Current head: 60a7b3a3ca41121cc09f4f63c3b9146afd010a0d. The three PRs remain independently owned. Their combined production code matches the v9.4.0 integration installed and verified at home at 15:30 UTC; the publication additionally corrects two optional-backend test runners. HA and the Predbat web UI reported v9.4.0, a fresh plan was generated without a runtime error, and protected settings were retained. Sustained physical EV delivery, deadline completion and real-tariff midnight savings remain unverified.

• Exact standalone head 60a7b3a3ca41121cc09f4f63c3b9146afd010a0d: 346/346 registered test groups passed, including slow suites and a freshly built/loaded native kernel. No optional-backend skips.
• Final local combination: 371/371 groups, no skips, with PREDBAT_JOINT_TEMPLATE_REQUIRED=1 and a fresh native kernel; tree efe99e7dc749a91d346dd68c9cd2a9637bf03d09. The shared-file resolutions preserve the independent contributions.
• Repository pre-commit hooks pass. Indexed source and Git trees were unchanged across each validation run.
• The installed production code also passed 25 isolated Linux groups plus kernel parity. The final publication differs from that deployment only in the two test runners, not runtime code.
• GitNexus impact and staged/release-relative change analyses were run. Broad planner paths remain high/critical impact; dynamic test-registry calls are not fully resolved by the graph.
• Source ownership was checked by subtracting the diagnostics and template contributions from the installed integration and recovering the joint-planning tree, with explicit shared-hunk resolutions.
• Local software checks and the home version/plan observation are not physical EV charging acceptance.

Current-head CI: run 38594150205: completed / action_required. Upstream execution awaits maintainer approval. Local validation is not an upstream CI pass.

The latest code is in the existing PR branches; no installation branch or private configuration is published.

Previous dated report — historical evidence, superseded status

Consolidated installation and validation update — 13 October 2026

Current-main rebase update: all three PRs now include upstream main at 1faff2288982fd3cd159822c99c2658697c48f52 (v9.3.8 plus the five subsequent upstream commits). Heads: #5486 6db22f4e, #5498 fcf54648, #5505 4546443f. The publish_car_plan conflict is resolved: the joint EV publication guard remains before native publication, and upstream keeps a separate plan list for each car. This rebase was validated locally and did not deploy or change the home installation. The dated home observations below describe the earlier installed revision, not a deployment or physical EV commissioning of these rebased heads. The EV compatibility review is complete on the published trees: the existing joint guards are preserved. Compatibility limit: upstream CarChargerControl is not a joint-lease executor and must not be enabled alongside the sole external EV executor. This is a software/source review, not physical EV commissioning. The reproduced boundary and remaining limitations are described in the joint-planning report.

Current-head CI: run 38375563028 for 6db22f4e reports action_required and awaits maintainer approval; local tests are not an upstream CI pass.

I now have the code from #5486, #5498 and #5505 installed together at home, with the household battery plan operating through Predbat's native control loop. This supersedes the original pre-deployment status, not this PR's diagnostics-only boundary. At 07:51 UTC, the latest ordinary cycle was 07:50 UTC: Demand, read-only off, no error, and the away car's relay/EV-slot signal off. Installed/protected hashes and standing settings were verified unchanged.

Scope and useful cases

• This PR (6db22f4e, rebased without feature changes) reports whole-site import peak, excess energy, minutes over the configured threshold and the checked forecast interval. It includes modelled EV load outside the house CT once, when it shares the grid connection. It does not alter optimisation, EV power, inverter commands or native-kernel dispatch.
• This helps identify house/heating + fixed-rate EV demand exceeding a chosen threshold, a battery at reserve unable to support that demand, and violations later in the forecast, rather than hiding them behind clipped import or an invented battery contribution. within_limit means forecast step averages, not instantaneous electrical protection.
• feat: experimental native joint EV and home battery planning #5498 (fcf54648) supplies the complementary joint search and guarded execution: availability-first household routing, paired nominal/PV10 replay, grid/PV headroom sharing, continuous/ASAP fixed-power charging and explicitly authorised partial delivery with an unchanged requested target. A car charging elsewhere is excluded before reading its SOC. Retained vehicle telemetry requires explicit session evidence; session renewal is never automatic.
• Add shared Deye/Solarman template with observed service sequences #5505 (4546443f, rebased without feature changes) supplies the reusable Deye/Solarman template and observed native service sequences, separately from diagnostics and planning.

Predbat already has EV planning; these additions address the coupled fixed-rate EV/domestic-battery constraint, not EV scheduling in general. The existing Plan table keeps its columns, colours, totals and calculation precision; ordinary household fallback has no production banner, with technical reasons only in Show Debug.

What was verified

This rebased diagnostics head passed 345/345 registered groups, including slow tests and a freshly rebuilt/loaded native kernel. The rebased #5498 public tree passed 364 registered test groups with a fresh native kernel. Five optional-backend groups checked safe rejection but skipped real-template scenarios when run without #5505. The local integration of all three public heads passed 367/367 groups, without skips, including the real shared-template fixtures. Pre-commit passed on both trees; the documentation build passed with unrelated existing warnings. The separately installed revision also passed native power-matrix and captured 48-hour replay checks. None of those offline checks actuated hardware.

All three PRs have now been rebased onto current upstream main. This diagnostic PR remains independent. The combined changes since the release-pinned validation are the five upstream commits, with both sides of the publication guard/per-car-list conflict preserved. Private captures, configuration and the combined installation branch are not published. GitNexus impact/change analysis was rerun after reindexing. The full PR changes still carry critical risk and dynamic-call relationships remain unresolved; this is not a graph all-clear. Current-head upstream CI remains approval-blocked as reported above; previous runs on older SHAs are not evidence for them.

Boundaries: household operation is observed; physical EV charging is still pending an at-home, connected, expressly authorised session. The installed session was not renewed, and EV authority remains disabled. Real departure SOC, realised savings, outage recovery and instantaneous enforcement have not been demonstrated. Existing currents, reserve/shutdown, EV pause/restart/cooldown, the separate executor cutoff and local-only inverter ownership were preserved. Peak Shaving is not a guaranteed hard grid clamp. This PR still does not close #2449 or #1199 by itself.