feat(config): add a performance_tweaks toggle and merge the two plan passes

[springfall2008]

Two related changes to the optimiser's final pass and how it is configured.

1. Merge tweak_plan and optimise_full_second_pass

They were the same loop — same window sort, same optimise_charge_limit/optimise_export calls, same keep_window_change_if_improved — differing only in:

• tweak_plan's count >= 8 cap
• the export-window start_orig reset, which only tweak_plan performed

Now one function, optimise_plan_pass(end_record, budget). calculate_second_pass selects budget=0 (every window in the record) instead of a second implementation. Net −48 lines.

The reset now applies on both paths. Without it a window trimmed by an earlier pass can only ever be trimmed further, so the pass cannot recover a window it narrowed on a plan that has since changed underneath it.

This part is provably behaviour-preserving. With PV90 held off, the merged pass reproduces main exactly across the 20-scenario random benchmark: all 20 identical on metric and cost, total runtime 38.21s vs 38.21s.

2. switch.predbat_performance_tweaks

A second gating toggle alongside expert_mode, defaulting Off. Behind it sit the options that trade planning time for plan quality — and unlike expert mode, everything behind it defaults to On. A user who never touches it gets the best plan Predbat can produce; the toggle exists so someone on slow hardware can reveal the switches and turn features off.

Moved behind it, both now defaulting to On:

• switch.predbat_calculate_second_pass (was expert-gated, default Off)
• switch.predbat_calculate_pv90_plan (was ungated, default Off)

Sharp edge worth knowing

get_ha_config substitutes an item's default when the item is disabled rather than returning None, so get_arg never reaches its self.args lookup. A hidden option is pinned to its default and cannot be held Off from apps.yaml without revealing the toggle. This is pre-existing behaviour for every expert-gated item, but it matters more now that hidden means "on". Documented and covered by tests.

Evidence for the default flips

PV90

The random benchmark scored every plan only on the central forecast, which cannot judge a hedge: a plan that weights PV90 deliberately spends money the central forecast says is wasted, so scoring it there marks it down however well it does on the day it was hedging for. This PR extends the harness to record each plan's cost in all three simulated futures, so the effect is reproducible from the repo rather than from a one-off script.

Main vs this branch, 20 scenarios (− = branch cheaper in that future):

future	avg	min	max	better	worse	unchanged
PV10 (cloudy, heavier load)	−4.66	−68.86	+14.98	7	6	7
nominal	−3.41	−33.91	+1.69	7	5	8
PV90 (sunny, lighter load)	−10.56	−53.38	+1.79	11	2	7

Cheaper on average in all three futures, and strongly so in the one PV90 exists for (11–2). Isolating PV90 alone — second pass off, both plans built from identical data — gives the same shape at smaller magnitude: −1.00 / −1.17 / −5.24, winning 10–2 in the sunny future.

Caveat on magnitude: the benchmark's P90 series is synthetic (pv90 = pv × 1.25, PV90_FACTOR fixed in the harness) and the sunny case also applies load_scaling90 = 0.7, so the size of the win is an artefact of those factors even though the direction is not.

Second pass

apply_scenario_to_predbat pinned calculate_second_pass to a literal False, so the random benchmark could never measure this switch — and silently ignored a template that set it, which made an earlier second-pass run look like a null result rather than an experiment that never happened. This PR changes the harness to read the CONFIG_ITEMS default instead, so the benchmark measures the shipped planning path.

With that fixed, main vs this branch on the real config read: metric average −3.1704, cost average −3.4125, 6 better / 7 worse / 7 unchanged. Those match the earlier stand-in measurement (forcing PLAN_PASS_WINDOW_BUDGET = 0, which is exactly what the switch selects) to within run-to-run noise, so both measurements agree.

Raising the budget also never regressed a scenario in isolation: at budget 40, 2 better / 0 worse / 18 unchanged; unbudgeted, 5 better / 0 worse / 15 unchanged.

Runtime cost

20-scenario random benchmark, total wall-clock across the suite:

config	total	mean	median	slowest	vs main
main	38.21s	1.910s	0.869s	8.87s	—
merged pass only	38.21s	1.910s	0.872s	8.85s	+0.0%
+ PV90 on	47.39s	2.369s	1.165s	11.19s	+24.0%
as shipped (both defaults on)	50.94s	2.547s	1.275s	11.62s	+33.3%

The final row is a real config read via the fixed harness, not a stand-in. Worst single scenario: +2.75s (2.07x). The cost scales with horizon length — a 48-hour plan pays more than these 24-hour scenarios (the #4478 capture went 11.0s → 15.6s for the second pass alone).

This is the whole reason for the toggle: users on a Pi can turn either feature off and get the old planning time back.

Golden cases

predbat_debug_pre_saving1's expected plan is regenerated. Those captures predate PV90 and do not pin pv_metric90_weight, so the new default switches PV90 on for them. The change is modest — exports slightly more (export_limits_best [7.0, 85] → [6.0, 85], target 12 → 10) with a mildly restructured charge plan, at the same nominal metric of 471.1906. agile1 is unchanged.

Testing

• Full ./run_all passes; pre-commit clean.
• New test_performance_tweaks covers the toggle, its gated items' defaults, hide/reveal behaviour, and the fetch_config_options read.
• test_export_commitment's two monotonicity tests now exercise the merged pass at budget=8 and budget=0.
• test_pv90's default-off assertions rewritten to force the Off case now that On is the default, plus a new test pinning the hidden-item-resolves-to-default mechanism.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR refactors Predbat’s optimiser “final pass” by merging the previous tweak/second-pass implementations into a single optimise_plan_pass(...) with a window budget, and introduces a new performance_tweaks gating switch that hides (but defaults-on) the more CPU-intensive plan-quality features.

Changes:

• Merge the old tweak/second-pass loops into optimise_plan_pass(end_record, budget=...), and drive the pass via PLAN_PASS_WINDOW_BUDGET vs “unlimited” for the full second pass.
• Add switch.predbat_performance_tweaks and move calculate_second_pass + calculate_pv90_plan behind it, flipping both defaults to On while hidden.
• Update docs and tests to reflect the new toggle, new defaults, and the merged pass behavior.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
docs/customisation.md	Documents the new performance tweaks toggle and updated defaults/visibility rules.
coverage/cases/predbat_debug_pre_saving1.yaml.expected.json	Updates golden output to match new default planning behavior.
apps/predbat/unit_test.py	Registers the new performance tweaks test suite.
apps/predbat/tests/test_pv90.py	Updates PV90 tests for new gating/defaults and adds coverage for hidden-default behavior.
apps/predbat/tests/test_performance_tweaks.py	Adds new unit tests covering the performance_tweaks gate, defaults, and hide/reveal semantics.
apps/predbat/tests/test_optimise_swap_export.py	Updates pass-order assertions for the merged plan pass.
apps/predbat/tests/test_export_commitment.py	Updates monotonicity tests to exercise optimise_plan_pass in budgeted and unbudgeted modes.
apps/predbat/plan.py	Implements optimise_plan_pass, introduces PLAN_PASS_WINDOW_BUDGET, and removes the duplicate second-pass implementation.
apps/predbat/config.py	Adds performance_tweaks config item and gates/updates defaults for second pass and PV90 switches.

Suppressed comments (1)

apps/predbat/plan.py:2895

• The optimise_plan_pass docstring says optimise_swap_export runs before this, but optimise_all_windows calls optimise_plan_pass before optimise_swap_export. This makes the rationale for re-baselining the metric misleading.

        The metric is measured from the plan we were handed rather than taken from the caller:
        optimise_swap_export runs before this and mutates the plan without its return value being used,
        so the caller's metric is already stale.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.