Energy demand profile: improve prediction accuracy for heating loadpoints - #28232
Energy demand profile: improve prediction accuracy for heating loadpoints#28232daniel309 wants to merge 215 commits into
Conversation
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 4 issues, and left some high level feedback:
- Access to
loadpointEnergyandloadpointSlotStartmaps happens from multiple goroutines inupdateLoadpoints/updateLoadpointConsumptionwithout synchronization, which risks concurrent map access panics; consider guarding these maps (and their contained state) with a mutex or encapsulating per-loadpoint state in a concurrency-safe structure. - In
applyTemperatureCorrection, the loopfor h := range 24is invalid in Go and will not compile; it should be replaced with an index loop likefor h := 0; h < 24; h++when buildingpastTempAvg.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Access to `loadpointEnergy` and `loadpointSlotStart` maps happens from multiple goroutines in `updateLoadpoints`/`updateLoadpointConsumption` without synchronization, which risks concurrent map access panics; consider guarding these maps (and their contained state) with a mutex or encapsulating per-loadpoint state in a concurrency-safe structure.
- In `applyTemperatureCorrection`, the loop `for h := range 24` is invalid in Go and will not compile; it should be replaced with an index loop like `for h := 0; h < 24; h++` when building `pastTempAvg`.
## Individual Comments
### Comment 1
<location path="core/site_optimizer.go" line_range="658-659" />
<code_context>
+ pastTempCount[h]++
+ }
+ }
+ pastTempAvg := make([]float64, 24)
+ for h := range 24 {
+ if pastTempCount[h] > 0 {
+ pastTempAvg[h] = pastTempSum[h] / float64(pastTempCount[h])
</code_context>
<issue_to_address>
**issue (bug_risk):** The `for h := range 24` loop is invalid Go and will not compile.
Use a standard indexed `for` loop here instead of ranging over an `int`, for example:
```go
pastTempAvg := make([]float64, 24)
for h := 0; h < 24; h++ {
if pastTempCount[h] > 0 {
pastTempAvg[h] = pastTempSum[h] / float64(pastTempCount[h])
}
}
```
</issue_to_address>
### Comment 2
<location path="core/site.go" line_range="927-928" />
<code_context>
)
- for _, lp := range site.loadpoints {
+ for i, lp := range site.loadpoints {
+ lpID := i // capture loop variable for goroutine
wg.Go(func() {
power := lp.UpdateChargePowerAndCurrents()
</code_context>
<issue_to_address>
**issue (bug_risk):** The goroutine still closes over the loop variable `lp`, which can cause data races and incorrect behavior.
You correctly capture `lpID`, but `lp` is still shared across iterations due to `range` semantics. Capture `lp` in a new local variable before starting the goroutine:
```go
for i, lp := range site.loadpoints {
lpID := i
lp := lp // capture lp for goroutine
wg.Go(func() {
power := lp.UpdateChargePowerAndCurrents()
// ...
})
}
```
This prevents the goroutine from seeing a different loadpoint than intended and avoids subtle concurrency bugs in the update methods.
</issue_to_address>
### Comment 3
<location path="core/site_optimizer.go" line_range="687-688" />
<code_context>
+ continue
+ }
+
+ h := ts.UTC().Hour()
+ tPastAvg := pastTempAvg[h]
+
+ // delta > 0: tomorrow colder than historical average → load increases
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Using a zero default for `tPastAvg` when there is no historical data for a given hour can skew the correction significantly.
When `pastTempCount[h] == 0`, `pastTempAvg[h]` stays at its zero value, which makes `tFuture` look much warmer than "history" and can drive overly strong negative corrections for those hours.
Consider skipping correction or using a safer fallback (e.g. overall past 24h average) when there’s no data:
```go
h := ts.UTC().Hour()
if pastTempCount[h] == 0 {
continue // or use a fallback average
}
tPastAvg := pastTempAvg[h]
```
This prevents over-correcting when the historical baseline is unknown.
```suggestion
h := ts.UTC().Hour()
if pastTempCount[h] == 0 {
// no historical data for this hour; skip correction (or use a fallback if available)
continue
}
tPastAvg := pastTempAvg[h]
```
</issue_to_address>
### Comment 4
<location path="core/site_optimizer.go" line_range="515" />
<code_context>
func (site *Site) homeProfile(minLen int) ([]float64, error) {
- // kWh over last 30 days
- profile, err := metrics.Profile(now.BeginningOfDay().AddDate(0, 0, -30))
+ from := now.BeginningOfDay().AddDate(0, 0, -7)
+
+ // kWh average over last 7 days - base load (excludes loadpoints)
</code_context>
<issue_to_address>
**issue (complexity):** Consider refactoring the new home profile and temperature-correction logic into small reusable helpers and pre-indexed data structures to keep the core control flow simple and readable.
The new logic is valid but it does increase complexity in a few focused places. You can reduce it without changing behavior by extracting small helpers and pre-indexing data.
### 1) Repeated “repeat profile until minLen then trim” logic
This pattern appears multiple times in `homeProfile` for both base and heater profiles:
```go
slots := make([]float64, 0, minLen+1)
for len(slots) <= minLen+24*4 { // allow for prorating first day
slots = append(slots, gt_base[:]...)
}
res := profileSlotsFromNow(slots)
if len(res) < minLen {
return nil, fmt.Errorf("minimum home profile length %d is less than required %d", len(res), minLen)
}
if len(res) > minLen {
res = res[:minLen]
}
```
and similarly for `heaterSlots`.
This can be encapsulated in a reusable helper:
```go
func repeatAndTrimProfile(profile []float64, minLen int) ([]float64, error) {
slots := make([]float64, 0, minLen+1)
for len(slots) <= minLen+24*4 {
slots = append(slots, profile...)
}
res := profileSlotsFromNow(slots)
if len(res) < minLen {
return nil, fmt.Errorf("minimum home profile length %d is less than required %d", len(res), minLen)
}
if len(res) > minLen {
res = res[:minLen]
}
return res, nil
}
```
Then `homeProfile` becomes simpler:
```go
gtBaseSlots, err := repeatAndTrimProfile(gt_base, minLen)
if err != nil {
return nil, err
}
var gtHeaterSlots []float64
if len(gt_heater_raw) > 0 {
if heater, err := repeatAndTrimProfile(gt_heater_raw, len(gtBaseSlots)); err == nil {
gtHeaterSlots = heater
}
}
```
This removes duplicated branching and makes the length-normalization logic easier to reason about.
### 2) Split `homeProfile` into clearer orchestration steps
`homeProfile` is currently doing: read base, read heater, normalize both, correct heater by temperature, merge, convert units. Extracting those as small helpers keeps the control flow readable:
```go
func (site *Site) buildBaseProfile(from time.Time, minLen int) ([]float64, error) {
base, err := metrics.Profile(from)
if err != nil {
return nil, err
}
return repeatAndTrimProfile(base, minLen)
}
func (site *Site) buildHeaterProfile(from time.Time, minLen int) []float64 {
raw := site.extractHeaterProfile(from, time.Now())
if len(raw) == 0 {
return nil
}
heater, err := repeatAndTrimProfile(raw, minLen)
if err != nil {
return nil
}
return site.applyTemperatureCorrection(heater)
}
```
`homeProfile` then reduces to an orchestrator:
```go
func (site *Site) homeProfile(minLen int) ([]float64, error) {
from := now.BeginningOfDay().AddDate(0, 0, -7)
base, err := site.buildBaseProfile(from, minLen)
if err != nil {
return nil, err
}
heater := site.buildHeaterProfile(from, len(base))
final := make([]float64, len(base))
for i := range base {
final[i] = base[i]
if heater != nil && i < len(heater) {
final[i] += heater[i]
}
}
return lo.Map(final, func(v float64, _ int) float64 { return v * 1e3 }), nil
}
```
Same behavior, but the main function reads as a high-level description of the steps.
### 3) Avoid O(N²) search in `applyTemperatureCorrection`
The inner loop that finds `tFuture` completely dominates the function’s complexity and obscures the main idea:
```go
for i := range profile {
ts := slotStart.Add(time.Duration(i) * tariff.SlotDuration)
var tFuture float64
found := false
for _, r := range rates {
if r.Start.Equal(ts) {
tFuture = r.Value
found = true
break
}
}
if !found {
continue
}
// ...
}
```
You can pre-index the forecast into a map and keep the slot loop single-level:
```go
func indexRatesByStart(rates api.Rates) map[time.Time]float64 {
m := make(map[time.Time]float64, len(rates))
for _, r := range rates {
m[r.Start] = r.Value
}
return m
}
```
Use it in `applyTemperatureCorrection`:
```go
forecastByTime := indexRatesByStart(rates)
slotStart := currentTime.Truncate(tariff.SlotDuration)
for i := range profile {
ts := slotStart.Add(time.Duration(i) * tariff.SlotDuration)
tFuture, ok := forecastByTime[ts]
if !ok {
continue
}
h := ts.UTC().Hour()
tPastAvg := pastTempAvg[h]
delta := tPastAvg - tFuture
result[i] = profile[i] * (1 + coeff*delta)
}
```
This both improves performance and reduces nesting, making the correction logic easier to follow.
### 4) Separate guard/validation from the core correction
The top of `applyTemperatureCorrection` is mostly guards and configuration checks. Extracting them into a small helper can clarify the “happy path”:
```go
func (site *Site) getWeatherRates() (api.Rates, float64, float64, error) {
weatherTariff := site.GetTariff(api.TariffUsageTemperature)
if weatherTariff == nil {
return nil, 0, 0, fmt.Errorf("no weather tariff")
}
rates, err := weatherTariff.Rates()
if err != nil || len(rates) == 0 {
return nil, 0, 0, fmt.Errorf("no weather rates")
}
threshold := site.HeatingThreshold
coeff := site.HeatingCoefficient
if threshold == 0 || coeff == 0 {
return nil, 0, 0, fmt.Errorf("heating config missing")
}
return rates, threshold, coeff, nil
}
```
Then:
```go
func (site *Site) applyTemperatureCorrection(profile []float64) []float64 {
rates, threshold, coeff, err := site.getWeatherRates()
if err != nil {
return profile
}
// ... 24h avg, hourly history, forecastByTime, main loop ...
}
```
This keeps `applyTemperatureCorrection` focused on the actual correction math rather than the early-exit conditions.
These refactors maintain all functionality but reduce branching and nesting, making the new behavior easier to understand and maintain.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
First thing is to get #23185 in. |
|
This contribution does not appear to meet our AI contribution guidelines. |
|
@naltatis this PR shows general understanding of evcc and has prior PRs. Fine for me. |
|
ill add more evidence for local testing and benefits once I have the build green. should be there is a bit. ai reviews addressed as follows |
|
ok, ready for review. because Base PR: #27780 (Weather Tariff) is open, only look at these 3 files below. The rest are the temperature tariff code changes that go away once that PR is merged. 
ill continue local testing and add evidence from the sqllite db, about migration (using my evcc.db file from latest) and most importantly, about the added precision of household load forecasts. With this PR optimizer forecasts are finally useable when you have a heatpump device registered in evcc. Before that they were unuseable because:
|
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- In applyTemperatureCorrection, the loop
for h := range 24will not compile in Go; it should be replaced with a standard indexed loop such asfor h := 0; h < 24; h++ { ... }. - The temperature correction currently relies on an exact
time.Timematch betweenslotStart.Add(i*SlotDuration)andratesByTimekeys; to avoid missed corrections due to timezone or truncation mismatches, consider deriving future slots directly from the tariff rates or using a nearest-slot lookup instead of exact equality.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In applyTemperatureCorrection, the loop `for h := range 24` will not compile in Go; it should be replaced with a standard indexed loop such as `for h := 0; h < 24; h++ { ... }`.
- The temperature correction currently relies on an exact `time.Time` match between `slotStart.Add(i*SlotDuration)` and `ratesByTime` keys; to avoid missed corrections due to timezone or truncation mismatches, consider deriving future slots directly from the tariff rates or using a nearest-slot lookup instead of exact equality.
## Individual Comments
### Comment 1
<location path="core/site_optimizer.go" line_range="659-660" />
<code_context>
+ pastTempCount[h]++
+ }
+ }
+ pastTempAvg := make([]float64, 24)
+ for h := range 24 {
+ if pastTempCount[h] > 0 {
+ pastTempAvg[h] = pastTempSum[h] / float64(pastTempCount[h])
</code_context>
<issue_to_address>
**issue (bug_risk):** The `for h := range 24` loop does not compile and should iterate over a slice or a numeric range explicitly.
`range` requires an array, slice, map, string, or channel. Here you probably want either `for h := range pastTempAvg { ... }` or `for h := 0; h < 24; h++ { ... }`. Given the fixed size, iterating over `pastTempAvg` is likely the most idiomatic option.
</issue_to_address>
### Comment 2
<location path="core/site.go" line_range="96-97" />
<code_context>
householdEnergy *meterEnergy
householdSlotStart time.Time
+ // per-loadpoint energy tracking for heating devices
+ loadpointEnergy map[int]*meterEnergy
+ loadpointSlotStart map[int]time.Time
+
</code_context>
<issue_to_address>
**issue (bug_risk):** Concurrent access to `loadpointEnergy` and `loadpointSlotStart` maps is not synchronized and can cause data races.
These maps are written in `updateLoadpointConsumption`, which runs in goroutines spawned by `updateLoadpoints`. Even if goroutines touch different keys, Go maps are not safe for concurrent writes and can panic (`concurrent map writes`) or introduce data races. Please either switch to a slice indexed by loadpoint ID or protect map access with synchronization (e.g., a shared mutex or a per-loadpoint struct with an embedded mutex).
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
update: db schema changes removed. see PR description. no more additional loadpoint column. 
|
andig
changed the title
|
@andig I removed any db schema change from this PR, so this is now independent of #23185. see Just tested compatibility with v0.303. works fine without issues. also, table "meter" remains lean which is a good thing. adding a varchar added significant volume to the file (20 bytes per each row...) the offset between meter ids and loadpoint ids is now 1000 to give enough space between the two for all practical numbers of meters and loadpoints. |
a5764b2 to
16203c8
Compare
|
evidence of the entire process working and about the impact on optimizer forecast and corrections. log messages of a working system: -> see how the heater profile slots got adjusted based on temperature forecast here is the result in the optimizer. see the wavy pattern of a modulating heatpump, how de-icing was running around 5am and how warm water was produced starting around noon. 
now here is the same optimizer profile without the temperature adjustments (you see differences in the range of 10% as per log messages above temperature didnt change that much between profile avg and tomorrow)
and finally, here is the exact same optimizer picture from vanilla evcc v0.303. Completely different household load and honestly not very useable/realistic. 
####################### @andig relevant code is in these 3 files. the remaining changes are all from the base PR (temperature tariff) 
|
Adds optional temperature correction to household load forecasts for the optimizer, allowing more accurate predictions when heating/cooling loads vary with outdoor temperature. Configuration (optional, in site config): - heatingThreshold: °C 24h avg above which corrections are disabled (default 12°C) - heatingCoefficient: fractional load change per °C delta (default 0.05 = 5%) Algorithm: 1. Gates on past 24h avg temperature vs threshold (heating active if below) 2. For each future slot, compares forecast temp to historical avg at same hour 3. Adjusts load: load *= (1 + coeff * (T_historical_avg - T_forecast)) - Colder forecast → higher load estimate - Warmer forecast → lower load estimate Requires the Temperature tariff (TariffUsageTemperature) to be configured. Changes household profile lookback from 30 days to 7 days because: - 7 days follows household rhythms more closely (weekly patterns) - Temperature changes too much over 30 days, making older data less relevant
- Separate heater load from base household consumption - Apply temperature correction only to heating devices (heat pumps, electric heaters) - Base loads (lighting, appliances) remain unchanged - Backward compatible: falls back to old behavior if no heating devices Changes: - Extended metrics DB to track per-loadpoint consumption - Added loadpoint energy tracking infrastructure in Site - Implemented profile extraction and aggregation functions - Modified homeProfile() to separate, correct, and merge profiles Addresses feedback on PR evcc-io#27780 that temperature adjustment should only apply to heating devices, not entire household consumption.
|
We're stuck at the profile selection. The blanked change to temp profile for some of the heatpumps like the LGTherma is imho plain wrong. |
|
|
||
| // homeProfile returns the predicted home base load in Wh for minLen 15min slots | ||
| // starting now, averaged over the past 4 weeks. | ||
| // starting now. Prefers same-weekday data from the past 4 weeks; falls back to |
There was a problem hiding this comment.
why did this happen? I can't remember any discussion that we wanted to change anything about the default profile.
There was a problem hiding this comment.
this is the conversation we had about this. Its in another review comment. Hard to find so I copied it here as picture.
I went ahead and added the logic (with fallback) for you to see how little of a change this is and how well it matches to home profile in general.
let me know, I can remove. But I think we would lose prediction accuracy without it
|
I also have the feeling that over the last week, we've not converged but more changes appeared that may need some more time to understand. |
I think there are 2 open items @andig 1. Profile selection
Our discussion on wiring above seemed to agree on providing defaults (via template includes), and then offer options for users to override in UI. Thats why I started to include the *switch .tpl includes in the warmwater and combined heatpump templates. I got cold feet when I found > 25 templates in The remaining templates will have to be categorized through additional separate PRs by the template owners that know the devices. 2. apply the DemandWeekly logic to home profile This discussion is in the review comment here: #28232 (comment). I think home profile is a posterchild usecase for the demandweekly logic, hence I added it. Let me know, I can also remove. |
The default imho- unless we have a very specific reason to deviate- is the standard (averaged) household profile. I've indeed lost track if we're now targeting that plus weekday+temperature (3 total) or if you've replaced the standard profile with weekday now?
I think that would make it 2 choices then? Looking for input here, but I don't see a reason to make that change (or make it now). /cc @iseeberg79 |
@andig This PR does two things:
Note that no heating device today is part of the household load. You have to explicitly tag a device with Demand* to add it, its load is ignored otherwise (see Also, I thought it makes sense for users having to define (via flag) if a device should be added to home profile explicitly. Another option would be to add all
So, to answer your 2nd question: there is no choice for the home profile base load after this PR. its using the weekly predictor (instead of 28-day avg today) and only if weekly doesnt have full data for all prior same weekdays, it falls back to 28-day avg. |
|
Difficult to jump in here after the fact — I tried to keep up with the discussion. To make sure I understand correctly: pointing me to the diff was to highlight that both PRs effectively adjust the same household consumption profile, just from different angles? I think there’s a good case for factoring in a short-term look-back rather than relying purely on a long-term average. The two approaches feel complementary to me. Over the summer holidays and the return to daily routine, actual consumption ran 10–20% off the long-term prediction; now that we’re back in our normal routine, it matches again (near 0% deviation). The forecast for battery levels has become noticeably more accurate as a result. I’d expect something similar could apply to heating and its relationship with outdoor temperature and potentially higher gain. To be precise about the models: my (now closed) PR uses an AR-style mean-reversion approach, whereas this PR is based on the degree-day method. I also compared against a weekday-based prediction model, but in my metrics it came out slightly less accurate — four weeks back means estimating a weekday profile from just 4 samples, which is too few (eight weeks would be better). In my data, without heating in the mix, the difference between a 28-day average and a weekday profile was negligible anyway. In my view, extending the historical consumption profile calls for a short-term adjustment on top. My personal take: there’s generally enough deviation worth accounting for — a long-term baseline adjusted by a short-term deviation factor seems like the right shape. I believe to shorten the period for home profiling isn't valid. That said, it’s still — necessarily — just an attempt at prediction… |
|
thank you @iseeberg79. I already have a python script that operates on evcc.db and statistically compares predictors based on actual loads from the db vs. predicted loads with data (and temperatures). I can share that for future predictor development, but I hope we can do this in a different PR. I dont think we should let more scope creep into this one ;-) @andig based on this message, let me know what is needed to close-out this PR? I think your feedback was
this would mean the Demand* profiles are unused and there is no change in behavior until someone explicitly adds the feature flags to a loadpoint / device .yaml. |
|
hi @FuR1u5.
No, this PR is solely about introducing a forecast infrastructure for the optimizer. This is to help predicting future loads, so that the optimizer can make better decisions.
see #32881 |
|
@andig implemented the changes suggested above: #28232 (comment) This is now as low touch as possible. No change to existing evcc behavior. Also, I have a python script that helps selecting the right Demand* feature for template developers as default. It runs over a evcc.db and does a statistical test and suggests the Demand* profile that yields the lowest error comparing prediction to real load. 
Not sure where to put it, so sharing as attachment for now |
| IntegratedDevice // charger - always connected - no vehicle, no charging sessions | ||
| SwitchDevice // charger - no current control - heat pumps or switch sockets | ||
| Heating // charger - heating device - soc ist temperature (°C) | ||
| DemandDaily // charger - demand forecast: 28-day daily average |
There was a problem hiding this comment.
Why was the daily added now? I'm wondering if we need it. Isn't this the default anyway, i.e. "no profile selected"?
There was a problem hiding this comment.
because of 2 reasons
- its what the home profile uses, so its an existing "predictor" used internally already
- my eval script that benchmarks the 3 predictors against each other shows its a good predictor for "noisy" loads, where there is no real pattern visible but high statistical noise instead. a slot-by-slot avg over 28 slots nicely smoothes this out.
Its not the default. A heatpump is excluded from the home profile (like today) unless the Demand* flag is added.
I had the impression we dont want to cause any change to how evcc and the optimizer behaves today with this PR. So unless you use the feature flag explicitly, nothing changes.
This is the most defensive approach.
Another idea would be to fold all integrateddevide (or integrateddevide and getgevicle()==nil) loadpoints into the home profile by default (and use the DemandDaily predictor implicitly) unless tagged with a Demand* profile explicitly. Let me know.
There was a problem hiding this comment.
Its not the default. A heatpump is excluded from the home profile (like today) unless the Demand* flag is added.
Thank you, found the same. Seems this is an actual bug today since we're excluding expected load. Imho we should not have an option for that but just make it the default (i.e. nothing set). I'd suggest not doing another PR but updating this to remove the param and make it the default fallback.
Wdyt? This would be the last item on my list that caught my eye yesterday when preparing to merge ;)
andig
left a comment
andig
left a comment
There was a problem hiding this comment.
Review of the demand-forecast changes. Builds clean and ./core/... + ./server/... pass on the PR head, so everything below is behaviour, not build breakage.
Two things look like blockers to me:
- The feature flags are unreachable as documented —
features:in YAML replaces the charger defaults, so addingdemandtemperaturedropsheatinganddemandProfilebails on its own guard (see inline comment). - The temperature correction inverts when the historical average is above
tRoom(see inline comment).
The remaining comments are about the heating loadpoint disappearing from the optimizer's model entirely in some states, plus accuracy and diagnosability nits.
🤖 Reviewed with Claude Code
| // demandProfile returns the heating demand profile of a heating loadpoint and whether | ||
| // it needs to be scaled by the outdoor temperature forecast. Returns nil when unavailable. | ||
| func (lp *Loadpoint) demandProfile() (*[96]float64, bool) { | ||
| if lp.chargeEnergy == nil || !lp.chargerHasFeature(api.Heating) { |
There was a problem hiding this comment.
As far as I can tell the three new features can't actually be reached in a real config.
Grepping *.go and *.yaml on this branch, nothing sets DemandDaily/DemandWeekday/DemandTemperature — only the definitions and this consumer. So the only route is a features: list in evcc.yaml, as in the PR description.
But features: replaces the charger's defaults rather than extending them. Decoding features: [demandtemperature] over the heatpump charger's default {Continuous, Heating, IntegratedDevice} yields [DemandTemperature] — so this very guard returns early on !chargerHasFeature(api.Heating), and the loadpoint additionally loses its Continuous/IntegratedDevice charge-control semantics.
Either the templates need to carry the tags, or the demand flags need to merge with the defaults instead of replacing them.
There was a problem hiding this comment.
yes, I have two options
- add the feature flags to one or more .tpl includes (heatpumpswitch.tpl, introduce warmwaterswitch.tpl, etc) and we all accept changed behavior (predictors active)
or (whats implemented today)
have template owners "inline" the entire .tpl file instead and add the Demand* flag
I had both implemented but then understood you wanted unchanged behavior (so no "wiring").
There was a problem hiding this comment.
wait, I found an option. Here is my solution now (latest commit)
A template author can now write:
features:
- continuous
- heating
- integrateddevice
- switchdevice
predictor:
- demandtemperature
There was a problem hiding this comment.
Pls- lets not make more changes at this point. Feature selection is not part of this PR, we'll do that later.
| func (site *Site) applyTemperatureCorrection(profile []float64) []float64 { | ||
| weatherTariff := site.GetTariff(api.TariffUsageTemperature) | ||
| if weatherTariff == nil { | ||
| return profile |
There was a problem hiding this comment.
Silent fallback: with demandtemperature set but no temperature tariff configured, this returns the uncorrected 7-day average and logs nothing.
The 7-day window also differs from DemandDaily's 28 days, so a misconfigured setup quietly gets a third profile that matches neither documented mode. A DEBUG or WARN line here would make that diagnosable.
There was a problem hiding this comment.
Two log lines added to applyTemperatureCorrection:
WARN when weatherTariff == nil: this is a misconfiguration — the device has demandtemperature set but no temperature tariff is wired up. A warning is appropriate because the user will see a silently degraded 7-day profile rather than the corrected one they expect.
DEBUG when rates is empty or errors: this is a transient condition (fetch failure, startup, cache miss) and not worth alarming users about, but useful for diagnostics.
…aniel309/evcc into feature/temperature-correction # Conflicts: # core/loadpoint_load_predictor.go
|
addressed all review comments. please have a look @andig |
|
note: ill be out the next 7 days, responses will be slow. |
|
|
||
| rates, err := weatherTariff.Rates() | ||
| if err != nil || len(rates) == 0 { | ||
| site.log.DEBUG.Printf("temperature correction: no rates available: %v", err) |
There was a problem hiding this comment.
| site.log.DEBUG.Printf("temperature correction: no rates available: %v", err) | |
| site.log.ERROR.Printf("temperature correction: no rates available: %v", err) |
17 participants
Summary
In winter, heating loads can dominate household electricity consumption and vary dramatically with outdoor temperature. Without accurate heating demand forecasts, the optimizer receives a severely understated household load estimate, leading to suboptimal or conflicting decisions for EV charging and battery scheduling.
This PR adds demand forecasting for heating loadpoints via two patterns:
demandtemperature— for loads that track outdoor temperature (room heating). Uses a 7-day historical average profile scaled by the outdoor temperature forecast.demandweekday— for loads that follow a weekly schedule (warm water, pool heaters). Uses the same-weekday profile from the prior week as-is.The home base load itself also uses the same-weekday pattern (past 4 weeks), falling back to the 28-day all-days average when insufficient data is available.
Builds on #27780 (open-meteo weather tariff).
Solution
Two new charger feature flags tell the optimizer how to forecast demand for a heating loadpoint:
demandtemperature— forecast based on the 7-day historical average load, scaled by the outdoor temperature forecast (suitable for room heating: heat pumps, electric radiators, floor heating)demandweekday— forecast based on the same weekday from the prior week, used as-is (suitable for schedule-driven loads: warm water boilers, pool heaters)Devices without either flag are excluded from demand forecasting entirely. The
heatingflag alone is not sufficient — one of the demand flags must also be set.How it works
The total household load passed to the optimizer is composed of two parts:
1. Home base load (
homeProfile)Uses the same-weekday profile from the past 4 weeks (~4 samples). Falls back to the 28-day all-days average when the weekday profile is incomplete.
2. Heating demand (
addHeatingDemand)For each heating loadpoint,
demandProfilereturns the historical load profile plus a flag indicating whether outdoor temperature correction should apply:demandtemperatureloadpoints — 7-day historical average profile, corrected by the temperature forecastdemandweekdayloadpoints — same-weekday profile from the prior week, used as-is (no temperature correction)The demand profiles from all heating loadpoints are summed on top of the home base load before being passed to the optimizer.
Temperature correction algorithm
Applied only to loadpoints with the
demandtemperaturefeature.Formula:
load[i] = load_avg[i] × clamp((T_room − T_forecast[i]) / (T_room − T_hist_avg[h]), 0.5, 2.0)Where:
T_room = 21°C(fixed reference)T_hist_avg[h]= average of all historical temperature readings at hourhfrom the weather tariff data[0.5, 2.0]to prevent extreme corrections from bad dataGate: If the forecast temperature is ≥ 18°C, that slot's load is zeroed (heating assumed off).
Safety skip: If the historical average denominator
(T_room − T_hist_avg)is < 0.5°C in absolute value (historical average ≈ room temperature, indicating the heating was never running at that hour), the slot is left uncorrected.Example: With historical average of 8°C at a given hour:
Configuration
Fully opt-in — no changes needed for existing setups.
Add the appropriate feature flag to the charger in
evcc.yaml:The
demandtemperaturecorrection requires a temperature tariff (TariffUsageTemperature) to be configured — e.g. the open-meteo weather tariff from #27780. Without it, the profile is used without scaling according to outdoor temperature forecast.Dependencies
TariffUsageTemperature) to be merged — this also removes the bulk of the file diff in this PRapi.Heatingfeature flag for loadpoint identificationapi.DemandTemperatureandapi.DemandWeekdayfeature flags (renamed fromPredictorProfileTemperature/PredictorProfileSameWeekday)Details for how PredictorProfileTemperature (a.k.a "outdoortemperaturesensitive") works
Base PR: #27780 (Weather Tariff)
This PR: Heater Profile Separation for Temperature Correction
note: this PR doesn't change the database schema. It is based on, and uses the "history for meters" work merged via #23185.
TL/DR
With this PR optimizer forecasts for the household load are significantly improved when you have a heatpump device registered in evcc.
Before that they were barely useable because:
evidence of the effects of this PR: #28232 (comment)
explanation of the mathematical function to estimate corrections: #28232 (comment)
Problem
The evopt energy optimizer uses a household load profile (
gt) to predict how much energy the home will consume in each future 15-minute slot. This profile is currently computed as a 30-day historical average — the same flat pattern is repeated regardless of weather conditions.This is a significant blind spot: heating and cooling loads are strongly temperature-dependent. On a cold winter day, a home with a heat pump or electric heating can consume 30–80% more energy than on a mild day. When the optimizer doesn't know this, it:
Important: The evcc optimizer is a mathematical solver that optimizes EV charging and battery schedules based on given inputs (solar forecast, grid prices, household load). It cannot forecast household load itself—it requires accurate future household load predictions as input. Without temperature-corrected heat pump load forecasts, the optimizer receives inaccurate household demand predictions, leading to suboptimal charging decisions that conflict with actual heating needs.
Solution Overview
This PR builds on #27780 (which added the
open-meteoweather tariff) and implements temperature-based correction of the household load profile with a critical improvement: the correction is applied only to heating device loads that are explicitly marked as temperature-sensitive.Key Innovation: Selective Temperature Correction
Not all heating devices have temperature-dependent loads. For example:
This PR implements a four-step process:
OutdoorTemperatureSensitivefeature flagNote on Historical Data Periods:
Temperature Correction Algorithm
The correction algorithm uses a physics-based model that relates heating load to the temperature difference between indoor and outdoor conditions:
which becomes:
where:
T_room= 21°C (constant room temperature)T_past_avg[h]= average temperature at hour-of-dayhover the past 7 daysT_forecast[i]= forecast temperature at the wall-clock time of slotiThis formula models heating load as proportional to the temperature difference that must be maintained. The correction factor represents the ratio of future heating demand to historical average demand.
Safety Check: If a forecast slot is > 18°C (indicating heating was likely turned off), the correction is skipped for that time slot.
Clamping: Correction factors are clamped to the range [0.5, 2.0] to prevent extreme corrections from bad data.
Example: With a 7-day historical average of 8°C at a given hour:
Configuration
The feature is fully opt-in — no changes needed for existing setups. Temperature correction is only active for heating devices explicitly marked as temperature-sensitive.
Basic Setup (requires base PR #27780)
Configuration Details
Temperature Tariff (required for correction):
Heating Device Features:
heating- Marks the device as a heating system (required for all heating devices)outdoortemperaturesensitive- Enables temperature correction for this specific device (optional)Important:
heatingfeature will have their consumption included in forecasts but without temperature correctionheatingandoutdoortemperaturesensitivewill have temperature correction appliedMultiple Heater Support
The implementation fully supports multiple heating devices with selective correction:
api.Heatingfeature are automatically identifiedapi.OutdoorTemperatureSensitiveget temperature correctionThis approach ensures that:
Dependencies
api.Heatingfeature flag for device identificationapi.OutdoorTemperatureSensitivefeature flag for selective correction