Using recommended recorder values breaks non predbat recording #5507

Activity

1. jparnellx

   assigned

   springfall2008

   on Oct 12, 2026
2. springfall2008 commented on Oct 12, 2026

   

   springfall2008

   on Oct 12, 2026

   Owner

   More actions

   Automated first-pass triage — a maintainer will review before any action is taken.

   Classification: bug (in Predbat's documentation) · Priority: priority_medium

   Bug statement: the recorder fix Predbat's docs and the #5485 thread recommend — adding include: entity_globs: - predbat.compare_tariff_* — turns Home Assistant's recorder into a whitelist, so every entity not matching an include rule (your thermostat, inverter, energy metres, …) silently stops being recorded and its long-term statistics break.

   What I verified (against current main, v9.3.7-21-g92478319, which is the shipped v9.3.8):

   • The Compare tab reads its day-by-day history back from HA's recorded history of the predbat.compare_tariff_<id> sensors — apps/predbat/compare.py:646 (self.prefix + ".compare_tariff_" + tariff_id, prefix defaults to predbat at apps/predbat/predbat.py:366); docs/compare.md:150 states the same.
   • Home Assistant's recorder filter (homeassistant/components/recorder/filters.py, _generate_filter_for_columns, which by its own docstring must "match exactly how homeassistant.helpers.entityfilter works") has two modes:
     • Excludes only (domains/globs in exclude:, plus plain include.entities entries, which win over excludes): everything not matching an exclude is still recorded. This is how your thermostat kept recording with the FAQ example before the change, while the predbat.compare_tariff_* sensors were dropped — they match the exclude: domains: - predbat rule — which is exactly the Compare only contains previous day #5485 symptom.
     • Any include filter (domains, entity_globs, or an include-only config): an entity matching no include rule falls through to exclude — the rule table you quoted, ending "Otherwise: exclude". The moment the glob went in, recording became: compare sensors yes (glob include), everything else no.
   • Your timeline matches this mode-flip step for step: (1) FAQ example without the glob → excludes-only mode → thermostat recorded, compare sensors not (Compare only contains previous day #5485); (2) glob added under include: → include-whitelist mode → compare sensors rescued, everything else silently un-recorded (this issue); (3) whole block removed → no filter → everything recorded (your current state, and why removing it fixed both).

   Root cause: docs/compare.md:151–152 and the FAQ recorder example in docs/faq.md ("exceed maximum size" item) — the entity_globs entry was added by #5487 for #5485 and shipped in v9.3.8 — recommend the glob without warning that defining any include filter changes the recorder's default from "record everything not excluded" to "record only what matches an include rule". The FAQ's existing caveat ("pay attention to the order in which inclusion and exclusion rules are applied") covers rule ordering, not this default-flip.

   The docs correction needs to say three different things to three different setups:

   1. No recorder: filter at all (your state since removing the config): add nothing — with no filters HA records everything, compare sensors included, and the Compare tab works. This is also the config you can just keep.
   2. Already running an include whitelist (your include: is the only meaningful filter section): add the glob to the existing include block — harmless there, because everything else was already whitelisted.
   3. Running the exclude-based FAQ example and want the rest of the house to keep recording: don't add the include glob. List the compare sensors explicitly under include.entities instead — plain entity entries are the one include form that can sit alongside domain excludes while keeping the "everything not excluded is recorded" default (verified against the recorder filter source: no domain/glob includes → excludes-plus-entity-includes mode):

   recorder:
     exclude:
       domains:
         - predbat
         - predheat
       entities:
         - sensor.predbat_load_ml_forecast
         # ... rest of the FAQ exclude list ...
     include:
       entities: # one entry per tariff id from apps.yaml compare_list — no entity_globs here
         - predbat.cost_today
         # ... rest of the FAQ include list ...
         - predbat.compare_tariff_<id>

   (The include.entities entries win over the predbat domain exclude, so the compare sensors stay recorded; the domain excludes keep doing their DB-size job.)

   Nothing further needed from you — your current no-filter config is safe for both features. Note the long-term statistics gap for the affected entities can't be back-filled, but HA resumes recording and long-term statistics automatically once the entities are recorded again.

   Related: #5485 (the original missing-Compare-history report whose fix created this trap), #2760 (why the FAQ example excludes the predbat domain at all — the compare entity attributes inflating the recorder DB). Longer term, the durable fix would be for the Compare tab to keep its history in Predbat's own storage rather than reading it back from the recorder — that removes the recorder-config dependency entirely, though it's a bigger change and the maintainer's call.
3. springfall2008

   added

   bugSomething isn't workingSomething isn't working

   documentationImprovements or additions to documentationImprovements or additions to documentation

   priority_medium

   BOT_TRIAGEDHas been through the triage botHas been through the triage bot

   on Oct 12, 2026
4. jparnellx commented on Oct 13, 2026

   

   jparnellx

   on Oct 13, 2026

   Last edited by gcoan

   Author

   More actions

   Confirmed that removing the glob include and specifying the individual tariffs resolves the issue.
   I now have history for non predbat stats and compare works

   recorder:
     exclude:
       domains:
         - predbat
         - predheat
       entities:
         - sensor.predbat_load_ml_forecast
         # ... rest of the FAQ exclude list ...
     include:
       entities: # one entry per tariff id from apps.yaml compare_list — no entity_globs here
         - predbat.cost_today
         # ... rest of the FAQ include list ...
         - predbat.compare_tariff_<id>
   
5. gcoan commented on Oct 13, 2026

   

   gcoan

   on Oct 13, 2026

   Collaborator

   More actions

   @jparnellx I think what you need is a slight variant of that, you shouldn't have to manually include every tariff id:

   recorder:
     exclude:
       domains:
         - predbat
         - predheat
       entities:
         - sensor.predbat_load_ml_forecast
         # ... rest of the FAQ exclude list ...
     include:
       entity_globs:
       # include the compare tariff entities
         - predbat.compare_tariff_*
       entities:
         - predbat.cost_today
         # ... rest of the FAQ include list ...
         - ...
   

   and btw, top tip, to include code like apps.yaml extracts put three back ticks before and after the text like this ```