fix(cron): freshness gate uses the value source's own observation time, judged on age alone (codex P2 x2)

- Latest.observedAt = inverter collectionTime when the inverter's live points override the station fields, else station lastUpdateTime (toSec normalises s/ms); null when the source gave none — updatedAt stays display-only
- freeze.ts: frozen = observedAt missing OR now-observedAt > STALE_AFTER_S; no monotonic marker (a 4-day-old reading that advances 5 min is still 4 days old); last_reading_ts dropped
- tests updated (105 total)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Author: THUNDER PHUM

.inbox/deyecloud/codex/review-088cb16.md

@@ -0,0 +1,38 @@
+# Review 088cb16 — HOLD (2 P2)
+
+Reviewed incident guard, health reporting, operator purge and adjacent ingestion/recovery paths. No application source, production data, branch, deploy or purge was changed. Existing main untracked `.mcp.json.bak-httptest` preserved.
+
+## P2: Gate on the timestamp of the data actually persisted
+
+Location: `src/worker/index.ts:167` (integration with `src/worker/deye.ts:390-408`).
+
+`Latest.updatedAt` always comes from station `lastUpdateTime`, or server `now` when absent. But `getLatestOpen` then overwrites watts/SOC with `/device/latest` values and never carries their `collectionTime` into that timestamp. The new guard therefore checks a different data source from the measurements it admits/rejects.
+
+Actual `getLatest` -> `pollAndStore` -> in-memory SQLite reproductions at 2026-09-22 17:20 BKK:
+
+- Station timestamp absent; inverter timestamp four days old with 2,631 W: writes a current `samples` row and `device_samples` row, deletes `frozen_since`, health healthy.
+- Fresh station timestamp; same four-day-old inverter: same fabricated current rows and false recovery.
+- Four-day-old station timestamp equal to marker; fresh inverter reports 4,000 W: drops the genuinely fresh sample and telemetry, remains frozen.
+
+Carry trustworthy observation provenance from the selected source(s), retaining unknown as unknown instead of substituting poll time for freshness evidence. If values can be mixed between sources, do not label the whole bundle fresh merely because one timestamp is fresh. Test both divergence directions and station timestamp absent. Normalize timestamp units at the provenance boundary; a synthetic millisecond station timestamp also bypassed the current seconds-based age guard.
+
+## P2: Reject known-stale observations even without a prior marker or when they advance
+
+Location: `src/lib/freeze.ts:13`.
+
+Known age >720 seconds is not sufficient to reject: rejection additionally requires a non-null stored marker and `readingTs <= lastStoredReadingTs`. Consequently:
+
+- First poll after deploying this new metadata key accepts a four-day-old reading, stamps it as current in `samples`/`device_samples`, and clears frozen state.
+- A delayed replay that advances from four days old to four days old +5 minutes also writes current rows and clears frozen state. A stream of delayed observations can continue doing this on each tick without ever being fresh.
+
+Both reproduced through the real worker and SQLite, not just the helper. Treat freshness separately from monotonic advancement/deduplication: a known-old observation cannot be written under today's poll timestamp or prove recovery. If historical ingestion is desired, use the original observation time in an explicit history path. Test missing marker + stale observation, advancing-but-stale observation, and genuinely fresh recovery.
+
+## Verified / non-blocking
+
+- Fresh run: 105/105 tests (14 files), typecheck, build, and `git diff --check` passed. Existing tests cover only the helper, not source composition or the purge route.
+- Established frozen station+device timestamps: no sample or telemetry written; repeated frozen ticks keep the first `frozen_since`; `_health` remains collector-healthy. Genuinely fresh recovery clears the marker and writes.
+- Self-heal is reached on genuine recovery. Existing cap is **2 days**, so a four-day gap needs explicit older-history backfill if wanted. It did not fabricate frames when the mocked history was empty.
+- Actual Hono purge endpoint: no login ->401; no configured PIN ->403; reversed or >31-day range ->400; default dry-run leaves data untouched; confirmed purge deletes only inclusive boundaries in both tables. Day/battery caches cleared; daily and year cache preserved.
+- No production purge executed. After deploying a corrected admission policy, dry-run and confirm exact incident bounds; especially preserve the last valid reading / first real recovery sample if endpoints are inclusive.
+
+Reproducer: `/home/l22/Desktop/deyecloud/.tmx-worktrees/codex/.summary/review-088cb16.cjs` (run with Node; transpiles actual TypeScript in memory, intercepts upstream requests, uses SQLite `:memory:`; external network forbidden).

src/lib/__tests__/freeze.test.ts

@@ -2,15 +2,15 @@ import { describe, it, expect } from "vitest";
 import { isFrozenReading } from "../freeze";
 
 const NOW = 1_790_000_000, STALE = 720;
-describe("isFrozenReading", () => {
-  it("first ever poll: nothing stored → not frozen", () => expect(isFrozenReading(NOW - 60, null, NOW, STALE).frozen).toBe(false));
-  it("reading advanced → not frozen even if old-ish", () => expect(isFrozenReading(NOW - 600, NOW - 900, NOW, STALE).frozen).toBe(false));
-  it("same reading, still within the stale window → not frozen yet (one slow upload is fine)", () => {
-    expect(isFrozenReading(NOW - 600, NOW - 600, NOW, STALE).frozen).toBe(false);
+describe("isFrozenReading — freshness of the value source, independent of monotonicity", () => {
+  it("fresh reading → stored", () => expect(isFrozenReading(NOW - 60, NOW, STALE).frozen).toBe(false));
+  it("exactly at the window edge → still stored", () => expect(isFrozenReading(NOW - STALE, NOW, STALE).frozen).toBe(false));
+  it("4-day-old reading → frozen, even if it 'advanced' by 5 min since last poll", () => {
+    const r = isFrozenReading(NOW - 4 * 86400 + 300, NOW, STALE);
+    expect(r.frozen).toBe(true); expect(r.reason).toContain("min ago");
   });
-  it("same reading and older than the window → FROZEN", () => {
-    const r = isFrozenReading(NOW - 3600, NOW - 3600, NOW, STALE);
-    expect(r.frozen).toBe(true); expect(r.reason).toContain("60 min");
+  it("no observation timestamp → cannot prove freshness → frozen", () => {
+    expect(isFrozenReading(null, NOW, STALE).frozen).toBe(true);
+    expect(isFrozenReading(0, NOW, STALE).frozen).toBe(true);
   });
-  it("no device timestamp → cannot judge → not frozen", () => expect(isFrozenReading(0, NOW - 3600, NOW, STALE).frozen).toBe(false));
 });

src/lib/__tests__/pointValue.test.ts

@@ -1,5 +1,5 @@
 import { describe, it, expect } from "vitest";
-import { pointValue } from "../../worker/deye";
+import { pointValue, toSec } from "../../worker/deye";
 
 describe("pointValue — a reading is a finite number, nothing else", () => {
   it("absent forms → undefined (never 0)", () => {
@@ -10,3 +10,11 @@ describe("pointValue — a reading is a finite number, nothing else", () => {
     expect(pointValue("79")).toBe(79); expect(pointValue(" -107.8 ")).toBeCloseTo(-107.8);
   });
 });
+
+describe("toSec — Deye timestamps in s or ms, absent → null", () => {
+  it("normalises", () => {
+    expect(toSec(1_790_000_000)).toBe(1_790_000_000);
+    expect(toSec("1790000000123")).toBe(1_790_000_000);
+    expect(toSec("")).toBeNull(); expect(toSec(0)).toBeNull(); expect(toSec(undefined)).toBeNull();
+  });
+});

src/lib/freeze.ts

@@ -1,17 +1,18 @@
-// Frozen-reading detection — pure, unit-tested.
+// Reading-freshness gate — pure, unit-tested.
 //
-// When the site's data logger drops off Deye Cloud, /station/latest keeps
-// answering with the LAST reading it has (same watts, same SOC, same
-// lastUpdateTime) for hours or days. Storing that every 5 minutes fabricates a
-// flat 4-day "production curve" (observed 2026-09-18→22: PV 2,631 W × 96 h).
-// A reading is only new evidence if its device timestamp advanced past the one
-// we last stored; if it hasn't and it is older than the stale window, the poll
-// carries no information about the present and must not be written as a sample.
+// When the site's data logger drops off Deye Cloud, /station/latest and
+// /device/latest keep answering with the LAST reading they have (same watts,
+// same SOC, same timestamp) for hours or days. Storing that every 5 minutes
+// fabricates a flat multi-day "production curve" (observed 2026-09-18→22:
+// PV 2,631 W × 96 h). A reading is evidence about NOW only if the timestamp of
+// the source that produced its values is inside the stale window — freshness is
+// judged on its own, never on whether the timestamp moved since last time (a
+// 4-day-old reading that "advances" by 5 min is still 4 days old), and a missing
+// timestamp cannot prove freshness at all.
 export interface FreezeCheck { frozen: boolean; reason: string | null }
-export function isFrozenReading(readingTs: number, lastStoredReadingTs: number | null, nowS: number, staleS: number): FreezeCheck {
-  if (!Number.isFinite(readingTs) || readingTs <= 0) return { frozen: false, reason: null }; // no device timestamp → cannot judge, keep old behaviour
-  if (lastStoredReadingTs != null && readingTs <= lastStoredReadingTs && nowS - readingTs > staleS) {
-    return { frozen: true, reason: `Deye reading unchanged since ${new Date(readingTs * 1000).toISOString()} (${Math.round((nowS - readingTs) / 60)} min)` };
-  }
+export function isFrozenReading(observedAt: number | null, nowS: number, staleS: number): FreezeCheck {
+  if (observedAt == null || !Number.isFinite(observedAt) || observedAt <= 0) return { frozen: true, reason: "source gave no observation timestamp" };
+  const age = nowS - observedAt;
+  if (age > staleS) return { frozen: true, reason: `reading observed ${Math.round(age / 60)} min ago (${new Date(observedAt * 1000).toISOString()})` };
   return { frozen: false, reason: null };
 }

src/worker/deye.ts

@@ -267,6 +267,8 @@ export async function getStationId(env: Env): Promise<string> {
 // A measure-point / field value is a reading only if it is a finite number.
 // null, "", "  ", "N/A" are ABSENT — Number("") is 0, which is how a missing SOC
 // used to become a fake 0 % reading.
+// Deye timestamps arrive in seconds or milliseconds depending on endpoint.
+export const toSec = (t: unknown): number | null => { const v = pointValue(t); if (v == null || v <= 0) return null; return v > 1e12 ? Math.floor(v / 1000) : Math.floor(v); };
 export function pointValue(raw: unknown): number | undefined {
   if (raw == null) return undefined;
   if (typeof raw === "string" && raw.trim() === "") return undefined;
@@ -282,6 +284,11 @@ export interface Latest {
   // false when Deye's day-history call failed: the *Today fields are then unknown
   // (0 placeholders) and must not be persisted as the day's totals.
   totalsOk: boolean;
+  // Observation time (unix s) of the SOURCE whose power values this record
+  // carries: the inverter's collectionTime when its live points overrode the
+  // station fields, else the station's lastUpdateTime. null = the source gave no
+  // usable timestamp — the reading's freshness is then unprovable.
+  observedAt: number | null;
   // false when neither the station nor the inverter reported SOC — soc is then a
   // 0 placeholder for the UI and must be persisted as NULL, never as a reading.
   socKnown: boolean;
@@ -387,7 +394,8 @@ async function getLatestOpen(env: Env, stationId?: string): Promise<Latest> {
     warningStatus: "NORMAL",
     totalsOk,
     selfSufficiency: selfSuff,
-    updatedAt: n(d.lastUpdateTime) || Math.floor(Date.now() / 1000),
+    updatedAt: n(d.lastUpdateTime) || Math.floor(Date.now() / 1000), // display only — never a freshness proof
+    observedAt: toSec(d.lastUpdateTime),
     raw: d,
   };
 
@@ -400,6 +408,8 @@ async function getLatestOpen(env: Env, stationId?: string): Promise<Latest> {
   try {
     const inv = await getInverterFlow(env, id);
     if (inv) {
+      // the stored power values now come from the inverter → so must the observation time
+      out.observedAt = inv.observedAt;
       if (inv.genPower != null) out.genPower = inv.genPower;
       if (inv.usePower != null) out.usePower = inv.usePower;
       if (inv.gridPower != null) { out.gridPower = inv.gridPower; out.gridStatus = inv.gridPower >= 0 ? "PURCHASE" : "REVERSE"; }
@@ -429,7 +439,7 @@ async function getInverterSn(env: Env, stationId: string): Promise<string> {
 async function getInverterFlow(
   env: Env,
   stationId: string
-): Promise<{ genPower?: number; usePower?: number; gridPower?: number; battPower?: number; soc?: number; genTotal?: number } | null> {
+): Promise<{ genPower?: number; usePower?: number; gridPower?: number; battPower?: number; soc?: number; genTotal?: number; observedAt: number | null } | null> {
   const sn = await getInverterSn(env, stationId);
   if (!sn) return null;
   const res = await deviceLatest(env, [sn]);
@@ -443,6 +453,7 @@ async function getInverterFlow(
     battPower: num("BatteryPower"),
     soc: num("SOC"),
     genTotal: num("TotalActiveProduction"),
+    observedAt: toSec(((res.deviceDataList && res.deviceDataList[0]) || {}).collectionTime),
   };
 }
 

src/worker/index.ts

@@ -163,13 +163,12 @@ async function pollAndStore(env: Env) {
   const day = bkkDay();
   // Frozen reading (logger offline, Deye re-serving its last value): record that
   // the poll itself succeeded, but write NO sample/daily — see lib/freeze.ts.
-  const lastReading = (await env.DB.prepare("SELECT v FROM meta WHERE k='last_reading_ts'").first()) as { v: string } | null;
-  const fz = isFrozenReading(Number(l.updatedAt) || 0, lastReading ? Number(lastReading.v) : null, Math.floor(Date.now() / 1000), STALE_AFTER_S);
+  const fz = isFrozenReading(l.observedAt, Math.floor(Date.now() / 1000), STALE_AFTER_S);
   if (fz.frozen) {
-    console.warn("poll: frozen reading — not stored:", fz.reason);
+    console.warn("poll: stale/frozen reading — not stored:", fz.reason);
     await env.DB.batch([
       env.DB.prepare("INSERT INTO meta (k,v) VALUES ('last_poll_ok',?) ON CONFLICT(k) DO UPDATE SET v=excluded.v").bind(String(ts)),
-      env.DB.prepare("INSERT INTO meta (k,v) VALUES ('frozen_since',?) ON CONFLICT(k) DO UPDATE SET v=v").bind(String(l.updatedAt)), // keep the FIRST frozen timestamp
+      env.DB.prepare("INSERT INTO meta (k,v) VALUES ('frozen_since',?) ON CONFLICT(k) DO UPDATE SET v=v").bind(String(l.observedAt ?? ts)), // keep the FIRST frozen timestamp
       env.DB.prepare("DELETE FROM meta WHERE k='last_poll_error'"),
     ]);
     // Alerts still run so the offline rule can fire from the device snapshot.
@@ -210,7 +209,6 @@ async function pollAndStore(env: Env) {
     env.DB.prepare("DELETE FROM meta WHERE k='last_poll_error'"),
     env.DB.prepare("DELETE FROM meta WHERE k='frozen_since'"),
     env.DB.prepare("INSERT INTO meta (k,v) VALUES ('last_poll_ok',?) ON CONFLICT(k) DO UPDATE SET v=excluded.v").bind(String(ts)),
-    env.DB.prepare("INSERT INTO meta (k,v) VALUES ('last_reading_ts',?) ON CONFLICT(k) DO UPDATE SET v=excluded.v").bind(String(Number(l.updatedAt) || ts)),
   ];
   if (l.totalsOk) stmts.push(
     env.DB.prepare(