Commit e898504
monitor: calibrate staleness detection to the 3h scan cadence
The map stopped moving at 2026-08-28T00:29Z and every signal this
repository publishes stayed green for the next seven hours: all seven
workflow badges passing, no alert issue, no red run. That is the exact
silent-failure mode health.yml was written to prevent, and it did not
fire because the monitor is calibrated to a round number instead of to
the cadence of the thing it monitors.
The arithmetic: the engine publishes every 3h, the monitor runs every
12h ("43 6,18 * * *") against a 12h threshold. Data that freezes just
after a check is 11.9h old at the next one — under the limit — and is
not reported until the check after that. Worst-case detection latency
is ~24h for a pipeline whose whole claim is a 3h refresh.
Three changes, no new dependencies:
* MAX_AGE_HOURS 12 -> 7, overridable via RADAR_MAX_AGE_HOURS. One
dropped scheduled run is normal on GitHub and must not page anyone;
two consecutive misses is a stalled pipeline. 7h is 6h of missed
scans plus grace for engine runtime and sync lag.
* health.yml cron "43 6,18 * * *" -> "43 */3 * * *", still offset from
the engine (:17) and the sync (:37). The check reads two small JSON
files over HTTPS; running it eight times a day costs seconds on a
public repository and cuts worst-case detection from ~24h to ~10h.
* sync.yml gains a freshness readout after the contract gate. The gate
proves the payload is well-formed; it cannot prove it is recent. A
frozen engine keeps serving last night's valid JSON, the sync finds
nothing to commit and exits 0. The readout prints the payload age on
every run and raises a ::warning:: past the threshold, so a stalled
upstream is visible in the run log itself rather than only in an
issue that may not open for hours.
This does not fix the outage. The sync path is healthy: it pulled the
00:24Z payload, the contract gate passed, and there has been nothing new
upstream since. The engine (private) is what stopped publishing. This
change is about the second defect the outage exposed — that nothing here
said so.1 parent 50e67bf commit e898504
3 files changed
Lines changed: 40 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
11 | 16 | | |
12 | 17 | | |
13 | 18 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
81 | 81 | | |
82 | 82 | | |
83 | 83 | | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
84 | 110 | | |
85 | 111 | | |
86 | 112 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
10 | 12 | | |
11 | 13 | | |
12 | 14 | | |
| |||
32 | 34 | | |
33 | 35 | | |
34 | 36 | | |
35 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
36 | 42 | | |
37 | 43 | | |
38 | 44 | | |
| |||
0 commit comments