avail-smg-pyramid spec: increment 8 (unpivot crash fix + polars-drift hazard)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GgLAzeRJeNeyizG4XHWRBK

Author: ryan-williams

specs/avail-smg-pyramid.md

@@ -131,6 +131,7 @@ Engine-vs-source parity, checked end-to-end through the deployed worker on 2026-
 5. ✅ **Cadence**: `gbfs-compact.yml` self-heal + `-f` fill; `/health` lists `smg-v1` (`HEALTH_PYRAMIDS` in `gbfs/api/src/health.ts`) and `/api/files` browses `smg-v1/`.
 6. ✅ **Base image bump** (2026-09-07): pyrmts `22a5f32` (= `72f2552` open periods + `specs/done/engine-fargate-mem-budget.md` + fill-mode deferral of shards over an absent open tile, addendum in `specs/done/engine-open-period-source.md`) → `pyrmts-engine:22a5f32` (arm64) → `ctbk-engine:81274948` (job-def rev 14); ctbk's `pyproject.toml` pins re-locked to `22a5f32` so the GHA submitter defaults `-b`. Validation on the shared job definition: uncapped `smg-v1 -f` and `avail-v6 -f` defer the trailing rungs over today (nothing written, exit 0); rides-v5 `-f` capped at the last published month is a no-op. `gbfs-compact.yml`'s fill step dropped `-b 20g` and the `-r` cap.
 7. ✅ **Closed-period fail-fast** (2026-09-07): pyrmts `a174254` (addendum 2 in `specs/done/engine-open-period-source.md`) extends the pre-walk check to every tile the fill would read, via a new `TiledSource.present_keys(tiles)` hook: shards over an absent *closed* tile are held and, past `--max-missing`, the fill exits 4 with **nothing written** — the guard that used to fire after the 0-row shards had landed. `MonthlyRidesSource.present_keys` answers from `available_months` (its tiles are on S3; the chassis default LISTs the pyramid's R2, which only got the open-month answer right by accident). → `pyrmts-engine:a174254` → `ctbk-engine:1cc7b912` (job-def rev 15). Validation on the real prefixes, R2 listings diffed before/after: `smg-v1 -f` 42 deferred / 0 held / exit 0; `avail-v6 -f` 1 deferred / exit 0; uncapped `rides-v5-{start,end} -f` → `fill: 8 held (closed-period source absent: normalized/202608.parquet …)` + 4 deferred over `202609`, exit 4, **no new keys** on either anchor (previously 12 zero-row relics each + a sweep).
+8. ✅ **`unpivot` crash fix + polars drift** (2026-09-08): the first nightly fill to BUILD (not defer) smg-v1 shards crashed (`gbfs-compact` exit 1). `SmgDailySource.parse` unpivoted with `value_name='state'` — same as an `on` column — which polars ≥ 1.44 rejects (`DuplicateError`); 1.41 tolerated it. The base image installs polars unpinned, so the `a174254` rebuild drifted to 1.44 (pyarrow 25) while local tests + the original backfill ran 1.41. Fixed the source (unpivot to `_state`, alias to `state`; commit `cedbf622` → `ctbk-engine:cedbf622`, job-def rev 16), verified under both polars versions; healed prod with a Batch fill (smg-v1 237 → 250 keys, 13 shards for 2026-09-06/07). Root hazard (image deps unpinned, so local ≠ prod) recorded in memory `engine-image-unpinned-deps`; pinning proposed, not yet done.
 
 ## Storage / cost