Skip to content

Commit 901e832

Browse files
wormeymanclaude
andauthored
docs: the container never slept - the ~8.5min tail was a backfill artifact (#120) (#155)
#120 published an "~8.5 minute" idle tail from `containersUsageAdaptiveGroups`, read ~14 minutes after the test render that produced it. Re-read now that the dataset has settled, the same window tells a different story: the placement that render woke (`a20877a2`, 2026-08-03T22:35Z) was still allocated 29.3 hours later - 352 of 352 five-minute buckets present, zero gaps, 98.1% of a 1 GiB instance's full allocation - against a total of 5 worker requests in that entire window. There is no tail. The container simply never sleeps, which is exactly what the SIGTERM diagnosis in #154 predicts: `@cloudflare/containers` sends SIGTERM and never escalates, node is PID 1 and ignored it, so the instance only ever goes away when a deploy replaces the placement. Two claims in CLAUDE.md were wrong as a result, and both were cost claims: - "That cost ~$28/month for weeks" implied the drain fix ended it. Billing says the instance ran at 100% of a 4 GiB day on EVERY full day from 2026-07-20 through 2026-08-03 (95.3 - 99.0 GiB-hours/day against the 96.0 a 4 GiB instance bills for 24h), including the five days after that fix deployed on 2026-07-29. The bill was still being paid; the 2026-08-03 downsize to `basic` cut it ~4x rather than ending it. The drain guard stays - the hazard is real - but it was never the load-bearing cause. - The disk-to-memory ratios "2.0" and "4.0" are dashboard units. Queried in bytes, which is what the GraphQL dataset returns, they read 1.86 and 3.73. Adds the SIGTERM cause and the backfill trap as their own bullets, so the next person reading absence-of-bucket as sleep has the rule in front of them. Measured via the `containersUsageAdaptiveGroups` and `workersInvocationsAdaptive` GraphQL datasets on account 3e467e5d..., 2026-08-05. NOT a behaviour change: the fix itself is 343df53, already on main and NOT yet deployed. Production still runs worker version 2e67470f and container app version 3 (image sha256:16205eda..., updated 2026-08-03T21:24:01Z), both of which predate the handler. Verified: pnpm run verify green, exit 0 - 1588 app tests passed / 3 skipped, 13 worker, 4 container (including "exits promptly on SIGTERM"), 0 warnings. Claude-Session: https://claude.ai/code/session_01QZvNS2H4cbaZk46ybA7hbj Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent d3248f9 commit 901e832

1 file changed

Lines changed: 37 additions & 5 deletions

File tree

CLAUDE.md

Lines changed: 37 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -758,16 +758,48 @@ before changing it:
758758
decrements its inflight-request counter when a proxied response body finishes
759759
piping. Dropping a response without reading it - which the 502 path used to do
760760
- leaves the counter above zero, `isActivityExpired()` returns false forever,
761-
and the instance never sleeps. That cost ~$28/month for weeks. Any new code
762-
path that discards a container response **must drain it first**; the guard is
763-
in `preview-service/worker/test/worker.spec.ts`.
761+
and the instance never sleeps. Any new code path that discards a container
762+
response **must drain it first**; the guard is in
763+
`preview-service/worker/test/worker.spec.ts`.
764+
765+
**That drain fix did not, on its own, stop the container being awake 24/7, and
766+
a note here used to imply it had.** Billing says the instance ran at 100% every
767+
full day from 2026-07-20 through 2026-08-03 - 95.6, 96.2, 98.2, 96.3, 96.0,
768+
96.8, 95.6, 97.0, 95.3, 96.1, 95.7, 96.3, 99.0 GiB-hours/day against the 96.0
769+
a 4 GiB instance bills for a whole day - including the five days _after_ the
770+
drain fix deployed on 2026-07-29. So the ~$28/month was still being paid; the
771+
2026-08-03 downsize to `basic` cut it ~4x rather than ending it. The sufficient
772+
explanation is the SIGTERM bug in the bullet below, which was present
773+
throughout. Keep the drain guard - the hazard is real - but do not credit it
774+
with the bill.
775+
776+
- **The container ignored SIGTERM, so it never stopped at all** (#120). Node runs
777+
as **PID 1** under the Dockerfile's exec-form `ENTRYPOINT`, and Linux gives PID
778+
1 no default signal dispositions. `@cloudflare/containers` stops an idle
779+
instance by sending SIGTERM and **never escalating to SIGKILL**, so with no
780+
handler the stop request was silently discarded and the instance only ever went
781+
away when a deploy replaced the placement. The handler and its regression test
782+
live in `preview-service/container/server.mjs` and `test/shutdown.test.mjs`.
783+
764784
- **To check what is actually running, read the billing metrics, not
765785
`wrangler containers instances`.** That command reported `state: running` with
766786
an 80-minute-old `created` timestamp during an hour when allocation was zero -
767787
it describes the placement, not whether you are paying. The
768788
`containersUsageAdaptiveGroups` GraphQL dataset is the truth, and the
769-
disk-to-memory ratio identifies the live instance type: **2.0** = `standard-1`
770-
(4 GiB / 8 GB), **4.0** = `basic` (1 GiB / 4 GB). See #120 for the idle tail.
789+
disk-to-memory ratio identifies the live instance type. Read it in **bytes**
790+
and the ratio is `1.86` = `standard-1` (4 GiB / 8 GB) and `3.73` = `basic`
791+
(1 GiB / 4 GB); the **2.0** and **4.0** this note used to quote are those same
792+
two numbers expressed in the mixed GiB/GB units the dashboard shows.
793+
794+
- **That dataset BACKFILLS, and a bucket that has not landed yet is
795+
indistinguishable from sleep.** This is not hypothetical: #120 read the
796+
5-minute buckets ~14 minutes after a test render, found nothing past 22:45Z,
797+
and published an "~8.5 minute" idle tail. Re-read once settled, that same
798+
window has **every** bucket present and the placement it woke stayed allocated
799+
for **29.3 hours straight - 352 of 352 buckets, no gaps** - on a total of
800+
**5** worker requests. The tail was never 8.5 minutes; there was no tail.
801+
Wait at least an hour before reading absence as sleep, and confirm with
802+
`placementId` continuity rather than bucket presence alone.
771803

772804
`wrangler` is not global - drive it through the workspace:
773805
`pnpm --filter @fmw/preview-worker exec wrangler <cmd>`.

0 commit comments

Comments
 (0)