Skip to content

Commit 1492f20

Browse files
committed
Count the cached tokens instead of throwing them away
The CLI reports cache_creation_input_tokens and cache_read_input_tokens separately from the fresh input count. The event schema already parsed both, and usageOf then dropped them on the floor. That mattered more than it looks. S1 to S4 share one session, so each stage resumes into the conversation the previous one built and sends most of its prompt as a cached read. The number this app recorded as a review's input usage was therefore a small fraction of what the model actually read, and there was no way to show how much the session chaining saved. Both counters now reach the stage rows and the review totals. Counted separately rather than folded into the input count, because a cached read costs a fraction of a fresh one and adding them together would overstate the price. The cost figure was always right, because it comes from the CLI's own total_cost_usd, which already accounts for caching. The fake CLI now reports non-zero cache counters, because a fixture that only ever produced zeros could not tell the difference between carrying these through and dropping them. Mutation checked: restoring the drop fails the test that follows the counters to the review row. Caching itself is the CLI's to do. What this app controls is whether consecutive stages share a session, and they do; the --resume flags that make it true are already asserted at the command line.
1 parent 68729ef commit 1492f20

18 files changed

Lines changed: 1798 additions & 10 deletions

docs/DECISIONS.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -567,3 +567,19 @@ verified evidence, in writing, here.
567567
- 2026-07-31 DECIDED (W6): the SSE stream takes a narrow watcher, not the
568568
manager. It reads a picture and listens for changes; nothing reachable from a
569569
route handler should be able to start or cancel a review.
570+
- 2026-07-31 FIXED: cached tokens are counted. The CLI reports
571+
`cache_creation_input_tokens` and `cache_read_input_tokens` separately, the
572+
event schema already parsed both, and `usageOf` then dropped them. A chained
573+
stage sends most of its prompt as a cached read, so the recorded input count
574+
was a small fraction of what the model actually read, and there was no way to
575+
show how much the session chaining saved. Both are now carried to the stage
576+
rows and the review totals. Counted separately rather than folded into the
577+
input count, because a cached read costs a fraction of a fresh one and adding
578+
them together would overstate the price.
579+
- 2026-07-31 NOTED: prompt caching itself is the CLI's, not this app's. What
580+
this app controls is whether consecutive stages share a session, and they do:
581+
S1 to S4 resume into one conversation, which is what lets the whole
582+
accumulated context be a cached read rather than a fresh send. S5 is
583+
deliberately excluded and pays full price for its prompt, which is the cost
584+
of an independent check and is worth it. The `--resume` flags that make this
585+
true are asserted at the command line by the scripted-pipeline test.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
ALTER TABLE `reviews` ADD `usage_cache_creation_tokens` integer DEFAULT 0 NOT NULL;--> statement-breakpoint
2+
ALTER TABLE `reviews` ADD `usage_cache_read_tokens` integer DEFAULT 0 NOT NULL;--> statement-breakpoint
3+
ALTER TABLE `stage_executions` ADD `cache_creation_tokens` integer DEFAULT 0 NOT NULL;--> statement-breakpoint
4+
ALTER TABLE `stage_executions` ADD `cache_read_tokens` integer DEFAULT 0 NOT NULL;

0 commit comments

Comments
 (0)