This repository was archived by the owner on Jun 8, 2026. It is now read-only.
Conversation
- Vendor model_prices_and_context_window.json + meta SHA at build time - Fix Opus 4.5/4.6/4.7 ~3x under-pricing inherited from prior seed - Add Opus floor + OpenAI cache-discount invariants at build time - Add `just desktop-refresh-pricing` recipe for weekly upstream sync - Preserve host-only overrides for fallback rows (big-pickle, *-default) and 3.5 Sonnet/3.5 Haiku/3.7 Sonnet/3 Opus/3 Haiku aliases LiteLLM omits - Document inline why Opus 4.x rows are pinned over LiteLLM (deviation) - Bump apps/desktop version 0.15.93 -> 0.15.94 Testing: - new fetch + invariant tests (17 cases): pass - existing agent-session-sync-service tests (26 cases): pass - full apps/desktop suite: 1938 tests pass - `pnpm build:agent-monitor --force` regenerates db.js with corrected Opus rows (15/75 input/output) and new fallback rows - `pnpm typecheck && pnpm lint`: clean Risks: - Vendored JSON growth (~50KB) — filtered to 5 vendor prefixes; sentinel + 20-row-floor checks block accidental near-empty refreshes - LiteLLM upstream regression caught by build-time invariants; refresh recipe re-runs the loader so a bad fetch never silently lands - HOST_ONLY_OVERRIDES intentionally diverge from LiteLLM for Opus 4.5-4.7 until upstream is corrected; comment includes follow-up trigger Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codex review of 5b8546d surfaced two P2 findings in scope for FEA-1431 (both about correctness of the pricing-source switchover for existing deployments): - HOST_ONLY_OVERRIDES now carries `claude-opus-4-2%` and `claude-sonnet-4-2%`. These shipped as host-curated defaults in 0.15.93 but are absent from LiteLLM's catalog; without overrides, sessions tagged with the legacy 4.2 model ids would fall through to the OpenCode `$0` fallback on a fresh DB. - One-shot legacy-row migration patched into the generated db.js. The startup top-up uses INSERT OR IGNORE, so existing users on 0.15.93 with mis-priced claude-opus-4-5/6/7 rows ($5/$25/0.5/6.25 — Sonnet rates) would keep the wrong price after upgrade. The new UPDATE rewrites only rows still matching the exact legacy bad tuple so user-edited rows are preserved. Third codex finding (GPT cache_write rate inconsistency between concrete `gpt-5-codex` and fallback `gpt-codex%`) intentionally deferred to FEA-1432, which owns OpenAI cache math. Codex's preferred fix (keep input-rate cache_write) is wrong: OpenAI publishes no cache-write surcharge — 0 is the correct value. FEA-1432 will zero the fallback to match. Testing: - 1834 + 104 = 1938 tests passing - typecheck + lint clean - pnpm build:agent-monitor regenerates db.js with both 4.2 aliases and the legacy Opus migration block Risks: - The legacy migration runs on every startup; it is a no-op once rows are corrected (or once a user has deliberately edited them away from the exact bad tuple). No perf concern. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add `cache_write_1h_per_mtok` 5th rate column to `model_pricing` so Anthropic's 5-minute and 1-hour ephemeral cache write tiers are priced separately. CREATE TABLE picks up the column for fresh DBs; an additive ALTER TABLE migration backfills existing DBs to input × 2.0 for every claude-* row currently at 0 (so upgraders are not stuck at $0 for 1h writes once parsers learn the split). - LiteLLM transformer (`fetch-litellm-pricing.mjs`) derives the new column: Anthropic = input × 2.0 (per Anthropic's rate card); OpenAI and Gemini = 0 (no analogous tier). For OpenAI rows the transformer also (a) clamps cache_read to input × 0.5 when LiteLLM's positive ratio falls outside [40%, 60%] (GPT-5 series ships at 10% upstream) and (b) zeroes cache_write — OpenAI has no cache-write surcharge. Zero cache_read values are preserved (they signal "no caching"). - Promote the FEA-1431 soft-warn OpenAI invariant to a hard build-time throw: gpt-* rows must have cache_read ≤ 55% of input AND cache_write = cache_write_1h = 0. Add Anthropic invariant: every priced claude-* row must have cache_write_1h ≥ input × 1.5. - HOST_ONLY_OVERRIDES regrown to 7-tuples. The gpt-codex% fallback pre-FEA-1432 carried cache_read=0.125 (10%) and cache_write=1.25 (a surcharge OpenAI does not bill). Now: input=1.25, output=10, cache_read=0.625 (50%), cache_write=0, cache_write_1h=0. Cursor, Copilot, and Anthropic fallbacks gain a 0 or input × 2.0 7th column. - Bump AGENT_SESSION_SYNC_SCHEMA_VERSION 1 → 2 with `cacheWrite1hTokens?: number | null` on SyncedAgentSessionTokenUsage. Cost compute in `estimateTokenUsageCostUsd` now sums all five components. v1 receivers tolerate the new field; legacy-shape removal tracked by FEA-1439. - Document the v1 parser limitation in token-usage.ts: Anthropic's response usage block does not split the cache-write tier (it is set by the request's cache_control.ttl), so v1 attributes every cache_creation_input_tokens to the 5-min bucket and synthesizes cache_write_1h_tokens = 0 downstream. Recovery of the per-tier split is tracked by FEA-1438. Testing: - pnpm -C apps/desktop test: 1949 pass / 0 fail (1845 tsx + 104 node) - pnpm -C apps/desktop typecheck: clean - pnpm -C apps/desktop lint: clean - pnpm -C apps/desktop build:agent-monitor: passes SQLite gate - New: test/openai-cache-math.test.ts (5 cases) — gpt-codex fallback shape, OpenAI cache_write contributes $0, Anthropic 1h billed at input × 2.0, 5-component cost sum, v1 backward compat. - Extended: test/fetch-litellm-pricing.test.ts adds Anthropic 1h derivation and OpenAI clamp/zeroing cases. - Extended: test/build-agent-monitor-pricing-invariants.test.ts pins the three hard FEA-1432 invariants on the merged steady state. Risks: - Upgrade DB migration backfills claude-* cache_write_1h via UPDATE conditional on the column currently being 0. Rows users manually set to 0 will be rewritten to input × 2.0; the conditional is the conservative trade-off (vs leaving every upgrader stuck at $0 for 1h cache writes once FEA-1438 lands per-tier parsing). - OpenAI cache_read clamping inflates effective billed cost for GPT-5 series sessions that currently use cached prompts at the upstream 10% rate — but that rate disagrees with OpenAI's published 50% discount. We treat published rate as truth. - Schema-version bump (1 → 2) follows the optional-field discipline, so a v1 relay receiver continues to accept the payload (the new field is ignored). Cloud-side hard removal tracked at FEA-1439. Deviations from plan: - Plan assumed FEA-1434 (schema v2) had landed first; it lives on a separate branch. We bump 1 → 2 instead of 2 → 3. The FEA-1434 merge will need to rebase its own bump to 3. - Plan step 8 boundary translation refers to a comment FEA-1434 added; since FEA-1434 is not in this branch, the equivalent forward-compat note now lives in this commit's contract file. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
FEA-1432's commit body referenced FEA-1438 in code comments as a placeholder for the follow-up that recovers the 5min/1h cache-write tier split from request correlation. The follow-up ticket was filed as FEA-1440. This commit substitutes the correct ticket id in all four comment sites. No behavior change. No tests touched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ant gap Claude reviewer flagged two findings on 5e17acf: Medium: apps/desktop/CLAUDE.md said the OpenAI cache discount check was "Soft-warn for now (FEA-1432 will tighten to a hard assertion once downstream cache math is fixed)" — but FEA-1432 IS that commit, and the invariant already throws. Also said "two invariants" when the build runs three (Opus floor, OpenAI cache, Anthropic 1h floor). Replaces the section with current behavior: three hard build-time assertions, explicit per- invariant bullet list. Low: the OpenAI invariant guard at build-agent-monitor.mjs:528 was `if (!pattern.startsWith("gpt-")) continue;` — the fetch transformer correctly treats o1- and o3- as OpenAI via isOpenAIKey(), but the build- time invariant would silently pass a future HOST_ONLY_OVERRIDES row for o1-pro or similar with a non-zero cache_write. Extends the guard to match isOpenAIKey()'s definition. Testing: - 28 pricing-specific tests still pass (build-agent-monitor-pricing- invariants + fetch-litellm-pricing + openai-cache-math suites) - pnpm build:agent-monitor regenerates db.js clean (SQLite gate PASS) - Pre-existing flaky integration tests (python3 health, shell path, symphony-loop wrappers) unrelated to pricing work — not regressions from this commit Risks: - None. CLAUDE.md is documentation; the invariant guard widening is protective and only changes behavior for theoretical future rows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
FEA-1431/1432 priced every Anthropic / OpenAI / Gemini model and synced
the per-session cost to the cloud, but the local Sessions UI never
read it back — the column showed "-" for every row. When a model had
no `model_pricing` match, the sidecar silently fabricated $0.00 so
unpriced models (local Ollama, OpenCode-hosted free tiers, custom
vendor IDs) looked identical to genuinely-zero sessions. This commit
surfaces the cost and turns the silent zero into a visible
diagnostic.
Highlights:
- Sidecar `routes/sessions.js` (patched by build-agent-monitor.mjs):
new `calculateSessionCostFea1433(sessionTokens, rules)` returns
`{ cost, unpriced_models, priced }`. When every model in a session
is unpriced, `cost` is `null` (not 0). The price-sort comparator
now anchors `null` to the end in both directions so diagnostic
rows stay visible without polluting an ascending sort. The session
detail handler attaches a `cost_breakdown` block with `priced`
per row so a future SessionDetail overlay (deferred — upstream
detail page is 39.7K, too heavy to overlay just for this) can
show "Estimated cost unavailable" inline.
- New sidecar route `GET /api/pricing/diagnostics/unpriced-models?limit=20`
returning the most-recently-seen distinct models in `token_usage`
that don't match any `model_pricing` LIKE pattern. Uses
`stmts.matchPricing` so a `claude-opus-4-%` rule covers every
dated variant without listing each.
- Sessions.tsx overlay: `session.cost == null` renders an em-dash
with a tooltip naming the unpriced model and pointing to
Settings → Pricing. `cost > 0` renders `fmtCost`; the genuine-zero
case still renders "-" so priced-but-unused sessions look
unchanged.
- Renderer Settings → Pricing sub-panel (`apps/desktop/src/renderer/
index.html`): lists unpriced models with an inline 5-rate form
(input, output, cache_read, cache_write 5min, cache_write 1h —
$/Mtok). Submits via the new `cost-pricing:add-rule` IPC.
- Two-file IPC split: `cost-pricing-client.ts` holds the Zod schema
+ HTTP client (no `electron` import → node:test can exercise it
directly); `cost-pricing-ipc.ts` registers ipcMain handlers.
fetchUnpricedModels + addPricingRule never throw — sidecar
unreachable returns null / { ok:false, error } so the renderer
degrades gracefully.
- Contract update (no schema bump per CLAUDE.md additive-optional
policy): `SyncedAgentSessionTokenUsage` gains optional
`priced?: boolean` and widens `estimatedCostUsd` to
`number | null`. Missing `priced` decodes as `true` for legacy
v1/v2 payloads. New `estimateTokenUsageCostBreakdown(row, rules)`
in agent-session-sync-service.ts returns
`{ priced, costUsd: number | null }`; the legacy
`estimateTokenUsageCostUsd` is preserved as a thin wrapper so
existing call sites (and the openai-cache-math regression tests)
keep working.
- types.ts overlay edits widen `Session.cost` to `number | null`
and add `unpriced_models?: string[] | null` so the React
component types match the new sidecar shape under
`noUncheckedIndexedAccess`.
Testing:
- pnpm -C apps/desktop typecheck: clean
- pnpm -C apps/desktop lint: clean
- pnpm -C apps/desktop build:agent-monitor: SQLite gate PASS,
patches re-apply on a clean .generated tree
- pnpm -C apps/desktop test: 1856 tests, 1856 pass on the second
steady-state run (1845 baseline + 11 new). Two pre-existing
parallel-run flaky tests under test/symphony-launcher.test.ts
occasionally surface; they exist on FEA-1432 main and are
unrelated to this change.
- New tests:
- test/cost-pricing-ipc.test.ts: 7 cases covering Zod rejection
before network IO, GET proxy + limit query param, PUT proxy
with defaults applied, sidecar-unreachable degradation, sidecar
HTTP-error surfacing.
- test/sidecar-cost-priced-flag.test.ts: 1 case requiring the
generated `pricing.calculateCost` and pinning the
`matched_rule: null` contract our session-list wrapper depends
on. Skips when `.generated/` doesn't exist yet (fresh checkout
before `pnpm build:agent-monitor`).
- test/agent-session-sync-service.test.ts: 3 added cases for
estimateTokenUsageCostBreakdown — priced=false on no match,
priced=true with correct USD math, and priced=true preserved
on zero-token rows (must not collapse to null).
Risks:
- `estimatedCostUsd` on the wire is now nullable. Legacy v1/v2
receivers that treated a missing field as 0 keep working
unchanged. Cloud-side receivers that strictly enforced
`number` will accept the optional `null` if their schema treats
the field as optional (it always was). Hard-typed receivers may
need a one-line widening — flagged in the contract comment.
- Sidecar `routes/sessions.js` patches anchor on the upstream
cost-assignment block. Future upstream bump that touches those
three lines will throw at build time with an actionable error
rather than silently regressing the diagnostic.
- Settings → Pricing form bypasses no client-side rate validation
beyond Zod (min 0, finite). A user can type an arbitrarily large
rate; that is the same trust model as the existing manual
upstream PUT /api/pricing endpoint.
Deviations from plan:
- Step 3 (Session detail breakdown UI) deferred. The upstream
SessionDetail.tsx is 39.7K; overlaying it just for a cache_write_1h
breakdown was disproportionate. The sidecar now emits the
`cost_breakdown` block on `GET /api/sessions/:id` so a future
overlay (or a smaller targeted edit) can land it without another
API change.
- Step 4 unpriced-models data source: planned via IPC bridge or
new sidecar route — chose the sidecar route per the plan's
preference ("keep logic close to the data").
Hindsight note on FEA-1432:
- The `openai-cache-math` regression suite still imports
`estimateTokenUsageCostUsd` directly. We keep that export as a
thin wrapper around `estimateTokenUsageCostBreakdown`, but a
follow-up could migrate those tests to the breakdown API and
drop the legacy wrapper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eeping Claude reviewer found 2 Medium + 2 Low on commit 66f38df. Medium #1 — Mixed-session cost was silently understated: calculateSessionCostFea1433 returns cost=null only when EVERY model is unpriced; sessions with some priced + some unpriced models returned a partial dollar total with no diagnostic signal. UI rendered the partial total via fmtCost(cost) with no asterisk, tooltip, or other indication. A session mixing claude-opus-4-5 (priced) and a local model (unpriced) displayed e.g. $0.04 when true cost was $0.13. Picked Option B from the review (preserve information over hiding it): Sessions.tsx now distinguishes the mixed case — cost > 0 with non-empty unpriced_models renders the partial $ + an amber asterisk + a tooltip explaining the total excludes N unpriced models. The em-dash diagnostic still fires when every model is unpriced. Medium #2 — Sessions.tsx comment contradicted implementation: Said "cost == null → at least one model unpriced", but sidecar only returns null when ALL models are unpriced. Per CLAUDE.md [mistake] rule on comment accuracy. Rewrote the block comment to describe the actual three states (em-dash, mixed-asterisk, full dollars) plus the no-usage zero case. Low #1 — Negative-limit bypass in diagnostics route: parseInt('-1', 10) || 20 → -1 (truthy), Math.min(-1, 200) → -1, SQLite LIMIT -1 returns all rows. Renderer always passes limit=20 so no current user-visible bug, but a direct local-sidecar caller could exfiltrate the full unpriced-models list. Clamped to [1, 200] via Math.max + Math.min. Low #2 — Missing FEA-1446 deferral marker: Step 3 per-token-category UI was deferred to FEA-1446; the data is emitted in the sidecar's session-detail cost_breakdown block but had no inline reference. Added a comment near the emit site so FEA-1446 has a clear linkback. Testing: - 36 pricing-related tests pass (build-agent-monitor-pricing-invariants + fetch-litellm-pricing + openai-cache-math + cost-pricing-ipc + sidecar-cost-priced-flag) - pnpm build:agent-monitor regenerates db.js with the new clamp - typecheck + lint clean - Pre-existing environment-flaky tests (python3 / shell-path / symphony-loop) unrelated to this work Risks: - The asterisk + tooltip rendering uses inline span with text-amber-400; if the sidecar UI later adopts a different visual-warning convention, this will need to follow. Low risk; no other UI uses this color today. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
thadeusb
reviewed
May 29, 2026
| // existing `harness` migration style (try a SELECT, ALTER TABLE on miss). | ||
| // The INSERT/addMissing patches below still pass the new column so a | ||
| // fresh DB and an upgraded DB converge on the same shape. | ||
| if (!source.includes("ADD COLUMN cache_write_1h_per_mtok")) { |
Contributor
There was a problem hiding this comment.
This ALTER is spliced in after addMissing(DEFAULT_PRICING), but the seed block above now prepares the 7-column INSERT (with cache_write_1h_per_mtok) before this runs. On a fresh DB the CREATE TABLE already has the column so it's fine. On any existing dashboard.db the table was created by an older version without it, and db.prepare() validates column names at compile time, so the INSERT prepare throws 'no column named cache_write_1h_per_mtok' before this migration ever executes. That kills db.js init, and since the column never gets added the next boot fails the exact same way. Every existing user's Agent Dashboard crash-loops on upgrade, which means the feature this PR ships is dead for anyone who isn't a fresh install. The ALTER has to run before the seed block, not after it. Nothing catches this today because every test creates the table with the column already present, so add an upgrade-path test that boots the patched db.js against an old-shape model_pricing table.
thadeusb
reviewed
May 29, 2026
| // FEA-1432: 1-hour Anthropic ephemeral cache tier. Optional in the form so | ||
| // OpenAI/Gemini rules can omit it; we always send a number (default 0) to | ||
| // the sidecar to keep the upstream upsert anchored on a stable shape. | ||
| cache_write_1h_per_mtok: z.number().finite().min(0).default(0), |
Contributor
There was a problem hiding this comment.
You validate and send cache_write_1h_per_mtok here, but the sidecar's PUT /api/pricing handler only destructures 6 fields and upsertPricing only has 6 rate columns. Neither got patched the way the seed INSERT did, so a manually added Claude rule silently stores cache_write_1h as 0. No visible effect today since cacheWrite1hTokens is always 0, but once FEA-1440 starts emitting real 1h tokens these hand-entered rules will under-price and nobody will know why.
thadeusb
reviewed
May 29, 2026
| /** | ||
| * FEA-1433: thin IPC binding for the Settings → Pricing surface. | ||
| * | ||
| * desktop:list-unpriced-models – GET /api/diagnostics/unpriced-models |
Contributor
There was a problem hiding this comment.
Says /api/diagnostics/unpriced-models but the route is mounted under /api/pricing, so the real path is /api/pricing/diagnostics/unpriced-models (which is what the client actually hits).
thadeusb
reviewed
May 29, 2026
| // Boundary translation: a relay receiver MAY accept v1 payloads (no | ||
| // `cacheWrite1hTokens` field) and treat the missing field as 0. The cloud | ||
| // removal ticket for the legacy-shape translation lives at FEA-1439. | ||
| export const AGENT_SESSION_SYNC_SCHEMA_VERSION = 2 as const; |
Contributor
There was a problem hiding this comment.
Bumping this to 2 means desktop starts stamping schemaVersion: 2 on every payload it sends to the cloud. Does the control plane already accept v2, or does it need to land there first so sync doesn't start getting rejected the moment this deploys? The doc comment covers a receiver tolerating v1, but the risk here is the other direction.
thadeusb
suggested changes
May 29, 2026
thadeusb
left a comment
thadeusb
left a comment
Contributor
There was a problem hiding this comment.
The cache_write_1h column migration gets spliced in after the seed INSERT that references it, so db.js init throws on any existing dashboard.db and the sidecar crash-loops for every upgrading user. Tests only ever CREATE TABLE with the column already there, so they stay green and miss it. Reorder the ALTER ahead of the seed block, add a real upgrade-path test, and fix the PUT upsert that drops the same column. Will re-review once that's reworked.
…s right This branch shipped an Opus 4.5/4.6/4.7 force-override in HOST_ONLY_OVERRIDES that pinned input=$15/output=$75 based on the *wrong* assumption that Anthropic priced new Opus models at the same rate as Opus 4.1. Anthropic actually re-priced Opus DOWN starting at 4.5 — to $5 input / $25 output / $0.50 cache_read / $6.25 5min cache_write / $10 1h cache_write per Mtok. LiteLLM upstream had it right all along. Source: https://platform.claude.com/docs/en/about-claude/pricing (confirmed via WebFetch and cross-checked against vendored apps/desktop/scripts/litellm-pricing.json which already carries 5/25/0.5/ 6.25/10 for Opus 4.5+). The downstream damage was 3× over-reporting on every Opus 4.5+ session. Users on the upgraded build saw cost numbers that triple-counted reality. This commit: 1. Removes the six HOST_ONLY_OVERRIDES rows that force-overrode Opus 4.5/4.6/4.7 (both unversioned and date-suffixed snapshots). The merged seed now follows LiteLLM's correct rates for these models. 2. Replaces the broken legacy migration in db.js with a REVERSE migration. The old migration detected the *correct* $5/$25 values and rewrote them to wrong $15/$75 values for upgraders. The new migration detects the wrong ($15/$75/$1.50/$18.75) state and resets all five rate columns (including cache_write_1h to $10) to the correct ($5/$25/$0.50/$6.25/$10) values. The reverse migration anchors AFTER the FEA-1432 cache_write_1h column-add+backfill so the column is guaranteed to exist when the UPDATE runs. 3. Removes the build-time "Opus 4.x floor" invariant. That invariant required input ≥ $10/Mtok for Opus 4.x rows — which now actively rejects correct data. The remaining two invariants (OpenAI cache discount + zero-write, Anthropic 1h cache floor) still apply. 4. Updates apps/desktop/CLAUDE.md to remove the Opus floor mention and document why it was removed. 5. Updates the two existing tests that pinned the old wrong values: - "Claude Opus 4.x rows price input ≥ $10/Mtok" → renamed and re- scoped to "Opus 4.1/4.2 retain their published $15/$75 list price" (Opus 4.1 and the deprecated Opus 4 stayed at $15). - "Opus floor invariant catches date-suffixed snapshot keys" → renamed to "Opus 4.5+ tracks Anthropic's published list price" with regex tightened to `\d{1,2}` so it doesn't trip on date-only snapshot keys like `claude-opus-4-20250514%` (deprecated base Opus 4). - Updates the openai-cache-math test to expect $10 (input × 2) for Opus 4.7 1h cache writes, not the old wrong $30. Testing: - 36 pricing-related tests pass (build-agent-monitor-pricing-invariants + fetch-litellm-pricing + openai-cache-math + sidecar-cost-priced-flag + cost-pricing-ipc) - pnpm build:agent-monitor regenerates db.js with Opus 4.7 at the correct [5, 25, 0.5, 6.25, 10] tuple in DEFAULT_PRICING and the reverse migration block in place - typecheck + lint clean Risks: - Reverse migration is conditional on the exact wrong-tuple match. A user who deliberately set their Opus 4.5+ rows to $15/$75 (very unlikely) would be reset on next launch. Acceptable: Anthropic does not bill at $15/$75 for these models, so anyone with that value is wrong. - Users on an existing 0.15.97-99 build of this branch will see their reported costs drop by ~3× on Opus 4.5+ sessions after upgrading. That's the bug correction, not a regression. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…cedence
Architectural cleanup of the host-side pricing data, prompted by the Opus
4.5+ pricing mistake earlier in this PR. The previous design had
HOST_ONLY_OVERRIDES win on collision with LiteLLM, which is exactly the
shape that let a bad override (Opus 4.5+ pinned to $15/$75) ship in
the first place. This commit makes that mistake structurally impossible.
Changes:
1. **Rename** HOST_ONLY_OVERRIDES → HOST_FALLBACKS throughout. The new
name describes what the rows are *for* (gap-fillers), not what they're
*against* (overrides). Comment block at the declaration site explains
the two legitimate categories: SYNTHETIC IDs (cursor-default, gpt-codex,
etc. that no vendor publishes) and COVERAGE GAPS (bare Claude aliases
like claude-3-5-sonnet% that LiteLLM only carries under date-suffixed
keys like claude-3-5-sonnet-20240620).
2. **Flip merge precedence** to LiteLLM-wins. loadHostDefaultPricing()
now seeds the merge map from LiteLLM first, and HOST_FALLBACKS rows
are only inserted for patterns the map does NOT already contain.
3. **Anti-override build assertion.** If any HOST_FALLBACKS row has a
model_pattern that already exists in LiteLLM, the build throws with
an actionable error message pointing to LiteLLM upstream as the fix
path. This catches my exact past mistake (force-overriding Opus 4.5+
to $15/$75) before it can ship.
The error message:
"HOST_FALLBACKS contains model_patterns that collide with LiteLLM: …
Remove these rows — LiteLLM is the source of truth for vendor pricing.
If a LiteLLM rate is genuinely wrong, fix it upstream at
https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json"
Current HOST_FALLBACKS contents (all legitimate per the new rule):
- Synthetic IDs: big-pickle%, opencode/big-pickle%, gpt-codex%,
cursor-default%, copilot-default%, opencode-default%
- Coverage gaps: claude-3-5-sonnet%, claude-3-5-haiku%,
claude-3-7-sonnet%, claude-3-haiku%, claude-3-opus%,
claude-opus-4-2%, claude-sonnet-4-2%
(all unversioned aliases that LiteLLM carries only as date-suffixed
keys — verified via WebFetch against LiteLLM upstream)
Testing:
- 36 pricing-specific tests pass; test file headers updated to describe
the new contract shape.
- pnpm build:agent-monitor passes the anti-override assertion (no
collisions in current state).
- typecheck + lint clean.
Risks:
- If LiteLLM upstream starts publishing the bare aliases we currently
fill (e.g. claude-3-7-sonnet%), the anti-override assertion will fail
the next build after the daily refresh — that's the expected outcome,
surfaces the deletion as a TODO, and ensures we follow upstream.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OpenAI re-priced cached input down to 10% of input for the GPT-5 family
(5, 5-codex, 5.4, 5.4-mini, 5.4-nano, 5.5). FEA-1432 was built on the
older "50% discount" assumption — both in the transformer (which CLAMPED
LiteLLM's correct 10% rates UP to 50%) and in the build invariant
(`cache_read ≤ input × 0.55`). Shipped state was 5× overcharging cached
input tokens for every GPT-5.x session.
Verified via the live OpenAI pricing page (gpt-5.5 shows $5 input /
$0.50 cached / $30 output = 10% cache_read ratio) and LiteLLM upstream
(gpt-5.4 / -mini / -nano all carry input × 0.1 cache_read).
Changes:
1. **Transformer clamp removed.** apps/desktop/scripts/fetch-litellm-pricing.mjs
no longer overrides cache_read; LiteLLM's value passes through verbatim.
cache_write (5-min) is still forced to 0 because that IS a vendor-stated
invariant (OpenAI publishes no cache-write surcharge), not a guess at
a ratio.
2. **Build invariant relaxed.** apps/desktop/scripts/build-agent-monitor.mjs
no longer asserts `cache_read ≤ input × 0.55`. The new sanity floor is
`cache_read ≤ input` — catches genuine regressions (a vendor pricing
cached input HIGHER than uncached would be absurd) without encoding any
assumption about the specific discount ratio.
3. **gpt-codex synthetic fallback updated** from $0.625 cache_read (old 50%
guess) to $0.125 (10% — matches the most likely underlying model,
GPT-5 Codex).
4. **Vendored LiteLLM JSON refreshed** via `pnpm refresh:pricing`. 176
rows (sha256 cc96ba1f6130…). Notable corrections post-refresh:
- gpt-5.4: cache_read $1.25 → $0.25 (5× drop, the bug correction)
- gpt-5.4-mini: $0.375 → $0.075
- gpt-5.4-nano: $0.1 → $0.02
- gpt-5.5: new row added at $5/$30/$0.50 cache_read (matches the live page)
- gpt-5/gpt-5-codex/gpt-5-mini/gpt-5-nano: cache_read also dropped to 10%
- gpt-4o: stayed at 50% — older tier, LiteLLM correctly reflects the
mixed state.
5. **Tests updated.** The "FEA-1432: OpenAI rows always carry cache_write
= 0 and cache_write_1h = 0" test was specifically about clamping
behavior — rewritten as "FEA-1431-bugfix: OpenAI cache_read passes
through LiteLLM unchanged; cache_writes always zero" with cases for
both the 10% (GPT-5 family) and 50% (GPT-4o) tiers, plus the
no-caching (cache_read=0) preservation case. The gpt-codex fallback
sanity check was updated to expect 10%.
6. **CLAUDE.md updated** to describe the new sanity floor + the fact that
LiteLLM is trusted for the actual discount ratio (currently 10% for
GPT-5 family, 50% for older).
Testing:
- 36 pricing tests pass
- pnpm build:agent-monitor passes the relaxed invariant cleanly
- typecheck + lint clean
- Vendored JSON's gpt-5.4 cache_read now reads $0.25 (was $1.25)
Risks:
- Users with active GPT-5.x sessions and prompt caching will see their
cached-input cost drop by 5× after this upgrade. That's the bug
correction, not a regression. No data migration needed — only the
pricing table changes; future sessions price correctly. Past sessions
still hold their old (5× too high) estimated_cost_usd in the cloud
relay; recompute would require re-syncing the session.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported crash on user's existing 0.15.93 DB:
Error: table model_pricing has no column named cache_write_1h_per_mtok
at Database.prepare (.../compat-sqlite.js:39:21)
at .../db.js:355:21 (the top-up's INSERT preparation)
The FEA-1432 ALTER TABLE migration was anchored on
`addMissing(DEFAULT_PRICING);\n}\n` and APPENDED after the call, meaning
the migration block landed BELOW the top-up loop in the generated db.js.
The patched top-up INSERT prepares with the new column name; node:sqlite
validates referenced columns at prepare time, so on an old DB the
sidecar crashed with ERR_SQLITE_ERROR before the ALTER ever ran. The
auto-restart hit its 5-attempt cap and the agent monitor never came up.
Fix: change the anchor to the `// Top-up: insert any default pattern`
comment and PREPEND the migration block before it. The reverse-Opus
migration (FEA-1431-bugfix from earlier in this PR) still anchors after
the FEA-1432 block, so the final order in the generated db.js is now:
1. CREATE TABLE IF NOT EXISTS (7 cols for fresh DBs)
2. FEA-1432 try/catch: ALTER TABLE ADD COLUMN + claude-% backfill
3. FEA-1431-bugfix: reverse Opus 4.5/4.6/4.7 from $15/$75 → $5/$25
4. Top-up: INSERT OR IGNORE with 7 cols (column now exists)
Inline comments in both migration blocks explain the ordering invariant
so it does not get accidentally re-ordered by a future patch.
Testing:
- 36 pricing tests pass
- pnpm build:agent-monitor regenerates db.js with the migrations now
positioned at lines 364 (ALTER), 375 (reverse), 428 (top-up) — order
is verified by grep against the generated output
- typecheck clean
Risks:
- For users who already hit the 5-restart cap on 0.15.101, the
app-startup state machine may have given up. Quitting Electron and
re-launching should fire the migration on the first try with this
build.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Superseded by a clean single-PR rebuild on Why: the override pipeline these PRs were built on was the root cause of the repeated overcharge bugs (Opus 4.5+ 3x, OpenAI cache-read 5x, phantom 1h-cache derivation). PRD-414 was reset to v2: adopt @pydantic/genai-prices as the single pricing source of truth and delete the override pipeline entirely. The library is trusted with no clamps/asserts/rewrites. The three-phase split (1431/1432/1433, 1434, 1435/1436) is being consolidated into one PR. FEA-1432 is now OBSOLETE (cache math is library-owned). Head commit archived locally as tag archive/pr-253-FEA-1431. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parent PRD: PRD-414 — Accurate token cost calculation & reconciliation across multi-vendor agents.
Summary
Phase 1 of the multi-vendor token cost work. Three stacked features that make the per-session cost numbers we already compute (a) match Anthropic/OpenAI list prices, (b) account for vendor-specific cache semantics, and (c) actually surface in the Sessions UI instead of being computed and thrown away.
model_prices_and_context_window.jsonas the canonical pricing source. Vendor at build time, refresh viajust desktop-refresh-pricing. Fixes ~3× under-pricing on Opus 4.5/4.6/4.7 inherited from the prior hand-curated seed; the Anthropic upstream rate in LiteLLM is also wrong here so HOST_ONLY_OVERRIDES pin them at $15/$75 with a documented deviation. Adds build-time invariants (Opus floor, OpenAI cache discount, Anthropic 1h floor) that throw if upstream regresses.cache_write_1h_per_mtokcolumn (Anthropic 1h tier = input × 2.0); zeroscache_writefor all OpenAI rows includinggpt-codex%fallback (OpenAI publishes no cache-write surcharge); clamps OpenAIcache_readto input × 0.5 only when LiteLLM reports a non-zero rate (preserveso1-prono-caching semantics). BumpsAGENT_SESSION_SYNC_SCHEMA_VERSION1→2.—em-dash + tooltip for unpriced models. Adds a Settings → Pricing surface listing models seen recently with no rule, with an inline "Add pricing rule" form. Surfaces partial-cost sessions with an amber asterisk + tooltip (Claude review fix). Sidecar exposes acost_breakdownblock on session detail for FEA-1446 to wire the per-token-category UI.Review chain:
Schema-version coordination
FEA-1432 bumps
AGENT_SESSION_SYNC_SCHEMA_VERSIONfrom 1 → 2. The sibling FEA-1434 PR (#PR-2) also bumps 1 → 2. Whichever PR merges first wins; the second needs to rebase to 2 → 3 before merging.Follow-up tickets filed during the work
token-usage.tsandagent-session-sync-service.ts)Test plan
🤖 Generated with Claude Code