[OPIK-8242] fix: price models with compact YYYYMMDD date suffixes

[aswynz]

Details

CostService.DATE_SUFFIX_PATTERN only stripped hyphen-separated date suffixes (-2025-12-17). Anthropic ships compact YYYYMMDD dates on every dated model ID (claude-haiku-4-5-20251001, claude-sonnet-4-5-20250929), so the date-suffix fallback in findModelPrice never fired for them.

Consequence: a compact-dated model ID priced correctly only if present verbatim in the price table. Needing any normalization step meant falling through every fallback to DEFAULT_COST and returning $0 — silently, with token counts still captured. Hyphenated-dated IDs degrade gracefully to the base model row; compact ones did not. Only 13 of 3,176 price rows carry compact dates, so the verbatim hit that masks this is the exception.

Three failure paths, all live in production:

1. alias_of entries were unreachable when dated. model_prices_overrides.json already has "claude-4-6-opus": {"alias_of": "claude-opus-4-6"} for reversed family/version ordering. anthropic/claude-4.6-opus prices at $5/$25; add the real date — anthropic/claude-4.6-opus-20260205 — and it silently became $0, because the compact date must be stripped before the alias can match.
2. New releases before the daily LiteLLM sync priced at $0 instead of falling back to their base model.
3. Not Anthropic-specific — the pattern is provider-agnostic. gpt-5.4-nano-20260205 missed too.

The fix makes both separators optional. Month/day ranges are retained, so an arbitrary 8-digit build number (-99999999, -20251345) is still not mistaken for a date and cannot collapse a distinct model onto another model's price row.

-    private static final String DATE_SUFFIX_PATTERN = "-\\d{4}-(0[1-9]|1[0-2])-(0[1-9]|[12]\\d|3[01])$";
+    private static final String DATE_SUFFIX_PATTERN = "-\\d{4}-?(0[1-9]|1[0-2])-?(0[1-9]|[12]\\d|3[01])$";

The pricing arithmetic itself is unchanged and was verified correct — across 11,951 correctly-priced production spans (7,504 with cache reads), stored cost matched input·p + cache_write·1.25p + cache_read·0.1p + output·q with zero mismatches. This defect was purely model-name resolution.

Production impact that surfaced it

One project (internal-ollie-assist-monitoring): 71,442 of ~131,000 LLM spans (54%) priced at $0, hiding ~$4,470 of spend. Zero of those spans had a non-zero cost — fully deterministic.

The user-visible symptom was a cost KPI reading +573,216% over 60 days while span volume grew only 21%. Re-pricing both windows:

60-day window	Reported	Missing	Actual
Current	$2,955.58	$349.38	$3,304.96
Previous	$4.49	$3,212.80	$3,217.29
Change	+65,723%		+2.7%

Actual spend was flat. Totals are understated; period-over-period deltas can be wildly overstated. Also affects online-evaluation spend budgets (BudgetGuard), which under-count against a cap.

Scope: every project, workspace and deployment — CostService is shared, provider-agnostic code with no per-tenant config. Present since the date-stripping fallback (#5018).

Change checklist

• Bug fix (non-breaking change which fixes an issue)
• Refactoring
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update

Issues

Resolves OPIK-8242

Testing

CostServiceTest: 124/124 pass with the fix; reverting only the production change (keeping the new tests) yields 8 failures, confirming they are genuine regression tests rather than tests written to pass.

Added:

• calculateCost_compactDateSuffixPricesSameAsBaseModel — asserts a compact-dated name prices at exactly the base model's rate, not merely non-zero, which is what distinguishes a real price-table hit from an accidental one. All five cases are real model IDs observed in production traffic.
• calculateCost_shouldNotStripNonDateEightDigitSuffix — guards the month/day ranges so a build number is never read as a date.
• calculateCost_shouldReturnZeroForUnknownModelWithCompactDateSuffix — genuinely unknown models still return $0.
• Four compact-date cases added to the existing provideModelNamesWithDateSuffixes provider.

Ran locally on JDK 26 with spotless enabled.

Follow-ups (deliberately not in this PR)

1. Historical backfill — total_estimated_cost is written at ingestion, so existing spans do not self-heal. Trend lines stay wrong until affected spans are re-priced.
2. Stop failing silently — an unresolved model should be observable (unpriced-span counter/metric, or a UI signal that a cost total is incomplete). This ran ~3 months across 71k spans in one project precisely because $0 is indistinguishable from "genuinely free".

Documentation

No user-facing docs change — internal pricing-resolution fix.

🤖 Generated with Claude Code

[github-actions]

⏱️ pre-commit per-hook timing

Hook	Description	Result	Duration
☕ spotless — java backend	Format Java code	✅	3.72s
🛡️ semgrep — java backend sql	Block SQL injection-prone string formatting	✅	1.58s
Total (2 ran)			5.30s

⏭️ 42 skipped (no matching files changed)

Hook	Description	Result
🐍 trim trailing whitespace — python sdk	Strip trailing whitespace	⏭️
🐍 fix end of files — python sdk	Ensure files end in a newline	⏭️
🐍 ruff — python sdk	Lint + autofix Python (ruff)	⏭️
🐍 ruff-format — python sdk	Format Python code (ruff)	⏭️
🐍 mypy — python sdk	Static type check	⏭️
🤖 trim trailing whitespace — optimizer	Strip trailing whitespace	⏭️
🤖 fix end of files — optimizer	Ensure files end in a newline	⏭️
🤖 check yaml — optimizer	Validate YAML syntax	⏭️
🤖 check json — optimizer	Validate JSON syntax	⏭️
🤖 check toml — optimizer	Validate TOML syntax	⏭️
🤖 check for added large files — optimizer	Block large files (>1MB)	⏭️
🔐 detect private key — optimizer	Block committed private keys	⏭️
🤖 check for merge conflicts — optimizer	Block merge-conflict markers	⏭️
🤖 check for case conflicts — optimizer	Block case-only name clashes	⏭️
🤖 pyupgrade — optimizer	Modernize Python syntax	⏭️
🤖 ruff — optimizer	Lint + autofix Python (ruff)	⏭️
🤖 ruff-format — optimizer	Format Python code (ruff)	⏭️
🤖 mypy — optimizer	Static type check	⏭️
📓 nbstripout — optimizer notebooks	Strip notebook output	⏭️
📝 markdownlint — optimizer	Lint Markdown	⏭️
🔤 codespell — optimizer	Fix common misspellings	⏭️
📊 radon cc — optimizer	Cyclomatic-complexity gate	⏭️
📊 radon raw — optimizer	Raw size metrics gate	⏭️
📊 xenon — optimizer	Fail on complexity thresholds	⏭️
📊 lizard — optimizer	Cyclomatic-complexity gate	⏭️
🧹 vulture — optimizer	Find dead code	⏭️
🛡️ trim trailing whitespace — guardrails	Strip trailing whitespace	⏭️
🛡️ fix end of files — guardrails	Ensure files end in a newline	⏭️
🛡️ ruff — guardrails	Lint + autofix Python (ruff)	⏭️
🛡️ ruff-format — guardrails	Format Python code (ruff)	⏭️
🛡️ mypy — guardrails	Static type check	⏭️
⚓ helm-docs	Regenerate Helm chart README	⏭️
block non-public FE plugins	Block non-public FE plugins	⏭️
🧪 pre-commit wrapper smoke tests	Self-test the wrapper scripts	⏭️
🧪 rebaseline script tests	Self-test the changelog re-baseline script	⏭️
🌐 eslint — frontend	Lint + autofix JS/TS	⏭️
🌐 typecheck — frontend	Whole-project tsc type check	⏭️
📘 eslint — typescript sdk	Lint + autofix JS/TS	⏭️
📘 typecheck — typescript sdk	Whole-project tsc type check	⏭️
⚙️ actionlint — github workflows	Lint GitHub Actions workflows	⏭️
🐳 hadolint — dockerfiles	Lint Dockerfiles	⏭️
🌈 zizmor — github workflows security	Security-scan GitHub Actions workflows	⏭️

[CometActions]

This change looks worth a test.

Your CostServiceTest cases pin the regex itself, so the fix is well covered at the unit level. The gap is one level up: no e2e test ever reaches CostService. The only cost assertion in the estate is trace-spans-depth.spec.ts:61 (@cap:traces.span-model-cost-tokens), and its fixture seeds total_cost: 0.00042 explicitly — SpanDAO:1888 takes the totalEstimatedCost() != null branch, so the price-table lookup is never exercised and that spec passes either way. So the exact user-visible symptom you fixed — a span logged with model/provider/usage and no cost showing $0.00 — has nothing asserting it. Seedable through the SDK with no provider key, since estimation is a static table lookup and needs no LLM call.

Would target traces.span-model-cost-tokens.

What it would check

1. Seed two llm spans via the SDK with usage but no total_cost: one model 'claude-4.5-haiku-20251001' and one 'claude-haiku-4-5', both provider 'anthropic'
2. Read total_estimated_cost from GET /v1/private/spans and confirm the compact-dated span equals the base-model span rather than 0 — comparing the two keeps the assertion stable against the daily LiteLLM price-table sync
3. Open the trace panel on the compact-dated span and confirm the cost renders as a real amount, not $0.00
4. Check the negative direction: a span with model 'gpt-5.2-12345678' should stay at zero cost, not silently collapse onto another model's row
5. Confirm a span that DOES supply total_cost still renders the supplied value, so the new test does not overlap what trace-spans-depth already asserts

Deploying a test environment for this PR and exploring it — results will follow in a comment.

also touches Backend (Java API / internal)

Run

Advisory, from the QA test radar. Nothing here blocks this PR, and anything it proposes is a draft for review.

[github-actions]

🔄 Test environment deployment process has started

Phase 1: Deploying base version 2.2.50-6534 (from main branch) if environment doesn't exist
Phase 2: Building new images from PR branch aswynz/OPIK-8242/fix-compact-date-suffix-cost
Phase 3: Will deploy newly built version after build completes

You can monitor the progress here.

[github-actions]

Backend Tests - Integration Group 15

 29 files  ±0   29 suites  ±0   8m 20s ⏱️ - 1m 21s
687 tests +1  686 ✅ ±0  0 💤 ±0  0 ❌ ±0  1 🔥 +1 
686 runs  ±0  686 ✅ ±0  0 💤 ±0  0 ❌ ±0 

For more details on these errors, see this check.

Results for commit 5608fb2. ± Comparison against base commit 867d65e.

[CometActions]

✅ Test environment is now available!

To configure additional Environment variables for your environment, run [Deploy Opik AdHoc Environment workflow] (https://github.com/comet-ml/comet-deployment/actions/workflows/deploy_opik_adhoc_env.yaml)

Access Information

• URL: https://pr-8139.dev.comet.com
• Cluster: comet-ml-development
• Namespace: pr-8139
• Version: 2.2.50-8139-merge-3182
• Application logs: View in Grafana

The deployment has completed successfully and the version has been verified.

[CometActions]

Explored this PR on its own test environment.

Worked all 5 ranked items in triage.json, plus 5 regex-boundary cases the PR message makes explicit claims about. 10 checks, 10 work, 0 suspicious, 0 blocked, 0 skipped. No defect found.

Per-item results

#	Item	Verdict	What I saw
1	Seed llm spans with usage and no total_cost	works	POST /v1/private/spans/batch 204; all 13 spans came back with the seeded model/provider/usage
2	Compact-dated span prices equal to the base model over the API	works	claude-4.5-haiku-20251001 → 0.6, identical to claude-haiku-4-5 → 0.6. This is the fix: the name needs dot→hyphen normalization and a compact-date strip before the claude-4-5-haiku → claude-haiku-4-5 alias hits. Under the old hyphen-only regex neither date-strip fallback fires and it lands on DEFAULT_COST
3	The PR message's headline example	works	claude-4.6-opus-20260205 → 3.0, identical to claude-opus-4-6 → 3.0
4	Not Anthropic-specific	works	gpt-5.2-20251217 (not in the price table) → 1.575, identical to gpt-5.2 → 1.575
5	Trace panel renders a real amount, not a missing cost	works	Span tree shows $0.6 on claude-4.5-haiku-20251001 and $3 on claude-4.6-opus-20260205, matching their base-model siblings — evidence/01-span-tree-costs.png
6	Logs → Spans Estimated cost column agrees with the API	works	Column reads $0.6 / $3 / $1.57 / - row-for-row against the API values; header stat Total cost $12.15 equals the sum of the priced spans — evidence/04-spans-table-cost-column-scrolled.png
7	Negative: 8-digit build number stays unpriced	works	gpt-5.2-12345678 → field absent from the API response, - in the table, nothing rendered in the tree. Critically it did not collapse onto gpt-5.2's row (1.575)
8	Negative: month out of range	works	gpt-5.2-20251317 (month 13) → unpriced, did not collapse onto gpt-5.2
9	Negative: day out of range	works	gpt-5.2-20251232 (day 32) → unpriced, did not collapse onto gpt-5.2
10	A client-supplied cost still wins	works	Span with total_estimated_cost: 0.00042 renders <$0.01 and returns 0.00042 verbatim — exactly what trace-spans-depth.spec.ts:61 already asserts, so a new test on the estimated path does not duplicate it

2 flows look worth a permanent test:

• traces.span-model-cost-tokens — A span logged with model/provider/usage and no client cost is priced from the price table, for a compact-dated model name
• traces.span-model-cost-tokens — An 8-digit suffix that is not a valid date leaves the span unpriced instead of collapsing onto another model's price row

Writing the spec now; a draft PR will follow.

Test env · Run

Advisory, from the QA test radar. Nothing here blocks this PR, and anything it proposes is a draft for review.

[CometActions]

Proposed a permanent test for this change.

Test proposal — opik PR #8139 (OPIK-8242)

No pull request was opened.

Not because the specs are unproven — both were written, run and passed. The GitHub token
available to this run (GH_TOKEN, the github-actions[bot] App installation) has
pull_requests: read but not pull_requests: write. Every route to a PR returns the same
hard 403:

POST /repos/comet-ml/opik/pulls
  → 403 {"message": "Resource not accessible by integration"}
gh pr create --repo comet-ml/opik --draft …
  → GraphQL: Resource not accessible by integration (createPullRequest)

OVERRIDE_GITHUB_TOKEN, DEFAULT_WORKFLOW_TOKEN and GITHUB_TOKEN are all byte-identical to
GH_TOKEN, so there is no second credential to fall back to. Reads work (PR #8139 was fetched
fine) and git push works — so the work is already on the remote, only the PR object is
missing. Nothing was merged and nothing was marked ready for review.

The branch is pushed and ready

branch: comet-qa-bot/OPIK-8242/e2e-span-estimated-cost
commit: b78fa520305440efc74ababd0f33a91ec6c3de49
base:   aswynz/OPIK-8242/fix-compact-date-suffix-cost   (#8139's head branch, still open)

The PR body is written and waiting at ./pr-body.md (it already ends with the
comet-qa-test-radar: propose marker). A human with pull-requests: write can open the draft
verbatim:

gh pr create --repo comet-ml/opik --draft \
  --base aswynz/OPIK-8242/fix-compact-date-suffix-cost \
  --head comet-qa-bot/OPIK-8242/e2e-span-estimated-cost \
  --title "[OPIK-8242] [QA] Proposed e2e specs: server-side span cost estimate for compact-dated model names" \
  --body-file pr-body.md

Or in one click:
aswynz/OPIK-8242/fix-compact-date-suffix-cost...comet-qa-bot/OPIK-8242/e2e-span-estimated-cost
— tick "Create draft pull request".

Base is #8139's branch, not main, on purpose. #8139 is still open and its head branch
still exists. On main, claude-4.5-haiku-20251001 still misses the price table and comes
back unpriced, so spec 1 would fail there for a real reason. Rebase onto main once #8139
merges.

The specs

Both in tests_end_to_end/e2e/tests/trace-explore/span-estimated-cost.spec.ts, tagged
@t2-cuj @area:traces @cap:traces.span-model-cost-tokens.

#	Spec	Surface	Verification
1	a compact-dated model name is priced as the model it names, over the API and in the UI	both	PASSED against https://pr-8139.dev.comet.com
2	an 8-digit suffix that is not a date leaves the span unpriced rather than priced as another model	api	PASSED against https://pr-8139.dev.comet.com

Candidates dropped: 0. candidates.json carried exactly two, both strength: strong and
verified_on_staging: true; both were written, and neither had to be dropped for failing or
for being un-runnable.

How they were verified

cd tests_end_to_end/e2e
npx playwright test tests/trace-explore/span-estimated-cost.spec.ts --reporter=list
#   2 passed (13.4s)

npx playwright test tests/trace-explore/ --reporter=list --workers=2
#   24 passed (1.6m)   ← whole feature directory: two shared POMs and the fixture chain
#                        head were touched

python3 tests_end_to_end/coverage/tag_lint.py \
  --taxonomy tests_end_to_end/coverage/taxonomy.yaml --estate tests_end_to_end
#   tag-lint: 60 specs checked, 1 exempt, 0 problem(s)

Each UI assertion was mutation-checked: the expected cost string and the - expectation
were each deliberately falsified and the run confirmed to fail on the live page (Expected "$0.69" / Received " $0.6", and Expected "MUTANT" / Received "-"). The green above is not a
locator quietly resolving to nothing.

The first full-directory run had one failure —
trace-filters.spec.ts › Feedback score filter narrows the table…, timing out in
LogsPage.openFilterChip. Pre-existing flake, not fallout: it passes on the unmodified tree
and passed on the second full-directory run, and nothing here touches openFilterChip.

Judgement calls a reviewer should check

• traces.toggle-spans-view left covered: false. Spec 1 does reach Logs → Spans, but by
  URL (mirroring the existing gotoThreads) and only to read a cost cell. It never clicks the
  toggle, so flipping that capability would report coverage nobody has.
• traces.span-model-cost-tokens stays covered: true and gains a note: naming its two
  branches. No capability is newly flipped — the value here is that the estimated branch was
  previously untested: trace-spans-depth.spec.ts seeds total_cost: 0.00042, which
  short-circuits CostService entirely.
• No data-testid was added to the front end. The span-tree cost block has no testid and
  no accessible name, so the POM scopes to the node's existing trace-tree-node-<name> testid
  and matches the leading $. A testid would be better and the POM says so; it was not added
  because the target environment serves a prebuilt frontend image, so the change could not
  have been verified in this run.
• The mixed-separator form (claude-4.5-haiku-2025-1001, which also prices now) is not
  asserted — the exploration read it as an accident of the fix rather than intended behaviour.

Environment notes

• npx tsc --noEmit does not run on this tree and did not before this branch: tsconfig.json
  still sets baseUrl, which the pinned typescript@7.0.2 has removed. Typechecked with an
  equivalent config minus baseUrl — the only two errors in the whole suite are pre-existing
  Duplicate identifier 'deleteDashboard' in core/backend/client.ts, neither in these files.
  Not fixed here; unrelated.
• The PR environment is an OSS install with no auth, but its hostname ends in comet.com, and
  the TS SDK treats any *.comet.com URL as cloud and refuses to construct without an API key.
  A dummy OPIK_API_KEY is enough — the backend ignores the header (curl returns 200 with or
  without it). Not an issue on the normal localhost:5173 target.
• uv was not on the runner; it was installed so Playwright's webServer could spawn the
  Python SDK bridge.

Standards read before writing

opik-main/.agents/skills/writing-e2e-tests/SKILL.md, its conventions.md, and
playwright-pom-discovery/SKILL.md — all present at the opik-main path and read from there
(they are QA's own standards, not part of the change under test). Existing specs matched
against: trace-explore/trace-spans-depth.spec.ts, trace-explore/trace-partial-update-merge.spec.ts,
dashboards/workspace-span-metrics.spec.ts, and the token-usage-spans fixture.

Run

Advisory, from the QA test radar. Nothing here blocks this PR, and anything it proposes is a draft for review.