fix(routing): table pricing with catalog fallback for off-table models, pooled latency bootstrap, fresh tier cache - #12792
Open
maxmad64bis wants to merge 1 commit into
Conversation
maxmad64bis
force-pushed
the
fix/batch7-pricing-db
branch
2 times, most recently
from
September 5, 2026 10:52
e41914b to
83b120f
Compare
…s, pooled latency bootstrap, fresh tier cache
maxmad64bis
force-pushed
the
fix/batch7-pricing-db
branch
from
September 5, 2026 11:25
83b120f to
eda7c0c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Free models outside the price table were billed as premium, and tier labels couldn't see prices an operator had overridden in settings. This PR fixes both sides of that: routing now checks the free-model catalog (the list of providers with a documented free tier) before falling back to premium prices, and tier classification reads live database prices with a freshness check so it can't drift silently.
Say you run a free model that isn't one of the 21 hardcoded entries — before this change it inherited $5/$15 per million tokens and got pushed to the bottom of cost-aware routing. Now it resolves to $0 and routes as free. The change is additive: the sync hot path keeps serving table values, and the new async tier lookup falls back to it if the database is unreachable, so nothing gets slower or stricter.
It also stops guessing 1500ms for every unknown model's latency (the pool's observed median is used instead, with a counter tracking how often the guess path fires), scores real per-candidate costs instead of zero, and computes budget caps against actual estimated tokens rather than a fixed 1000.
Related Issues
Validation
npm run lintTests Added Or Updated
tests/unit/pricing-fallback.test.ts(new, 4 tests): catalog free off-table → $0, prefixed id normalization, paid off-table → 5.0/15.0 fallback, table hits untouchedtests/unit/combo-bootstrap-latency.test.ts(new, 5 tests): median ignores corrupt entries, empty stats → undefined, counters, table-hit vs miss accounting, pure warn gatetests/unit/tier-pricing-cache.test.ts(new, 4 tests): async tier reflects DB writes incl.resetAllPricing, DB-down fallback equals sync, sync path untouched by DBtests/unit/virtualFactory-valuation.test.ts(new, 4 tests): reliability absent/null/0.2/errorRate, real table costs via helpertests/unit/auto-combo-budget.test.ts(new, 2 tests): default math byte-identical, 32k tokens enforce 32x tighter budget viaassert.throwstests/unit/complexity-router.test.ts,tests/integration/manifest-routing.test.ts,open-sse/services/__tests__/manifestAdapter.test.ts: awaited for the new async tier lookupsscripts/check/check-pricing-freshness.mjs(new): STRICT gate failing whenproviderCostData.tsis untouched for 90+ daysnpm run test:combo:matrix: 11 failures identical before/after this change (pre-existing on base, verified via stash comparison)npm run lint: exit 2 on base and branch alike — pre-existing unused-var errors incombo.ts/manifestAdapter.ts/tierResolver.ts(none introduced here; full-suite count unchanged). Note: lint needednpm install es-abstract@1.23.9 --no-save --no-package-lockworktree-only to start at all (loader bug, noted per repo conventions).Coverage Notes
open-sse/only: every new branch is covered by the tests above (table-first/catalog/fallback ingetModelPricing, median + counters + warn gate, async tier + fallback counter, reliability/cost/budget in the virtual factory and engine).catchpaths, exercised implicitly by the DB-down test.Reviewer Notes
classifyTiersync is untouched;getTargetTierstays sync;classifyTierAsyncis migrated onto one real call-site (generateRoutingHints, awaited by its 3 consumers). If the async path ever misbehaves, the sync fallback returns byte-identical values.cost-savermode actually discriminates. Worth a glance at theresolveVirtualCosthelper (open-sse/services/autoCombo/virtualFactory.ts).DEFAULT_MODEL_P95_MStable still has 8 entries (21 in the pricing table — separate tables, separate concerns); unlisted models now bootstrap from the pool median instead of a flat 1500ms, and the freshness gate keeps the pricing table honest going forward.reliabilityweight defaults to 0, so scoring only discriminates once mode packs carry it (fix(auto-combo): give every mode pack quality and reliability weights #12731).