Add Ramp Router support: detect a Router-pointed client and bill the model that served the call - #34
Add Ramp Router support: detect a Router-pointed client and bill the model that served the call#34anassg-lago wants to merge 5 commits into
Conversation
…model that served the call The Python twin of lago-agent-sdk-js#36, ported from the rebased branch so it carries the full history's decisions at once: - Detection: a parsed-HOST arm after _provider_hint_for's path table — Router serves every provider it fronts through one dedicated host with no distinguishing path, and a substring test would be spoofable. - provider:provider-model[:service-tier] candidates parse into a bare model plus extras (router_provider, service_tier); first-colon split so Fireworks path-shaped ids survive; unknown trailing segments stay in the model. - ramp_router is TOKEN_BILLED (BYOK requests bill $0 undetectably; every observed catalog input rate is empty) and carries a MEASURED OPENAI_SHAPED_APIS entry: Router normalizes the numbers to OpenAI's convention even for Anthropic-served models — warm cache_control call, cached block inside an unchanged input_tokens; reasoning inside output. Both decisions pinned by tests that fail on revert. - Fixtures byte-identical with the JS repo (ten live captures, scrubbed); capture_ramp_router.py is the script that reproduces them, including the probes the first capture got wrong (candidates must be provider:model strings; caching only activates with an explicit cache_control part). - No gateway adapter, deliberately: Router exposes no programmatic usage surface; the note in gateway/__init__.py carries the seam and the reason. - Live-verified against a real Router account into a real Lago instance: plain, streamed and models-fallback calls billed the SERVED dated-snapshot model as provider=ramp_router token events in price mode with no error noise. Python nuance, documented in the README: the typed client rejects the models kwarg, so a fallback list goes through extra_body.
The README had grown to 526 lines, most of it the four gateway deep-dives. It now keeps a short quickstart per provider and per gateway; the full guides — backfill mechanics, attribution, measured billing caveats, account setup — move verbatim to docs/cloudflare.md, docs/databricks.md, docs/snowflake.md, docs/ramp-router.md, and the coverage/token-semantics tables to docs/providers.md.
Centered logo header with a dark-mode variant (the same wordmark the other Lago repos use), title and tagline; the ASCII diagram's box borders and connector spine now actually line up.
The logo already carries the name; the package id folds into the tagline line instead of standing alone as an h1.
sarkissianraffi
left a comment
There was a problem hiding this comment.
Ran the suite on this branch: 817 passed. Read the adapter, wrapper, semantics wiring, docs, capture script, and every fixture. Hygiene swept the tree and the branch history: clean, and the scrub-as-you-write capture script is the right systemic answer — value-first key redaction with a pattern backstop, content blanked not deleted, provenance annotated on every fixture. That is the standard the earlier capture scripts should adopt.
The hard design calls are right and well-evidenced. Treating Router as a provider in its own right instead of resolving to the served vendor, with both reasons stated (BYOK-vs-list price ambiguity, and the measured fact that Router normalizes NUMBERS to OpenAI's convention even for Anthropic-served models). Parsed-host detection instead of a substring row, with the spoof case written down. The tier whitelist so an unknown trailing segment cannot rename a model. The first-colon split for Fireworks paths. The documented ABSENCE of a gateway adapter, checked against the whole doc corpus, is exactly how to scope honestly.
One blocker, one line to fix, flagged inline: the total_tokens guard runs BEFORE the Router block reassigns api, so for a Router call the guard reads token_semantics("ramp_router", "responses") — all-additive — while compute_cost and deoverlapped_token_total read the reassigned api = "ramp_router" and get subset semantics. That is the exact divergence token_semantics.py exists to make impossible; its docstring says the guard and the money paths cannot answer the convention question differently, and on this surface they do. Reachability today is nil (every capture reports total = input + output), but the guard exists for the day Router misreports, and on that day it under-folds by cache_read + reasoning. Move the reassignment (or compute the effective api) above the guard, and pin it with a Router payload whose declared total exceeds input + output — no current test covers that shape, so the fix is test-safe and closes an untested branch at once.
Two non-blocking notes: one inline on the candidate-parse heuristic, and the fixture-hygiene guard gained no Router patterns — defensible since the capture script scrubs at the source, but say so in the guard's comment so the next surface knows which of the two mechanisms is load-bearing, and fold it into the source-file guard extension already requested on #28.
Fix the ordering and this is approved. Same finding applies to the JS twin (#36), commented there.
| # `/v1/responses` (`/v1/chat/completions` 404s), so a `chat_completions` value | ||
| # here is drift worth seeing rather than a case to handle. | ||
| extras["router_surface"] = api | ||
| api = RAMP_ROUTER_PROVIDER |
There was a problem hiding this comment.
The totals guard above (line 390) runs before this reassignment, so a Router call is guarded with ("ramp_router", "responses") — every subset check False — while the emit path bills the same row under api="ramp_router" with subset semantics. The guard's accounted sum then over-counts by cache_read + reasoning, and a genuine positive remainder is under-folded or suppressed: the disarmed-guard case token_semantics.py warns about, reintroduced one surface later. Move this reassignment (or compute the effective api) above the guard, and pin it with a Router payload whose declared total exceeds input + output. Blocker, one line plus a test.
| rest = model_id[first_colon + 1 :] | ||
| # A head that is not a plausible provider token — a path, or something long — means | ||
| # this is an opaque id that merely happens to contain a colon, not a candidate. | ||
| if not rest or not _ROUTER_PROVIDER_SEGMENT.match(head): |
There was a problem hiding this comment.
Non-blocking: an opaque account-specific id that happens to contain a colon with a short lowercase head — my-model:v2 — passes this check and splits, billing the model as v2 with router_provider: "my-model". The tier arm got a whitelist for exactly this reason; the provider arm relies on shape alone. Cheap hedge: record the unsplit id in extras whenever this branch splits, so a misparse is recoverable at reconciliation instead of invisible.
The Python twin of lago-agent-sdk-js#36, ported from that branch's rebased, live-verified state — so the pair merges together per the repo rule.
What it carries
_provider_hint_for's path table (Router's only signal is its dedicated host;"api.router.com" in base_urlwould be spoofable, so the host is parsed).provider:provider-model[:service-tier]→ bare model +extras.router_provider/extras.service_tier; first-colon split; tier matched against Router's documented set only.ramp_routerinTOKEN_BILLED_PROVIDERS(BYOK bills $0 undetectably; every catalog input rate observed is empty) and inOPENAI_SHAPED_APIS— measured, not guessed: on an Anthropic-served model a warmcache_controlcall reports the cached block inside an unchangedinput_tokens, reasoning insideoutput(fixtures 06b/07).capture_ramp_router.py— the script that reproduces them, including the corrected probes (candidates must beprovider:modelstrings; caching only activates with an explicitcache_controlpart).gateway/__init__.pycarries the seam and the reason.Live evidence
Driven against a real Router account into a real Lago instance: plain, streamed and
models-fallback calls all billed the SERVED dated-snapshot model asprovider=ramp_routertoken events, in price mode, with zero error-hook noise. One Python-ecosystem nuance surfaced live and is documented in the README: the typed client rejects the non-standardmodelskwarg, so a fallback list goes throughextra_body={"models": [...]}.Rebase notes
_provider_hint_forgains the host arm after the path table;wrap_openai_clientis otherwise untouched — the INT-246 eventId plumbing and stream paths are unchanged.extract_openai_nativegains the Ramp block between the total_tokens guard and the return; the guard's arithmetic is untouched.TOKEN_BILLED_PROVIDERS,KNOWN_PROVIDERSandOPENAI_SHAPED_APISeach gain one entry.