Skip to content

Add Ramp Router support: detect a Router-pointed client and bill the model that served the call - #34

Open
anassg-lago wants to merge 5 commits into
mainfrom
feat/ramp-router-gateway
Open

Add Ramp Router support: detect a Router-pointed client and bill the model that served the call#34
anassg-lago wants to merge 5 commits into
mainfrom
feat/ramp-router-gateway

Conversation

@anassg-lago

Copy link
Copy Markdown
Collaborator

The Python twin of lago-agent-sdk-js#36, ported from that branch's rebased, live-verified state — so the pair merges together per the repo rule.

What it carries

  • Detection: a parsed-host arm after _provider_hint_for's path table (Router's only signal is its dedicated host; "api.router.com" in base_url would be spoofable, so the host is parsed).
  • Candidate parsing: provider:provider-model[:service-tier] → bare model + extras.router_provider/extras.service_tier; first-colon split; tier matched against Router's documented set only.
  • Billing decisions, both test-pinned (revert fails): ramp_router in TOKEN_BILLED_PROVIDERS (BYOK bills $0 undetectably; every catalog input rate observed is empty) and in OPENAI_SHAPED_APISmeasured, not guessed: on an Anthropic-served model a warm cache_control call reports the cached block inside an unchanged input_tokens, reasoning inside output (fixtures 06b/07).
  • Fixtures byte-identical with the JS repo (ten live captures, scrubbed and swept), plus capture_ramp_router.py — the script that reproduces them, including the corrected probes (candidates must be provider:model strings; caching only activates with an explicit cache_control part).
  • No gateway adapter, deliberately — Router exposes no programmatic usage surface; gateway/__init__.py carries the seam and the reason.
  • 55 tests, mirroring the JS names one for one.

Live evidence

Driven against a real Router account into a real Lago instance: plain, streamed and models-fallback calls all billed the SERVED dated-snapshot model as provider=ramp_router token events, in price mode, with zero error-hook noise. One Python-ecosystem nuance surfaced live and is documented in the README: the typed client rejects the non-standard models kwarg, so a fallback list goes through extra_body={"models": [...]}.

Rebase notes

  • _provider_hint_for gains the host arm after the path table; wrap_openai_client is otherwise untouched — the INT-246 eventId plumbing and stream paths are unchanged.
  • extract_openai_native gains the Ramp block between the total_tokens guard and the return; the guard's arithmetic is untouched.
  • TOKEN_BILLED_PROVIDERS, KNOWN_PROVIDERS and OPENAI_SHAPED_APIS each gain one entry.
  • No signature changes anywhere.

…model that served the call

The Python twin of lago-agent-sdk-js#36, ported from the rebased branch so it
carries the full history's decisions at once:

- Detection: a parsed-HOST arm after _provider_hint_for's path table — Router
  serves every provider it fronts through one dedicated host with no
  distinguishing path, and a substring test would be spoofable.
- provider:provider-model[:service-tier] candidates parse into a bare model
  plus extras (router_provider, service_tier); first-colon split so Fireworks
  path-shaped ids survive; unknown trailing segments stay in the model.
- ramp_router is TOKEN_BILLED (BYOK requests bill $0 undetectably; every
  observed catalog input rate is empty) and carries a MEASURED
  OPENAI_SHAPED_APIS entry: Router normalizes the numbers to OpenAI's
  convention even for Anthropic-served models — warm cache_control call,
  cached block inside an unchanged input_tokens; reasoning inside output.
  Both decisions pinned by tests that fail on revert.
- Fixtures byte-identical with the JS repo (ten live captures, scrubbed);
  capture_ramp_router.py is the script that reproduces them, including the
  probes the first capture got wrong (candidates must be provider:model
  strings; caching only activates with an explicit cache_control part).
- No gateway adapter, deliberately: Router exposes no programmatic usage
  surface; the note in gateway/__init__.py carries the seam and the reason.
- Live-verified against a real Router account into a real Lago instance:
  plain, streamed and models-fallback calls billed the SERVED dated-snapshot
  model as provider=ramp_router token events in price mode with no error
  noise. Python nuance, documented in the README: the typed client rejects
  the models kwarg, so a fallback list goes through extra_body.
The README had grown to 526 lines, most of it the four gateway
deep-dives. It now keeps a short quickstart per provider and per
gateway; the full guides — backfill mechanics, attribution, measured
billing caveats, account setup — move verbatim to docs/cloudflare.md,
docs/databricks.md, docs/snowflake.md, docs/ramp-router.md, and the
coverage/token-semantics tables to docs/providers.md.
Centered logo header with a dark-mode variant (the same wordmark the
other Lago repos use), title and tagline; the ASCII diagram's box
borders and connector spine now actually line up.
The logo already carries the name; the package id folds into the
tagline line instead of standing alone as an h1.

@sarkissianraffi sarkissianraffi left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran the suite on this branch: 817 passed. Read the adapter, wrapper, semantics wiring, docs, capture script, and every fixture. Hygiene swept the tree and the branch history: clean, and the scrub-as-you-write capture script is the right systemic answer — value-first key redaction with a pattern backstop, content blanked not deleted, provenance annotated on every fixture. That is the standard the earlier capture scripts should adopt.

The hard design calls are right and well-evidenced. Treating Router as a provider in its own right instead of resolving to the served vendor, with both reasons stated (BYOK-vs-list price ambiguity, and the measured fact that Router normalizes NUMBERS to OpenAI's convention even for Anthropic-served models). Parsed-host detection instead of a substring row, with the spoof case written down. The tier whitelist so an unknown trailing segment cannot rename a model. The first-colon split for Fireworks paths. The documented ABSENCE of a gateway adapter, checked against the whole doc corpus, is exactly how to scope honestly.

One blocker, one line to fix, flagged inline: the total_tokens guard runs BEFORE the Router block reassigns api, so for a Router call the guard reads token_semantics("ramp_router", "responses") — all-additive — while compute_cost and deoverlapped_token_total read the reassigned api = "ramp_router" and get subset semantics. That is the exact divergence token_semantics.py exists to make impossible; its docstring says the guard and the money paths cannot answer the convention question differently, and on this surface they do. Reachability today is nil (every capture reports total = input + output), but the guard exists for the day Router misreports, and on that day it under-folds by cache_read + reasoning. Move the reassignment (or compute the effective api) above the guard, and pin it with a Router payload whose declared total exceeds input + output — no current test covers that shape, so the fix is test-safe and closes an untested branch at once.

Two non-blocking notes: one inline on the candidate-parse heuristic, and the fixture-hygiene guard gained no Router patterns — defensible since the capture script scrubs at the source, but say so in the guard's comment so the next surface knows which of the two mechanisms is load-bearing, and fold it into the source-file guard extension already requested on #28.

Fix the ordering and this is approved. Same finding applies to the JS twin (#36), commented there.

# `/v1/responses` (`/v1/chat/completions` 404s), so a `chat_completions` value
# here is drift worth seeing rather than a case to handle.
extras["router_surface"] = api
api = RAMP_ROUTER_PROVIDER

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The totals guard above (line 390) runs before this reassignment, so a Router call is guarded with ("ramp_router", "responses") — every subset check False — while the emit path bills the same row under api="ramp_router" with subset semantics. The guard's accounted sum then over-counts by cache_read + reasoning, and a genuine positive remainder is under-folded or suppressed: the disarmed-guard case token_semantics.py warns about, reintroduced one surface later. Move this reassignment (or compute the effective api) above the guard, and pin it with a Router payload whose declared total exceeds input + output. Blocker, one line plus a test.

rest = model_id[first_colon + 1 :]
# A head that is not a plausible provider token — a path, or something long — means
# this is an opaque id that merely happens to contain a colon, not a candidate.
if not rest or not _ROUTER_PROVIDER_SEGMENT.match(head):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: an opaque account-specific id that happens to contain a colon with a short lowercase head — my-model:v2 — passes this check and splits, billing the model as v2 with router_provider: "my-model". The tier arm got a whitelist for exactly this reason; the provider arm relies on shape alone. Cheap hedge: record the unsplit id in extras whenever this branch splits, so a misparse is recoverable at reconciliation instead of invisible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants