Skip to content

Commit aa9bdc6

Browse files
rickstaaclaude
andcommitted
feat(vllm): register single-shot so the call pays as it runs
vllm registered as persistent while the README apologised for it: the app is one request in, one response out, with no state to keep between calls. It stayed persistent because a single-shot call could not keep paying, so metering it needed a session the gateway held open by hand. That capability is on the pinned SDK branch now. call_runner starts a funding loop when the price is metered, and for a streamed response the stream owns that loop, so an SSE generation pays for as long as tokens flow. The orchestrator reserves a session around the call and releases it when the response returns. So the gateway drops from three SDK calls to two: discover, then call. This also fills the empty cell in the axis table, single-shot paired with metered pricing, which nothing showed before. One behaviour becomes visible: a single-shot call holds a capacity slot for its duration, so a second concurrent request gets 503 from the orchestrator. That escaped as an opaque aiohttp 500, so it is now handed back as a JSON error an OpenAI client can read. Closes #4, closes #5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent c8a3708 commit aa9bdc6

4 files changed

Lines changed: 75 additions & 69 deletions

File tree

README.md

Lines changed: 11 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -37,13 +37,13 @@ Need a schema that isn't here? [Open an issue](https://github.com/livepeer/runne
3737

3838
## Examples
3939

40-
| Example | Goal | Registration | Mode | Transport | Pricing |
41-
| ------------------------------ | ------------------------------------------------------------------------------------- | ------------ | ---------------------------------- | ----------------- | ------- |
42-
| [`hello-world`](./hello-world) | The simplest app: one request, one response | dynamic | single-shot | HTTP (JSON) | fixed |
43-
| [`tiles`](./tiles) | Capacity fan-out — one call per tile | dynamic | single-shot | HTTP (base64 PNG) | fixed |
44-
| [`api-proxy`](./api-proxy) | Pass calls through to a hosted API — the operator holds the key, callers pay per call | static | single-shot | HTTP (JPEG bytes) | fixed |
45-
| [`echo`](./echo) | Realtime video, transformed and echoed back | dynamic | persistent | trickle | hour |
46-
| [`vllm`](./vllm) | Drop-in OpenAI API; the client stays unmodified | static | persistent (single-shot by nature) | HTTP + SSE | hour |
40+
| Example | Goal | Registration | Mode | Transport | Pricing |
41+
| ------------------------------ | ------------------------------------------------------------------------------------- | ------------ | ----------- | ----------------- | ------- |
42+
| [`hello-world`](./hello-world) | The simplest app: one request, one response | dynamic | single-shot | HTTP (JSON) | fixed |
43+
| [`tiles`](./tiles) | Capacity fan-out — one call per tile | dynamic | single-shot | HTTP (base64 PNG) | fixed |
44+
| [`api-proxy`](./api-proxy) | Pass calls through to a hosted API — the operator holds the key, callers pay per call | static | single-shot | HTTP (JPEG bytes) | fixed |
45+
| [`echo`](./echo) | Realtime video, transformed and echoed back | dynamic | persistent | trickle | hour |
46+
| [`vllm`](./vllm) | Drop-in OpenAI API; the client stays unmodified | static | single-shot | HTTP + SSE | hour |
4747

4848
Start with `hello-world` (the smallest end-to-end path); the others each layer on one new idea. More will follow, including a full example that exercises every feature. Each is self-contained and runs **offchain** (free, no wallet); most also run **on-chain** (paid) — see each README.
4949

@@ -74,18 +74,15 @@ flowchart LR
7474

7575
Chosen _at_ registration (above); **defaults to `persistent`** — set on both `register_runner(...)` and in `runners.json`. The examples set it explicitly.
7676

77-
- **Persistent** — a held-open session the client reserves and releases, billed per second of wall-clock (or once, with fixed pricing). Best for realtime / streaming. (`echo`, `vllm`)
78-
- **Single-shot** — one request in, one response out; the orchestrator reserves a session per call and releases it when the response returns, so the client manages no session at all. Best for batch / request-response. (`hello-world`, `tiles`, `api-proxy`)
79-
80-
> [!NOTE]
81-
> The `vllm` example is single-shot by nature but stays **persistent** for now: it meters per second across a reserved session, and true per-token billing is brokerage for the gateway/signer layer.
77+
- **Persistent** — a held-open session the client reserves and releases, billed per second of wall-clock (or once, with fixed pricing). Best for realtime / streaming. (`echo`)
78+
- **Single-shot** — one request in, one response out; the orchestrator reserves a session per call and releases it when the response returns, so the client manages no session at all. Best for batch / request-response. With metered pricing the call pays for as long as it runs, so the work need not be short. (`hello-world`, `tiles`, `api-proxy`, `vllm`)
8279

8380
## Calling your app
8481

8582
The client side depends on the runner's mode:
8683

87-
- **Single-shot****discover → call**: find the app via `runner_selector`, then one `call_runner`. The orchestrator reserves a session for the call and releases it when the response returns; on the paid path `call_runner` answers the 402 payment challenge inline. (`hello-world`, `tiles`, `api-proxy`)
88-
- **Persistent****discover → reserve → call → release**: reserve a session (`reserve_session`), call it — `call_runner`, streamed frames, or a WebSocket, depending on transport — then release it (`stop_runner_session`), which settles payment on-chain. (`echo`, `vllm`)
84+
- **Single-shot****discover → call**: find the app via `runner_selector`, then one `call_runner`. The orchestrator reserves a session for the call and releases it when the response returns; on the paid path `call_runner` answers the 402 payment challenge inline. (`hello-world`, `tiles`, `api-proxy`, `vllm`)
85+
- **Persistent****discover → reserve → call → release**: reserve a session (`reserve_session`), call it — `call_runner`, streamed frames, or a WebSocket, depending on transport — then release it (`stop_runner_session`), which settles payment on-chain. (`echo`)
8986

9087
Each example's `client.py` shows its exact calls — grep `# Livepeer:` to find them.
9188

vllm/README.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,17 +5,14 @@ Runs an OpenAI-compatible LLM on the Livepeer network and consumes it with the *
55
| | |
66
| ------------ | ------------------------------------------ |
77
| App id | `vllm/qwen2.5-0.5b-instruct` |
8-
| Runner mode | persistent (single-shot by nature) |
8+
| Runner mode | single-shot (one session per call) |
99
| Registration | static (orchestrator config + health poll) |
1010
| Transport | HTTP + SSE (OpenAI `/v1/chat/completions`) |
11-
| Pricing | hour (metered per second of session) |
11+
| Pricing | hour (metered per second of the call) |
1212
| Port | 8000 (vLLM), 8080 (gateway) |
1313

1414
**Requires an NVIDIA GPU** for vLLM. The default model (`Qwen/Qwen2.5-0.5B-Instruct`) is tiny so it fits a modest card can be overridden with `VLLM_MODEL`. Prerequisites (Docker, `uv`, the not-yet-released SDK) and the shared on-chain/payment setup are in the [repo README](../README.md).
1515

16-
> [!NOTE]
17-
> This app is single-shot by nature but currently registers as **persistent**. It will switch to **single-shot** once [#5](https://github.com/livepeer/runner-app-examples/issues/5) lands.
18-
1916
## How it's wired
2017

2118
vLLM is a **static runner**: the orchestrator reads `runners.json` via `-liveRunnerConfig`, health-polls `http://vllm:8000/health`, and reverse-proxies OpenAI requests straight to vLLM — no registrar, no heartbeat, no SDK in the app, nothing to build.
@@ -27,7 +24,9 @@ Two sides:
2724

2825
The local gateway is a _client-side_ component, so it runs on the host like the client, not in the infra compose.
2926

30-
The gateway is the **only** Livepeer-aware piece in the whole path — and it's tiny: three SDK calls, `reserve_session``call_runner``stop_runner_session` (grep `# Livepeer:` in [gateway.py](gateway.py)). They exist _purely_ because an OpenAI client has no idea how to discover a runner or settle Livepeer's payments. Move that glue into the gateway and everything else — `client.py`, any OpenAI SDK, `curl` — stays 100% stock OpenAI, oblivious to Livepeer.
27+
The gateway is the **only** Livepeer-aware piece in the whole path — and it's tiny: two SDK calls, `runner_selector``call_runner` (grep `# Livepeer:` in [gateway.py](gateway.py)). They exist _purely_ because an OpenAI client has no idea how to discover a runner or settle Livepeer's payments. Move that glue into the gateway and everything else — `client.py`, any OpenAI SDK, `curl` — stays 100% stock OpenAI, oblivious to Livepeer.
28+
29+
The runner is **single-shot**, so the orchestrator reserves a session around each call and releases it when the response returns; the gateway never manages one, which is why there is no third SDK call. Pricing is still metered, so the call pays for as long as it runs and a long generation costs what it takes. With `capacity: 1`, a second request arriving while one is in flight gets a 503, which the gateway hands back as a JSON error rather than an opaque 500.
3130

3231
## Run offchain (free)
3332

@@ -76,4 +75,4 @@ kill %1; docker compose -f compose.yml -f compose.onchain.yml down
7675

7776
The client is **unchanged** — only the gateway gets `--signer`; it pays per call through the remote signer, so the consumer never sees discovery or payment. The price is set in `runners.json`.
7877

79-
Pricing note: the orchestrator meters compute per **second**, not per token. Probabilistic payments are made up front, so token counts can't drive protocol pricing. Per-token billing is left to the signer/gateway layer, which sees `usage` in every response and can bill users per token while paying the orchestrator per second.
78+
Pricing note: the orchestrator meters compute per **second** of the call, not per token. Probabilistic payments are made up front, so token counts can't drive protocol pricing. Per-token billing is left to the signer/gateway layer, which sees `usage` in every response and can bill users per token while paying the orchestrator per second.

vllm/gateway.py

Lines changed: 57 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -10,17 +10,17 @@
1010
export OPENAI_BASE_URL=http://localhost:8080/v1 OPENAI_API_KEY=unused
1111
# then plain `openai`, curl, or any SDK just works
1212
13-
Each request: reserve a session, forward the body, release the session. call_runner does
14-
the 402 payment challenge internally, so the client never sees discovery or payment.
15-
(Release matters: the runner has capacity 1, so an unreleased session would block the
16-
next call.)
13+
Each request: discover the runner, forward the body. The runner is single-shot, so the
14+
orchestrator reserves a session for the call and releases it when the response returns --
15+
the gateway manages no session at all. call_runner does the 402 payment challenge
16+
internally, so the client never sees discovery or payment. Pricing is metered, so the
17+
call keeps paying for as long as it runs, which for a long generation is the point.
1718
1819
Livepeer integration (grep `# Livepeer:`):
19-
1. reserve_session() — discover the runner, reserve a session
20-
2. call_runner() — forward the request through the orchestrator (pays 402)
21-
3. stop_runner_session() — release the session
20+
1. runner_selector() — discover the runner advertising this app
21+
2. call_runner() — forward the request through the orchestrator (pays 402)
2222
23-
These three calls are the *entire* Livepeer surface. They live here, and only here,
23+
These two calls are the *entire* Livepeer surface. They live here, and only here,
2424
because an OpenAI client can't do discovery or settle payments itself — so `client.py`
2525
(and any OpenAI SDK/curl) stays 100% stock, unaware of Livepeer.
2626
@@ -34,12 +34,12 @@
3434

3535
import argparse
3636
import logging
37-
from contextlib import suppress
3837

3938
from aiohttp import web
4039

41-
from livepeer_gateway.live_runner import call_runner, stop_runner_session
42-
from livepeer_gateway.selection import reserve_session
40+
from livepeer_gateway.errors import LivepeerHTTPError
41+
from livepeer_gateway.live_runner import call_runner
42+
from livepeer_gateway.selection import runner_selector
4343

4444
APP_ID = "vllm/qwen2.5-0.5b-instruct"
4545

@@ -71,49 +71,59 @@ def main() -> None:
7171
async def _forward(request: web.Request) -> web.StreamResponse:
7272
payload = await request.json()
7373
runner_path = request.path # e.g. /v1/chat/completions
74-
session = await reserve_session(
74+
cursor = await runner_selector( # Livepeer: 1
7575
discovery_url=args.discovery, # omit if the signer does discovery itself
7676
app=APP_ID,
77+
)
78+
runner = cursor.candidates[0]
79+
runner_url = runner.url.rstrip("/") + runner_path
80+
81+
# When the OpenAI client asks for stream=True the runner replies with
82+
# text/event-stream; pipe those chunks straight through with stream=True
83+
# so tokens reach the client as they arrive instead of buffering the blob.
84+
if payload.get("stream"):
85+
async with await call_runner( # Livepeer: 2 (streaming)
86+
runner=runner, # discovery metadata tells call_runner the price unit
87+
runner_url=runner_url,
88+
payload=payload,
89+
signer_url=signer_url,
90+
stream=True,
91+
) as stream:
92+
resp = web.StreamResponse(
93+
status=stream.status,
94+
headers={
95+
"Content-Type": stream.content_type or "text/event-stream"
96+
},
97+
)
98+
await resp.prepare(request)
99+
async for (
100+
chunk
101+
) in stream.aiter_bytes(): # raw bytes -> keep SSE framing
102+
await resp.write(chunk)
103+
await resp.write_eof()
104+
return resp
105+
106+
result = await call_runner( # Livepeer: 2
107+
runner=runner, # discovery metadata tells call_runner the price unit
108+
runner_url=runner_url,
109+
payload=payload,
77110
signer_url=signer_url,
78-
) # Livepeer: 1
111+
)
112+
return web.json_response(result.data)
79113

114+
async def _forward_or_error(request: web.Request) -> web.StreamResponse:
115+
# A single-shot call holds a capacity slot for its duration, so a busy runner
116+
# answers 503. Hand that back as JSON an OpenAI client can read.
80117
try:
81-
runner_url = session.app_url.rstrip("/") + runner_path
82-
83-
# When the OpenAI client asks for stream=True the runner replies with
84-
# text/event-stream; pipe those chunks straight through with stream=True
85-
# so tokens reach the client as they arrive instead of buffering the blob.
86-
if payload.get("stream"):
87-
async with await call_runner( # Livepeer: 2 (streaming)
88-
runner_url=runner_url,
89-
payload=payload,
90-
signer_url=signer_url,
91-
stream=True,
92-
) as stream:
93-
resp = web.StreamResponse(
94-
status=stream.status,
95-
headers={
96-
"Content-Type": stream.content_type or "text/event-stream"
97-
},
98-
)
99-
await resp.prepare(request)
100-
async for (
101-
chunk
102-
) in stream.aiter_bytes(): # raw bytes -> preserve SSE framing
103-
await resp.write(chunk)
104-
await resp.write_eof()
105-
return resp
106-
107-
result = await call_runner(
108-
runner_url=runner_url, payload=payload, signer_url=signer_url
109-
) # Livepeer: 2
110-
return web.json_response(result.data)
111-
finally:
112-
with suppress(Exception):
113-
await stop_runner_session(session) # Livepeer: 3
118+
return await _forward(request)
119+
except LivepeerHTTPError as exc:
120+
return web.json_response(
121+
{"error": {"message": str(exc), "type": "livepeer_error"}},
122+
status=exc.status_code,
123+
)
114124

115125
app = web.Application()
116-
app.router.add_post("/v1/{tail:.*}", _forward) # forward every OpenAI path
126+
app.router.add_post("/v1/{tail:.*}", _forward_or_error) # forward every OpenAI path
117127
log.info(
118128
"gateway on http://%s:%d/v1 -> %s (signer=%s)",
119129
args.host,

vllm/runners.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
"app": "vllm/qwen2.5-0.5b-instruct",
66
"runner_url": "http://vllm:8000",
77
"health_url": "/health",
8-
"mode": "persistent",
8+
"mode": "single-shot",
99
"capacity": 1,
1010
"price_info": { "price": 0.01 }
1111
}

0 commit comments

Comments
 (0)