Skip to content

Commit 609135a

Browse files
rickstaaclaude
andauthored
feat(realtime-transcription): add the WebSocket example (#57)
Adds a WebSocket example: realtime speech-to-text where audio streams up and transcripts stream back on one socket, which HTTP and SSE cannot do. Fills the last transport the README declared but never showed. A small aiohttp server wraps faster-whisper; the orchestrator proxies the upgrade through unchanged. Dynamic, persistent, USD per hour. The model is pinned at large-v3-turbo, so the example is GPU-only. Input is a 16 kHz mono WAV, or `-` to pipe a microphone through ffmpeg. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent b889a07 commit 609135a

11 files changed

Lines changed: 623 additions & 12 deletions

File tree

.github/workflows/images.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ on:
1616
- "hello-world/**"
1717
- "echo/**"
1818
- "tiles/**"
19+
- "realtime-transcription/**"
1920
- ".github/workflows/images.yml"
2021
pull_request:
2122
paths: *image_paths
@@ -34,7 +35,7 @@ jobs:
3435
strategy:
3536
fail-fast: false
3637
matrix:
37-
example: [hello-world, echo, tiles]
38+
example: [hello-world, echo, tiles, realtime-transcription]
3839
steps:
3940
- uses: actions/checkout@v7
4041

README.md

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -31,19 +31,20 @@ The orchestrator is a **transparent reverse proxy**: every endpoint you expose i
3131
- **HTTP** request/response — the common case. (`hello-world`, `tiles`, `api-proxy`)
3232
- **HTTP + SSE** — streamed / token responses. (`vllm`)
3333
- **Trickle** — continuous realtime video in/out. (`echo`)
34-
- **WebSocket** — long-lived bidirectional sessions. (external: `scope`)
34+
- **WebSocket** — long-lived bidirectional sessions. (`realtime-transcription`)
3535

3636
Need a schema that isn't here? [Open an issue](https://github.com/livepeer/runner-app-examples/issues).
3737

3838
## Examples
3939

40-
| Example | Goal | Registration | Mode | Transport | Pricing |
41-
| ------------------------------ | ------------------------------------------------------------------------------------- | ------------ | ----------- | ----------------- | ------- |
42-
| [`hello-world`](./hello-world) | The simplest app: one request, one response | dynamic | single-shot | HTTP (JSON) | fixed |
43-
| [`tiles`](./tiles) | Capacity fan-out — one call per tile | dynamic | single-shot | HTTP (base64 PNG) | fixed |
44-
| [`api-proxy`](./api-proxy) | Pass calls through to a hosted API — the operator holds the key, callers pay per call | static | single-shot | HTTP (JPEG bytes) | fixed |
45-
| [`echo`](./echo) | Realtime video, transformed and echoed back | dynamic | persistent | trickle | hour |
46-
| [`vllm`](./vllm) | Drop-in OpenAI API; the client stays unmodified | static | single-shot | HTTP + SSE | hour |
40+
| Example | Goal | Registration | Mode | Transport | Pricing |
41+
| ---------------------------------------------------- | ------------------------------------------------------------------------------------- | ------------ | ----------- | ----------------- | ------- |
42+
| [`hello-world`](./hello-world) | The simplest app: one request, one response | dynamic | single-shot | HTTP (JSON) | fixed |
43+
| [`tiles`](./tiles) | Capacity fan-out — one call per tile | dynamic | single-shot | HTTP (base64 PNG) | fixed |
44+
| [`api-proxy`](./api-proxy) | Pass calls through to a hosted API — the operator holds the key, callers pay per call | static | single-shot | HTTP (JPEG bytes) | fixed |
45+
| [`echo`](./echo) | Realtime video, transformed and echoed back | dynamic | persistent | trickle | hour |
46+
| [`vllm`](./vllm) | Drop-in OpenAI API; the client stays unmodified | static | single-shot | HTTP + SSE | hour |
47+
| [`realtime-transcription`](./realtime-transcription) | Audio up, transcripts back, on one socket | dynamic | persistent | WebSocket | hour |
4748

4849
Start with `hello-world` (the smallest end-to-end path); the others each layer on one new idea. More will follow, including a full example that exercises every feature. Each is self-contained and runs **offchain** (free, no wallet); most also run **on-chain** (paid) — see each README.
4950

@@ -53,7 +54,7 @@ This set stays **minimal and curated**: it covers each value of the axes above (
5354

5455
How the app attaches to the orchestrator:
5556

56-
- **Dynamic** — the app self-registers via the SDK (`register_runner`) and heartbeats; the orchestrator drops it when heartbeats stop. Best for apps that come and go. (`hello-world`, `echo`)
57+
- **Dynamic** — the app self-registers via the SDK (`register_runner`) and heartbeats; the orchestrator drops it when heartbeats stop. Best for apps that come and go. (`hello-world`, `echo`, `realtime-transcription`)
5758
- **Static** — the orchestrator is configured with the app's URL in a `runners.json` and health-polls it; the app needs no SDK. Best for fixed, long-running deployments. (`vllm`, `api-proxy`)
5859

5960
Both forms also take an optional **`metadata`** string: up to 1 KB of app-controlled UTF-8 for detail the protocol doesn't model, echoed in `/discovery` and never read by the orchestrator. Clients read it off the discovered runner, whose `raw` holds that runner's discovery entry: `cursor.candidates[0].raw["metadata"]` after `runner_selector`, `session.runner.raw["metadata"]` after `reserve_session`. Anything a caller **selects or pays differently for** belongs in the app id instead: discovery filters on `app` and `gpu`, never on metadata, which is why no example here uses it.
@@ -76,15 +77,15 @@ flowchart LR
7677

7778
Chosen _at_ registration (above); **defaults to `persistent`** — set on both `register_runner(...)` and in `runners.json`. The examples set it explicitly.
7879

79-
- **Persistent** — a held-open session the client reserves and releases, billed per second of wall-clock (or once, with fixed pricing). Best for realtime / streaming. (`echo`)
80+
- **Persistent** — a held-open session the client reserves and releases, billed per second of wall-clock (or once, with fixed pricing). Best for realtime / streaming. (`echo`, `realtime-transcription`)
8081
- **Single-shot** — one request in, one response out; the orchestrator reserves a session per call and releases it when the response returns, so the client manages no session at all. Best for batch / request-response. With metered pricing the call pays for as long as it runs, so the work need not be short. (`hello-world`, `tiles`, `api-proxy`, `vllm`)
8182

8283
## Calling your app
8384

8485
The client side depends on the runner's mode:
8586

8687
- **Single-shot****discover → call**: find the app via `runner_selector`, then one `call_runner`. The orchestrator reserves a session for the call and releases it when the response returns; on the paid path `call_runner` answers the 402 payment challenge inline. (`hello-world`, `tiles`, `api-proxy`, `vllm`)
87-
- **Persistent****discover → reserve → call → release**: reserve a session (`reserve_session`), call it — `call_runner`, streamed frames, or a WebSocket, depending on transport — then release it (`stop_runner_session`), which settles payment on-chain. (`echo`)
88+
- **Persistent****discover → reserve → call → release**: reserve a session (`reserve_session`), call it — `call_runner`, streamed frames, or a WebSocket, depending on transport — then release it (`stop_runner_session`), which settles payment on-chain. (`echo`, `realtime-transcription`)
8889

8990
Each example's `client.py` shows its exact calls — grep `# Livepeer:` to find them.
9091

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Copy to .env (gitignored) and fill in. Never commit secrets.
2+
# Keystore dirs: absolute paths OUTSIDE this repo, mounted read-only.
3+
4+
NETWORK=arbitrum-one-mainnet
5+
ETH_RPC_URL=https://arb1.arbitrum.io/rpc
6+
7+
# Signer (payer): needs an on-chain deposit + reserve.
8+
SIGNER_KEYSTORE_DIR=/absolute/path/to/signer-keystore
9+
SIGNER_ETH_ACCT=0xYourSignerAddress
10+
SIGNER_ETH_PASSWORD=your-signer-keystore-password
11+
12+
# Orchestrator operating key (split-key): needs ETH for gas to redeem tickets.
13+
ORCH_KEYSTORE_DIR=/absolute/path/to/operator-keystore
14+
ORCH_ETH_ACCT=0xYourOperatorAddress
15+
ORCH_ETH_PASSWORD=your-operator-keystore-password
16+
# Registered orch = ticket recipient (-ethOrchAddr); empty = use the operating key.
17+
ORCH_ONCHAIN_ADDR=0xYourRegisteredOrchestrator
18+
19+
# Runner price (on-chain): USD per hour, metered per second while the socket is
20+
# open. Keep under ~0.67: the signer signs at most 100 tickets per payment, and
21+
# the demo orchestrator runs -ticketEV=1e10 (fee / ticketEV).
22+
PRICE=0.01
23+
# Signer's max-price cap (payer side), per billing unit. Metered here, so the
24+
# unit is one second and must exceed PRICE / 3600 (0.000111USD is ~0.40/hour).
25+
MAX_PRICE_PER_UNIT=0.000111USD

realtime-transcription/.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Local audio and demo assets: the README shows how to make sample.wav.
2+
*.wav
3+
*.mp4
4+
*.srt
5+
media/

realtime-transcription/Dockerfile

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Realtime-transcription example app (persistent WebSocket, GPU speech-to-text).
2+
FROM python:3.12-slim
3+
4+
# Flush stdout/stderr immediately so output isn't block-buffered in `docker logs`.
5+
ENV PYTHONUNBUFFERED=1
6+
7+
RUN apt-get update \
8+
&& apt-get install -y --no-install-recommends git \
9+
&& rm -rf /var/lib/apt/lists/*
10+
11+
# livepeer-gateway SDK isn't on PyPI yet; install from Git. CTranslate2 (under
12+
# faster-whisper) loads cuBLAS/cuDNN from the nvidia wheels, so no CUDA base image
13+
# is needed -- the host driver comes in via the compose `deploy` reservation.
14+
RUN pip install --no-cache-dir \
15+
faster-whisper numpy \
16+
nvidia-cublas-cu12 nvidia-cudnn-cu12 \
17+
"livepeer-gateway @ git+https://github.com/livepeer/livepeer-python-gateway@ja/live-runner"
18+
19+
ENV LD_LIBRARY_PATH=/usr/local/lib/python3.12/site-packages/nvidia/cublas/lib:/usr/local/lib/python3.12/site-packages/nvidia/cudnn/lib
20+
21+
WORKDIR /app
22+
COPY runner.py client.py ./
23+
24+
EXPOSE 8989
25+
26+
ENTRYPOINT ["python", "runner.py"]

realtime-transcription/README.md

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Realtime transcription app (WebSocket speech-to-text)
2+
3+
Realtime speech-to-text on the Livepeer network over a **WebSocket** — the client streams audio _up_ and gets transcripts streamed _back_ on one socket. This is the example where WebSockets are genuinely required: HTTP can't stream audio upstream, and SSE is one-directional (server→client). The app is a small aiohttp server wrapping `faster-whisper` that **self-registers** (dynamic) — the dynamic, WebSocket counterpart to the static HTTP vLLM example.
4+
5+
| | |
6+
| ------------ | ----------------------------------------- |
7+
| App id | `livepeer-example/realtime-transcription` |
8+
| Runner mode | persistent (held-open WebSocket session) |
9+
| Registration | dynamic (self-registers via the SDK) |
10+
| Model | `large-v3-turbo` (faster-whisper, fixed) |
11+
| Transport | WebSocket (`/transcribe`) |
12+
| Port | 8989 |
13+
14+
**Requires an NVIDIA GPU.** The model is fixed at `large-v3-turbo`: it swaps large-v3's 32-layer decoder for 4, so it runs far below realtime on a 3090 while staying near large-v3 quality. The device is pinned with it (`cuda`/`float16`) rather than exposed as a flag: on CPU the model loads but falls behind a live stream, which is the one thing this example is about. Prerequisites (Docker, `uv`, the not-yet-released SDK) and the shared on-chain/payment setup are in the [repo README](../README.md).
15+
16+
## How it's wired
17+
18+
The app is **dynamically registered**: it self-registers with the orchestrator via `register_runner` ([runner.py](runner.py)) and exposes a `GET /transcribe` WebSocket, whose upgrade the orchestrator proxies straight through — your app speaks standard WS, nothing Livepeer-specific in the socket. The client calls it with `reserve_session``ws_connect``stop_runner_session` ([client.py](client.py)) — reserve a session, open a `wss://` socket to the session URL, stream audio up / transcripts back, release. Grep `# Livepeer:` in either file to see the exact calls.
19+
20+
On-chain, the reserved session is the billing unit: `reserve_session` pays at reserve and the meter runs while the socket is open — continuous connection = continuous billing, the right model for live audio.
21+
22+
**Staying realtime is the constraint,** and it is why the model is a constant rather than a setting. The app degrades by stretching out partials rather than dropping audio, so a model that cannot keep pace buys accuracy with unbounded lag instead of failing loudly. Serving a different model is a different app, with its own price and its own app id — which is why the id names the capability (`realtime-transcription`) and not the technique.
23+
24+
**Realtime design:** the receive loop only appends audio (never blocking on the model); a background worker transcribes the _current utterance_ (bounded to 15s) every ~0.5s, emits partials, and finalizes on trailing silence or max length — so cost stays bounded no matter how long the stream runs, instead of re-transcribing an ever-growing buffer. It uses the low-latency Whisper preset (`beam_size=1`, no cross-segment conditioning). For production-grade streaming you'd reach for a LocalAgreement approach (whisper_streaming / WhisperLive).
25+
26+
Wire protocol on `/transcribe`:
27+
28+
- client → server: binary frames of **16 kHz mono PCM (int16)**
29+
- client → server: text `eos` to finish
30+
- server → client: JSON `{"text": "...", "final": false|true, "start": <sec>, "end": <sec>}`
31+
32+
Cumulative, not incremental: each partial carries the whole utterance so far and may revise earlier words, so a client replaces rather than appends. That matches Deepgram and Vosk, and it is the honest shape for a decoder that re-runs over the buffer. Delta protocols (OpenAI's `transcript.text.delta`) only become correct once decoding is append-only, which is what the LocalAgreement approach below buys you.
33+
34+
## Audio
35+
36+
Input must be **16 kHz mono WAV**. Fetch 21s of NASA podcast speech, public domain under 17 U.S.C. 105:
37+
38+
```sh
39+
curl -sL https://images-assets.nasa.gov/audio/Ep401_Artemis_II_Launch/Ep401_Artemis_II_Launch~128k.mp3 \
40+
| ffmpeg -ss 900 -t 21 -i pipe:0 -ar 16000 -ac 1 sample.wav
41+
```
42+
43+
Or bring your own, replacing `input.mp3` / picking your capture device:
44+
45+
```sh
46+
ffmpeg -i input.mp3 -ar 16000 -ac 1 sample.wav # convert
47+
ffmpeg -f alsa -i default -ar 16000 -ac 1 -t 20 sample.wav # record (macOS: -f avfoundation -i :0)
48+
```
49+
50+
Use a clip with a couple of sentences and a pause between them: the app finalizes on trailing silence, so that is what shows partials turning into finals more than once. The NASA clip is trimmed to three such sentences.
51+
52+
Or skip the file and talk into a microphone: pass `-` and pipe raw PCM in, which streams until you Ctrl-C.
53+
54+
```sh
55+
ffmpeg -f alsa -i default -ar 16000 -ac 1 -f s16le - \
56+
| uv run client.py --discovery https://localhost:8935/discovery -
57+
```
58+
59+
(macOS: `-f avfoundation -i :0`. If `default` fails, name the device: `arecord -l` then `-i plughw:1,0`.)
60+
61+
## Run offchain (free)
62+
63+
```sh
64+
docker compose up -d --build # first run downloads the whisper model
65+
curl -sk https://localhost:8935/discovery | jq '.[].runners[].app' # confirm livepeer-example/realtime-transcription registered
66+
uv run client.py --discovery https://localhost:8935/discovery sample.wav
67+
docker compose down
68+
```
69+
70+
The client reserves a session, opens a WebSocket through the orchestrator, streams the WAV in real-time-paced chunks, and prints partial transcripts as they arrive plus a final one per utterance.
71+
72+
## Run on-chain (paid)
73+
74+
Layer `compose.onchain.yml` to add a remote signer and run the orchestrator on-chain. Needs an Ethereum RPC, a funded signer wallet (deposit + reserve), and an orchestrator wallet — see [On-chain (paid) setup](../README.md#on-chain-paid-setup) in the repo README.
75+
76+
```sh
77+
cp .env.example .env # fill in RPC, network, keystore paths, accounts, pricing
78+
docker compose -f compose.yml -f compose.onchain.yml up -d --build
79+
uv run client.py --discovery https://localhost:8935/discovery \
80+
--signer http://localhost:7936 sample.wav
81+
docker compose -f compose.yml -f compose.onchain.yml down
82+
```
83+
84+
`reserve_session` pays for the session through the remote signer; the WebSocket then streams over it. Because the session is metered, keep clips short for the demo — a long-lived socket keeps billing for its duration.
85+
86+
## Run without Docker
87+
88+
Start an orchestrator built from go-livepeer `v0.9.0` or newer (see [Build from source](https://docs.livepeer.org/v1/orchestrators/guides/install-go-livepeer#build-from-source)), then the app and client directly (the app needs `faster-whisper` installed):
89+
90+
```sh
91+
./livepeer -orchestrator -useLiveRunners -serviceAddr localhost:8935 -orchSecret abcdef -v 6
92+
uv run runner.py --orchestrator https://localhost:8935 --orchSecret abcdef
93+
uv run client.py sample.wav
94+
```

0 commit comments

Comments
 (0)