[Marketen]

All gateway changes for guest chat in one PR. Replaces #19, #20 and #21. Companion PRs: dappnode/dappnode-nexus#156 (Metering holds and guest chat) and dappnode/infrastructure#309 (settings).

Deploy order: dappnode-nexus#156 (Metering) first. Metering ignores unknown request fields, and a Metering without the estimate holds the context window, as today. But a Metering without guest_allowed makes every model read as closed to guests (part 4), which only matters once guest chat is on.

1. Send an upper-bound prompt estimate with each credit hold (was #19)

Metering held every request's prompt as the model's full context window, which rejected small balances even for short chats. The gateway now sends an upper bound of the real prompt, computed on the body after PII masking:

estimate = body bytes + 1,024 + Σ over image parts (16,384 − part bytes)

Tokenizers never produce more tokens than UTF-8 bytes. No estimate is sent for audio, video, files, unknown part types or /v1/systemone; Metering then holds the full context window. Full rules: dappnode-nexus docs/METERING_HOLDS.md.

2. Charge requests whose client hangs up before the answer ends (was #20)

A request is charged when the provider reports usage, at the end of the answer. Before, a client that hung up early made the gateway cut the provider off. No usage arrived and the hold was released unpaid, while the provider still billed us. Anyone could get answers for free by disconnecting just before the end.

Providers have no API to look up a cancelled request's cost. So once credit is reserved, the provider call no longer follows the client's connection: the answer runs to the end and its reported usage is charged, for every account.

• Pressing Stop no longer lowers the bill; max_tokens bounds it.
• A stalled provider still ends through the existing PROVIDER_SILENCE_TIMEOUT_SECONDS (15 min).
• No fallback answer is generated for a client that has hung up.

3. Guest chat requests always carry an output limit (was #21)

A guest is an ordinary account (is_guest from Metering) with a small credit and no login. The only difference: a guest chat request always carries an output limit of at most GUEST_MAX_OUTPUT_TOKENS (default 8,192), written into the request so the provider enforces it. Without one, Metering would hold the model's maximum output (46 cents on Flash), far more than a guest's 3-cent credit.

• 8,192 covers almost every answer: production DeepSeek V4.1 Flash answers averaged about 1,100 output tokens over a week, and the longest was about 15,000.
• At 1,024, a hard question spent the whole limit reasoning and returned an empty answer (checked against the real DeepSeek API).
• When the guest set no limit, the gateway writes max_completion_tokens. The provider adapter sends max_tokens to DeepSeek, which ignores max_completion_tokens. Checked against the real APIs (2026-11-04): Novita, Phala, Doubleword, MiniMax, Dappnode and Tinfoil enforce both fields.

Other accounts' requests go upstream as sent. Without max_tokens, Metering holds the model's maximum output (dappnode-nexus#156), which the provider can't exceed.

4. Guest keys call only models open to guests

Routing runs before credit is checked, and the Jev router (nexus/auto in prod) calls TypeSafe's paid API, so an exhausted guest key could keep making us pay TypeSafe. A free model costs a guest nothing against its credit, so a guest could use prod's free Qwen without limit. Guest keys now reach only chat completions on models an operator ticked Guests can use this model in the admin UI (guest_allowed from Metering). Anything else gets 403 guest_model_not_allowed before routing, PII masking or Metering run. Routers are never public models, so they are always refused.

Tests

• go test -race ./... and go vet ./... pass.
• TestStreamingUsageRetention (real HTTP through the production OpenAI adapter) now proves a client that hangs up mid-answer is charged.
• New tests: input_estimate_test.go, proxy_hangup_test.go (each fails with its part of the fix removed), output_limit_test.go (guest limit written and held; signed-in requests untouched) and guest_models_test.go (unticked model, router, free model, other endpoint refused with no routing, reservation or provider call; fails with the check removed).
• End to end with real DeepSeek: with the guest limit set to 16, a guest asking for 300 lines got 16 tokens and finish_reason: length.
• Local end-to-end run against real DeepSeek with dappnode-nexus#156: results in a comment there.

Squash on merge: the branch history includes a default limit for every request that was later removed.

🤖 Generated with Claude Code