Skip to content

Guest chat: guest requests always carry an output limit - #21

Closed
Marketen wants to merge 1 commit into
feat/request-sized-reservationsfrom
feat/guest-accounts
Closed

Marketen wants to merge 1 commit into
feat/request-sized-reservationsfrom
feat/guest-accounts

Conversation

@Marketen

@Marketen Marketen commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Part of guest chat: a landing-page visitor chats without signing up. Companion PRs: dappnode/dappnode-nexus (control plane, metering, UI) and dappnode/infrastructure (settings). Supersedes #18. Stacked on #19, which supplies the prompt estimate every hold uses.

What this changes

A guest is an ordinary account (is_guest from Metering) with a small credit and no login. Its requests are ordinary requests with one difference: a chat request always carries an output limit of at most GUEST_MAX_OUTPUT_TOKENS (default 8,192), written into the request so the provider enforces it.

Why: Metering holds credit for the request's output limit. Without one, the request goes upstream unbounded and a longer answer is charged as debt (dappnode-nexus#151). A guest can't pay a debt, so the limit keeps a guest's answer within its hold.

Why 8,192 and not lower:

  • Over a week of production traffic on DeepSeek V4.1 Flash, answers averaged about 1,100 output tokens; 28% used 1,024 or more, and the longest was about 15,000.
  • Reasoning counts toward the limit. Against the real DeepSeek API, a hard question at 1,024 tokens spent all of them reasoning and returned an empty answer.

Everything else is untouched: any model the credit covers, tools, images, the key's PII mode. #18 restricted models, fields and request size; none of that is needed once the hold is honest.

Tests

  • guest_output_test.go: the limit is written (absent, larger, smaller kept, max_completion_tokens name kept), Metering holds for it, and other fields pass through; signed-in requests get no limit.
  • The is_guest flag survives the Metering response, and an older Metering without it means "not a guest".
  • go test ./... and go vet ./... pass.

🤖 Generated with Claude Code

Guest chat accounts (is_guest, from Metering) are landing-page visitors with
a small credit and no login. Their requests are ordinary requests with one
difference: a chat request always carries an output limit of at most
GUEST_MAX_OUTPUT_TOKENS (default 8,192), written into the request so the
provider enforces it. Metering holds credit for that limit, so a guest's
answer can never cost more than its hold; without a limit the request would
go upstream unbounded and could leave a debt nobody pays.

8,192 covers almost every answer, reasoning included: over a week of
production traffic on DeepSeek V4.1 Flash, answers averaged about 1,100
output tokens and the longest was about 15,000. A 1,024 limit left hard
questions with an empty answer, all of it spent on reasoning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Marketen

Marketen commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Merged into #22.

@Marketen Marketen closed this Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant