[paultancre-bt]

Follow-up to https://github.com/braintrustdata/braintrust/pull/24075, which made the playground and "+ Experiment" dialog send max_concurrency to remote eval dev servers. This PR makes the dev server honor it.

Companion JS SDK PR: braintrustdata/braintrust-sdk-javascript#2730

Context

The playground Max concurrency slider and the "+ Experiment" Max concurrency field had no effect on remote eval tasks. braintrustdata/braintrust#24075 (merged) made the UI send max_concurrency in the POST /eval request to the dev server, but parse_eval_body never read the field and run_eval built the EvalAsync kwargs only from the registered evaluator, so it was ignored. The value set on the Eval in code always won. A customer lowered the slider to avoid Azure 429s and saw no change (Pylon 20534, COR-288).

Description

Parse max_concurrency from the eval request and pass it to EvalAsync.

The request value can lower the evaluator's limit but never raise it (min(request, evaluator); if the evaluator sets no limit, the request value applies). This is deliberate: the playground always sends a value and the slider defaults to 10, so letting the request override outright would silently raise every code-set limit below 10 (e.g. max_concurrency=5 → 10) as soon as users upgrade the SDK. The trade-off is that the UI can't push concurrency above what's in code.

Other details:

• null or a missing value keeps the evaluator's limit, which also covers older UIs that don't send the field.
• Fractional values are rounded down (the "+ Experiment" field is a free number input validated only as >= 1). Values below 1 or non-numbers are rejected with a 400.

Testing

• New tests cover: request lowers the limit, request can't raise it, request applies when the evaluator has no limit, null or omitted request keeps the evaluator's limit, rounding down, and invalid values. They fail without the fix and pass with it.
• nox -s test_cli passes (43 passed, 2 skipped); make lint is clean.
• End to end: ran the real dev server (eval --dev) from this branch against a Braintrust org, with a 12-row eval whose task records how many calls are in flight, and sent POST /eval requests shaped like the playground's. Same run against the released Python 0.49.2 (PyPI) for comparison:

Evaluator limit in code	Request max_concurrency	Expected peak	This branch	Released
5	2	2	2	5
5	10	5	5	5
5	null	5	5	5
5	omitted	5	5	5
5	2.5	2	2	5
none	3	3	3	12
none	omitted	12	12	12

Refs: COR-288

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-05T11:35:53.513404Z	d96b8a6	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review" or "Codex (@codex) security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d96b8a6634

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "Codex (@codex) address that feedback".