[Fix] Remove min_tokens from prefill request to avoid max_tokens conflict in PD disaggregation mode

[NinSang]

Problem

When using the PD disaggregation mode, the proxy server modifies the prefill request by setting max_tokens=1 (since the prefill stage only
needs to compute KV cache, not generate full output). However, if the original request contains a min_tokens parameter, it is passed through
unchanged. This causes a conflict where min_tokens > max_tokens (e.g., min_tokens=100, max_tokens=1), resulting in an Internal Server Error
(HTTP 500) from the vLLM backend.

Reproduction:

curl -s -X POST http://:/v1/chat/completions
-H 'Content-Type: application/json'
-d '{"model":"qwen3.6","messages":[{"role":"user","content":"你好"}],"stream":true,"max_tokens":500,"min_tokens":100}'

The same request without min_tokens works correctly.

Fix

Remove min_tokens from the prefill request data in both places where max_tokens is overridden to 1:

1. send_request_to_service() — utility function for sending requests to backend services
2. _handle_completions() — prefill request construction in PD disaggregation flow

The min_tokens constraint is still respected in the decode stage, since the original request (with min_tokens intact) is forwarded to the
decoder for full output generation.

Diff

diff --git a/ucm/pd/toy_proxy_server.py b/ucm/pd/toy_proxy_server.py
index f338bf5..efa7593 100644
--- a/ucm/pd/toy_proxy_server.py
+++ b/ucm/pd/toy_proxy_server.py
@@ -200,6 +200,8 @@ async def send_request_to_service(
req_data["max_tokens"] = 1
if "stream_options" in req_data:
del req_data["stream_options"]

• if "min_tokens" in req_data:

•    del req_data["min_tokens"]
  

  headers = {
  "Authorization": f"Bearer {os.environ.get('OPENAI_API_KEY')}",
  "X-Request-Id": request_id,
  @@ -252,6 +254,8 @@ async def _handle_completions(api: str, request: Request):
  prefill_req_data["max_tokens"] = 1
  if "stream_options" in prefill_req_data:
  del prefill_req_data["stream_options"]

•        if "min_tokens" in prefill_req_data:
  

•            del prefill_req_data["min_tokens"]
  
         response = await prefill_client_info["client"].post(
             api, json=prefill_req_data, headers=headers
  

Files Changed

• ucm/pd/toy_proxy_server.py — 2 locations, +4 lines