Skip to content

BYOK: autopilot nudge turn re-serializes previously-sent transcript items, breaking prompt caching #4500

Description

@dzamoshchin

Describe the bug

On the --autopilot completion-nudge turn, the CLI rebuilds the whole input array from parsed internal state instead of resending prior items byte-for-byte (BYOK, responses wire API). Item ids are preserved, bytes are not. The same tool call across requests:

request 1: '{"command": "echo \u00e9", "description": "unicode \u00e9 kept?"}'
request 5: '{"command":"echo é","description":"unicode é kept?"}'

The collapsed escape shows a JSON.parse / JSON.stringify round trip. Tool outputs are also truncated on the same turn.

Prompt caching is exact-prefix, so it breaks at the first rewritten item. Measured against the Responses API (gpt-5-mini), changing only the whitespace in the first tool call's arguments, same prompt_cache_key:

conversation prompt tokens cached (identical resend) cached (after rewrite) tokens lost
10k 18,060 17,920 5,632 12,288 (68%)
100k 114,820 114,688 5,632 109,056 (95%)

The post-rewrite value is constant (the static prefix ahead of the rewritten item), so the loss grows with session length. This recurs on every nudge turn. Non-nudge turns and sessions without --autopilot are byte-stable.

Separately, on ordinary turns, multi-part reasoning summary arrays are echoed back space-joined into one part.

Affected version

1.0.79

Steps to reproduce the behavior

  1. Point the CLI at an OpenAI-compatible endpoint behind a request-logging proxy (COPILOT_PROVIDER_BASE_URL, COPILOT_PROVIDER_TYPE=openai, COPILOT_PROVIDER_WIRE_API=responses)
  2. Run copilot -p "..." --allow-all-tools --no-ask-user -s --stream off --autopilot with a prompt that produces a few tool calls, and have the model answer once without calling task_complete so the nudge fires
  3. Diff the input arrays across logged requests: on the nudge turn, previously sent function_call.arguments are re-serialized and a rejected call's output is truncated

Expected behavior

Previously sent items are resent byte-for-byte, with only new items appended.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:modelsModel selection, availability, switching, rate limits, and model-specific behaviorarea:non-interactiveNon-interactive mode (-p), CI/CD, ACP protocol, and headless automation

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions