[openai] Record cache-write and audio token usage

[1fanwang]

Description

Chat Completions telemetry omits cache-write and audio token counts, so users cannot observe those reported usage breakdowns. This records them for normal and streamed calls without changing aggregate input/output totals; streamed usage is applied during cleanup.

Text and image counts are not recorded: the SDK schema defines them, but the provider check below only confirms that Chat Completions returned audio and cache-write counts.

Fixes #603.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How has this been tested?

Testing Done

The OpenAI 3.13.0 SDK parsed synthetic JSON/SSE through a mock HTTP transport on macOS with Python 3.14.7. These fixtures check extraction, not provider behavior.

A provider check ran against an earlier head of this PR with Python 3.12 and OpenAI SDK 3.13.0. It made two Chat Completions calls with retries disabled, a 30-second timeout, and completion limits of 128 audio tokens and 8 text tokens. API usage matched the exported span attributes. The probe prints the usage counts shown below, then a final status line. To repeat it, set OPENAI_API_KEY through a secret manager, save the source below as provider_probe.py, and run:

uvx --with tox-uv tox -e py312-test-instrumentation-genai-openai-latest --notest
.tox/py312-test-instrumentation-genai-openai-latest/bin/python provider_probe.py

{"span_usage": {"gen_ai.usage.audio.input_tokens": 5, "gen_ai.usage.audio.output_tokens": 18, "gen_ai.usage.input_tokens": 19, "gen_ai.usage.output_tokens": 25}}
{"api_usage": {"audio_input_tokens": 5, "audio_output_tokens": 18}, "case": "audio"}
{"span_usage": {"gen_ai.usage.cache_write.input_tokens": 2404, "gen_ai.usage.input_tokens": 2407, "gen_ai.usage.output_tokens": 8, "gen_ai.usage.reasoning.output_tokens": 8}}
{"api_usage": {"cache_write_tokens": 2404}, "case": "cache_write"}

Save the source below as sdk_probe.py. From the repository root:

uv run tox -e py314-test-instrumentation-genai-openai-latest --notest
before=$(mktemp -d)
git archive d859ebce191f93fe4675bdf354f55b03bdbe468c instrumentation/opentelemetry-instrumentation-genai-openai/src | tar -x -C "$before"
PYTHONPATH="$before/instrumentation/opentelemetry-instrumentation-genai-openai/src" .tox/py314-test-instrumentation-genai-openai-latest/bin/python sdk_probe.py
.tox/py314-test-instrumentation-genai-openai-latest/bin/python sdk_probe.py

Both probe runs exited 0. Before:

{"path": "sync-response", "usage": {"gen_ai.usage.cache_read.input_tokens": 5, "gen_ai.usage.input_tokens": 100, "gen_ai.usage.output_tokens": 20, "gen_ai.usage.reasoning.output_tokens": 3}}
{"path": "sync-stream", "usage": {"gen_ai.usage.cache_read.input_tokens": 5, "gen_ai.usage.input_tokens": 100, "gen_ai.usage.output_tokens": 20, "gen_ai.usage.reasoning.output_tokens": 3}}
{"path": "async-response", "usage": {"gen_ai.usage.cache_read.input_tokens": 5, "gen_ai.usage.input_tokens": 100, "gen_ai.usage.output_tokens": 20, "gen_ai.usage.reasoning.output_tokens": 3}}
{"path": "async-stream", "usage": {"gen_ai.usage.cache_read.input_tokens": 5, "gen_ai.usage.input_tokens": 100, "gen_ai.usage.output_tokens": 20, "gen_ai.usage.reasoning.output_tokens": 3}}

After:

{"path": "sync-response", "usage": {"gen_ai.usage.audio.input_tokens": 10, "gen_ai.usage.audio.output_tokens": 2, "gen_ai.usage.cache_read.input_tokens": 5, "gen_ai.usage.cache_write.input_tokens": 10, "gen_ai.usage.input_tokens": 100, "gen_ai.usage.output_tokens": 20, "gen_ai.usage.reasoning.output_tokens": 3}}
{"path": "sync-stream", "usage": {"gen_ai.usage.audio.input_tokens": 10, "gen_ai.usage.audio.output_tokens": 2, "gen_ai.usage.cache_read.input_tokens": 5, "gen_ai.usage.cache_write.input_tokens": 10, "gen_ai.usage.input_tokens": 100, "gen_ai.usage.output_tokens": 20, "gen_ai.usage.reasoning.output_tokens": 3}}
{"path": "async-response", "usage": {"gen_ai.usage.audio.input_tokens": 10, "gen_ai.usage.audio.output_tokens": 2, "gen_ai.usage.cache_read.input_tokens": 5, "gen_ai.usage.cache_write.input_tokens": 10, "gen_ai.usage.input_tokens": 100, "gen_ai.usage.output_tokens": 20, "gen_ai.usage.reasoning.output_tokens": 3}}
{"path": "async-stream", "usage": {"gen_ai.usage.audio.input_tokens": 10, "gen_ai.usage.audio.output_tokens": 2, "gen_ai.usage.cache_read.input_tokens": 5, "gen_ai.usage.cache_write.input_tokens": 10, "gen_ai.usage.input_tokens": 100, "gen_ai.usage.output_tokens": 20, "gen_ai.usage.reasoning.output_tokens": 3}}

Reproducer source: sdk_probe.py

import asyncio
import json

import httpx2
from openai import AsyncOpenAI, OpenAI

from opentelemetry.instrumentation.genai.openai import OpenAIInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.sdk.trace.sampling import ALWAYS_ON
from opentelemetry.test_util_genai.instrumentor import instrument


def respond(request: httpx2.Request) -> httpx2.Response:
    usage: dict[str, object] = {
        "prompt_tokens": 100, "completion_tokens": 20, "total_tokens": 120,
        "prompt_tokens_details": {
            "cached_tokens": 5, "cache_write_tokens": 10, "audio_tokens": 10,
            "text_tokens": 70, "image_tokens": 20,
        },
        "completion_tokens_details": {
            "audio_tokens": 2, "reasoning_tokens": 3, "text_tokens": 18,
        },
    }
    common: dict[str, object] = {
        "id": "chatcmpl-probe", "created": 1, "model": "gpt-4",
    }
    if json.loads(request.content).get("stream"):
        chunks = [
            {**common, "object": "chat.completion.chunk", "choices": [
                {"index": 0, "delta": {"content": "hello"}, "finish_reason": "stop"}
            ]},
            {**common, "object": "chat.completion.chunk", "choices": [], "usage": usage},
        ]
        return httpx2.Response(
            status_code=200, headers={"content-type": "text/event-stream"},
            content="".join(f"data: {json.dumps(chunk)}\n\n" for chunk in chunks)
            + "data: [DONE]\n\n",
        )
    return httpx2.Response(
        status_code=200,
        json={**common, "object": "chat.completion", "usage": usage, "choices": [
            {"index": 0, "message": {"role": "assistant", "content": "hello"},
             "finish_reason": "stop"}
        ]},
    )


exporter = InMemorySpanExporter()
provider = TracerProvider(sampler=ALWAYS_ON)
provider.add_span_processor(SimpleSpanProcessor(exporter))


def report(path: str) -> None:
    (span,) = exporter.get_finished_spans()
    attributes = {key: value for key, value in span.attributes.items()
                  if key.startswith("gen_ai.usage.")}
    print(json.dumps({"path": path, "usage": attributes}, sort_keys=True))
    exporter.clear()


async def run_async() -> None:
    async with AsyncOpenAI(
        api_key="test", base_url="http://openai.test/v1", max_retries=0,
        http_client=httpx2.AsyncClient(transport=httpx2.MockTransport(respond)),
    ) as client:
        for streaming in (False, True):
            response = await client.chat.completions.create(
                model="gpt-4", messages=[{"role": "user", "content": "hello"}],
                stream=streaming,
                **({"stream_options": {"include_usage": True}} if streaming else {}),
            )
            if streaming:
                async for _ in response:
                    pass
            report(path=f"async-{'stream' if streaming else 'response'}")


with instrument(
    instrumentor=OpenAIInstrumentor(), tracer_provider=provider,
    logger_provider=None, meter_provider=None,
):
    with OpenAI(
        api_key="test", base_url="http://openai.test/v1", max_retries=0,
        http_client=httpx2.Client(transport=httpx2.MockTransport(respond)),
    ) as client:
        for streaming in (False, True):
            response = client.chat.completions.create(
                model="gpt-4", messages=[{"role": "user", "content": "hello"}],
                stream=streaming,
                **({"stream_options": {"include_usage": True}} if streaming else {}),
            )
            if streaming:
                list(response)
            report(path=f"sync-{'stream' if streaming else 'response'}")
    asyncio.run(run_async())
provider.shutdown()

• Test A: The SDK probe above exports cache-write and audio counts on sync/async responses and streams, with unchanged totals and no text/image attributes.
• Test B: Two live Chat Completions responses reported audio and cache-write counts that matched the exported span attributes, shown above.

Reproducer source: provider_probe.py

from __future__ import annotations

import base64
import io
import json
import math
import struct
import wave
from typing import Any

import openai
from openai import OpenAI

from opentelemetry.instrumentation.genai.openai import (
    OpenAIInstrumentor,
    utils,
)
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
    InMemorySpanExporter,
)
from opentelemetry.sdk.trace.sampling import ALWAYS_ON
from opentelemetry.test_util_genai.instrumentor import instrument


def audio_sample() -> str:
    buffer = io.BytesIO()
    sample_rate = 24_000
    sample_count = sample_rate // 2
    with wave.open(buffer, "wb") as audio_file:
        audio_file.setnchannels(1)
        audio_file.setsampwidth(2)
        audio_file.setframerate(sample_rate)
        audio_file.writeframes(
            b"".join(
                struct.pack(
                    "<h",
                    int(
                        4_000
                        * math.sin(2 * math.pi * 440 * index / sample_rate)
                    ),
                )
                for index in range(sample_count)
            )
        )
    return base64.b64encode(buffer.getvalue()).decode("ascii")


def span_usage(exporter: InMemorySpanExporter) -> dict[str, int]:
    (span,) = exporter.get_finished_spans()
    usage: dict[str, Any] = {
        key: value
        for key, value in (span.attributes or {}).items()
        if key.startswith("gen_ai.usage.")
    }
    if not all(type(value) is int for value in usage.values()):
        raise AssertionError(f"usage attributes must be integers: {usage}")
    print(json.dumps({"span_usage": usage}, sort_keys=True), flush=True)
    exporter.clear()
    return usage


def main() -> None:
    if (
        "instrumentation/opentelemetry-instrumentation-genai-openai/src/"
        not in utils.__file__
    ):
        raise AssertionError(
            "probe did not import the PR's OpenAI instrumentation"
        )
    exporter = InMemorySpanExporter()
    provider = TracerProvider(sampler=ALWAYS_ON)
    provider.add_span_processor(SimpleSpanProcessor(exporter))
    with instrument(
        instrumentor=OpenAIInstrumentor(),
        tracer_provider=provider,
        logger_provider=None,
        meter_provider=None,
    ):
        with OpenAI(max_retries=0, timeout=30.0) as client:
            audio_response = client.chat.completions.create(
                model="gpt-audio-1.5",
                modalities=["text", "audio"],
                audio={"voice": "alloy", "format": "wav"},
                max_completion_tokens=128,
                store=False,
                messages=[
                    {
                        "role": "user",
                        "content": [
                            {"type": "text", "text": "Say beep."},
                            {
                                "type": "input_audio",
                                "input_audio": {
                                    "data": audio_sample(),
                                    "format": "wav",
                                },
                            },
                        ],
                    }
                ],
            )
            if audio_response.usage is None:
                raise AssertionError("audio response did not include usage")
            audio_details = audio_response.usage
            audio_input = audio_details.prompt_tokens_details.audio_tokens
            audio_output = audio_details.completion_tokens_details.audio_tokens
            if type(audio_input) is not int or type(audio_output) is not int:
                raise AssertionError(
                    "provider audio usage fields must be integers"
                )
            if audio_input <= 0 or audio_output <= 0:
                raise AssertionError(
                    "provider did not report positive audio usage"
                )
            audio_span_usage = span_usage(exporter=exporter)
            if (
                audio_span_usage.get("gen_ai.usage.audio.input_tokens")
                != audio_input
                or audio_span_usage.get("gen_ai.usage.audio.output_tokens")
                != audio_output
            ):
                raise AssertionError(
                    "audio usage attributes differ from API usage"
                )
            print(
                json.dumps(
                    {
                        "case": "audio",
                        "api_usage": {
                            "audio_input_tokens": audio_input,
                            "audio_output_tokens": audio_output,
                        },
                    },
                    sort_keys=True,
                ),
                flush=True,
            )

            cache_response = client.chat.completions.create(
                model="gpt-6-luna",
                max_completion_tokens=8,
                prompt_cache_options={"mode": "explicit"},
                store=False,
                messages=[
                    {
                        "role": "user",
                        "content": [
                            {
                                "type": "text",
                                "text": "OpenTelemetry usage field validation. "
                                * 400,
                                "prompt_cache_breakpoint": {
                                    "mode": "explicit"
                                },
                            }
                        ],
                    }
                ],
            )
            if cache_response.usage is None:
                raise AssertionError("cache response did not include usage")
            cache_write = (
                cache_response.usage.prompt_tokens_details.cache_write_tokens
            )
            if type(cache_write) is not int:
                raise AssertionError(
                    "provider cache-write usage must be an integer"
                )
            if cache_write <= 0:
                raise AssertionError(
                    "provider did not report positive cache-write usage"
                )
            cache_span_usage = span_usage(exporter=exporter)
            if cache_span_usage.get(
                "gen_ai.usage.cache_write.input_tokens"
            ) != (cache_write):
                raise AssertionError(
                    "cache-write attribute differs from API usage"
                )
            print(
                json.dumps(
                    {
                        "case": "cache_write",
                        "api_usage": {"cache_write_tokens": cache_write},
                    },
                    sort_keys=True,
                ),
                flush=True,
            )
    provider.shutdown()
    print(
        json.dumps(
            {
                "openai_sdk_version": openai.__version__,
                "instrumentation_source": utils.__file__,
                "provider_calls": 2,
                "status": "passed",
            },
            sort_keys=True,
        ),
        flush=True,
    )


if __name__ == "__main__":
    main()

Local conformance scenarios skipped because Weaver is unavailable.

Checklist

See CONTRIBUTING.md
for the style guide, changelog guidance, and more.

• Followed the style guidelines of this project
• Changelog updated if the change requires an entry
• Unit tests added
• Documentation updated (changelog; no public API or configuration change)

[opentelemetry-pr-dashboard]

Pull request dashboard status

Merged · refreshed 2026-10-15 22:27 UTC

Status above doesn't look right?

• Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

[Copilot]

🔵 Needs a closer look

Add integer type assertions for exported usage attributes in the regression tests.

Pull request overview

Adds OpenAI Chat Completions cache-write and modality token usage recording for synchronous, asynchronous, and streaming calls.

Changes:

• Extracts detailed usage fields into telemetry.
• Adds local HTTP/SSE regression coverage.
• Adds a changelog entry.

File summaries

File	Description
instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_chat_token_usage.py	Tests usage-detail variants across call modes.
instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/utils.py	Extracts detailed usage fields.
instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/patch.py	Applies extraction to non-streaming responses.
instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/chat_wrappers.py	Applies extraction to streaming responses.
instrumentation/opentelemetry-instrumentation-genai-openai/.changelog/603.added	Documents the feature.

Review details

Suppressed comments (1)

instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_chat_token_usage.py:222

• This only checks equality, so a bool or float usage value would still pass (True == 1 and 1.0 == 1). The new semconv usage attributes must be integers; assert the types of the exported values here so the regression test catches invalid attribute values.

    assert actual == expected

• Files reviewed: 5/5 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[lmolkova]

Please make sure to follow existing test practices in this repo

[github-actions]

This PR has been automatically marked as stale because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 days of this comment.
If you're still working on this, please add a comment or push new commits.

[1fanwang]

Please make sure to follow existing test practices in this repo

@lmolkova the tests now use the instrumentation fixtures and SDK response types for normal and streamed calls, including async coverage. The body distinguishes the synthetic SDK probe from live provider evidence; no provider recording is claimed.