Skip to content

fix(openai): handle null choices in chat completion responses - #1745

Merged
hassiebp merged 1 commit into
mainfrom
fix/openai-choices-none
Jul 8, 2026
Merged

hassiebp merged 1 commit into
mainfrom
fix/openai-choices-none

Conversation

@hassiebp

@hassiebp hassiebp commented Jul 8, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

LFE-10734: the OpenAI wrapper crashes when a chat completion response contains "choices": null, which some OpenAI-compatible providers/gateways return. The OpenAI v1 SDK constructs response models leniently, so the None reaches _get_langfuse_data_from_default_response, where len(choices) raises:

TypeError: object of type 'NoneType' has no len()

The exception is re-raised by the wrapper's error handler, so the user's create() call fails even though the underlying API request succeeded.

Root cause

response.get("choices", []) only falls back to the default when the key is missing — an explicit null value returns None as-is.

Fix

Use response.get("choices") or [] in:

  • the non-streaming chat path (len(choices) crash from the issue)
  • the non-streaming legacy completion path (same pattern)
  • the streaming chunk path (for choice in choices would crash the same way)
  • the embedding data extraction (identical latent bug)

Tests

  • Added a regression test for a non-streaming chat completion with choices=None: the call succeeds, the response is returned unchanged, and the generation span is exported with model and usage (no ERROR level).
  • Added a regression test for a stream containing a choices=None chunk: iteration completes and the span still captures the output and finish reason from subsequent chunks.

🤖 Generated with Claude Code

Greptile Summary

This PR fixes a TypeError crash in the OpenAI wrapper when an API response contains an explicit "choices": null (or "data": null), as returned by some OpenAI-compatible gateways. The root cause was dict.get("key", []) returning None when the key is present but explicitly null, while dict.get("key") or [] correctly falls back to an empty list in that case.

  • Four call sites in langfuse/openai.py are updated: the streaming chunk path, the legacy completion path, the chat completion path, and the embedding data extraction path.
  • Two regression tests are added — one for a non-streaming chat response with choices=None, and one for a stream containing a choices=None chunk — verifying that neither the call nor span export fails.

Confidence Score: 5/5

Safe to merge — minimal, targeted change that fixes a real crash without altering any happy-path behavior.

The four changed lines are strictly additive guard rails; [] or [] and some_list or [] both produce the same result as before, so existing callers are unaffected. The regression tests exercise both the streaming and non-streaming code paths that previously crashed. No other code paths appear to share this null-dereference pattern.

No files require special attention.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Client
    participant LangfuseWrapper as Langfuse OpenAI Wrapper
    participant OpenAI as OpenAI-Compatible API

    Client->>LangfuseWrapper: chat.completions.create(...)
    LangfuseWrapper->>OpenAI: HTTP POST /v1/chat/completions
    OpenAI-->>LangfuseWrapper: "{"choices": null, "usage": {...}}"

    alt Before fix
        LangfuseWrapper->>LangfuseWrapper: "choices = response.get("choices", []) → None"
        LangfuseWrapper->>LangfuseWrapper: len(None) → TypeError raised
        LangfuseWrapper-->>Client: Exception (call fails)
    else After fix
        LangfuseWrapper->>LangfuseWrapper: "choices = response.get("choices") or [] → []"
        LangfuseWrapper->>LangfuseWrapper: "len([]) == 0, skip extraction"
        LangfuseWrapper->>LangfuseWrapper: Export span with model + usage only
        LangfuseWrapper-->>Client: response (unchanged)
    end
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Client
    participant LangfuseWrapper as Langfuse OpenAI Wrapper
    participant OpenAI as OpenAI-Compatible API

    Client->>LangfuseWrapper: chat.completions.create(...)
    LangfuseWrapper->>OpenAI: HTTP POST /v1/chat/completions
    OpenAI-->>LangfuseWrapper: "{"choices": null, "usage": {...}}"

    alt Before fix
        LangfuseWrapper->>LangfuseWrapper: "choices = response.get("choices", []) → None"
        LangfuseWrapper->>LangfuseWrapper: len(None) → TypeError raised
        LangfuseWrapper-->>Client: Exception (call fails)
    else After fix
        LangfuseWrapper->>LangfuseWrapper: "choices = response.get("choices") or [] → []"
        LangfuseWrapper->>LangfuseWrapper: "len([]) == 0, skip extraction"
        LangfuseWrapper->>LangfuseWrapper: Export span with model + usage only
        LangfuseWrapper-->>Client: response (unchanged)
    end
Loading

Reviews (1): Last reviewed commit: "fix(openai): handle null choices in chat..." | Re-trigger Greptile

Some OpenAI-compatible providers and gateways return "choices": null
in the response body. Since dict.get() only falls back to its default
for missing keys (not explicit None values), the wrapper crashed with
"TypeError: object of type 'NoneType' has no len()" and failed the
user's call even though the underlying API request succeeded.

Guard the choices extraction in both the non-streaming and streaming
paths (and the analogous embedding data extraction) with `or []`.

Fixes LFE-10734

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Straightforward defensive fix — swaps dict.get(key, []) for dict.get(key) or [] to handle explicit null values from OpenAI-compatible providers, with regression tests for both non-streaming and streaming paths.

Extended reasoning...

Overview

This PR modifies four call sites in langfuse/openai.py where the wrapper extracted choices (or data for embeddings) from response payloads. It replaces response.get("choices", []) with response.get("choices") or [] in the non-streaming chat, non-streaming completion, streaming chunk, and embedding paths. Two regression tests are added to tests/unit/test_openai.py covering the non-streaming chat case and a stream containing a choices=None chunk.

Security risks

None. The change is a defensive null-check on already-received response data — no auth, crypto, permissions, or input validation surface is affected. It reduces error surface rather than expanding it.

Level of scrutiny

Low. This is a minimal, mechanical fix following an established idiom for handling explicitly-null JSON values. The behavior change is well-scoped (falsy → empty list), the four modified sites are structurally identical, and the tests verify the intended behavior (no crash, span still exported with model/usage). The bug hunter found nothing.

Other factors

The PR description clearly identifies the root cause (dict.get default only triggers on missing keys, not explicit None) and applies the fix uniformly across all analogous call sites, including latent versions of the same bug in the completion and embedding paths. Test coverage is proportionate — one test per meaningfully distinct code path.

@hassiebp
hassiebp merged commit 6659125 into main Jul 8, 2026
30 of 32 checks passed
@hassiebp
hassiebp deleted the fix/openai-choices-none branch July 8, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant