Repository navigation
fix(openai): handle null choices in chat completion responses - #1745
Conversation
Some OpenAI-compatible providers and gateways return "choices": null in the response body. Since dict.get() only falls back to its default for missing keys (not explicit None values), the wrapper crashed with "TypeError: object of type 'NoneType' has no len()" and failed the user's call even though the underlying API request succeeded. Guard the choices extraction in both the non-streaming and streaming paths (and the analogous embedding data extraction) with `or []`. Fixes LFE-10734 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@claude review |
There was a problem hiding this comment.
Straightforward defensive fix — swaps dict.get(key, []) for dict.get(key) or [] to handle explicit null values from OpenAI-compatible providers, with regression tests for both non-streaming and streaming paths.
Extended reasoning...
Overview
This PR modifies four call sites in langfuse/openai.py where the wrapper extracted choices (or data for embeddings) from response payloads. It replaces response.get("choices", []) with response.get("choices") or [] in the non-streaming chat, non-streaming completion, streaming chunk, and embedding paths. Two regression tests are added to tests/unit/test_openai.py covering the non-streaming chat case and a stream containing a choices=None chunk.
Security risks
None. The change is a defensive null-check on already-received response data — no auth, crypto, permissions, or input validation surface is affected. It reduces error surface rather than expanding it.
Level of scrutiny
Low. This is a minimal, mechanical fix following an established idiom for handling explicitly-null JSON values. The behavior change is well-scoped (falsy → empty list), the four modified sites are structurally identical, and the tests verify the intended behavior (no crash, span still exported with model/usage). The bug hunter found nothing.
Other factors
The PR description clearly identifies the root cause (dict.get default only triggers on missing keys, not explicit None) and applies the fix uniformly across all analogous call sites, including latent versions of the same bug in the completion and embedding paths. Test coverage is proportionate — one test per meaningfully distinct code path.
Problem
LFE-10734: the OpenAI wrapper crashes when a chat completion response contains
"choices": null, which some OpenAI-compatible providers/gateways return. The OpenAI v1 SDK constructs response models leniently, so theNonereaches_get_langfuse_data_from_default_response, wherelen(choices)raises:The exception is re-raised by the wrapper's error handler, so the user's
create()call fails even though the underlying API request succeeded.Root cause
response.get("choices", [])only falls back to the default when the key is missing — an explicitnullvalue returnsNoneas-is.Fix
Use
response.get("choices") or []in:len(choices)crash from the issue)for choice in choiceswould crash the same way)dataextraction (identical latent bug)Tests
choices=None: the call succeeds, the response is returned unchanged, and the generation span is exported with model and usage (no ERROR level).choices=Nonechunk: iteration completes and the span still captures the output and finish reason from subsequent chunks.🤖 Generated with Claude Code
Greptile Summary
This PR fixes a
TypeErrorcrash in the OpenAI wrapper when an API response contains an explicit"choices": null(or"data": null), as returned by some OpenAI-compatible gateways. The root cause wasdict.get("key", [])returningNonewhen the key is present but explicitlynull, whiledict.get("key") or []correctly falls back to an empty list in that case.langfuse/openai.pyare updated: the streaming chunk path, the legacy completion path, the chat completion path, and the embedding data extraction path.choices=None, and one for a stream containing achoices=Nonechunk — verifying that neither the call nor span export fails.Confidence Score: 5/5
Safe to merge — minimal, targeted change that fixes a real crash without altering any happy-path behavior.
The four changed lines are strictly additive guard rails;
[] or []andsome_list or []both produce the same result as before, so existing callers are unaffected. The regression tests exercise both the streaming and non-streaming code paths that previously crashed. No other code paths appear to share this null-dereference pattern.No files require special attention.
Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant Client participant LangfuseWrapper as Langfuse OpenAI Wrapper participant OpenAI as OpenAI-Compatible API Client->>LangfuseWrapper: chat.completions.create(...) LangfuseWrapper->>OpenAI: HTTP POST /v1/chat/completions OpenAI-->>LangfuseWrapper: "{"choices": null, "usage": {...}}" alt Before fix LangfuseWrapper->>LangfuseWrapper: "choices = response.get("choices", []) → None" LangfuseWrapper->>LangfuseWrapper: len(None) → TypeError raised LangfuseWrapper-->>Client: Exception (call fails) else After fix LangfuseWrapper->>LangfuseWrapper: "choices = response.get("choices") or [] → []" LangfuseWrapper->>LangfuseWrapper: "len([]) == 0, skip extraction" LangfuseWrapper->>LangfuseWrapper: Export span with model + usage only LangfuseWrapper-->>Client: response (unchanged) end%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant Client participant LangfuseWrapper as Langfuse OpenAI Wrapper participant OpenAI as OpenAI-Compatible API Client->>LangfuseWrapper: chat.completions.create(...) LangfuseWrapper->>OpenAI: HTTP POST /v1/chat/completions OpenAI-->>LangfuseWrapper: "{"choices": null, "usage": {...}}" alt Before fix LangfuseWrapper->>LangfuseWrapper: "choices = response.get("choices", []) → None" LangfuseWrapper->>LangfuseWrapper: len(None) → TypeError raised LangfuseWrapper-->>Client: Exception (call fails) else After fix LangfuseWrapper->>LangfuseWrapper: "choices = response.get("choices") or [] → []" LangfuseWrapper->>LangfuseWrapper: "len([]) == 0, skip extraction" LangfuseWrapper->>LangfuseWrapper: Export span with model + usage only LangfuseWrapper-->>Client: response (unchanged) endReviews (1): Last reviewed commit: "fix(openai): handle null choices in chat..." | Re-trigger Greptile