Repository navigation
Conversation
sglang/vLLM-style reasoning parsers can leave message.content as None when thinking is truncated or empty after </think>, which then crashes find_code_blocks via re.finditer. Fall back to reasoning_content.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--reasoning-parser qwen3) can returnmessage.content = Nonewhile putting the thinking trace inmessage.reasoning_content.find_code_blocks→re.finditer, and also into message history /RLMIteration.response, which crashes mid-run with:TypeError: expected string or bytes-like object, got 'NoneType'._extract_content()inOpenAIClientsocompletion/acompletionalways return astr: prefercontent, else fall back toreasoning_content, else"".Related to #166. Complementary to #175 (which guards
find_code_blocks); this is closer to Option B suggested in that issue — normalize at the LM-client boundary so every downstream consumer sees a string, and so reasoning text is not discarded whencontentis empty/None.Why not only #175?
#175 stops the regex crash, but
Nonecan still leak into trajectory / history / partial-answer tracking. For reasoning backends the model may have produced useful thinking with no post-</think>body; falling back toreasoning_contentpreserves that progress instead of treating the turn as empty.Root cause (verified)
Servers that split think vs answer often implement something like
content = text if text else None.contentbecomesNonewhen:finish_reason == "length");</think>is closed but no further tokens (or only whitespace) are emitted;</think>never appears before EOS, so the whole output is classified as reasoning.Reproduced against sglang + Qwen3.5 with forced small
max_tokens(truncation during think →content=None,reasoning_contentpopulated).Changes
rlm/clients/openai.py:_extract_content()+ use it in sync/async completion paths.Out of scope / follow-ups
message.contentpattern still exists inazure_openai.py/portkey.py— happy to extend in a follow-up if desired.find_code_blocksguard is still useful as a second line of defense for non-OpenAI clients (e.g. Cerebras as in find_code_blocks() crashes with TypeError when LM response is None #166).