Repository navigation
fix(python): preserve final JSON Lines records without a trailing newline - #126
Draft
Kayvan-Zahiri wants to merge 1 commit into
Draft
Kayvan-Zahiri wants to merge 1 commit into
Kayvan-Zahiri wants to merge 1 commit into
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Python JSON Lines reader drops the final record when the response ends without a trailing newline. The EOF check uses
position, but that cursor is reset to zero after each chunk. The remaining buffer is therefore never decoded. A single-record response can silently produce no events, and a malformed final record is silently discarded.The JSON Lines format recommends a final newline but does not require one.
What changed
Check for non-whitespace buffered data at EOF in both synchronous and asynchronous reader templates, then pass it through the existing decoder. Empty streams and whitespace-only tails retain their existing behavior; malformed final JSON raises the decoder's error.
Add 50 offline regression cases to the existing Python primary streaming tests: newline/no-newline, multiple records, LF/CRLF, UTF-8, empty input, whitespace tails, malformed final JSON, and different byte-chunk boundaries. Add the
jsonlResponsespatch changeset using the repository command.Regenerated the Python review SDK with the supported command; it contains no JSON Lines operation and produces no tracked diff. Separately generated a minimal synthetic JSON Lines SDK before and after the template change and regenerated the primary test SDK.
Testing
timeoutconfiguration warning because pytest-timeout is not installed in the isolated test environment.npm run format,make check-template-pythonv2, review SDK regeneration/integrity validation, andgit diff --checkpass.make lintpassed its public-input guard, then stopped becausegolangci-lintis unavailable. The repository-wide lint gate and Docker-backed multi-target integration matrix are not claimed as passing.Public-safety check
git diff --check.Summary by cubic
Fixes the Python JSON Lines reader dropping the final record when a response ends without a trailing newline. The EOF check relied on a position cursor that resets to zero after each chunk, so remaining buffered data was never decoded. Buffered non-whitespace data at EOF now passes through the existing decoder; empty streams and whitespace-only tails behave as before, and malformed final JSON raises the decoder's error instead of being silently discarded.
Adds 50 offline regression cases covering newline/no-newline, multiple records, LF/CRLF, UTF-8, empty input, whitespace tails, malformed final JSON, and chunk boundaries, plus a patch changeset for
jsonlResponses.Written for commit 1a007a5. Summary will update on new commits.