Skip to content

Fix send_all byte offsets for typed memoryviews - #3544

Closed
AryanRG wants to merge 2 commits into
python-trio:mainfrom
AryanRG:fix/send-all-memoryview-bytes
Closed

AryanRG wants to merge 2 commits into
python-trio:mainfrom
AryanRG:fix/send-all-memoryview-bytes

Conversation

@AryanRG

@AryanRG AryanRG commented Oct 16, 2026

Copy link
Copy Markdown

Fixes #3543.

SocketStream.send_all() and POSIX FdStream.send_all() treat the byte count returned by a partial write as a memoryview element offset. With a contiguous typed view, the retry can skip data and return successfully after sending only part of the buffer. A real-pipe reproduction on macOS sent 524,288 bytes from a 1,048,576-byte H view on unchanged main.

Cast nonempty views to a one-dimensional byte view before slicing. This also handles multidimensional and scalar views. Preserve empty strided-view handling and release the internal views without releasing the caller's view. A release note is included.

Validation on macOS:

  • Six focused regression cases fail against base 4630d02872c0aa5778ce0afd702c5d532a2ccf60 and pass with the fix. Two empty-view compatibility cases pass in both versions. Tests cap real socket/pipe writes at three bytes for deterministic partial writes.
  • Both affected modules: 26 passed on Python 3.10 and 3.12.
  • Python 3.10 installed-wheel suite, including slow tests: 879 passed, 25 skipped, 2 xfailed.
  • Mypy over all 147 source files passes for Darwin, Linux, and Windows targets. Applicable configured pre-commit hooks pass.
  • Independent AI review: approved after an initial empty-strided-view regression was fixed and covered by tests.

Earlier full-suite attempts used the shell's Conda environment for Jedi or lacked the virtualenv's tools on PATH. The isolated wheel run above used the correct active virtualenv. Windows/Linux runtime execution remains for CI.

AI disclosure: this issue investigation, implementation, tests, and PR description were produced by an autonomous OpenAI Codex agent at the account owner's request. Validation and independent review were performed by AI agents; no human review is claimed.

Partial writes return bytes rather than memoryview elements. Normalize nonempty buffers to byte views before retrying while retaining empty-view handling and explicit view cleanup.

Fixes python-trio#3543. Developed and validated by an autonomous OpenAI Codex agent; independently reviewed by an AI agent.
@codecov

codecov Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00000%. Comparing base (4630d02) to head (1c8419e).

Additional details and impacted files
@@               Coverage Diff               @@
##                 main        #3544   +/-   ##
===============================================
  Coverage   100.00000%   100.00000%           
===============================================
  Files             128          128           
  Lines           19474        19529   +55     
  Branches         1323         1328    +5     
===============================================
+ Hits            19474        19529   +55     
Files with missing lines Coverage Δ
src/trio/_highlevel_socket.py 100.00000% <100.00000%> (ø)
src/trio/_tests/test_highlevel_socket.py 100.00000% <100.00000%> (ø)
src/trio/_tests/test_unix_pipes.py 100.00000% <100.00000%> (ø)
src/trio/_unix_pipes.py 100.00000% <100.00000%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Render inherited send_all method names as inline literals because their Sphinx object targets do not exist. The strict docs build reproduces the two warnings before this change and passes afterward; untouched base docs pass too.

Developed and validated by an autonomous OpenAI Codex agent; independently reviewed by an AI agent.
@A5rocks

A5rocks commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Hi! Thanks for the bug report and PR. I agree this is a bug, but I'm not willing to code review code someone else has not even reviewed. I think some parts of this PR is questionable (why not instantly cast? why not rely on truthiness instead of doing .nbytes?) so I'll close it.

@A5rocks A5rocks closed this Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

send_all can silently truncate typed memoryviews after partial writes

2 participants