Repository navigation
Question: RecordBatchReader.close() behavior #643
Description
When working with the arrow_reader() function on a cursor object, the returned type is a RecordBatchReader. I was hoping that I could call close() on this object partway through iteration and have it cancel any server-side work and clean up the underlying resources, but instead this appears to be a no-op. (Or at least, I don't see any effect on this end.)
So: What is close() supposed to do today, if anything? And would it possible for it to do what I'm asking for?
Activity
github-actions commentedon Jun 24, 2026
Hi Adam Machanic (@amachanic), thank you for opening this issue!
Our team will review it shortly. We aim to triage all new issues within 24-48 hours and get back to you.
If you have additional information to share, please feel free to update the issue.
Thank you for your patience!
subrata-ms commentedon Jun 25, 2026
Hi Adam Machanic (@amachanic) , Thanks for raising the bug.
Effectively, [reader.close()] today is a client-side, pyarrow-internal no-op with respect to mssql_python and the database. The only paths that actually free anything are:
However, this looks like a valid scenario, and we are evaluating the design and potential fix.
amachanic commentedon Jun 25, 2026
Subrata (@subrata-ms) Thank you for the prompt reply!
Just to let you know my overall goal, in case it helps with design and/or prioritization: I have a data access layer which is effectively a thin wrapper on top of mssql-python. I've added a function supporting
arrow_reader()-- i.e. it allows tabular queries to stream back data in record batches. I've implemented that as a context manager, and I'd like to put some cleanup code in afinallyblock: Ideally if the client were iterating over several million of rows and some upstream (Python) exception occurred close to the beginning, I'd like it to be able to cancel, rather than forcing it to pull back all of the rows to get things back to a steady state. Closing the cursor is not an ideal option here, as there might be other things going on such as open transactions that I may or may not want to commit.Thanks again, I really appreciate your attention here.
FIX: Implementing RecordBatchReader.Close() functionality for Arrow (#…
subrata-ms commentedon Aug 4, 2026
Hi Adam Machanic (@amachanic), This change was merged into the main branch today and will be available in the upcoming mssql-python driver release.
Closing this bug for now. Please reopen it if you encounter any issues during testing or usage.
RELEASE:1.13.0 (#707)