Repository navigation
Fix CuPy indexing, assignment and scalar truth conversion - #11797
wpyaoxueros wants to merge 7 commits into
Conversation
Prepared with Codex assistance. Public PR authorship and required human review remain pending. Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
headtr1ck
left a comment
There was a problem hiding this comment.
Thanks! I ran this on a GPU (cupy 14.2.0): all tests in test_cupy.py and the cupy cases in test_duck_array_wrapping.py pass, while 10 of the new tests and test_idxmax/test_idxmin[True-cupy] fail on main, so this fixes everything from #11739.
The main thing I'd like to change is that the fix is quite specific to cupy, while the underlying problem is general: pandas can only be indexed with numpy indexers, so any non-numpy duck array indexer has to be converted at the pandas boundary.
PandasIndex.iselalready does this generically, butPandasIndexingAdapter._index_getonly converts cupy arrays, and does it twice (key, thenindexer.tuplefor the multidimensional case). Converting the indexer once at the start of_index_getwith the same rule as inPandasIndex.isel(is_duck_array(k) and not isinstance(k, np.ndarray)→to_numpy(k)) removes both the special case and the duplication. I tried that locally and the cupy tests still pass.- Could you add a short comment to
__bool__on why extension arrays are special-cased? They have no__bool__, sobool()falls back tolen(), andbool(pd.array([False]))would beTrue. - Optional:
NumpyIndexingAdapter._safe_setitemstill readsarray.flags.writeable, which cupy's flags don't have, so otherValueErrors from cupy assignments still turn into anAttributeError.getattr(array.flags, "writeable", True)would fix that. - The whats-new entry can drop "while preserving pandas extension-array behavior", that's not a user-visible change.
The docs failure is the pandas 3.1 issue fixed in #11796, merging main should fix it.
Normalize non-NumPy indexers once at the pandas boundary, retain CuPy assignment errors, and explain extension-array truth conversion. Add sparse-indexer and CuPy assignment-error regressions. Co-authored-by: Codex <noreply@openai.com>
headtr1ck
left a comment
There was a problem hiding this comment.
Thanks, all points are addressed! I reran the cupy tests on a GPU (cupy 14.2.0) after merging main: test_cupy.py and the cupy cases in test_duck_array_wrapping.py pass, as do the indexing, variable, dataarray, dataset and indexes tests.
Co-authored-by: Codex <noreply@openai.com>
Description
Codex on behalf of @wpyaoxueros. Closes #11739.
Contributor confirmation, translated from Chinese: "I have reviewed the changes and found no issues."
CuPy indexers currently fail when they reach pandas-backed coordinates, basic assignment can mix NumPy values with CuPy storage, and scalar truth conversion can trigger an implicit conversion to NumPy. This patch explicitly converts indexers at the pandas boundary, converts assignment values to the target array backend, and uses the underlying array's truth conversion while preserving pandas extension-array handling.
Regression coverage includes vector/multidimensional/argmax indexers, scalar/slice/
.locassignment, scalar CuPy truth values,idxmin/idxmax, and pandas nullable-boolean controls.Remote validation against base
fbc4cf19b919ce23fd0f7d419e9c141f046788a9:These checks ran on the designated Linux GPU workstation. The full repository suite and type checking were not rerun on this commit. The required release-note entry has been added. The contributor has confirmed review of the changes; the remote checks below were executed by Codex.
Documentation CI on
6cbb300failed in the unchangeddoc/internals/time-coding.mdtimestamp example with pandas 3.1.0. A standalone remote pandas-only check reproducesOutOfBoundsDatetimefor thesandmsunits without importing xarray; the same check passes with pandas 3.0.6. The follow-up commit only moves this PR's release note into the unreleased section. This is a known documentation-check failure, not a claim that the complete documentation build passes.Checklist
whats-new.rstAI Disclosure
Tool: Codex. Task scope: address the CuPy indexing, assignment and scalar-bool failures reported in #11739, with regression tests and validation on the remote workstation. Codex prepared the six-file patch, ran the reported checks, prepared this description and submitted the PR at the contributor's explicit request. The contributor's own review confirmation is quoted above. No personal execution of the agent-run tests is claimed.