Skip to content

fix(scan): apply equality deletes from an all-void partition spec globally - #3449

Draft
JeonDaehong wants to merge 1 commit into
apache:mainfrom
JeonDaehong:fix-void-spec-global-eq-deletes
Draft

JeonDaehong wants to merge 1 commit into
apache:mainfrom
JeonDaehong:fix-void-spec-global-eq-deletes

Conversation

@JeonDaehong

@JeonDaehong JeonDaehong commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

None, found while reading DeleteFileIndex.

What changes are included in this PR?

The spec says equality deletes stored with an unpartitioned spec are global. Java decides this with spec.isUnpartitioned(), which is also true when every field is a void transform (what a v1 table keeps after dropping a partition field). Rust checked whether the partition tuple was empty, so such deletes were indexed under a [null] partition and never matched data files from the earlier, partitioned spec. Deleted rows came back.

DeleteFileContext now carries the delete file's spec, and the index uses PartitionSpec::is_unpartitioned. If the spec is missing, it falls back to the old check. Position deletes are unchanged: Java scopes them by spec and partition too.

Are these changes tested?

Yes. test_void_spec_equality_delete_is_global fails on main (0 deletes applied) and passes here. cargo test -p iceberg --lib, clippy and fmt are clean.

AI Disclosure

I used Claude Code to find this, write the change and run the tests. I reviewed it before opening the PR.

…bally

A spec whose fields are all void transforms is unpartitioned, as PartitionSpec::is_unpartitioned and Java PartitionSpec.isUnpartitioned say. The delete file index only checked whether the partition tuple was empty, so equality deletes written under such a spec were indexed by a null partition and never matched data files from an earlier, partitioned spec.
@JeonDaehong JeonDaehong changed the title fix(scan): apply equality deletes from an all-void partition spec glo… fix(scan): apply equality deletes from an all-void partition spec globally Oct 14, 2026
@JeonDaehong

Copy link
Copy Markdown
Contributor Author

@CTTY @mbutrovich could you take a look when you have time? Small fix in DeleteFileIndex: equality deletes under an all-void partition spec are global in Java but were never applied to older-spec data files here.

@JeonDaehong
JeonDaehong marked this pull request as draft October 16, 2026 06:23
@JeonDaehong

Copy link
Copy Markdown
Contributor Author

Converted to draft to stay within the new limit of 3 open PRs (#3465). The change is still ready for review; I will mark it ready again when one of my other PRs lands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant