fix(spec): drop v3-only data file fields from the v2 manifest schema

[DeviousCardi]

Which issue does this PR close?

• Closes v2 manifest Avro schema includes v3-only data file fields 142, 144 and 145 #3399.

What changes are included in this PR?

The v2 manifest Avro schema declared first_row_id (142), content_offset (144) and content_size_in_bytes (145). The spec defines these only for v3, so a DataFile with any of them set wrote v3 values into a v2 manifest.

The v2 data_file schema now has the same fields as Java's V2Metadata.fileType, keeping referenced_data_file (143). The writer previously used the v2 schema for v3 manifests too, so v3 manifests now get their own manifest_schema_v3, which keeps all four fields.

Things the diff doesn't show:

• v3-only values set on a DataFile are dropped when it is written to a v2 manifest, not rejected. Java behaves the same way: its v2 DataFileWrapper exposes only the v2 positions. The v1 writer already drops them the same way.
• Reading is unchanged. Manifest::parse_avro reads with the writer schema, so v2 manifests that older iceberg-rust versions wrote with these fields still parse, and the values are returned as before. Gating v3 fields when reading v1/v2 is a separate item on EPIC: v3 Support Tracking #2411.
• Nothing on a v2 table depends on these fields. Row lineage is v3-only, and deletion vectors (which use the content fields) are v3-only.
• write_data_files_to_avro and read_data_files_from_avro use data_file_schema_v2 for FormatVersion::V2, so they also stop carrying the three fields for v2.

Are these changes tested?

Yes, with unit tests:

• A new writer test writes the same DataFile, with all four fields set, to a v2 and a v3 manifest. It checks the data_file field ids in the written Avro schema and the data file read back from each manifest.
• test_parse_manifest_without_each_field now writes with the v3 schema and runs for both v2 and v3 metadata. Its v2 case is a manifest as older versions wrote it, with 142/144/145 present.

AI Disclosure

The code and tests were written with Claude Code (Claude Opus 5.5), then reviewed by the author. Areas a reviewer should check:

• Dropping the values silently on the v2 path, versus returning an error from add_file/add_entry. I followed Java, but an error would surface caller bugs earlier.
• read_data_files_from_avro with FormatVersion::V2 now uses a reader schema without the three fields, so it returns None for them even if the bytes contain them. I found no callers outside tests; if something external depends on this, it is a behaviour change.
• The v3 field order (first_row_id, referenced_data_file, content_offset, content_size_in_bytes) is the one that already existed, and it matches Java's V3Metadata.fileType.

🤖 Generated with Claude Code