Hello!
Thansk for this crate!
I'm writing concat tool to merge multiple parquet files into one.
When reading CompressedPages with PageReader, I get back CompessedPages with .selected_rows = None.
When I try to write those pages, write_page expects Some(selected_rows), and my program ultimately fails with "All data pages must declare the number of rows on it" --
|
Error::oos("All data pages must declare the number of rows on it") |
.
Is this hard requirement of parquet format? Can I pass 0 instead, or do I have to decompress/deser & ser/compress pages to copy them to new file?
Hello!
Thansk for this crate!
I'm writing
concattool to merge multiple parquet files into one.When reading
CompressedPages withPageReader, I get backCompessedPages with.selected_rows = None.When I try to write those pages,
write_pageexpectsSome(selected_rows), and my program ultimately fails with "All data pages must declare the number of rows on it" --parquet2/src/write/row_group.rs
Line 69 in 7a5fc27
Is this hard requirement of parquet format? Can I pass 0 instead, or do I have to decompress/deser & ser/compress pages to copy them to new file?