Repository navigation
Commit 8ee9e09
catalog: carry a format branch's reader options on the wire
ScanBranch.format_options was a declared, documented field with no wire column.
Both ends were built and neither was connected: a worker could set it, and the
C++ rewriter has always consumed it as the reader's named arguments — but
nothing carried it between them, so the map arrived permanently empty.
Because the rewriter ASSIGNS the reader's named arguments from it rather than
merging, that made it impossible to pass a format branch's reader a single
option. A CSV branch could name its files and could not say they were
pipe-delimited. No error; the reader just ran with defaults.
Encoded as a 1-row IPC batch whose column names are the option names — the same
shape the arguments blob uses, and for the same reason: an option value may be
any Arrow type, so no static schema could describe the map.
Also fixes ScanBranch.deserialize, which dropped format_name and
format_locations entirely, so a deserialized format branch came back looking
like a malformed function branch. Invisible only because the C++ client is the
sole consumer and parses the columns itself.
Fixture: multi_branch_format, exercised by the new catalog test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 798fc31 commit 8ee9e09
4 files changed
Lines changed: 84 additions & 0 deletions
File tree
- tests
- data/split_tokens
- vgi
- _test_fixtures
- catalog
Binary file not shown.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
97 | 97 | | |
98 | 98 | | |
99 | 99 | | |
| 100 | + | |
100 | 101 | | |
101 | 102 | | |
102 | 103 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
966 | 966 | | |
967 | 967 | | |
968 | 968 | | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
969 | 978 | | |
970 | 979 | | |
971 | 980 | | |
| |||
1638 | 1647 | | |
1639 | 1648 | | |
1640 | 1649 | | |
| 1650 | + | |
| 1651 | + | |
| 1652 | + | |
| 1653 | + | |
| 1654 | + | |
| 1655 | + | |
| 1656 | + | |
| 1657 | + | |
| 1658 | + | |
| 1659 | + | |
| 1660 | + | |
| 1661 | + | |
| 1662 | + | |
| 1663 | + | |
| 1664 | + | |
| 1665 | + | |
| 1666 | + | |
| 1667 | + | |
| 1668 | + | |
| 1669 | + | |
| 1670 | + | |
| 1671 | + | |
| 1672 | + | |
| 1673 | + | |
| 1674 | + | |
| 1675 | + | |
| 1676 | + | |
| 1677 | + | |
| 1678 | + | |
| 1679 | + | |
| 1680 | + | |
| 1681 | + | |
| 1682 | + | |
| 1683 | + | |
| 1684 | + | |
| 1685 | + | |
| 1686 | + | |
| 1687 | + | |
1641 | 1688 | | |
1642 | 1689 | | |
1643 | 1690 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1011 | 1011 | | |
1012 | 1012 | | |
1013 | 1013 | | |
| 1014 | + | |
| 1015 | + | |
| 1016 | + | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
| 1020 | + | |
| 1021 | + | |
| 1022 | + | |
| 1023 | + | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
| 1030 | + | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
1014 | 1037 | | |
1015 | 1038 | | |
1016 | 1039 | | |
| |||
1098 | 1121 | | |
1099 | 1122 | | |
1100 | 1123 | | |
| 1124 | + | |
| 1125 | + | |
| 1126 | + | |
| 1127 | + | |
| 1128 | + | |
1101 | 1129 | | |
1102 | 1130 | | |
1103 | 1131 | | |
| |||
1126 | 1154 | | |
1127 | 1155 | | |
1128 | 1156 | | |
| 1157 | + | |
1129 | 1158 | | |
1130 | 1159 | | |
1131 | 1160 | | |
| |||
1173 | 1202 | | |
1174 | 1203 | | |
1175 | 1204 | | |
| 1205 | + | |
| 1206 | + | |
| 1207 | + | |
| 1208 | + | |
| 1209 | + | |
| 1210 | + | |
| 1211 | + | |
1176 | 1212 | | |
1177 | 1213 | | |
1178 | 1214 | | |
| |||
0 commit comments