Repository navigation
Commit 6a315a4
authored
Load an older PTD file in the Python reader instead of rejecting it (pytorch#22117)
### Summary
The Python `FlatTensorSerializer.deserialize` refused any PTD file whose
`version` did not **exactly** equal `_FLAT_TENSOR_VERSION`:
```python
if flat_tensor.version != _FLAT_TENSOR_VERSION:
raise NotImplementedError(...)
```
That rejects an **older** file too, which contradicts:
- the append-only schema policy in `schema/README.md` (older files stay
loadable), and
- the C++ runtime readers, which accept anything `<=` their supported
version and only refuse a file **newer** than they understand
(`Program::load`, `FlatTensorDataMap::load`).
This aligns the Python PTD reader with that policy: compare with `>`
instead of `!=`, so an older or equal file loads and only a newer one is
refused. The error message now says the file is newer than this reader
supports.
### Context
This was called out as a follow-up during review of pytorch#22114 (which adds
the `<=` gate on the C++ side). It keeps the Python export/tooling path
consistent so a future non-zero PTD version won't regress it.
### Test plan
Two tests in `extension/flat_tensor/test/test_serialize.py`:
- `test_deserialize_refuses_newer_version` — a bumped-version file is
refused.
- `test_deserialize_accepts_older_version` — a file older than the
reader still loads (the case the old `!=` wrongly rejected).
Both are behavior-only; no schema or format change. Version constants
are unchanged (still `0`), so this is inert for existing files and only
changes behavior once a non-zero version is ever stamped.1 parent d181029 commit 6a315a4
2 files changed
Lines changed: 51 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
404 | 404 | | |
405 | 405 | | |
406 | 406 | | |
407 | | - | |
408 | | - | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
409 | 411 | | |
410 | | - | |
| 412 | + | |
411 | 413 | | |
412 | 414 | | |
413 | 415 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
| |||
31 | 32 | | |
32 | 33 | | |
33 | 34 | | |
| 35 | + | |
34 | 36 | | |
35 | 37 | | |
36 | 38 | | |
| |||
282 | 284 | | |
283 | 285 | | |
284 | 286 | | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
285 | 331 | | |
286 | 332 | | |
287 | 333 | | |
| |||
0 commit comments