Repository navigation
Commit 4c3d3bd
authored
[CoreAI] Record file sizes and bundle digests in the delegate manifest (pytorch#23316)
The Core AI runtime checks the payloads it reads against the manifest
and keys its asset cache on bundle contents. This PR records both in the
delegate manifest.
* Asset metadata (`compiler/preprocess.py`): the new `_asset_metadata`
walks the staged bundles (`model.aimodel` or `model.<arch>.aimodelc`)
and returns:
* `files`: each file's path relative to the asset root, mapped to its
byte size. This replaces the previous inline-only list of paths.
* `bundle_digests`: one SHA-256 per bundle, over a fixed prefix, the
file count, and each file's path length, path, size and bytes in UTF-8
path order.
* `_deliver` merges this metadata into the manifest before choosing a
delivery, so every delivery reports identical metadata.
* Tests (`test_preprocess.py`):
* `AssetMetadataTest` checks that `files` exactly covers the delivered
files and sizes, with one digest per bundle, for source and AOT.
* A second test checks that changing a file's bytes changes its digest
when the size stays the same.
* The real-toolchain AOT sidecar test no longer greps the `.pte` for a
compiled filename, which now appears in the manifest by design. Its
`named_data_store_output is None` check still covers "contents not
embedded".
Test plan: `python -m pytest backends/apple/coreai` gives 125 passed
(coreai-core 1.0.0b3, coreai-torch 0.4.3).1 parent b908580 commit 4c3d3bd
2 files changed
Lines changed: 77 additions & 11 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| |||
262 | 263 | | |
263 | 264 | | |
264 | 265 | | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
265 | 295 | | |
266 | 296 | | |
267 | 297 | | |
| |||
281 | 311 | | |
282 | 312 | | |
283 | 313 | | |
284 | | - | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
285 | 320 | | |
286 | 321 | | |
287 | 322 | | |
| |||
352 | 387 | | |
353 | 388 | | |
354 | 389 | | |
355 | | - | |
356 | | - | |
| 390 | + | |
| 391 | + | |
357 | 392 | | |
358 | 393 | | |
359 | | - | |
360 | 394 | | |
361 | 395 | | |
362 | | - | |
363 | | - | |
364 | 396 | | |
365 | | - | |
| 397 | + | |
366 | 398 | | |
367 | 399 | | |
368 | 400 | | |
369 | | - | |
370 | 401 | | |
371 | | - | |
| 402 | + | |
372 | 403 | | |
373 | 404 | | |
374 | 405 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| |||
253 | 254 | | |
254 | 255 | | |
255 | 256 | | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
256 | 293 | | |
257 | 294 | | |
258 | 295 | | |
| |||
542 | 579 | | |
543 | 580 | | |
544 | 581 | | |
545 | | - | |
546 | 582 | | |
547 | 583 | | |
548 | 584 | | |
549 | 585 | | |
550 | 586 | | |
551 | | - | |
552 | 587 | | |
553 | 588 | | |
554 | 589 | | |
| |||
0 commit comments