Repository navigation
Commit 0da8c7c
authored
fix(serve): repack source_code for image_uri/ModelTrainer builds (#6112)
* fix(serve): repack source_code for image_uri/ModelTrainer builds
ModelBuilder classified any image_uri build with a model artifact and
source_code (no model/inference_spec) as passthrough and dropped the
inference code, so build() produced a model.tar.gz without code/ and
register() yielded a package that could not serve. This regressed the
classic v2 Model(model_data, entry_point, source_dir) repack behavior.
In _build_for_passthrough(), distinguish a pure image-only deployment
from an "image + model artifact + source_code" build. For the latter,
keep the source code and bridge model_path to s3_model_data_url so the
existing _upload_code(repack=True) path repacks the code into the model
artifact, producing a self-contained tarball. Pure-image (Nova/BYOC)
passthrough behavior is unchanged. ModelTrainer builds are covered too
since they are normalized to model_path before validation.
Also pass script_dependencies (a list derived from SourceCode) to
repack_model instead of the deprecated self.dependencies auto-detect
dict, which repack_model would otherwise iterate as filesystem paths.
* fix(serve): honor source_code for image_uri/model-server builds
ModelBuilder classified any image_uri build without an in-memory model
or inference_spec as passthrough, ignoring source_code. _build_for_
passthrough() then cleared source_dir/entry_point, so custom inference
code was silently dropped: no repack occurred, the container pointed at
the raw model artifact, register()/deploy() produced a model with no
serving code. Regressed from v2 Model(model_data, entry_point,
source_dir) + _RepackModelStep; introduced by #5969 (commit ebeb54c)
which added the passthrough nulling.
Fix the passthrough classification at its source: do not treat a build
as passthrough when source_code is set (both 1P and non-1P image
branches). Such builds route to the normal model-server path and repack
the code into the artifact.
Also:
- Relax the _build_for_model_server guard to accept s3_model_data_url
(a prebuilt training artifact) alongside model/MLflow/inference_spec.
- In _upload_code repack, pass script_dependencies (a list) instead of
the deprecated self.dependencies auto-detect dict, which repack_model
would otherwise iterate as filesystem paths.
- Update _build_for_model_server unit tests to set s3_model_data_url
so the missing-parameter cases still validate.
* nit: clean up long comments not necessary for customer
* fix(serve): repack source_code inside passthrough path
Restore v2-parity repack behavior for image_uri + model artifact +
source_code builds by repacking inside _build_for_passthrough(), instead
of gating these builds out of passthrough into the model-server path.
Routing such builds to _build_for_model_server exposed pre-existing gaps
in the server builders (e.g. _build_for_djl reads hf_model_config, which
is unset when model is None) and diverged from the path that worked
before serve 1.15.0. Repacking within passthrough matches the 1.12.0
behavior and keeps the fix self-contained.
- _build_validations: drop the source_code passthrough exclusion so these
builds stay on the passthrough path
- _build_for_passthrough: when source_code and a model artifact are both
present, bridge model_path to s3_model_data_url and repack into the
artifact; otherwise keep the pure image-only passthrough behavior
- _build_for_model_server: revert the s3_model_data_url guard relaxation
- tests: revert the now-unneeded servers guard test changes
Verified end-to-end (build -> deploy -> invoke) for both the ticket
scenario (image_uri + model_server=DJL_SERVING + source_code +
s3_model_data_url) and the #6105 scenario (no model_server).
* test(serve): cover source_code repack in passthrough build
Add unit tests for _build_for_passthrough repacking source_code into the
model artifact (and bridging model_path to s3_model_data_url), a regression
test for pure image-only passthrough, and a test that _upload_code repack
passes script_dependencies instead of the deprecated dependencies dict.
Add a build-only integ test that verifies build() repacks code/inference.py
into the artifact and wires up the script-mode env vars.
* fix(serve): warn on unrepackable source_code; address review nits
- Warn (instead of silently dropping) when source_code has an entry_script
but no source_dir, so the code cannot be repacked into the artifact
- Drop the redundant str() guard on model_path for consistency
- Add unit tests: is_repack() is True after the repack branch, and the
entry_script-without-source_dir warning path1 parent 317e291 commit 0da8c7c
4 files changed
Lines changed: 323 additions & 6 deletions
File tree
- sagemaker-serve
- src/sagemaker/serve
- tests
- integ
- unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1879 | 1879 | | |
1880 | 1880 | | |
1881 | 1881 | | |
| 1882 | + | |
| 1883 | + | |
| 1884 | + | |
| 1885 | + | |
| 1886 | + | |
| 1887 | + | |
| 1888 | + | |
| 1889 | + | |
| 1890 | + | |
| 1891 | + | |
| 1892 | + | |
| 1893 | + | |
| 1894 | + | |
| 1895 | + | |
| 1896 | + | |
| 1897 | + | |
| 1898 | + | |
| 1899 | + | |
| 1900 | + | |
| 1901 | + | |
| 1902 | + | |
| 1903 | + | |
| 1904 | + | |
| 1905 | + | |
| 1906 | + | |
| 1907 | + | |
| 1908 | + | |
| 1909 | + | |
| 1910 | + | |
| 1911 | + | |
| 1912 | + | |
| 1913 | + | |
| 1914 | + | |
| 1915 | + | |
| 1916 | + | |
| 1917 | + | |
1882 | 1918 | | |
1883 | 1919 | | |
1884 | 1920 | | |
| |||
2282 | 2318 | | |
2283 | 2319 | | |
2284 | 2320 | | |
| 2321 | + | |
| 2322 | + | |
2285 | 2323 | | |
2286 | 2324 | | |
2287 | 2325 | | |
2288 | 2326 | | |
2289 | 2327 | | |
2290 | 2328 | | |
2291 | 2329 | | |
2292 | | - | |
| 2330 | + | |
2293 | 2331 | | |
2294 | 2332 | | |
2295 | 2333 | | |
2296 | 2334 | | |
2297 | 2335 | | |
2298 | 2336 | | |
2299 | | - | |
| 2337 | + | |
2300 | 2338 | | |
2301 | 2339 | | |
2302 | 2340 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
386 | 386 | | |
387 | 387 | | |
388 | 388 | | |
389 | | - | |
390 | | - | |
391 | | - | |
392 | | - | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
393 | 416 | | |
394 | 417 | | |
395 | 418 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
| |||
455 | 456 | | |
456 | 457 | | |
457 | 458 | | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
458 | 572 | | |
459 | 573 | | |
460 | 574 | | |
| |||
0 commit comments