Skip to content

Commit 0da8c7c

Browse files
authored
fix(serve): repack source_code for image_uri/ModelTrainer builds (#6112)
* fix(serve): repack source_code for image_uri/ModelTrainer builds ModelBuilder classified any image_uri build with a model artifact and source_code (no model/inference_spec) as passthrough and dropped the inference code, so build() produced a model.tar.gz without code/ and register() yielded a package that could not serve. This regressed the classic v2 Model(model_data, entry_point, source_dir) repack behavior. In _build_for_passthrough(), distinguish a pure image-only deployment from an "image + model artifact + source_code" build. For the latter, keep the source code and bridge model_path to s3_model_data_url so the existing _upload_code(repack=True) path repacks the code into the model artifact, producing a self-contained tarball. Pure-image (Nova/BYOC) passthrough behavior is unchanged. ModelTrainer builds are covered too since they are normalized to model_path before validation. Also pass script_dependencies (a list derived from SourceCode) to repack_model instead of the deprecated self.dependencies auto-detect dict, which repack_model would otherwise iterate as filesystem paths. * fix(serve): honor source_code for image_uri/model-server builds ModelBuilder classified any image_uri build without an in-memory model or inference_spec as passthrough, ignoring source_code. _build_for_ passthrough() then cleared source_dir/entry_point, so custom inference code was silently dropped: no repack occurred, the container pointed at the raw model artifact, register()/deploy() produced a model with no serving code. Regressed from v2 Model(model_data, entry_point, source_dir) + _RepackModelStep; introduced by #5969 (commit ebeb54c) which added the passthrough nulling. Fix the passthrough classification at its source: do not treat a build as passthrough when source_code is set (both 1P and non-1P image branches). Such builds route to the normal model-server path and repack the code into the artifact. Also: - Relax the _build_for_model_server guard to accept s3_model_data_url (a prebuilt training artifact) alongside model/MLflow/inference_spec. - In _upload_code repack, pass script_dependencies (a list) instead of the deprecated self.dependencies auto-detect dict, which repack_model would otherwise iterate as filesystem paths. - Update _build_for_model_server unit tests to set s3_model_data_url so the missing-parameter cases still validate. * nit: clean up long comments not necessary for customer * fix(serve): repack source_code inside passthrough path Restore v2-parity repack behavior for image_uri + model artifact + source_code builds by repacking inside _build_for_passthrough(), instead of gating these builds out of passthrough into the model-server path. Routing such builds to _build_for_model_server exposed pre-existing gaps in the server builders (e.g. _build_for_djl reads hf_model_config, which is unset when model is None) and diverged from the path that worked before serve 1.15.0. Repacking within passthrough matches the 1.12.0 behavior and keeps the fix self-contained. - _build_validations: drop the source_code passthrough exclusion so these builds stay on the passthrough path - _build_for_passthrough: when source_code and a model artifact are both present, bridge model_path to s3_model_data_url and repack into the artifact; otherwise keep the pure image-only passthrough behavior - _build_for_model_server: revert the s3_model_data_url guard relaxation - tests: revert the now-unneeded servers guard test changes Verified end-to-end (build -> deploy -> invoke) for both the ticket scenario (image_uri + model_server=DJL_SERVING + source_code + s3_model_data_url) and the #6105 scenario (no model_server). * test(serve): cover source_code repack in passthrough build Add unit tests for _build_for_passthrough repacking source_code into the model artifact (and bridging model_path to s3_model_data_url), a regression test for pure image-only passthrough, and a test that _upload_code repack passes script_dependencies instead of the deprecated dependencies dict. Add a build-only integ test that verifies build() repacks code/inference.py into the artifact and wires up the script-mode env vars. * fix(serve): warn on unrepackable source_code; address review nits - Warn (instead of silently dropping) when source_code has an entry_script but no source_dir, so the code cannot be repacked into the artifact - Drop the redundant str() guard on model_path for consistency - Add unit tests: is_repack() is True after the repack branch, and the entry_script-without-source_dir warning path
1 parent 317e291 commit 0da8c7c

4 files changed

Lines changed: 323 additions & 6 deletions

File tree

‎sagemaker-serve/src/sagemaker/serve/model_builder.py‎

Lines changed: 40 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1879,6 +1879,42 @@ def _build_for_passthrough(self) -> Model:
18791879

18801880
self.secret_key = ""
18811881

1882+
model_artifact_uri = None
1883+
if self.model_path and self.model_path.startswith("s3://"):
1884+
model_artifact_uri = self.model_path
1885+
elif isinstance(self.s3_model_data_url, str) and self.s3_model_data_url.startswith(
1886+
"s3://"
1887+
):
1888+
model_artifact_uri = self.s3_model_data_url
1889+
1890+
has_source_code = bool(
1891+
getattr(self, "entry_point", None) and getattr(self, "source_dir", None)
1892+
)
1893+
1894+
# Repack source_code into the model artifact so build() produces a
1895+
# self-contained model.tar.gz (code under code/).
1896+
if has_source_code and model_artifact_uri:
1897+
if not (
1898+
isinstance(self.s3_model_data_url, str)
1899+
and self.s3_model_data_url.startswith("s3://")
1900+
):
1901+
self.s3_model_data_url = model_artifact_uri
1902+
self.s3_upload_path = None
1903+
1904+
if self.mode in LOCAL_MODES:
1905+
self._prepare_for_mode()
1906+
1907+
return self._create_model()
1908+
1909+
if getattr(self, "entry_point", None) and model_artifact_uri:
1910+
# entry_point provided without a source_dir: repack cannot bundle the
1911+
# code, so it would be dropped. Warn instead of silently ignoring it.
1912+
logger.warning(
1913+
"source_code was provided without a source_dir; the inference code "
1914+
"will not be repacked into the model artifact. Provide "
1915+
"SourceCode(source_dir=...) to bundle custom inference code."
1916+
)
1917+
18821918
if self.model_path and self.model_path.startswith("s3://"):
18831919
self.s3_upload_path = self.model_path
18841920
else:
@@ -2282,21 +2318,23 @@ def _upload_code(self, key_prefix: str, repack: bool = False) -> None:
22822318
script_name=os.path.basename(self.entry_point),
22832319
)
22842320

2321+
repack_dependencies = self.script_dependencies or []
2322+
22852323
logger.info(
22862324
"Repacking model artifact (%s), script artifact "
22872325
"(%s), and dependencies (%s) "
22882326
"into single tar.gz file located at %s. "
22892327
"This may take some time depending on model size...",
22902328
self.s3_model_data_url,
22912329
self.source_dir,
2292-
self.dependencies,
2330+
repack_dependencies,
22932331
repacked_model_data,
22942332
)
22952333

22962334
repack_model(
22972335
inference_script=self.entry_point,
22982336
source_directory=self.source_dir,
2299-
dependencies=self.dependencies,
2337+
dependencies=repack_dependencies,
23002338
model_uri=self.s3_model_data_url,
23012339
repacked_model_uri=repacked_model_data,
23022340
sagemaker_session=self.sagemaker_session,
Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License"). You
4+
# may not use this file except in compliance with the License. A copy of
5+
# the License is located at
6+
#
7+
# http://aws.amazon.com/apache2.0/
8+
#
9+
# or in the "license" file accompanying this file. This file is
10+
# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
11+
# ANY KIND, either express or implied. See the License for the specific
12+
# language governing permissions and limitations under the License.
13+
"""Integration test for ModelBuilder passthrough source_code repack.
14+
15+
Covers the fix where an image_uri build with a model artifact and custom
16+
source_code repacks the code into the artifact (instead of silently dropping
17+
it). This calls build() only (no deploy) so it runs in seconds.
18+
"""
19+
from __future__ import absolute_import
20+
21+
import io
22+
import os
23+
import tarfile
24+
import tempfile
25+
import uuid
26+
27+
import boto3
28+
import pytest
29+
30+
from sagemaker.serve.model_builder import ModelBuilder
31+
from sagemaker.serve.mode.function_pointers import Mode
32+
from sagemaker.core.training.configs import SourceCode
33+
from sagemaker.core.helper.session_helper import Session, get_execution_role
34+
35+
MODEL_NAME_PREFIX = "mb-passthrough-repack"
36+
37+
38+
def _upload_raw_artifact(s3_client, bucket, prefix):
39+
"""Upload a minimal model.tar.gz containing only a model file (no code/)."""
40+
tmp = tempfile.mkdtemp()
41+
with open(os.path.join(tmp, "model.json"), "w") as f:
42+
f.write('{"weights": [1, 1, 1, 1]}')
43+
tar_path = os.path.join(tmp, "model.tar.gz")
44+
with tarfile.open(tar_path, "w:gz") as t:
45+
t.add(os.path.join(tmp, "model.json"), arcname="model.json")
46+
key = f"{prefix}/model.tar.gz"
47+
s3_client.upload_file(tar_path, bucket, key)
48+
return f"s3://{bucket}/{key}", key
49+
50+
51+
def _make_source_code_dir():
52+
d = tempfile.mkdtemp()
53+
with open(os.path.join(d, "inference.py"), "w") as f:
54+
f.write(
55+
"def model_fn(model_dir):\n return None\n"
56+
"def predict_fn(data, model):\n return data\n"
57+
)
58+
with open(os.path.join(d, "requirements.txt"), "w") as f:
59+
f.write("joblib\n")
60+
return d
61+
62+
63+
def _tar_members(s3_client, s3_uri):
64+
_, _, rest = s3_uri.partition("s3://")
65+
bucket, _, key = rest.partition("/")
66+
body = s3_client.get_object(Bucket=bucket, Key=key)["Body"].read()
67+
return tarfile.open(fileobj=io.BytesIO(body), mode="r:gz").getnames()
68+
69+
70+
@pytest.mark.slow_test
71+
def test_build_repacks_source_code_into_artifact():
72+
"""build() with image_uri + model artifact + source_code repacks code/ into
73+
the model.tar.gz. No deploy - runs in seconds."""
74+
session = Session()
75+
region = session.boto_region_name
76+
bucket = session.default_bucket()
77+
role = get_execution_role(sagemaker_session=session)
78+
s3_client = boto3.client("s3", region_name=region)
79+
80+
unique_id = uuid.uuid4().hex[:8]
81+
prefix = f"{MODEL_NAME_PREFIX}/{unique_id}"
82+
s3_keys = []
83+
core_model = None
84+
85+
try:
86+
from sagemaker.core import image_uris
87+
88+
image_uri = image_uris.retrieve(
89+
framework="sklearn",
90+
region=region,
91+
version="1.2-1",
92+
instance_type="ml.m5.large",
93+
image_scope="inference",
94+
)
95+
96+
artifact_uri, key = _upload_raw_artifact(s3_client, bucket, prefix)
97+
s3_keys.append(key)
98+
src_dir = _make_source_code_dir()
99+
100+
model_builder = ModelBuilder(
101+
image_uri=image_uri,
102+
source_code=SourceCode(source_dir=src_dir, entry_script="inference.py"),
103+
s3_model_data_url=artifact_uri,
104+
role_arn=role,
105+
instance_type="ml.m5.large",
106+
sagemaker_session=session,
107+
)
108+
model_builder.model_path = f"/tmp/sagemaker/model-builder/{unique_id}"
109+
os.makedirs(model_builder.model_path, exist_ok=True)
110+
model_builder.dependencies = []
111+
112+
core_model = model_builder.build(
113+
model_name=f"{MODEL_NAME_PREFIX}-{unique_id}", mode=Mode.SAGEMAKER_ENDPOINT
114+
)
115+
116+
# A repack must have produced a new artifact (not the raw one)
117+
repacked = model_builder.repacked_model_data
118+
assert repacked is not None
119+
assert repacked != artifact_uri
120+
s3_keys.append(repacked.partition(f"s3://{bucket}/")[2])
121+
122+
# The repacked artifact must contain the inference code under code/
123+
members = _tar_members(s3_client, repacked)
124+
assert any(m.endswith("code/inference.py") for m in members), members
125+
assert any(m.endswith("model.json") for m in members), members
126+
127+
# Script-mode env vars must be wired up on the model container
128+
env = core_model.primary_container.environment or {}
129+
assert env.get("SAGEMAKER_PROGRAM") == "inference.py"
130+
assert env.get("SAGEMAKER_SUBMIT_DIRECTORY") == "/opt/ml/model/code"
131+
132+
finally:
133+
if core_model is not None:
134+
try:
135+
core_model.delete()
136+
except Exception: # noqa
137+
pass
138+
for k in s3_keys:
139+
try:
140+
s3_client.delete_object(Bucket=bucket, Key=k)
141+
except Exception: # noqa
142+
pass

‎sagemaker-serve/tests/unit/test_model_builder_build.py‎

Lines changed: 27 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -386,10 +386,33 @@ def test_upload_code_local_mode(self, mock_determine):
386386

387387
self.assertIsNone(builder.uploaded_code)
388388

389-
@unittest.skip("Complex mocking required for repack_model with file system operations")
390-
def test_upload_code_with_repack(self):
391-
"""Test _upload_code with repacking."""
392-
pass
389+
@patch("sagemaker.serve.model_builder.repack_model")
390+
@patch("sagemaker.core.workflow.is_pipeline_variable", return_value=False)
391+
@patch("sagemaker.core.s3.determine_bucket_and_prefix")
392+
def test_upload_code_repack_uses_script_dependencies(
393+
self, mock_determine, mock_is_pipeline, mock_repack
394+
):
395+
"""_upload_code(repack=True) must pass script_dependencies (a list) to
396+
repack_model, not the deprecated self.dependencies auto-detect dict."""
397+
mock_determine.return_value = ("test-bucket", "test-prefix")
398+
399+
builder = ModelBuilder(
400+
model=Mock(),
401+
role_arn="arn:aws:iam::123456789012:role/TestRole",
402+
sagemaker_session=self.mock_session,
403+
)
404+
builder.bucket = None
405+
builder.entry_point = "inference.py"
406+
builder.source_dir = "/path/to/code"
407+
builder.script_dependencies = ["/path/to/requirements.txt"]
408+
builder.dependencies = {"auto": True} # deprecated dict, must NOT be used
409+
builder.s3_model_data_url = "s3://bucket/model.tar.gz"
410+
builder.model_kms_key = None
411+
412+
builder._upload_code("test-prefix", repack=True)
413+
414+
mock_repack.assert_called_once()
415+
assert mock_repack.call_args.kwargs["dependencies"] == ["/path/to/requirements.txt"]
393416

394417
@unittest.skip("Complex file system mocking - os.stat requires real file paths")
395418
@patch('sagemaker.core.s3.determine_bucket_and_prefix')

‎sagemaker-serve/tests/unit/test_model_builder_methods.py‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
from sagemaker.core.deserializers import JSONDeserializer, TorchTensorDeserializer
2424
from sagemaker.serve.constants import Framework
2525
from sagemaker.serve.mode.function_pointers import Mode
26+
from sagemaker.core.training.configs import SourceCode
2627

2728

2829
class TestModelBuilderSimpleMethods:
@@ -455,6 +456,119 @@ def test_build_for_passthrough_sets_s3_upload_path_none_when_no_model_path(
455456
assert builder.s3_upload_path is None
456457

457458

459+
class TestBuildForPassthroughSourceCodeRepack:
460+
"""Tests for _build_for_passthrough() repacking source_code into the model artifact.
461+
462+
When source_code is supplied together with a model artifact, build() must repack
463+
the code into the artifact instead of dropping it.
464+
"""
465+
466+
def _make_mock_session(self):
467+
mock_session = Mock()
468+
mock_session.boto_region_name = "us-west-2"
469+
mock_session.config = {}
470+
mock_session.boto_session = Mock()
471+
mock_session.boto_session.region_name = "us-west-2"
472+
return mock_session
473+
474+
def _make_builder(self, **kwargs):
475+
return ModelBuilder(
476+
image_uri="123456789.dkr.ecr.us-west-2.amazonaws.com/my-image:latest",
477+
mode=Mode.SAGEMAKER_ENDPOINT,
478+
role_arn="arn:aws:iam::123456789012:role/TestRole",
479+
source_code=SourceCode(source_dir="./code", entry_script="inference.py"),
480+
sagemaker_session=self._make_mock_session(),
481+
**kwargs,
482+
)
483+
484+
@patch.object(ModelBuilder, "_create_model")
485+
def test_repacks_and_preserves_script_mode_vars_with_s3_model_data_url(self, mock_create):
486+
"""source_code + s3_model_data_url artifact -> repack path, script-mode vars kept."""
487+
mock_create.return_value = Mock()
488+
489+
builder = self._make_builder(s3_model_data_url="s3://bucket/model.tar.gz")
490+
491+
builder._build_for_passthrough()
492+
493+
# script-mode vars must NOT be cleared (that would skip repack)
494+
assert builder.entry_point == "inference.py"
495+
assert builder.source_dir == "./code"
496+
assert builder.s3_upload_path is None
497+
mock_create.assert_called_once()
498+
499+
@patch.object(ModelBuilder, "_create_model")
500+
def test_bridges_model_path_to_s3_model_data_url(self, mock_create):
501+
"""When only model_path (S3) is given, it is bridged to s3_model_data_url for repack."""
502+
mock_create.return_value = Mock()
503+
504+
builder = self._make_builder()
505+
builder.model_path = "s3://bucket/model.tar.gz"
506+
507+
builder._build_for_passthrough()
508+
509+
assert builder.s3_model_data_url == "s3://bucket/model.tar.gz"
510+
assert builder.entry_point == "inference.py"
511+
assert builder.source_dir == "./code"
512+
mock_create.assert_called_once()
513+
514+
@patch.object(ModelBuilder, "_create_model")
515+
def test_repack_branch_makes_is_repack_true(self, mock_create):
516+
"""After the repack branch runs, is_repack() must return True so
517+
_prepare_container_def_base triggers _upload_code(repack=True)."""
518+
mock_create.return_value = Mock()
519+
520+
builder = self._make_builder(s3_model_data_url="s3://bucket/model.tar.gz")
521+
522+
builder._build_for_passthrough()
523+
524+
assert builder.is_repack() is True
525+
526+
@patch.object(ModelBuilder, "_create_model")
527+
def test_pure_image_passthrough_without_source_code_still_clears_vars(self, mock_create):
528+
"""No source_code -> pure image passthrough clears script-mode vars (unchanged)."""
529+
mock_create.return_value = Mock()
530+
531+
builder = ModelBuilder(
532+
image_uri="123456789.dkr.ecr.us-west-2.amazonaws.com/my-image:latest",
533+
mode=Mode.SAGEMAKER_ENDPOINT,
534+
role_arn="arn:aws:iam::123456789012:role/TestRole",
535+
s3_model_data_url="s3://bucket/model.tar.gz",
536+
sagemaker_session=self._make_mock_session(),
537+
)
538+
539+
builder._build_for_passthrough()
540+
541+
assert builder.entry_point is None
542+
assert builder.source_dir is None
543+
mock_create.assert_called_once()
544+
545+
@patch.object(ModelBuilder, "_create_model")
546+
def test_entry_point_without_source_dir_warns_and_does_not_repack(self, mock_create):
547+
"""entry_script without source_dir cannot be repacked: warn, don't repack."""
548+
mock_create.return_value = Mock()
549+
550+
builder = ModelBuilder(
551+
image_uri="123456789.dkr.ecr.us-west-2.amazonaws.com/my-image:latest",
552+
mode=Mode.SAGEMAKER_ENDPOINT,
553+
role_arn="arn:aws:iam::123456789012:role/TestRole",
554+
source_code=SourceCode(entry_script="inference.py"),
555+
s3_model_data_url="s3://bucket/model.tar.gz",
556+
sagemaker_session=self._make_mock_session(),
557+
)
558+
# sanity: no source_dir -> repack branch is not eligible
559+
assert builder.source_dir is None
560+
assert builder.entry_point == "inference.py"
561+
562+
with patch("sagemaker.serve.model_builder.logger") as mock_logger:
563+
builder._build_for_passthrough()
564+
mock_logger.warning.assert_called_once()
565+
566+
# fall-through nulls script-mode vars (no repack occurred)
567+
assert builder.entry_point is None
568+
assert builder.source_dir is None
569+
mock_create.assert_called_once()
570+
571+
458572
class TestGetDockerClient:
459573
"""Tests for _get_docker_client() Studio Docker client initialization.
460574

0 commit comments

Comments
 (0)