Skip to content

Commit 5f0a651

Browse files
author
Mohamed Zeidan
committed
fix(serve): decouple the SMD orchestrator entrypoint from the container's sagemaker
Deploying a CustomOrchestrator with ModelBuilder fails its container health check at startup with `AttributeError: 'NoneType' object has no attribute 'encode'`, so the Inference Component never reaches InService. The generated `code/inference.py` did `from sagemaker.serve.validations. check_integrity import perform_integrity_check`, which resolves against the *container's* installed sagemaker rather than the SDK that produced the artifacts. `sagemaker-distribution.json` maps SMD inference to 3.2.0 only, and that image bundles sagemaker 2.245.0, whose `perform_integrity_check` still reads `SAGEMAKER_SERVE_SECRET_KEY` and calls `.encode()` on it. The host no longer sets that variable, and must not: the secret key was removed on purpose in #5654 / #5656 as the fix for CVE-2026-8596, where it was exposed in plaintext through the Describe APIs. So the missing `return` in `prepare_for_smd()` is not the defect, and restoring it would reintroduce the vulnerability -- the version coupling is the defect. Inline the digest comparison in the entrypoint instead. It is the same plain SHA-256 check against `metadata.json` that the host writes, so the integrity posture is unchanged, but the script no longer imports anything from sagemaker and therefore cannot drift against whatever version the image carries. A second, independent mismatch sat behind the first: `_save_model_inference_ spec` pickled `(inference_spec, schema_builder)`, and a v3 SchemaBuilder pickles with references to `sagemaker.core.*` serializer modules that a 2.x install cannot import, so `cloudpickle.load` failed even once the integrity check passed. The orchestrator entrypoint discards the second element of that tuple, so pickle `(inference_spec, None)` for custom orchestrators. Also drop the six `self.secret_key = prepare_for_*(...)` assignments and their `# fix out of scope` / pylint-disable comments, and correct the `-> str` annotations to `-> None` on all four `prepare_for_*` functions. Those functions record a digest and have had nothing to return since the CVE fix; the lying annotation is what led two earlier PRs (#6221, #6222) to try to resurrect the secret key. Every builder already initializes `self.secret_key = ""` before calling them, so this is not a behavior change. Verified by generating the artifacts with this branch and executing the resulting `inference.py` under a sagemaker 2.245.0 interpreter -- the version bundled in the image the SDK selects. Before: the reported traceback. After: the orchestrator loads and serves. Tampered `serve.pkl` and missing `metadata.json` are still rejected. Closes #6200
1 parent 9d819b4 commit 5f0a651

11 files changed

Lines changed: 410 additions & 49 deletions

File tree

‎sagemaker-serve/src/sagemaker/serve/model_builder.py‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1944,7 +1944,14 @@ def _save_model_inference_spec(self) -> None:
19441944
code_path = Path(self.model_path).joinpath("code")
19451945

19461946
if self.inference_spec:
1947-
save_pkl(code_path, (self.inference_spec, self.schema_builder))
1947+
if isinstance(self.inference_spec, (CustomOrchestrator, AsyncCustomOrchestrator)):
1948+
# The SMD orchestrator entrypoint discards the second element of the
1949+
# tuple, and a SchemaBuilder pickles with references to sagemaker.core
1950+
# modules that an older sagemaker in the serving container cannot
1951+
# import. Leave it out so serve.pkl stays loadable there.
1952+
save_pkl(code_path, (self.inference_spec, None))
1953+
else:
1954+
save_pkl(code_path, (self.inference_spec, self.schema_builder))
19481955
elif self.model:
19491956
if isinstance(self.model, str):
19501957
self.framework = None

‎sagemaker-serve/src/sagemaker/serve/model_builder_servers.py‎

Lines changed: 6 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -180,9 +180,7 @@ def _build_for_torchserve(self) -> Model:
180180

181181
# Prepare TorchServe artifacts for local container mode
182182
if self.mode == Mode.LOCAL_CONTAINER and self.model_path:
183-
# NOTE: prepare_* annotated -> str but returns None (latent bug; fix out of scope)
184-
# pylint: disable-next=assignment-from-no-return
185-
self.secret_key = prepare_for_torchserve(
183+
prepare_for_torchserve(
186184
model_path=self.model_path,
187185
shared_libs=self.shared_libs,
188186
dependencies=self.dependencies,
@@ -191,9 +189,7 @@ def _build_for_torchserve(self) -> Model:
191189
inference_spec=self.inference_spec,
192190
)
193191
if self.mode == Mode.SAGEMAKER_ENDPOINT and self.model_path:
194-
# NOTE: prepare_* annotated -> str but returns None (latent bug; fix out of scope)
195-
# pylint: disable-next=assignment-from-no-return
196-
self.secret_key = prepare_for_torchserve(
192+
prepare_for_torchserve(
197193
model_path=self.model_path,
198194
shared_libs=self.shared_libs,
199195
dependencies=self.dependencies,
@@ -666,9 +662,7 @@ def _build_for_tensorflow_serving(self) -> Model:
666662
raise ValueError("image_uri is required for TensorFlow Serving deployment")
667663

668664
# Prepare TensorFlow Serving artifacts for local container mode
669-
# NOTE: prepare_* annotated -> str but returns None (latent bug; fix out of scope)
670-
# pylint: disable-next=assignment-from-no-return
671-
self.secret_key = prepare_for_tf_serving(
665+
prepare_for_tf_serving(
672666
model_path=self.model_path,
673667
shared_libs=self.shared_libs,
674668
dependencies=self.dependencies,
@@ -788,9 +782,7 @@ def _build_for_smd(self) -> Model:
788782
cpu_or_gpu = self._get_processing_unit()
789783
self.image_uri = self._get_smd_image_uri(processing_unit=cpu_or_gpu)
790784

791-
# NOTE: prepare_* annotated -> str but returns None (latent bug; fix out of scope)
792-
# pylint: disable-next=assignment-from-no-return
793-
self.secret_key = prepare_for_smd(
785+
prepare_for_smd(
794786
model_path=self.model_path,
795787
shared_libs=self.shared_libs,
796788
dependencies=self.dependencies,
@@ -834,9 +826,7 @@ def _build_for_transformers(self) -> Model:
834826
self._create_conda_env()
835827

836828
if self.mode in [Mode.LOCAL_CONTAINER] and self.model_path:
837-
# NOTE: prepare_* annotated -> str but returns None (latent bug; fix out of scope)
838-
# pylint: disable-next=assignment-from-no-return
839-
self.secret_key = prepare_for_mms(
829+
prepare_for_mms(
840830
model_path=self.model_path,
841831
shared_libs=self.shared_libs,
842832
dependencies=self.dependencies,
@@ -845,9 +835,7 @@ def _build_for_transformers(self) -> Model:
845835
inference_spec=self.inference_spec,
846836
)
847837
if self.mode == Mode.SAGEMAKER_ENDPOINT and self.model_path:
848-
# NOTE: prepare_* annotated -> str but returns None (latent bug; fix out of scope)
849-
# pylint: disable-next=assignment-from-no-return
850-
self.secret_key = prepare_for_mms(
838+
prepare_for_mms(
851839
model_path=self.model_path,
852840
shared_libs=self.shared_libs,
853841
dependencies=self.dependencies,

‎sagemaker-serve/src/sagemaker/serve/model_server/multi_model_server/prepare.py‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -81,18 +81,20 @@ def prepare_for_mms(
8181
session: Session,
8282
image_uri: str,
8383
inference_spec: InferenceSpec = None,
84-
) -> str:
85-
"""Prepares for InferenceSpec, writes inference.py, and captures dependencies for secret_key.
84+
) -> None:
85+
"""Prepares for InferenceSpec, writes inference.py, and captures dependencies.
8686
87-
Args:to
87+
Nothing is returned: the SHA-256 digest of ``serve.pkl`` is recorded in
88+
``metadata.json`` and verified against it in the container, so there is no secret to
89+
hand back to the caller.
90+
91+
Args:
8892
model_path (str) : Argument
8993
shared_libs (List[]) : Argument
9094
dependencies (dict) : Argument
9195
session (Session) : Argument
9296
inference_spec (InferenceSpec, optional) : Argument
9397
(default is None)
94-
Returns:
95-
( str ) : secret_key
9698
"""
9799
model_path = Path(model_path)
98100
if not model_path.exists():

‎sagemaker-serve/src/sagemaker/serve/model_server/smd/custom_execution_inference.py‎

Lines changed: 45 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -10,24 +10,40 @@
1010
# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
1111
# ANY KIND, either express or implied. See the License for the specific
1212
# language governing permissions and limitations under the License.
13-
"""This module is for SageMaker inference.py."""
13+
"""This module is for SageMaker inference.py.
14+
15+
This file is copied into the model artifacts as ``code/inference.py`` and runs inside
16+
the SageMaker Distribution serving container, whose installed ``sagemaker`` version is
17+
set by the image rather than by the SDK that built the artifacts. It therefore must not
18+
import from ``sagemaker``: the integrity check is inlined below instead of imported from
19+
``sagemaker.serve.validations.check_integrity`` so that it cannot drift against the
20+
container's copy of that module.
21+
"""
1422

1523
from __future__ import absolute_import
1624
import asyncio
25+
import hashlib
26+
import hmac
27+
import json
28+
import logging
1729
import os
1830
import platform
19-
import cloudpickle
20-
import logging
2131
from pathlib import Path
22-
from sagemaker.serve.validations.check_integrity import perform_integrity_check
32+
import cloudpickle
2333

2434
logger = LOGGER = logging.getLogger("sagemaker")
2535

36+
DEFAULT_CODE_DIRECTORY = "/opt/ml/model/code"
37+
38+
39+
def _get_code_directory() -> Path:
40+
"""Returns the directory holding serve.pkl and metadata.json."""
41+
return Path(os.getenv("SAGEMAKER_INFERENCE_CODE_DIRECTORY") or DEFAULT_CODE_DIRECTORY)
42+
2643

2744
def initialize_custom_orchestrator():
2845
"""Initializes the custom orchestrator."""
29-
code_dir = os.getenv("SAGEMAKER_INFERENCE_CODE_DIRECTORY", None)
30-
serve_path = Path(code_dir).joinpath("serve.pkl")
46+
serve_path = _get_code_directory().joinpath("serve.pkl")
3147
with open(str(serve_path), mode="rb") as pkl_file:
3248
return cloudpickle.load(pkl_file)
3349

@@ -48,12 +64,32 @@ def _py_vs_parity_check():
4864
)
4965

5066

67+
def _perform_integrity_check(buffer: bytes, metadata_path: Path):
68+
"""Validates the integrity of bytes by comparing the SHA-256 hash value."""
69+
if not Path.exists(metadata_path):
70+
raise ValueError("Path to metadata.json does not exist")
71+
72+
with open(str(metadata_path), "rb") as md:
73+
try:
74+
expected_hash_value = json.loads(md.read()).get("sha256_hash")
75+
except json.JSONDecodeError:
76+
raise ValueError("Corrupt metadata file. It is not a valid json file.")
77+
78+
if not expected_hash_value:
79+
raise ValueError("metadata.json does not contain a sha256_hash value")
80+
81+
actual_hash_value = hashlib.sha256(buffer).hexdigest()
82+
83+
if not hmac.compare_digest(expected_hash_value, actual_hash_value):
84+
raise ValueError("Integrity check for the serialized function or data failed.")
85+
86+
5187
def _pickle_file_integrity_check():
52-
with open("/opt/ml/model/code/serve.pkl", "rb") as f:
88+
code_directory = _get_code_directory()
89+
with open(str(code_directory.joinpath("serve.pkl")), "rb") as f:
5390
buffer = f.read()
5491

55-
metadata_path = Path("/opt/ml/model/code/metadata.json")
56-
perform_integrity_check(buffer=buffer, metadata_path=metadata_path)
92+
_perform_integrity_check(buffer=buffer, metadata_path=code_directory.joinpath("metadata.json"))
5793

5894

5995
_run_preflight_diagnostics()

‎sagemaker-serve/src/sagemaker/serve/model_server/smd/prepare.py‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,19 +23,20 @@ def prepare_for_smd(
2323
shared_libs: List[str],
2424
dependencies: dict,
2525
inference_spec: InferenceSpec = None,
26-
) -> str:
26+
) -> None:
2727
"""Prepares artifacts for SageMaker model deployment.
2828
29-
Args:to
29+
Writes the inference entrypoint, dependencies and a ``metadata.json`` holding the
30+
SHA-256 digest of ``serve.pkl``. Nothing is returned: the digest is verified in the
31+
container against ``metadata.json``, so there is no secret to hand back to the
32+
caller.
33+
34+
Args:
3035
model_path (str) : Argument
3136
shared_libs (List[]) : Argument
3237
dependencies (dict) : Argument
3338
inference_spec (InferenceSpec, optional) : Argument
3439
(default is None)
35-
36-
Returns:
37-
( str ) :
38-
3940
"""
4041
model_path = Path(model_path)
4142
if not model_path.exists():

‎sagemaker-serve/src/sagemaker/serve/model_server/tensorflow_serving/prepare.py‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,16 +20,17 @@ def prepare_for_tf_serving(
2020
model_path: str,
2121
shared_libs: List[str],
2222
dependencies: Dict[str, Any],
23-
) -> str:
23+
) -> None:
2424
"""Prepares the model for serving.
2525
26+
Nothing is returned: the SHA-256 digest of ``serve.pkl`` is recorded in
27+
``metadata.json`` and verified against it in the container, so there is no secret to
28+
hand back to the caller.
29+
2630
Args:
2731
model_path (str): Path to the model directory.
2832
shared_libs (List[str]): List of shared libraries.
2933
dependencies (Dict[str, Any]): Dictionary of dependencies.
30-
31-
Returns:
32-
str: Secret key.
3334
"""
3435

3536
_model_path = Path(model_path)

‎sagemaker-serve/src/sagemaker/serve/model_server/torchserve/prepare.py‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -26,20 +26,20 @@ def prepare_for_torchserve(
2626
session: Session,
2727
image_uri: str,
2828
inference_spec: InferenceSpec = None,
29-
) -> str:
29+
) -> None:
3030
"""This is a one-line summary of the function.
3131
32-
Args:to
32+
Nothing is returned: the SHA-256 digest of ``serve.pkl`` is recorded in
33+
``metadata.json`` and verified against it in the container, so there is no secret to
34+
hand back to the caller.
35+
36+
Args:
3337
model_path (str) : Argument
3438
shared_libs (List[]) : Argument
3539
dependencies (dict) : Argument
3640
session (Session) : Argument
3741
inference_spec (InferenceSpec, optional) : Argument
3842
(default is None)
39-
40-
Returns:
41-
( str ) :
42-
4343
"""
4444
model_path = Path(model_path)
4545
if not model_path.exists():

0 commit comments

Comments
 (0)