Skip to content

Commit 96edfbf

Browse files
author
Eli Davidson
committed
test(serve): state expected request values inline, trim to key assertions
Remove the expected-value helpers whose conditional re-implemented the accept_eula fold rule, making the tests a parallel implementation of the code under test. Expectations are now hand-written literals stated in each test body: one exact-shape pin for the propagation case, and targeted assertions (ModelAccessConfig value or absence, HostingEulaKey absence, resolved bucket URI) for the EULA cases. Mutation coverage is unchanged: the naive-propagation mutant still fails the same 5 tests.
1 parent 0f05fe6 commit 96edfbf

1 file changed

Lines changed: 59 additions & 150 deletions

File tree

‎sagemaker-serve/tests/unit/servers/test_model_builder_servers.py‎

Lines changed: 59 additions & 150 deletions
Original file line numberDiff line numberDiff line change
@@ -1010,101 +1010,36 @@ class TestJumpStartAdditionalModelDataSourcesUserFlow(unittest.TestCase):
10101010
"""End-to-end tests for additional model data sources, asserting the
10111011
CreateModel request shape.
10121012
1013-
Each test uses the documented single-call customer API:
1014-
1015-
builder = ModelBuilder.from_jumpstart_config(
1016-
jumpstart_config=JumpStartConfig(
1017-
model_id="...", accept_eula=True, inference_config_name="..."),
1018-
compute=Compute(instance_type="..."), ...)
1019-
builder.build()
1020-
1021-
and asserts on what the SDK sends to the CreateModel API -- the
1022-
``container_defs`` captured from ``sagemaker_session.create_model`` -- not
1023-
on builder internals. This is the request boto receives, so the assertions
1024-
cover the full downstream impact: primary container image, model data
1025-
source, and AdditionalModelDataSources.
1013+
Each test drives the documented single-call customer API
1014+
(ModelBuilder.from_jumpstart_config(...) then build()) and asserts on the
1015+
``container_defs`` captured from ``sagemaker_session.create_model`` -- the
1016+
request boto receives -- never on builder internals. Expected values are
1017+
hand-written literals so the wire contract is stated, not computed.
10261018
10271019
The spec fixtures under tests/unit/servers/data/jumpstart_specs/ are real,
1028-
unmodified specs captured from the production JumpStart content bucket
1029-
(jumpstart-cache-prod-us-west-2):
1020+
unmodified specs captured from the production JumpStart content bucket:
10301021
10311022
- pytorch-ic-mobilenet-v2: public model, no additional data sources.
1032-
- openai-reasoning-gpt-oss-20b: public model whose default (lmi-optimized)
1033-
config carries an ungated EAGLE speculative-decoding source -- the model
1034-
that surfaced the propagation bug this suite guards.
1035-
- meta-textgeneration-llama-3-1-70b: gated model whose lmi-optimized
1036-
config carries a GATED draft_model source
1037-
(hosting_eula_key=fmhMetadata/eula/llama3_2Eula.txt).
1038-
1039-
Patching is centralized in setUp on the same seams the legacy master-v2
1040-
JumpStartModel tests patched:
1041-
1042-
1. The spec-fetch boundary: JumpStartModelsAccessor.get_model_specs routes
1043-
by model id into the captured spec files (master-v2's
1044-
PROTOTYPICAL_MODEL_SPECS_DICT pattern); _get_manifest serves the
1045-
captured manifest headers.
1046-
2. The AWS boundary: a mock session (whose create_model call is the
1047-
assertion target), IAM role validation, artifact staging
1048-
(_prepare_for_mode), and the post-create Model.get describe call.
1049-
1050-
Everything in between runs for real: JumpStart model-id detection,
1051-
JumpStartModelSpecs parsing, inference-config resolution, the
1052-
get_init_kwargs factory (image URI resolution, content-bucket injection,
1053-
PascalCase shaping), the propagation in _build_for_jumpstart, the
1054-
accept_eula application in container_def (shared with the primary model),
1055-
and _prepare_container_def_base assembling the CreateModel request.
1023+
- openai-reasoning-gpt-oss-20b: default (lmi-optimized) config carries an
1024+
ungated EAGLE speculative-decoding source -- the model that surfaced the
1025+
propagation bug this suite guards.
1026+
- meta-textgeneration-llama-3-1-70b: lmi-optimized config carries a GATED
1027+
draft_model source (hosting_eula_key=fmhMetadata/eula/llama3_2Eula.txt).
1028+
1029+
setUp patches only the spec-fetch boundary (get_model_specs/_get_manifest
1030+
serve the captured files, master-v2's PROTOTYPICAL_MODEL_SPECS_DICT
1031+
pattern) and the AWS boundary (mock session whose create_model call is the
1032+
assertion target, role validation, artifact staging, post-create
1033+
describe). Everything in between runs for real: model-id detection, spec
1034+
parsing, inference-config resolution, the get_init_kwargs factory
1035+
(image URI resolution, content-bucket injection, PascalCase shaping), and
1036+
the HostingEulaKey strip + accept_eula fold in _build_for_jumpstart.
10561037
"""
10571038

10581039
SPEC_DIR = Path(__file__).parent / "data" / "jumpstart_specs"
10591040

10601041
ROLE_ARN = "arn:aws:iam::123456789012:role/SageMakerRole"
10611042

1062-
# The content buckets the sources must resolve into (the raw specs publish
1063-
# bucket-less key prefixes): public sources land in the public content
1064-
# bucket, gated sources in the private one.
1065-
PUBLIC_CONTENT_BUCKET = "jumpstart-cache-prod-us-west-2"
1066-
PRIVATE_CONTENT_BUCKET = "jumpstart-private-cache-prod-us-west-2"
1067-
1068-
@classmethod
1069-
def _spec_additional_source(cls, model_id, config_name="lmi-optimized"):
1070-
"""The raw additional data source exactly as published in the captured
1071-
spec fixture: snake_case keys, bucket-less s3_uri key prefix, and
1072-
hosting_eula_key present when the source is gated."""
1073-
with open(cls.SPEC_DIR / f"{model_id}.json") as f:
1074-
spec = json.load(f)
1075-
sources = spec["inference_config_components"][config_name][
1076-
"hosting_additional_data_sources"
1077-
]["speculative_decoding"]
1078-
assert len(sources) == 1, f"expected one source in {model_id}/{config_name}"
1079-
return sources[0]
1080-
1081-
@classmethod
1082-
def _expected_create_model_source(cls, model_id, bucket, accept_eula=None):
1083-
"""The shape the CreateModel request must carry for the fixture's
1084-
source. Data values (channel name, compression, data type, key prefix)
1085-
come from the spec fixture so it stays the single source of truth; the
1086-
transformation under test is spelled out declaratively here instead of
1087-
reusing production code (which would make the assertion a tautology):
1088-
1089-
- snake_case spec keys become the PascalCase API keys
1090-
- the bucket-less key prefix resolves into the given content bucket
1091-
- HostingEulaKey never appears (the API rejects it)
1092-
- accept_eula, when set, is folded into S3DataSource.ModelAccessConfig
1093-
exactly like the primary model's ModelDataSource
1094-
"""
1095-
raw = cls._spec_additional_source(model_id)
1096-
expected = {
1097-
"ChannelName": raw["channel_name"],
1098-
"S3DataSource": {
1099-
"CompressionType": raw["s3_data_source"]["compression_type"],
1100-
"S3DataType": raw["s3_data_source"]["s3_data_type"],
1101-
"S3Uri": f"s3://{bucket}/{raw['s3_data_source']['s3_uri']}",
1102-
},
1103-
}
1104-
if accept_eula is not None:
1105-
expected["S3DataSource"]["ModelAccessConfig"] = {"AcceptEula": accept_eula}
1106-
return expected
1107-
11081043
def setUp(self):
11091044
import sagemaker.serve.model_builder as model_builder_module
11101045
from sagemaker.core.jumpstart.accessors import JumpStartModelsAccessor
@@ -1182,120 +1117,94 @@ def test_model_without_additional_sources_sends_none_to_create_model(self):
11821117
self.assertNotIn("AdditionalModelDataSources", container_def)
11831118

11841119
def test_ungated_additional_source_reaches_create_model_without_eula(self):
1185-
"""The bug this suite guards: an ungated speculative-decoding source
1186-
in the model's default config must reach CreateModel even though the
1187-
customer never touches accept_eula."""
1120+
"""The bug this suite guards: an ungated speculative-decoding source in
1121+
the model's default config must reach CreateModel even though the
1122+
customer never touches accept_eula. This is the one exact-shape pin:
1123+
PascalCase keys, key prefix resolved into the public content bucket,
1124+
no HostingEulaKey, no ModelAccessConfig."""
11881125
self._build("openai-reasoning-gpt-oss-20b", instance_type="ml.g7e.2xlarge")
11891126

11901127
container_def = self._create_model_container_def()
11911128
self.assertEqual(
11921129
container_def["AdditionalModelDataSources"],
11931130
[
1194-
self._expected_create_model_source(
1195-
"openai-reasoning-gpt-oss-20b", bucket=self.PUBLIC_CONTENT_BUCKET
1196-
)
1131+
{
1132+
"ChannelName": "eagle",
1133+
"S3DataSource": {
1134+
"CompressionType": "None",
1135+
"S3DataType": "S3Prefix",
1136+
"S3Uri": "s3://jumpstart-cache-prod-us-west-2/"
1137+
"lmi-eagle-heads/models/gpt-oss-20b-p-eagle/",
1138+
},
1139+
}
11971140
],
11981141
)
11991142

12001143
def test_accept_eula_applies_to_every_model_data_source_uniformly(self):
12011144
"""The single accept_eula knob folds the same ModelAccessConfig into
1202-
the primary model AND each additional source, exactly like the primary
1203-
model's ModelDataSource handling. Gatedness is not the SDK's call: the
1204-
service determines it from the artifact's bucket and ignores the config
1205-
on ungated sources."""
1145+
the primary model AND the (ungated) additional source. Gatedness is
1146+
not the SDK's call: the service resolves it from the bucket and
1147+
ignores the config on ungated sources."""
12061148
self._build(
12071149
"openai-reasoning-gpt-oss-20b", accept_eula=True, instance_type="ml.g7e.2xlarge"
12081150
)
12091151

12101152
container_def = self._create_model_container_def()
1211-
self.assertEqual(
1212-
container_def["AdditionalModelDataSources"],
1213-
[
1214-
self._expected_create_model_source(
1215-
"openai-reasoning-gpt-oss-20b",
1216-
bucket=self.PUBLIC_CONTENT_BUCKET,
1217-
accept_eula=True,
1218-
)
1219-
],
1220-
)
1153+
(source,) = container_def["AdditionalModelDataSources"]
1154+
self.assertEqual(source["S3DataSource"]["ModelAccessConfig"], {"AcceptEula": True})
12211155
self.assertEqual(
12221156
container_def["ModelDataSource"]["S3DataSource"]["ModelAccessConfig"],
12231157
{"AcceptEula": True},
12241158
)
12251159

12261160
def test_gated_additional_source_with_accepted_eula_sends_model_access_config(self):
1227-
"""Accepting the EULA folds ModelAccessConfig into the gated
1228-
additional source AND the primary model in the CreateModel request."""
1161+
"""Accepting the EULA folds ModelAccessConfig into the gated draft
1162+
model source, whose URI resolves into the private content bucket. The
1163+
spec-internal HostingEulaKey never reaches the request."""
12291164
self._build(
12301165
"meta-textgeneration-llama-3-1-70b",
12311166
accept_eula=True,
12321167
inference_config_name="lmi-optimized",
12331168
instance_type="ml.p4d.24xlarge",
12341169
)
12351170

1236-
container_def = self._create_model_container_def()
1237-
self.assertEqual(
1238-
container_def["AdditionalModelDataSources"],
1239-
[
1240-
self._expected_create_model_source(
1241-
"meta-textgeneration-llama-3-1-70b",
1242-
bucket=self.PRIVATE_CONTENT_BUCKET,
1243-
accept_eula=True,
1244-
)
1245-
],
1246-
)
1171+
(source,) = self._create_model_container_def()["AdditionalModelDataSources"]
1172+
self.assertEqual(source["ChannelName"], "draft_model")
1173+
self.assertEqual(source["S3DataSource"]["ModelAccessConfig"], {"AcceptEula": True})
12471174
self.assertEqual(
1248-
container_def["ModelDataSource"]["S3DataSource"]["ModelAccessConfig"],
1249-
{"AcceptEula": True},
1175+
source["S3DataSource"]["S3Uri"],
1176+
"s3://jumpstart-private-cache-prod-us-west-2/meta-textgeneration/"
1177+
"meta-textgeneration-llama-3-2-1b/artifacts/inference-prepack/v1.0.0/",
12501178
)
1179+
self.assertNotIn("HostingEulaKey", source)
12511180

12521181
def test_gated_additional_source_without_eula_sends_no_model_access_config(self):
1253-
"""No EULA decision: the request carries the gated source without any
1254-
ModelAccessConfig. Enforcement is the service's: the SageMaker control
1255-
plane resolves the private-bucket URI as gated and rejects CreateModel
1256-
with an EULA validation error, the same as for a gated primary model."""
1182+
"""No EULA decision: the gated source goes out with no
1183+
ModelAccessConfig (and no HostingEulaKey). Enforcement is the
1184+
service's: the control plane resolves the private bucket as gated and
1185+
rejects CreateModel with an EULA validation error."""
12571186
self._build(
12581187
"meta-textgeneration-llama-3-1-70b",
12591188
inference_config_name="lmi-optimized",
12601189
instance_type="ml.p4d.24xlarge",
12611190
)
12621191

1263-
container_def = self._create_model_container_def()
1264-
self.assertEqual(
1265-
container_def["AdditionalModelDataSources"],
1266-
[
1267-
self._expected_create_model_source(
1268-
"meta-textgeneration-llama-3-1-70b", bucket=self.PRIVATE_CONTENT_BUCKET
1269-
)
1270-
],
1271-
)
1192+
(source,) = self._create_model_container_def()["AdditionalModelDataSources"]
1193+
self.assertNotIn("ModelAccessConfig", source["S3DataSource"])
1194+
self.assertNotIn("HostingEulaKey", source)
12721195

12731196
def test_gated_additional_source_with_rejected_eula_sends_acceptance_false(self):
12741197
"""Explicit accept_eula=False is transmitted faithfully as
1275-
ModelAccessConfig={"AcceptEula": False} on every source, exactly like
1276-
the primary model. The service rejects the gated sources."""
1198+
ModelAccessConfig={"AcceptEula": False}; the service rejects it."""
12771199
self._build(
12781200
"meta-textgeneration-llama-3-1-70b",
12791201
accept_eula=False,
12801202
inference_config_name="lmi-optimized",
12811203
instance_type="ml.p4d.24xlarge",
12821204
)
12831205

1284-
container_def = self._create_model_container_def()
1285-
self.assertEqual(
1286-
container_def["AdditionalModelDataSources"],
1287-
[
1288-
self._expected_create_model_source(
1289-
"meta-textgeneration-llama-3-1-70b",
1290-
bucket=self.PRIVATE_CONTENT_BUCKET,
1291-
accept_eula=False,
1292-
)
1293-
],
1294-
)
1295-
self.assertEqual(
1296-
container_def["ModelDataSource"]["S3DataSource"]["ModelAccessConfig"],
1297-
{"AcceptEula": False},
1298-
)
1206+
(source,) = self._create_model_container_def()["AdditionalModelDataSources"]
1207+
self.assertEqual(source["S3DataSource"]["ModelAccessConfig"], {"AcceptEula": False})
12991208

13001209
def test_unselected_config_sources_do_not_leak_into_create_model(self):
13011210
"""Config resolution gates which sources apply: the same gated model on

0 commit comments

Comments
 (0)