Repository navigation
Commit e698fc6
fix(serve): propagate additional_model_data_sources for JumpStart models (#6151)
* fix(serve): propagate additional_model_data_sources for JumpStart models
ModelBuilder._build_for_jumpstart() copied image_uri, env,
enable_network_isolation, model_reference_arn, and model_data from the
resolved JumpStart init_kwargs, but never propagated
additional_model_data_sources. Any JumpStart model whose spec declares
an extra S3 data channel (e.g. an EAGLE speculative decoding draft
model, a LoRA adapter, or a tokenizer override) therefore had that
channel silently dropped from the CreateModel call, and the container
failed at runtime trying to read artifacts from a path that was never
mounted.
Propagate additional_model_data_sources from init_kwargs onto the
builder so _prepare_container_def_base attaches it to the CreateModel
request. Each source is filtered to only the keys the CreateModel API
accepts (ChannelName, S3DataSource); JumpStart specs may include extra
metadata keys such as HostingEulaKey that the API rejects.
Adds unit tests for both the propagation+filtering path and the no-op
case when the spec declares no additional data sources.
* fix(serve): enforce accept_eula on gated additional model data sources
A source carrying a truthy HostingEulaKey is gated and requires license
acceptance, exactly like the base model. container_def folds accept_eula
into the base model's ModelAccessConfig but passes additional sources
through untouched, so the acceptance is translated onto each gated
source here: fold ModelAccessConfig={AcceptEula: True} into its
S3DataSource when accept_eula is set, raise the same error the base
model raises when it is not. Ungated sources just drop the
JumpStart-internal HostingEulaKey the CreateModel API rejects.
Test coverage mirrors the v2 helper's case inventory: multiple public
sources, ungated ignores acceptance, mixed public+gated, gated accept,
gated missing/false acceptance raises, plus a raw-spec-JSON end-to-end
test driving the real JumpStartModelSpecs parsing and factory shaping.
* test(serve): drive additional-source EULA tests through the user flow
Replace the mock-harness additional_model_data_sources tests with
user-flow tests that exercise the documented user contract: construct a
real ModelBuilder, set accept_eula, and call build(). Only the spec
fetch boundary (JumpStartModelsAccessor.get_model_specs / _get_manifest)
and the AWS boundary (_prepare_for_mode / _create_model, mock session)
are mocked, mirroring how master-v2 tested JumpStartModel.
Raw spec JSON fixtures live in data/jumpstart_specs.json as a base spec
plus overlay variants (public model without/with public sources, gated
sources, mixed), reusable across tests. Everything between the two
boundaries runs for real: JumpStart detection, JumpStartModelSpecs
parsing, the get_init_kwargs factory shaping, and the EULA translation
in _build_for_jumpstart.
* test(serve): use real captured JumpStart specs as test fixtures
Replace the synthetic base-spec-plus-variants fixture with real,
unmodified specs captured from the production JumpStart content bucket,
mirroring master-v2's PROTOTYPICAL_MODEL_SPECS_DICT pattern: the spec
accessor routes by model id into one captured spec file per model, and
each test calls the SDK exactly like a customer does.
Fixture models (tests/unit/servers/data/jumpstart_specs/):
- pytorch-ic-mobilenet-v2 v3.0.25: public, no additional data sources
- openai-reasoning-gpt-oss-20b v3.38.0: ungated EAGLE speculative
decoding source in its default (lmi-optimized) config -- the model
that surfaced the propagation bug
- meta-textgeneration-llama-3-1-70b v2.16.2: gated draft_model source
(hosting_eula_key) in its lmi-optimized config, selected in tests via
the public set_deployment_config API
All patching is centralized in setUp (spec fetch, manifest, and the
AWS boundary), so test bodies contain only public API calls:
ModelBuilder(model=<real id>, instance_type=...), accept_eula, and
build(). Real inference-config resolution now runs in every test.
* test(serve): assert the CreateModel request shape via the single-call API
Restructure the additional-model-data-source tests on three axes:
1. Test names describe the case under test (gated source with accepted
EULA sends ModelAccessConfig, unselected config sources do not leak,
etc.), not the fixture model.
2. Each test drives the documented single-call customer API:
ModelBuilder.from_jumpstart_config(JumpStartConfig(model_id,
accept_eula, inference_config_name), compute=Compute(...)) followed
by build(). No attribute pokes and no separate config-selection call.
3. Assertions target the CreateModel request itself: _create_model is
no longer patched, so the real _create_sagemaker_model and
_prepare_container_def_base run, and every test inspects the
container_defs captured from sagemaker_session.create_model. This
covers the full downstream impact: AdditionalModelDataSources
presence/absence/shape, ModelAccessConfig on both the gated
additional source and the primary ModelDataSource, and
create_model never being called when the EULA gate raises.
* test(serve): derive expected sources from the spec fixtures
Replace the hand-copied expected-source class constants with a helper
that reads the captured spec fixture and applies the expected
transformation declaratively (PascalCase key mapping, content-bucket
resolution, HostingEulaKey absence, EULA folding). The fixture stays
the single source of truth for data values; only the behavior under
test remains spelled out in the test, avoiding both duplication and
tautological assertions.
* refactor(core): apply accept_eula to additional sources in container_def
Move the additional-model-data-source EULA handling out of the
JumpStart-specific block in _build_for_jumpstart and into
_apply_accept_eula_to_additional_model_data_sources next to
container_def, the single place the primary model's accept_eula
already becomes ModelAccessConfig. Both the primary model and the
additional sources are now driven by the same accept_eula flag in
adjacent code, and every container-assembly call site (including the
deployment-config path that never passes through _build_for_jumpstart)
gets the gated-source enforcement. _build_for_jumpstart is reduced to
pure propagation. No behavior change on the JumpStart path: all 63
serve tests pass unchanged, plus 103 core session_helper tests.
* refactor(serve): inline additional-source EULA fold in _build_for_jumpstart
Scope the EULA handling down to mirror the primary model exactly: strip
the JumpStart-internal HostingEulaKey (not part of the CreateModel API
shape) and fold accept_eula into each source's ModelAccessConfig when
set. No client-side gatedness check or raise: the control plane
determines gatedness from the artifact's bucket and rejects CreateModel
with its own EULA validation error, for the primary model and
additional sources alike.
This reverts the sagemaker-core half of the previous commit:
session_helper.py returns to upstream and the whole change lives in the
JumpStart builder that produces the spec-derived sources. Tests assert
the new contract on the CreateModel request: acceptance folded when
accept_eula is set (True or False), no ModelAccessConfig when unset.
* test(serve): state expected request values inline, trim to key assertions
Remove the expected-value helpers whose conditional re-implemented the
accept_eula fold rule, making the tests a parallel implementation of
the code under test. Expectations are now hand-written literals stated
in each test body: one exact-shape pin for the propagation case, and
targeted assertions (ModelAccessConfig value or absence, HostingEulaKey
absence, resolved bucket URI) for the EULA cases. Mutation coverage is
unchanged: the naive-propagation mutant still fails the same 5 tests.
---------
Co-authored-by: Eli Davidson <elleedee@amazon.com>1 parent 3c6d362 commit e698fc6
6 files changed
Lines changed: 7090 additions & 0 deletions
File tree
- sagemaker-serve
- src/sagemaker/serve
- tests/unit/servers
- data/jumpstart_specs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1014 | 1014 | | |
1015 | 1015 | | |
1016 | 1016 | | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
| 1020 | + | |
| 1021 | + | |
| 1022 | + | |
| 1023 | + | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
| 1030 | + | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
1017 | 1050 | | |
1018 | 1051 | | |
1019 | 1052 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
0 commit comments