Repository navigation
Commit 58b31e0
authored
feat(serve): support fine-tuned models in deployment-config API (#6041)
Make ModelBuilder.list_deployment_configs / get_deployment_config /
set_deployment_config work for fine-tuned (model-customization) models
in addition to base/JumpStart models, dispatching internally on
_is_model_customization(). For fine-tuned models the recipe's published
HostingConfigs are the source of truth; selection is by instance type
(recipe configs are largely unnamed).
The base "deployment config" vs recipe "hosting config" distinction is
internal and does not surface: fine-tuned configs are normalized to a
shape compatible with the base response (DeploymentConfigName plus a
nested DeploymentArgs block, plus BenchmarkMetrics/AccelerationConfigs
and an additive IsDefault flag). The normalized fine-tuned shape always
populates the DeploymentArgs keys (None when unset); the base response
may omit unset keys (its serializer drops empty slots), so the fine-tuned
shape is a superset — callers consuming both pathways should use .get()
for the optional keys (documented on the methods).
set_deployment_config applies the whole matching config (image,
environment, compute requirements) at build time. A caller-provided
image_uri still takes precedence over the config's ImageUri (documented).
Both branches fail fast on bad input: the fine-tuned branch requires
instance_type and raises on an unpublished or ambiguous instance; the
base branch requires config_name AND instance_type and now also rejects
an unpublished config name or an instance the config does not support
(validated against JumpStart metadata) instead of silently recording a
no-op selection.
Instance-type matching honors the config's full offered set, not only
its default. A base config is a multi-instance bundle: list_deployment_
configs(instance_type=X) filters against each config's supported-instance
metadata (supported_inference_instance_types) and materializes matched
configs FOR X, so a config that supports X but defaults to another
instance is neither discarded nor materialized at the wrong instance.
Recipe hosting configs are per-instance bundles by contract, but
SupportedInstanceTypes, when present, is honored for selection and
filtering. list/get/set agree for the same selection: get_deployment_
config() materializes the pinned instance (matching what list returns and
what build deploys), including when the pinned instance came from
SupportedInstanceTypes (differs from the config's default). The pinned
instance is preserved through build. Configs published with only
DefaultInstanceType are matched consistently end to end. The build env
merge tolerates a config that publishes an explicit null Environment.
Config discovery (recipe-level with a top-level HostingConfigs fallback)
is centralized in _extract_hosting_configs_from_hub() and used by BOTH
the selection API and the build path, so a top-level config that is
listable/selectable is also applied at build. Nova models are routed to
their dedicated build path first (per-tier SMI validation + Nova env
precedence), so the top-level fallback never diverts them. An explicit
selection stores a deep copy of the raw config, is applied exactly at
build (or raises if no longer published), and returned configs are copied
so caller mutation cannot corrupt internal state.
This is the SDK half of a paired change; the SageMaker agent
model-deployment skill consumes this unified API. Internal CR:
https://code.amazon.com/reviews/CR-2899036431 parent 87dc675 commit 58b31e0
2 files changed
Lines changed: 1865 additions & 48 deletions
File tree
- sagemaker-serve
- src/sagemaker/serve
- tests/unit
0 commit comments