Skip to content

Commit 58b31e0

Browse files
authored
feat(serve): support fine-tuned models in deployment-config API (#6041)
Make ModelBuilder.list_deployment_configs / get_deployment_config / set_deployment_config work for fine-tuned (model-customization) models in addition to base/JumpStart models, dispatching internally on _is_model_customization(). For fine-tuned models the recipe's published HostingConfigs are the source of truth; selection is by instance type (recipe configs are largely unnamed). The base "deployment config" vs recipe "hosting config" distinction is internal and does not surface: fine-tuned configs are normalized to a shape compatible with the base response (DeploymentConfigName plus a nested DeploymentArgs block, plus BenchmarkMetrics/AccelerationConfigs and an additive IsDefault flag). The normalized fine-tuned shape always populates the DeploymentArgs keys (None when unset); the base response may omit unset keys (its serializer drops empty slots), so the fine-tuned shape is a superset — callers consuming both pathways should use .get() for the optional keys (documented on the methods). set_deployment_config applies the whole matching config (image, environment, compute requirements) at build time. A caller-provided image_uri still takes precedence over the config's ImageUri (documented). Both branches fail fast on bad input: the fine-tuned branch requires instance_type and raises on an unpublished or ambiguous instance; the base branch requires config_name AND instance_type and now also rejects an unpublished config name or an instance the config does not support (validated against JumpStart metadata) instead of silently recording a no-op selection. Instance-type matching honors the config's full offered set, not only its default. A base config is a multi-instance bundle: list_deployment_ configs(instance_type=X) filters against each config's supported-instance metadata (supported_inference_instance_types) and materializes matched configs FOR X, so a config that supports X but defaults to another instance is neither discarded nor materialized at the wrong instance. Recipe hosting configs are per-instance bundles by contract, but SupportedInstanceTypes, when present, is honored for selection and filtering. list/get/set agree for the same selection: get_deployment_ config() materializes the pinned instance (matching what list returns and what build deploys), including when the pinned instance came from SupportedInstanceTypes (differs from the config's default). The pinned instance is preserved through build. Configs published with only DefaultInstanceType are matched consistently end to end. The build env merge tolerates a config that publishes an explicit null Environment. Config discovery (recipe-level with a top-level HostingConfigs fallback) is centralized in _extract_hosting_configs_from_hub() and used by BOTH the selection API and the build path, so a top-level config that is listable/selectable is also applied at build. Nova models are routed to their dedicated build path first (per-tier SMI validation + Nova env precedence), so the top-level fallback never diverts them. An explicit selection stores a deep copy of the raw config, is applied exactly at build (or raises if no longer published), and returned configs are copied so caller mutation cannot corrupt internal state. This is the SDK half of a paired change; the SageMaker agent model-deployment skill consumes this unified API. Internal CR: https://code.amazon.com/reviews/CR-289903643
1 parent 87dc675 commit 58b31e0

2 files changed

Lines changed: 1865 additions & 48 deletions

File tree

0 commit comments

Comments
 (0)