feat: hub search and metadata, trainer job handles, typed deploy options, endpoint diagnostics, Bedrock and S3 helpers

docs/api/sagemaker_core.rst

@@ -52,6 +52,18 @@ Utilities
    :members:
    :undoc-members:
 
+.. automodule:: sagemaker.core.instance_types
+   :members:
+   :undoc-members:
+
+.. automodule:: sagemaker.core.hub_content
+   :members:
+   :undoc-members:
+
+.. automodule:: sagemaker.core.hub_catalog
+   :members:
+   :undoc-members:
+
 Exceptions
 ----------
 

docs/model_customization/model_customization.rst

@@ -154,14 +154,15 @@ Key Features
 
     # Re-attach (HyperPod — needs cluster name for log group resolution)
     from sagemaker.train import SFTTrainer
+    from sagemaker.core.resources import TrainingJob
     from sagemaker.core.training.configs import HyperPodCompute
 
     trainer = SFTTrainer(
         model="nova-textgeneration-micro",
         training_dataset="s3://unused",
         compute=HyperPodCompute(cluster_name="my-cluster", instance_type="ml.p5.48xlarge")
     )
-    trainer._latest_training_job = "my-hp-job"
+    trainer.latest_training_job = TrainingJob.get("my-hp-job")
     df = trainer.show_metrics()
 
 **Monitoring: stream_logs()**

docs/training/index.rst

@@ -299,13 +299,65 @@ Train pre-configured models from the SageMaker JumpStart hub using ``ModelTraine
    # Complex queries with filters
    text_gen = search_public_hub_models(query="@task:text-generation")
 
+**Reading a Hub Model's Metadata:**
+
+.. code-block:: python
+
+   from datetime import timedelta
+
+   from sagemaker.core.resources import HubContent
+
+   # Every model of the public hub, with the fields the hub tags it with. get_all serves the
+   # hub catalog cache, built like the JumpStart models cache: an LRU cache of hubs in memory
+   # (6 hour horizon) backed by one JSON file per hub. The first call describes every model
+   # on a thread pool, a call inside the horizon makes no request, and a call after it lists
+   # the hub and describes only the models whose version changed.
+   small_sft_models = [
+       content.hub_content_name
+       for content in HubContent.get_all(hub_name="SageMakerPublicHub", hub_content_type="Model")
+       if content.supports_technique("sft") and content.model_size in ("<1b", "1b-10b")
+   ]
+
+   # The ListHubContents filters apply to the cached catalog, so a filtered call costs no request.
+   newest_qwen3 = HubContent.get_all(
+       hub_name="SageMakerPublicHub",
+       hub_content_type="Model",
+       name_contains="qwen3",
+       sort_by="CreationTime",
+       sort_order="Descending",
+   )
+   HubContent.set_cache_kwargs(cache_expiration_horizon=timedelta(days=1))
+
+   # One model; the document fills in fields the 50-keyword cap dropped
+   content = HubContent.get(
+       hub_name="SageMakerPublicHub",
+       hub_content_type="Model",
+       hub_content_name="meta-textgeneration-llama-3-2-1b",
+   )
+   print(content.license, content.provider, content.tasks, content.fine_tuning_techniques())
+
+   # Server-side search on the document fields, paged for you. A Metadata value matches the
+   # document spelling, case-sensitive: Apache-2.0 and meta, not apache-2.0 and Meta.
+   from sagemaker.core.hub_content import HubContentFilter
+
+   apache_meta_models = HubContent.search(
+       "SageMakerPublicHub",
+       filters=[
+           HubContentFilter.metadata("license", "Apache-2.0"),
+           HubContentFilter.metadata("provider", "meta"),
+       ],
+   )
+
 Key points:
 
 - ``from_jumpstart_config()`` auto-configures training image, instance type, and default hyperparameters
 - Override any default hyperparameters while keeping proven defaults for the rest
 - JumpStart provides built-in datasets so you can start training immediately
 - Supports HuggingFace, XGBoost, CatBoost, LightGBM, and many more frameworks
 - Use ``list_jumpstart_models()`` and ``search_public_hub_models()`` to discover available models
+- ``HubContent`` exposes ``tasks``, ``license``, ``provider``, ``model_size``, ``context_window``, ``languages``, ``capabilities``, and ``recipes`` read from its search keywords and document, so no caller parses ``HubContentSearchKeywords`` by hand
+- ``HubContent.search()`` filters a hub server side on ``Metadata.<Field>`` through ``HubContentFilter``, inside the 20-filter and 1024-character service limits, and returns the matching ``HubContent`` list. A ``Metadata`` value is case-sensitive and matches the document spelling (``Apache-2.0``, ``meta``), while the keyword accessors return the lowercase keyword
+- ``HubContent.get_all()`` serves a hub from ``HubCatalogCache``, the same ``LRUCache`` shape as ``JumpStartModelsCache`` (20 hubs, 6 hour horizon), backed by one JSON file per hub under ``SAGEMAKER_HUB_CACHE_DIR`` or the user cache directory. It describes only the contents whose ``HubContentVersion`` changed, on a thread pool of the host default size with adaptive retries, and applies the ``ListHubContents`` filters to the cached catalog. ``refresh_cache()`` lists again and reports the counts, ``clear_cache()`` drops the memory and the file, and ``set_cache_kwargs()`` tunes the horizon, the item cap, the worker count, and the directory
 
 :doc:`Full example notebook <../v3-examples/training-examples/jumpstart-training-example>`
 
@@ -434,7 +486,7 @@ Provide an ordered list of candidate instance types and the platform launches th
 
 .. code-block:: python
 
-   training_job = model_trainer._latest_training_job
+   training_job = model_trainer.latest_training_job
    training_job.refresh()
 
    resource_config = training_job.resource_config

migration.md

@@ -478,6 +478,7 @@ pipeline = Pipeline(
 | Fine-tuning SDK (SFT, RLVR, RLAIF) | Not available | V3.1.0: Standardized fine-tuning techniques | 🆕 NEW IN V3 |
 | Model customization | Manual training scripts | model-customization-examples/ | 🆕 NEW IN V3 |
 | Nova recipe training | Not available | V3.4.0: Nova recipe support in ModelTrainer | 🆕 NEW IN V3 |
+| Attach a trainer to an existing job | `Estimator.attach(job_name)` | `trainer.latest_training_job = TrainingJob.get(job_name)`. The property accepts a `TrainingJob` or `None` and raises `TypeError` for a job name string. `trainer.train()` returns the created `TrainingJob` | 🆕 NEW IN V3 |
 
 ### Data Processing
 
@@ -531,6 +532,9 @@ pipeline = Pipeline(
 | Fine-tune foundation models | `JumpStartEstimator(model_id=...).fit()` | V3: jumpstart-training-example.ipynb | 🔄 REPLACED |
 | E2E training + inference | Separate steps | jumpstart-e2e-training-example.ipynb | 🔄 REPLACED |
 | Marketplace algorithms | `sagemaker.algorithm.AlgorithmEstimator` | `sagemaker.core.resources.Algorithm` (13 classes) | 🔄 REPLACED |
+| List hub models | `list_jumpstart_models()` (manifest) | `HubContent.get_all(hub_name, hub_content_type)` serves the hub from an `LRUCache` with the JumpStart cache defaults (20 hubs, 6 hour horizon), backed by a JSON file per hub. It describes only the models whose version changed, on a thread pool, and applies the `ListHubContents` filters to the cached catalog | 🆕 NEW IN V3 |
+| Read model metadata | `JumpStartModelSpecs` fields | `HubContent.license`, `.provider`, `.model_size`, `.tasks`, `.capabilities`, `.recipes`, `.supports_technique("sft")` read from the hub keywords and document | 🆕 NEW IN V3 |
+| Search hub models server side | None (`search_public_hub_models` filters client side) | `HubContent.search(hub_name, filters=[HubContentFilter.metadata("license", "Apache-2.0")])` runs `Search(Resource="HubContent")` | 🆕 NEW IN V3 |
 
 ## Configuration Management
 
@@ -559,16 +563,16 @@ compute = Compute(
     instance_type="ml.m5.xlarge",
     instance_count=1,
     volume_size_in_gb=30,
-    use_spot_instances=True,
-    max_wait_time_in_seconds=7200
+    enable_managed_spot_training=True
 )
 networking = Networking(
     subnets=["subnet-12345"],
     security_group_ids=["sg-12345"],
     enable_inter_container_traffic_encryption=True
 )
 stopping_condition = StoppingCondition(
-    max_runtime_in_seconds=3600
+    max_runtime_in_seconds=3600,
+    max_wait_time_in_seconds=7200
 )
 model_trainer = ModelTrainer(
     training_image="...",

sagemaker-core/src/sagemaker/core/common_utils.py

@@ -1688,12 +1688,74 @@ def create_paginator_config(max_items: int = None, page_size: int = None) -> Dic
     }
 
 
-def format_tags(tags: Tags) -> List[TagsDict]:
-    """Process tags to turn them into the expected format for Sagemaker."""
-    if isinstance(tags, dict):
-        return [{"Key": str(k), "Value": str(v)} for k, v in tags.items()]
+def _tag_text(value: Any) -> Union[str, PipelineVariable]:
+    """Return a tag key or value as the service string, keeping a pipeline variable intact.
+
+    The service model marks ``Tag.Value`` as a necessary member (0 to 256 characters), so a
+    missing value becomes the empty string instead of ``None``.
+    """
+    if value is None:
+        return ""
+    if is_pipeline_variable(value):
+        return value
+    return str(value)
+
+
+def _tag_dict(key: Any, value: Any) -> TagsDict:
+    return {"Key": _tag_text(key), "Value": _tag_text(value)}
+
+
+def _normalize_tag(tag: Any) -> TagsDict:
+    """Return one tag as a boto-style ``{"Key": ..., "Value": ...}`` dict of strings.
+
+    Accepts the boto form, the ``{"key": ..., "value": ...}`` form that the
+    ``sagemaker.core.resources`` classes take, a ``(key, value)`` pair, and a ``Tag``
+    shape object (anything with ``key`` and ``value`` attributes). The key and the value
+    come out as ``str`` whatever the input representation, except a ``PipelineVariable``,
+    which the request serializer resolves later. Any other input raises ``TypeError`` so a
+    malformed tag fails at the call site instead of as a ``KeyError`` deep in a deploy
+    path.
+    """
+    if isinstance(tag, dict):
+        if "Key" in tag:
+            return _tag_dict(tag["Key"], tag.get("Value"))
+        if "key" in tag:
+            return _tag_dict(tag["key"], tag.get("value"))
+        raise TypeError(
+            "A tag dict must have a 'Key' (or 'key') entry, "
+            f"got {sorted(tag)}. Pass a {{key: value}} mapping to set many tags at once."
+        )
+    if isinstance(tag, (tuple, list)) and len(tag) == 2:
+        return _tag_dict(tag[0], tag[1])
+    if hasattr(tag, "key") and hasattr(tag, "value"):
+        return _tag_dict(tag.key, tag.value)
+    raise TypeError(
+        "A tag must be a {'Key': ..., 'Value': ...} dict, a {'key': ..., 'value': ...} dict, "
+        f"a (key, value) pair, or a Tag object, got {type(tag).__name__}."
+    )
+
 
-    return tags
+def format_tags(tags: Optional[Tags]) -> Optional[List[TagsDict]]:
+    """Process tags to turn them into the expected format for Sagemaker.
+
+    Every accepted representation comes out as a list of boto-style
+    ``{"Key": ..., "Value": ...}`` dicts with string keys and values:
+
+    * a ``{key: value}`` mapping, one tag per entry;
+    * a single tag dict or ``Tag`` object;
+    * a list of tag dicts in either casing, ``(key, value)`` pairs, ``Tag`` objects, or a mix.
+
+    ``None`` is returned unchanged.
+    """
+    if tags is None:
+        return tags
+    if isinstance(tags, dict):
+        if "Key" in tags or "key" in tags:
+            return [_normalize_tag(tags)]
+        return [_tag_dict(key, value) for key, value in tags.items()]
+    if isinstance(tags, (list, tuple)):
+        return [_normalize_tag(tag) for tag in tags]
+    return [_normalize_tag(tags)]
 
 
 def _get_resolved_path(path):
@@ -2124,38 +2186,43 @@ def tag_exists(tag: TagsDict, curr_tags: Optional[Tags]) -> bool:
     if curr_tags is None:
         return False
 
-    for curr_tag in curr_tags:
-        if tag["Key"] == curr_tag["Key"]:
+    key = _normalize_tag(tag)["Key"]
+    for curr_tag in format_tags(curr_tags):
+        if key == curr_tag["Key"]:
             return True
 
     return False
 
 
-def _validate_new_tags(new_tags: Optional[Tags], curr_tags: Optional[Tags]) -> Optional[Tags]:
-    """Validates new tags against existing tags.
+def _validate_new_tags(
+    new_tags: Optional[Tags], curr_tags: Optional[Tags]
+) -> Optional[List[TagsDict]]:
+    """Merge new tags into the current tags, without a duplicate key.
+
+    The two inputs take any representation ``format_tags`` accepts. The result is the
+    merged list in boto form, so a caller that stores it (``ModelBuilder.add_tags``)
+    holds one shape whatever shape it received. ``None`` comes back only when the two
+    inputs are ``None``.
 
     Args:
         new_tags (Optional[Tags]): The new tags.
         curr_tags (Optional[Tags]): The current tags.
 
     Returns:
-        Optional[Tags]: The updated tags.
+        Optional[List[TagsDict]]: The merged tags in boto form.
     """
-    if curr_tags is None:
-        return new_tags
+    if new_tags is None:
+        return format_tags(curr_tags) if curr_tags is not None else None
 
-    if curr_tags and isinstance(curr_tags, dict):
-        curr_tags = [curr_tags]
+    if curr_tags is None:
+        return format_tags(new_tags)
 
-    if isinstance(new_tags, dict):
-        if not tag_exists(new_tags, curr_tags):
-            curr_tags.append(new_tags)
-    elif isinstance(new_tags, list):
-        for new_tag in new_tags:
-            if not tag_exists(new_tag, curr_tags):
-                curr_tags.append(new_tag)
+    merged = format_tags(curr_tags)
+    for new_tag in format_tags(new_tags):
+        if not tag_exists(new_tag, merged):
+            merged.append(new_tag)
 
-    return curr_tags
+    return merged
 
 
 def remove_tag_with_key(key: str, tags: Optional[Tags]) -> Optional[Tags]:
@@ -2170,11 +2237,9 @@ def remove_tag_with_key(key: str, tags: Optional[Tags]) -> Optional[Tags]:
     """
     if tags is None:
         return tags
-    if isinstance(tags, dict):
-        tags = [tags]
 
     updated_tags = []
-    for tag in tags:
+    for tag in format_tags(tags):
         if tag["Key"] != key:
             updated_tags.append(tag)
 

sagemaker-core/src/sagemaker/core/config_schema.py

@@ -1,3 +1,5 @@
+"""JSON schema of the SageMaker Python SDK configuration file."""
+
 SAGEMAKER_PYTHON_SDK_CONFIG_SCHEMA = {
     "$schema": "https://json-schema.org/draft/2020-12/schema",
     "type": "object",