Skip to content

Commit 1e1cbcf

Browse files
authored
feat: actionable guidance for removed v2 interfaces (#6004)
* feat: actionable guidance for removed v2 interfaces In v3 the v2 top-level modules (sagemaker.estimator, sagemaker.model, ...) were removed, so importing them fails with a bare "ModuleNotFoundError: No module named 'sagemaker.estimator'" that gives the caller no path forward. This adds an actionable migration experience. - raise_removed_in_v3() helper in sagemaker.core.deprecations: emits a DeprecationWarning and raises a ModuleNotFoundError naming the v3 replacement (and the exact import where the migration guide publishes one), plus the migration-guide link. - 12 curated "tombstone" modules under sagemaker/ for the high-traffic removed interfaces (estimator, model, predictor, base_predictor, predictor_async, transformer, tuner, processing, pipeline, multidatamodel, automl, algorithm). These ship with sagemaker-core. - A fallback meta-path finder (_RemovedV2ModuleFinder) registered from sagemaker.core.__init__ as a last resort: any other removed sagemaker.<name> gets a generic "not available in v3, see migration guide" message instead of a bare error. Appended to sys.meta_path so it never shadows real v3 packages or the tombstones, with a _KNOWN_V3_TOPLEVEL guard protecting real packages (core/train/serve/mlops/lineage/ai_registry). Adds tests/unit/test_removed_v2_modules.py (36 tests) covering tombstone messages and the fallback finder (registration, ordering, non-shadowing of real packages, pass-through of non-sagemaker and nested names). * refactor: log removed-module guidance at debug, not warning The raised ModuleNotFoundError is the loud, authoritative signal (it stops execution and carries the full message with the v3 replacement + migration link). Logging the same message at WARNING duplicated it noisily on a path that always raises. Downgrade the logger call to debug in both raise_removed_in_v3 (tombstones) and the fallback finder (catch-all), leaving a breadcrumb for log-captured environments without the noise. The DeprecationWarning and the raise are unchanged. * docs: simplify fallback message to 'It may have been removed' * test: assert finder pass-through via find_spec, not real import test_finder_does_not_shadow_real_v3_packages imported sagemaker.train / sagemaker.serve to prove the fallback finder does not shadow them. That assumed those packages are installed -- but the sagemaker-core unit-test CI job installs sagemaker-core only, so the import raised ModuleNotFoundError and the test failed. Assert the actual contract instead: the finder's find_spec() returns None (pass-through) for real v3 top-level packages, which needs no import and holds in a core-only environment. Also covers mlops and lineage. * feat: point removed-interface errors at verified v3 code + docs Address PR feedback: name the actual v3 replacement (identifiable in the repo) and link its API docs, instead of vague package references. - Add a v3_docs argument to raise_removed_in_v3 so each message carries a direct readthedocs API link alongside the copy-pasteable import. - Update all 12 tombstones to verified-importable v3 targets + docs URLs: estimator/algorithm -> sagemaker.train.ModelTrainer model/multidatamodel -> sagemaker.serve.ModelBuilder predictor/base_predictor -> sagemaker.core.resources.Endpoint predictor_async -> sagemaker.serve.ModelBuilder transformer -> sagemaker.core.resources.TransformJob tuner -> sagemaker.core.resources.HyperParameterTuningJob processing -> sagemaker.core.resources.ProcessingJob pipeline -> sagemaker.mlops.workflow.pipeline.Pipeline automl -> sagemaker.core.resources.AutoMLJob Corrects two paths that did not resolve in the shipped v3 code (the migration guide's sagemaker.mlops.pipeline and sagemaker.mlops.processing .DataProcessor): Pipeline is at sagemaker.mlops.workflow.pipeline, and processing maps to the core ProcessingJob resource. All imports and docs URLs were verified against the installed packages and readthedocs. Tests assert the import statement and docs URL are present in every message; updated expected symbols for tuner and processing. * fix: fallback finder only guides for known removed v2 modules Previously the fallback finder emitted "was removed in v3" guidance for ANY top-level sagemaker.<name> that was not a real v3 package -- including typos and hallucinated imports like `sagemaker.foobar`, which never existed and so should not claim to have been removed. Gate the finder on _REMOVED_V2_MODULES, the set of top-level modules that actually existed in v2 (derived from the master-v2 branch, minus names still present in v3). Unknown names now fall through to a plain ModuleNotFoundError with no deprecation warning and no migration link. Because the name is now known to be a real v2 module, the message states "was removed" (was "may have been removed"). Adds tests: known removed names -> guidance; unknown names -> plain error with no warning/link; finder find_spec pass-through for unknown names. * fix: register fallback finder on any sagemaker import, not just core The fallback finder was registered only from sagemaker/core/__init__.py, so a removed module imported as the very first sagemaker statement (e.g. `import sagemaker.estimator` with nothing else loaded) missed the guidance and got a bare ModuleNotFoundError -- the finder had not been installed yet. Register it from sagemaker-core's namespace sagemaker/__init__.py, which runs on any `import sagemaker.*` (these are pkgutil-style namespace packages whose __init__ executes). Fully guarded so it can never interfere with importing the package. Tombstone modules were unaffected (real files), and normal usage already pulled in core transitively; this closes the first-import edge case. Adds a fresh-interpreter regression test asserting a removed module gets guidance even as the first sagemaker import. * refactor: consolidate removed-module guidance into the finder Address review: the 12 per-module "tombstone" files were not strictly necessary -- the meta-path finder receives the full module name, so it can produce the same precise guidance from a lookup table. - Add _V3_REPLACEMENTS: curated (replacement, import, docs) for the 12 high-traffic removed modules. find_spec now emits the precise message for these, a generic "was removed" message for other _REMOVED_V2_MODULES, and a plain ModuleNotFoundError for unknown names (typos). - Delete the 12 tombstone modules under src/sagemaker/. Behavior is unchanged for users (verified: same precise messages), with one mechanism instead of files + finder. - Add a debug log when the finder registers (review comment). - Add name-set invariants (review comment "what prevents a collision?"): _REMOVED_V2_MODULES is disjoint from real v3 top-level packages, all curated keys are in the removed set, and a real module under a removed name still wins over the appended finder (never shadowed). 47 unit tests pass; black + flake8 clean. * feat: expand precise v3 guidance to 34 modules + validate targets in CI Now that guidance is data-driven (no per-module files), extend the curated _V3_REPLACEMENTS map from 12 to 34 removed v2 modules -- every framework estimator, inference/serverless/jumpstart interface, core resource (clarify/wrangler/spark/model_monitor/feature_store/experiments/model_card), relocated utility (image_uris/s3/serializers/deserializers), workflow, and inputs/session. Each replacement import and docs URL was verified against installed v3 and readthedocs. The remaining removed modules (internal utils, hard-removed frameworks like mxnet/chainer/rl) keep the generic message -- they have no user-facing v3 replacement to point at. Maintenance guards (recommendation from review): - test_curated_v3_imports_resolve: imports every replacement target (via pytest.importorskip so it runs in the all-extras lane and skips cleanly in the core-only job), so a future rename/move fails CI instead of shipping a dead import string. - test_curated_docs_url_matches_import_module_root: the docs URL is a derivation of the import's module path, asserted to share the same top-level package so the two cannot drift apart. 82 unit tests pass; black + flake8 clean.
1 parent 17b2679 commit 1e1cbcf

4 files changed

Lines changed: 665 additions & 1 deletion

File tree

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,16 @@
11
"""Namespace package for SageMaker."""
2-
__path__ = __import__('pkgutil').extend_path(__path__, __name__)
2+
3+
__path__ = __import__("pkgutil").extend_path(__path__, __name__)
4+
5+
# Register the fallback finder that gives actionable migration guidance for v2
6+
# modules removed in v3. Doing it here (in the namespace package init, which
7+
# runs on any ``import sagemaker.*``) ensures the guidance is active even when a
8+
# removed module is the very first sagemaker import in the process (e.g.
9+
# ``import sagemaker.estimator``), before ``sagemaker.core`` is otherwise loaded.
10+
# Fully guarded so it can never interfere with importing the package.
11+
try:
12+
from sagemaker.core.deprecations import register_removed_module_finder as _register
13+
14+
_register()
15+
except Exception: # pylint: disable=W0703 # noqa: E722
16+
pass

‎sagemaker-core/src/sagemaker/core/__init__.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,18 @@
11
"""SageMaker Core package for low-level resource management and SDK foundations."""
2+
23
from sagemaker.core.utils.utils import enable_textual_rich_console_and_traceback
4+
from sagemaker.core.deprecations import register_removed_module_finder
35

46

57
enable_textual_rich_console_and_traceback()
68

9+
# Install the meta-path finder that gives actionable migration guidance for v2
10+
# modules removed in v3. sagemaker-core is the universal dependency of every v3
11+
# package, so registering here guarantees the finder is active whenever the SDK
12+
# is used (the namespace package __init__ also registers it for the cold
13+
# first-import case).
14+
register_removed_module_finder()
15+
716
# Job management
817
from sagemaker.core.job import _Job # noqa: F401
918
from sagemaker.core.processing import ( # noqa: F401

‎sagemaker-core/src/sagemaker/core/deprecations.py‎

Lines changed: 327 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,340 @@
1313
"""Module for deprecation abstractions."""
1414
from __future__ import absolute_import
1515

16+
import importlib.abc
1617
import logging
18+
import sys
1719
import warnings
1820

1921
logger = logging.getLogger(__name__)
2022

2123
V2_URL = "https://sagemaker.readthedocs.io/en/stable/v2.html"
2224

25+
# Migration guide for users moving from the v2 SDK to v3.
26+
V3_MIGRATION_URL = "https://github.com/aws/sagemaker-python-sdk/blob/master/migration.md"
27+
28+
# Real top-level ``sagemaker.*`` names that ship in v3. The fallback finder must
29+
# never intercept these -- even when a package (e.g. sagemaker-train) is simply
30+
# not installed, its absence should surface the normal error, not a bogus
31+
# "removed in v3" message.
32+
_KNOWN_V3_TOPLEVEL = frozenset({"core", "train", "serve", "mlops", "lineage", "ai_registry"})
33+
34+
# Top-level ``sagemaker.<name>`` modules that existed in v2 but were removed in
35+
# v3 (some relocated under ``sagemaker.core.*``). Derived from the v2 top-level
36+
# module surface (the ``master-v2`` branch), minus names that still exist in v3.
37+
# The fallback finder only emits migration guidance for THESE names, so a typo
38+
# or hallucinated import (e.g. ``sagemaker.foobar``) gets a plain
39+
# ``ModuleNotFoundError`` rather than a misleading "was removed" message. V2 is
40+
# in maintenance, so this surface is effectively frozen.
41+
_REMOVED_V2_MODULES = frozenset(
42+
{
43+
"_studio",
44+
"accept_types",
45+
"algorithm",
46+
"amazon",
47+
"amtviz",
48+
"analytics",
49+
"apiutils",
50+
"async_inference",
51+
"automl",
52+
"aws_batch",
53+
"base_deserializers",
54+
"base_predictor",
55+
"base_serializers",
56+
"batch_inference",
57+
"chainer",
58+
"clarify",
59+
"cli",
60+
"collection",
61+
"compute_resource_requirements",
62+
"config",
63+
"container_base_model",
64+
"content_types",
65+
"dataset_definition",
66+
"debugger",
67+
"deprecations",
68+
"deserializers",
69+
"djl_inference",
70+
"drift_check_baselines",
71+
"enums",
72+
"environment_variables",
73+
"estimator",
74+
"exceptions",
75+
"experiments",
76+
"explainer",
77+
"feature_store",
78+
"fw_utils",
79+
"git_utils",
80+
"huggingface",
81+
"hyperparameters",
82+
"image_uri_config",
83+
"image_uris",
84+
"inference_recommender",
85+
"inputs",
86+
"instance_group",
87+
"instance_types",
88+
"instance_types_gpu_info",
89+
"interactive_apps",
90+
"iterators",
91+
"job",
92+
"jumpstart",
93+
"lambda_helper",
94+
"local",
95+
"logs",
96+
"metadata_properties",
97+
"metric_definitions",
98+
"mlflow",
99+
"model",
100+
"model_card",
101+
"model_life_cycle",
102+
"model_metrics",
103+
"model_monitor",
104+
"model_uris",
105+
"modules",
106+
"multidatamodel",
107+
"mxnet",
108+
"network",
109+
"parameter",
110+
"partner_app",
111+
"payloads",
112+
"pipeline",
113+
"predictor",
114+
"predictor_async",
115+
"processing",
116+
"pytorch",
117+
"remote_function",
118+
"resource_requirements",
119+
"rl",
120+
"s3",
121+
"s3_utils",
122+
"script_uris",
123+
"serializer_utils",
124+
"serializers",
125+
"serverless",
126+
"session",
127+
"session_settings",
128+
"sklearn",
129+
"spark",
130+
"sparkml",
131+
"stabilityai",
132+
"telemetry",
133+
"tensorflow",
134+
"training_compiler",
135+
"transformer",
136+
"tuner",
137+
"user_agent",
138+
"utilities",
139+
"utils",
140+
"vpc_utils",
141+
"workflow",
142+
"wrangler",
143+
"xgboost",
144+
}
145+
)
146+
147+
_DOCS_BASE = "https://sagemaker.readthedocs.io/en/stable/api/generated/"
148+
149+
# Curated removed v2 modules -> precise v3 guidance:
150+
# (replacement, exact import, docs module path). The finder emits a specific
151+
# message (exact class + copy-pasteable import + docs link) for these names, and
152+
# a generic "was removed" message for every other name in _REMOVED_V2_MODULES.
153+
# Every import target and docs page below is verified against installed v3 and
154+
# readthedocs; the import targets are additionally guarded by a unit test.
155+
_CORE_RES = "sagemaker.core.resources"
156+
157+
158+
def _trainer():
159+
return (
160+
"`ModelTrainer`",
161+
"from sagemaker.train import ModelTrainer",
162+
"sagemaker.train.model_trainer",
163+
)
164+
165+
166+
def _builder(note=""):
167+
repl = "`ModelBuilder`" + (f" ({note})" if note else "")
168+
return (repl, "from sagemaker.serve import ModelBuilder", "sagemaker.serve.model_builder")
169+
170+
171+
def _core_resource(cls):
172+
return (f"the `{cls}` resource", f"from {_CORE_RES} import {cls}", _CORE_RES)
173+
174+
175+
def _core_module(mod):
176+
return (f"`{mod}`", f"from sagemaker.core import {mod}", f"sagemaker.core.{mod}")
177+
178+
179+
_V3_REPLACEMENTS = {
180+
# Training: estimators (generic + framework) + marketplace algorithms -> ModelTrainer
181+
"estimator": _trainer(),
182+
"algorithm": _trainer(),
183+
"pytorch": _trainer(),
184+
"tensorflow": _trainer(),
185+
"huggingface": _trainer(),
186+
"sklearn": _trainer(),
187+
"xgboost": _trainer(),
188+
# Inference: models / predictors / serverless / async / jumpstart -> ModelBuilder
189+
"model": _builder(),
190+
"multidatamodel": _builder(),
191+
"predictor_async": _builder("async deploy"),
192+
"serverless": _builder("serverless deploy"),
193+
"async_inference": _builder("async deploy"),
194+
"jumpstart": _builder("JumpStart models"),
195+
# sagemaker-core resources
196+
"predictor": _core_resource("Endpoint"),
197+
"base_predictor": _core_resource("Endpoint"),
198+
"transformer": _core_resource("TransformJob"),
199+
"tuner": _core_resource("HyperParameterTuningJob"),
200+
"processing": _core_resource("ProcessingJob"),
201+
"clarify": _core_resource("ProcessingJob"),
202+
"wrangler": _core_resource("ProcessingJob"),
203+
"spark": _core_resource("ProcessingJob"),
204+
"model_monitor": _core_resource("MonitoringSchedule"),
205+
"feature_store": _core_resource("FeatureGroup"),
206+
"experiments": _core_resource("Experiment"),
207+
"model_card": _core_resource("ModelCard"),
208+
"automl": _core_resource("AutoMLJob"),
209+
# Relocated utility modules under sagemaker.core
210+
"image_uris": _core_module("image_uris"),
211+
"s3": _core_module("s3"),
212+
"serializers": _core_module("serializers"),
213+
"deserializers": _core_module("deserializers"),
214+
# Pipelines / workflow -> sagemaker.mlops
215+
"pipeline": (
216+
"`Pipeline`",
217+
"from sagemaker.mlops.workflow.pipeline import Pipeline",
218+
"sagemaker.mlops.workflow.pipeline",
219+
),
220+
"workflow": (
221+
"`Pipeline`",
222+
"from sagemaker.mlops.workflow.pipeline import Pipeline",
223+
"sagemaker.mlops.workflow.pipeline",
224+
),
225+
# Training inputs -> configs; Session helper
226+
"inputs": (
227+
"`InputData`",
228+
"from sagemaker.train.configs import InputData",
229+
"sagemaker.train.configs",
230+
),
231+
"session": (
232+
"`Session`",
233+
"from sagemaker.core.helper.session_helper import Session",
234+
"sagemaker.core.helper.session_helper",
235+
),
236+
}
237+
238+
239+
def raise_removed_in_v3(module, replacement=None, v3_import=None, v3_docs=None):
240+
"""Warn and then raise an actionable error for a v2 module removed in v3.
241+
242+
The v2 SDK exposed top-level modules (e.g. ``sagemaker.estimator``) that no
243+
longer exist in v3. Importing one would otherwise fail with a bare
244+
``ModuleNotFoundError: No module named 'sagemaker.estimator'`` that gives the
245+
caller no path forward. This helper is called from ``_RemovedV2ModuleFinder``
246+
for removed names: it emits a ``DeprecationWarning`` and then raises a
247+
``ModuleNotFoundError`` whose message names the exact v3 replacement, the
248+
import to copy-paste, and a direct link to that replacement's API docs (plus
249+
the migration guide).
250+
251+
Args:
252+
module (str): The removed v2 module path, e.g. ``"sagemaker.estimator"``.
253+
replacement (str): Human readable v3 replacement, e.g. ``"ModelTrainer"``.
254+
Optional.
255+
v3_import (str): The exact v3 import statement, e.g.
256+
``"from sagemaker.train import ModelTrainer"``. Quoted verbatim so the
257+
caller can copy-paste it. Optional.
258+
v3_docs (str): Direct URL to the v3 replacement's API documentation, e.g.
259+
the generated ``sagemaker.train.model_trainer`` page. Optional.
260+
261+
Raises:
262+
ModuleNotFoundError: always, after emitting the deprecation warning.
263+
"""
264+
msg = f"`{module}` was removed in the SageMaker Python SDK v3."
265+
if replacement:
266+
msg += f" Use {replacement}."
267+
if v3_import:
268+
msg += f" ({v3_import})"
269+
if v3_docs:
270+
msg += f"\nDocs: {v3_docs}"
271+
msg += f"\nSee {V3_MIGRATION_URL} for the migration guide."
272+
273+
warnings.warn(msg, DeprecationWarning, stacklevel=2)
274+
# The raised ModuleNotFoundError below is the loud, authoritative signal
275+
# (it stops execution and carries the full message). Log at debug only, to
276+
# leave a breadcrumb for log-captured environments without duplicating the
277+
# message at WARNING level.
278+
logger.debug(msg)
279+
raise ModuleNotFoundError(msg, name=module)
280+
281+
282+
class _RemovedV2ModuleFinder(importlib.abc.MetaPathFinder):
283+
"""Meta-path finder that gives actionable guidance for removed v2 modules.
284+
285+
A single hook handles all removed top-level ``sagemaker.<name>`` modules:
286+
- names in ``_V3_REPLACEMENTS`` get a **precise** message (exact v3 class,
287+
copy-pasteable import, and API-docs link),
288+
- other names in ``_REMOVED_V2_MODULES`` get a **generic** "was removed"
289+
message, and
290+
- any other name (typo, hallucinated import) falls through to Python's
291+
plain ``ModuleNotFoundError`` -- we never claim something "was removed"
292+
when it never existed.
293+
294+
It is registered by *appending* to ``sys.meta_path``, so it only runs after
295+
the normal import machinery fails to locate the module. That ordering
296+
guarantees it never shadows a real module: if v3 ever ships a top-level
297+
module whose name matches a removed v2 one, the real module resolves first
298+
and this finder is never consulted for it.
299+
"""
300+
301+
def find_spec(self, fullname, path=None, target=None):
302+
"""Emit guidance only for known removed v2 top-level ``sagemaker`` modules."""
303+
if not fullname.startswith("sagemaker."):
304+
return None
305+
leaf = fullname[len("sagemaker.") :]
306+
# Only guard top-level names; never touch real v3 subpackages.
307+
if "." in leaf or leaf in _KNOWN_V3_TOPLEVEL:
308+
return None
309+
# Only guard names that were actually v2 modules. Unknown names (typos,
310+
# hallucinated imports) fall through to a plain ModuleNotFoundError so we
311+
# never claim something "was removed" when it never existed.
312+
if leaf not in _REMOVED_V2_MODULES:
313+
return None
314+
315+
if leaf in _V3_REPLACEMENTS:
316+
# Curated, high-traffic module -> precise guidance.
317+
replacement, v3_import, docs_module = _V3_REPLACEMENTS[leaf]
318+
raise_removed_in_v3(
319+
module=fullname,
320+
replacement=replacement,
321+
v3_import=v3_import,
322+
v3_docs=f"{_DOCS_BASE}{docs_module}.html",
323+
)
324+
325+
# Other removed v2 module -> generic guidance.
326+
msg = (
327+
f"`{fullname}` was removed in the SageMaker Python SDK v3. "
328+
"It may have moved to a new location."
329+
f"\nSee {V3_MIGRATION_URL} for the migration guide."
330+
)
331+
warnings.warn(msg, DeprecationWarning, stacklevel=2)
332+
# See raise_removed_in_v3: the raised error is the loud signal; log at
333+
# debug to avoid duplicating the message at WARNING level.
334+
logger.debug(msg)
335+
raise ModuleNotFoundError(msg, name=fullname)
336+
337+
338+
def register_removed_module_finder():
339+
"""Install the fallback finder for removed v2 modules (idempotent).
340+
341+
Appends a single ``_RemovedV2ModuleFinder`` to ``sys.meta_path`` so it acts
342+
as a last resort. Safe to call multiple times -- it installs at most one
343+
instance per process.
344+
"""
345+
if any(isinstance(f, _RemovedV2ModuleFinder) for f in sys.meta_path):
346+
return
347+
sys.meta_path.append(_RemovedV2ModuleFinder())
348+
logger.debug("Registered SageMaker v2 removed-module guidance finder on sys.meta_path.")
349+
23350

24351
def _warn(msg, sdk_version=None):
25352
"""Generic warning raiser referencing V2

0 commit comments

Comments
 (0)