Repository navigation
Commit 60857c1
authored
fix: Make codestyle-doc-tests pass in all four submodules (#6294)
* fix: Restore lint configuration lost in the v3 monorepo split
The codestyle-doc-tests CI job runs flake8, pylint, pydocstyle, black
and doc8 in each of the four submodules, and every one of them has
failed since the v3 split because the lint configuration did not
survive the move:
- .pydocstylerc lived at the v2 repo root and was dropped, so pydocstyle
ran with its defaults (~6,000 findings in sagemaker-core alone).
Restored from master-v2 at the root, where every submodule's
``pydocstyle src/sagemaker`` picks it up.
- The generated API surface (sagemaker/core/resources.py, 35k lines, and
shapes/shapes.py, 15k lines) was never excluded from flake8, pylint or
pydocstyle and dominated the violation counts. It is now excluded in
all three. pylint's ignore-paths must use forward slashes and ``[.]``
because pylint rewrites backslashes to derive the Windows pattern.
- The pylint env installs nothing, so cross-submodule imports of the
shared ``sagemaker`` namespace package produced ~700
no-name-in-module errors. Every submodule's src/ is now on PYTHONPATH
in that env.
- The pylint disable list was missing a comma after W0719, which made
pylint read ``W0719 W1404`` as one token and silently drop both.
Also disable W0718, W1203 and R0801, which the v3 code base violates
by design (documented inline), and align max-line-length with the
flake8 setting of 120 since black formats at 100 but does not split
long strings or comments.
- sagemaker-core's ``[tool.black] exclude`` replaced black's default
exclusions, so the CI black-check env scanned its own .tox tree
(2,500+ third-party files). Switched to extend-exclude.
- FI11 (missing ``from __future__ import absolute_import``) is a no-op
on Python 3 and is now ignored rather than added to ~350 files.
- Every submodule's sphinx env used ``changedir = doc`` but only
sagemaker-core has a docs/ directory. Fixed the path for core; for
the other three the env is a documented no-op so the shared
``tox -e sphinx,doc8`` step succeeds.
* style: Format all submodules with black
Mechanical: `black ./` in each submodule with the pinned CI version
(26.3.1) and the existing line-length of 100. No code changes.
* style: Remove unused imports, f-string prefixes and invalid escapes
Mechanical ruff autofix for F401 (unused import), F541 (f-string
without placeholders), F811 (redefinition of an unused import) and
W605 (invalid escape sequence), followed by black. Re-exports in
__init__.py and fixture imports in conftest.py were left alone, as were
notebooks and the generated files.
* fix(core): Make flake8, pydocstyle and pylint pass in sagemaker-core
Behavior-preserving lint fixes so the codestyle-doc-tests job passes for
this submodule: flake8 0, pydocstyle 0, pylint 9.92 (gate 9.9).
Real defects the linters surfaced, fixed minimally:
- model_monitor/model_monitoring.py: run_baseline built BaseliningJob
from two undefined names; four describe_processing_job calls in
BaseliningJob and MonitoringExecution used an undefined
processing_job_name. Now use the actual locals / self attributes.
- lineage/artifact.py: List["Context"] forward reference had no import;
added under TYPE_CHECKING.
- utils/utils.py, training/configs.py, remote_function/job.py: duplicate
module-level definitions where the later one silently won; the dead
earlier copy is removed.
Docstrings were added for the public modules, classes and functions
pydocstyle flagged and reshaped to the D212/D205 layout the restored
.pydocstylerc expects. Duplicate test classes that shadowed an earlier
copy with the same name are renamed (...Part1/...Part2) so both are
collected; those earlier copies were never run before and may need
attention if they fail. undefined-all-variable is disabled around the
__all__ lists in experiments/__init__.py and utils/__init__.py, which
export lazily through PEP 562 __getattr__ that pylint cannot see.
* fix(train): Make flake8, pydocstyle and pylint pass in sagemaker-train
Behavior-preserving lint fixes so the codestyle-doc-tests job passes for
this submodule: flake8 0, pydocstyle 0, pylint 9.91 (gate 9.9).
Real defect the linters surfaced: remote_function/invoke_function.py
was a stale copy of the sagemaker-core entry point that still passed
``hmac_key=`` to StoredFunction (whose parameter is ``signing_key``)
and to handle_error (which takes no key). Both would raise TypeError
when the remote-function job actually ran; the unit test mocked
StoredFunction so it never surfaced. The file is now identical to the
core version apart from the namespace, and the test kwargs follow.
Also: evaluate/execution.py imported ``datetime`` inside a method that
already used the module-level ``datetime`` earlier in the same scope,
which made every earlier use an unbound local; the four
``from sagemaker.train import logger`` imports go through a PEP 562
__getattr__ that simply returns sagemaker.core.utils.utils.logger, so
they import it from there directly; redundant in-function re-imports,
a self-assignment, an unused import and ``elif``/``else`` after
``return``/``raise`` were removed.
* fix(serve): Make flake8, pydocstyle and pylint pass in sagemaker-serve
Behavior-preserving lint fixes so the codestyle-doc-tests job passes for
this submodule: flake8 0, pydocstyle 0, pylint 9.92 (gate 9.9).
Real defects the linters surfaced, fixed minimally:
- model_builder_utils.py: the JumpStart gated-bucket path called the
zero-argument accessor JumpStartModelsAccessor.get_jumpstart_content_bucket
with a region and would raise TypeError; it now calls the region-aware
jumpstart.utils.get_jumpstart_content_bucket already imported in the
module.
- Exceptions raised with a (format, arg) tuple instead of a formatted
message (``raise ValueError("... %s", x)``) now format the message.
Left in place with an inline NOTE for the owning team rather than
guessed at: utils/lineage_utils.py calls Artifact.create with the
legacy sagemaker.lineage keyword arguments, but Artifact now resolves to
the generated core class with a different signature; and several
``prepare_*`` helpers are annotated ``-> str`` but return None, so the
assignments of their result are suppressed as assignment-from-no-return.
The ModelBuilder classes assign most of their state in build()/deploy()
rather than __init__, so attribute-defined-outside-init is disabled on
those specific classes with a reason instead of declaring hundreds of
placeholder attributes.
* fix(mlops): Make flake8, pydocstyle and pylint pass in sagemaker-mlops
Behavior-preserving lint fixes so the codestyle-doc-tests job passes for
this submodule: flake8 0, pydocstyle 0, pylint 9.91 (gate 9.9).
Real defects the linters surfaced, fixed minimally:
- workflow/retry.py: ``(a is None) == b is None`` is a chained
comparison that Python reads as ``((a is None) == b) and (b is None)``
and is always false, so RetryPolicy.to_request never rejected a policy
with both or neither of max_attempts / expire_after_mins. The
parenthesised form now performs the intended exclusive-or check. Every
in-repo caller passes exactly one of the two.
- feature_processor/lineage/_feature_processor_lineage.py: a ValueError
was constructed but never raised, so the "exactly one output feature
group" check was a no-op.
- local/pipeline_entities.py: ``type(x) != y`` is now ``type(x) is not y``
(identical for type objects, and what E721 asks for).
* docs: Fix doc8 findings (trailing whitespace, missing final newlines)
Four findings from the doc8 step that runs after the lint envs in the
codestyle-doc-tests job.
* ci: Stop tox from building an sdist for the sphinx and doc8 envs
With the lint envs green, the codestyle-doc-tests job reached its second
command, ``tox -e sphinx,doc8``, for the first time and failed in all
four submodules before running either env:
error: option --formats not recognized
ERROR: FAIL could not package project
tox 3 (what the CodeBuild image provides) builds the project sdist with
``build_sdist(..., {"--global-option": ["--formats=gztar"]})`` whenever
a selected env installs the package. setuptools dropped support for that
option in 69.0, and ``[build-system] requires = ["setuptools>=64"]``
resolves to 84.0.0 today (verified: 68.2.2 builds, 69.5.1+ fails). The
five lint envs all set ``skip_install = true`` and were unaffected.
doc8 only reads .rst files, so it now skips the install in every
submodule. sagemaker-core's sphinx env still needs an importable package
for autodoc; it now installs the project through pip (``deps =
{toxinidir}``), the same PEP 517 path ``pip install -e .[test]`` already
uses successfully earlier in the job, instead of tox's sdist step.
* ci: Ignore the installed package's egg-info in doc8
The codestyle-doc-tests job runs ``pip install -e .[test]`` before tox,
which writes ``src/sagemaker_<pkg>.egg-info/SOURCES.txt`` (no trailing
newline) into the tree. doc8's ignore list still named the v2 package,
``src/sagemaker_utils.egg-info``, so in all four submodules the only
doc8 finding in CI was D005 on that generated file. Ignore
``src/*.egg-info`` instead.
* fix(mlops): Return a real bool from _is_collection_column; drop stale patch
Two unit-test failures introduced by the lint pass:
- feature_utils._is_collection_column is annotated ``-> bool`` but
returned ``Series.any()``, a numpy.bool_. The tests compared with
``== True``; changing them to ``is True`` (E712) exposed the mismatch.
Wrap the result in ``bool()`` so the function honours its annotation.
- test_feature_scheduler.test_to_pipeline patched
``_config_uploader.TrainingInput``, a name that module imported but
never used. Removing the unused import (F401) made the patch fail with
AttributeError. The patch never affected the test (the mock argument
was unused), so the decorator and its parameter are removed.
* fix(serve): Restore the import that test_import_deprecation_warning exercises
The ruff F401 pass removed
``from sagemaker.serve.serverless.serverless_inference_config import
ServerlessInferenceConfig`` from inside the test because the name is not
used, but that import is the action under test: it is what emits the
DeprecationWarning the assertions look for. Without it the test recorded
zero warnings. Restored with a noqa and a comment saying why it stays.
* test(core): Patch os.name in the Windows _HostingContainer.down test
test_hosting_container_down_windows was one of the duplicate test
definitions this branch un-shadowed, so it ran for the first time in CI
and failed with ``ValueError: Invalid PID``. It patched
``platform.system`` to return "Windows", but ``_HostingContainer.down``
branches on ``os.name != "nt"``, so on the Linux runner it still called
``kill_child_processes`` with a Mock pid. Patch ``os.name`` in the module
under test instead, which exercises the Windows branch the test describes.
* style: Black-format two test files merged from master
test_load_or_create.py (core) and test_pipeline_class.py (mlops) landed
on master via #6256 after this branch's black pass and were pulled in
unformatted by the merge commit, failing black-check in CI.
---
X-AI-Prompt: fix failing codestyle-doc-tests reruns on PR #6294
X-AI-Tool: Kiro1 parent 4bcca35 commit 60857c1
970 files changed
Lines changed: 25658 additions & 21161 deletions
File tree
- sagemaker-core
- docs
- src/sagemaker
- core
- apiutils
- clarify
- compute_resource_requirements
- config
- debugger
- deserializers
- experiments
- helper
- image_retriever
- inference_recommender
- interactive_apps
- jumpstart
- artifacts
- factory
- hub
- lineage
- local
- mlflow
- model_card
- model_monitor
- modules
- local_core
- train
- container_drivers
- common
- distributed_drivers
- scripts
- sm_recipes
- partner_app
- remote_function
- core
- runtime_environment
- s3
- serializers
- shapes
- spark
- telemetry
- tools
- model_card
- training_compiler
- training
- utilities
- utils
- code_injection
- workflow
- lineage
- tests
- integ
- helper
- image_retriever
- jumpstart
- remote_function
- helpers
- unit
- experiments
- generated
- helper
- image_uris
- interactive_apps
- jumpstart
- artifacts
- hub
- lineage
- local
- model_monitor
- modules
- local_core
- train
- container_drivers/distributed_drivers
- remote_function
- runtime_environment
- serializers
- session
- telemetry
- tools
- utils
- workflow
- sagemaker-mlops
- src/sagemaker
- mlops
- feature_store
- feature_processor
- lineage
- local
- workflow
- tests
- data/feature_store/feature_processor
- integ
- code
- pipeline
- pytorch_processing
- s3_source_dir_processing
- feature_store/feature_processor
- workflow
- unit
- local
- sagemaker/mlops/feature_store
- feature_processor
- lineage
- workflow
- sagemaker-serve
- src/sagemaker
- serve
- ai_inference_recommender
- async_inference
- builder
- compute_resource_requirements
- detector
- model_format/mlflow
- model_server
- djl_serving
- in_process_model_server
- multi_model_server
- smd
- tei
- tensorflow_serving
- tgi
- torchserve
- triton
- mode
- serverless
- spec
- utils
- validations
- tests
- integ
- unit
- async_inference
- batch_inference
- builder
- detector
- marshalling
- model_format
- model_server
- mode
- serverless
- servers
- spec
- test_ai_inference_recommender
- utils
- validations
- sagemaker-train
- src/sagemaker
- ai_registry
- train
- aws_batch
- common_utils
- container_drivers
- common
- distributed_drivers
- scripts
- evaluate
- local
- remote_function
- core
- runtime_environment
- rft
- adapters
- sm_recipes
- tests
- data
- local_script
- params_script
- integ
- ai_registry
- train
- aws_batch
- code
- shallow
- unit
- ai_registry
- train
- aws_batch
- common_utils
- container_drivers
- scripts
- evaluate
- local
- remote_function
- sm_recipes
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
28 | 38 | | |
29 | 39 | | |
30 | 40 | | |
| |||
109 | 119 | | |
110 | 120 | | |
111 | 121 | | |
112 | | - | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
113 | 125 | | |
114 | 126 | | |
| 127 | + | |
115 | 128 | | |
116 | 129 | | |
117 | 130 | | |
| |||
245 | 258 | | |
246 | 259 | | |
247 | 260 | | |
248 | | - | |
249 | | - | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
250 | 265 | | |
251 | 266 | | |
252 | 267 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
| 2 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
85 | | - | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
86 | 88 | | |
87 | 89 | | |
88 | 90 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
7 | 6 | | |
8 | 7 | | |
9 | 8 | | |
| |||
14 | 13 | | |
15 | 14 | | |
16 | 15 | | |
17 | | - | |
18 | | - | |
| 16 | + | |
| 17 | + | |
19 | 18 | | |
20 | 19 | | |
21 | 20 | | |
22 | 21 | | |
23 | | - | |
| 22 | + | |
24 | 23 | | |
25 | 24 | | |
26 | | - | |
| 25 | + | |
27 | 26 | | |
28 | 27 | | |
29 | | - | |
| 28 | + | |
30 | 29 | | |
31 | 30 | | |
32 | 31 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| |||
0 commit comments