Skip to content

Commit 323ffb6

Browse files
committed
Merge origin/master into fix/codestyle-doc-tests-ci
Resolves conflicts with aws#6293, aws#6281, aws#6272, aws#6290 and aws#6204. Every conflict was upstream adding code next to lines this branch had reformatted, so each hunk takes the upstream side and the file is then re-run through black and the ruff F401/F541/W605 pass, which is how the rest of the branch was produced. Two upstream test files needed the same treatment this branch applied elsewhere: sagemaker-train/tests/unit/ai_registry/test_dataset.py and test_evaluator_domain_id.py assign DataSet.create()/Evaluator.create() results that are never read (flake8 F841); those assignments are dropped. test_evaluator_domain_id.py now uses @pytest.fixture upstream, so the `import pytest` this branch had removed as unused is back. Verified on the merged tree with the CI commands: black --check, flake8 and pydocstyle clean in all four submodules; pylint 9.92 / 9.91 / 9.92 for core / train / serve (mlops untouched by upstream).
2 parents 9024f9d + 3094164 commit 323ffb6

28 files changed

Lines changed: 1026 additions & 43 deletions

‎CONTRIBUTING.md‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ information to effectively respond to your bug report or contribution.
1616
* [Run the Unit Tests](#run-the-unit-tests)
1717
* [Run the Integration Tests](#run-the-integration-tests)
1818
* [Make and Test Your Change](#make-and-test-your-change)
19+
* [Instrument New Features with Telemetry](#instrument-new-features-with-telemetry)
1920
* [Lint Your Change](#lint-your-change)
2021
* [Commit Your Change](#commit-your-change)
2122
* [Send a Pull Request](#send-a-pull-request)
@@ -118,6 +119,27 @@ If you are writing or modifying a test that creates a SageMaker job (training, t
118119
1. If your changes include documentation changes, please see the [Documentation Guidelines](#documentation-guidelines).
119120
1. If you include integration tests, do not mark them as canaries if they will not run in all regions.
120121

122+
### Instrument New Features with Telemetry
123+
124+
When you add or change a **public** API in one of the V3 modules (`sagemaker-core`,
125+
`sagemaker-train`, `sagemaker-serve`, `sagemaker-mlops`), instrument its public entry points
126+
with usage telemetry using the SDK's standard `@_telemetry_emitter` decorator from
127+
`sagemaker.core.telemetry`. Consistent telemetry helps us understand feature usage and
128+
prioritize investment, and applying the decorator does not change your feature's behavior.
129+
130+
To make this repeatable, the repository includes an **Agent SOP** — a step-by-step procedure
131+
that an AI coding agent (or a person) can follow — at
132+
[`agent-sops/add-feature-telemetry.sop.md`](./agent-sops/add-feature-telemetry.sop.md). It:
133+
134+
1. Locates the public entry points in a target file.
135+
1. Resolves the appropriate `Feature` enum value, creating a new one if none fits.
136+
1. Applies the `@_telemetry_emitter(Feature.<X>, "<name>")` decorator.
137+
1. Adds matching unit tests and verifies the module builds — without altering behavior.
138+
139+
To use it, point your agent at the SOP and provide its parameters (at minimum `target_path`
140+
and `feature_name`); the SOP documents the rest. The SOP is a guide you invoke on demand — it
141+
does **not** run automatically on pull requests — so you can equally follow its steps by hand.
142+
121143
### Lint Your Change
122144

123145
Before submitting, ensure your code meets our quality and style guidelines. Run:
Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
# Add Telemetry to a SageMaker Python SDK Feature
2+
3+
## Overview
4+
5+
This SOP instruments a SageMaker Python SDK V3 feature with usage telemetry using the
6+
SDK's standard `@_telemetry_emitter` decorator from `sagemaker.core.telemetry`. It is
7+
intended for partner teams who contribute features and need a clear, consistent,
8+
low-overhead way to add telemetry to their own public APIs.
9+
10+
Use this SOP when a new (or existing) public method or module-level function in one of the
11+
V3 modules (`sagemaker-core`, `sagemaker-train`, `sagemaker-serve`, `sagemaker-mlops`)
12+
should report usage. The outcome is: the feature's public entry points carry the
13+
`@_telemetry_emitter(Feature.<X>, "<name>")` decorator, a valid `Feature` is used (creating
14+
one if none fits), matching unit tests exist, and the module builds green — without changing
15+
the feature's behavior.
16+
17+
Do NOT use this SOP to design new telemetry infrastructure or to emit metrics to a custom
18+
backend; it only applies the existing SDK telemetry contract to feature code.
19+
20+
## Parameters
21+
22+
- **target_path** (required): Repo-relative path to the feature's Python file or package
23+
directory to instrument (e.g., `sagemaker-train/src/sagemaker/train/ai_registry/dataset.py`).
24+
- **feature_name** (required): The telemetry feature to attribute usage to. Either the name
25+
of an existing `Feature` enum member (e.g., `MODEL_CUSTOMIZATION`, `FEATURE_STORE`,
26+
`MODEL_TRAINER`) or a new `UPPER_SNAKE_CASE` name to create if no existing member fits.
27+
- **entry_points** (optional): Explicit list of public entry points to instrument, each as
28+
`Class.method` or `module.function`. If omitted, the SOP detects the public entry points
29+
in `target_path`.
30+
- **telemetry_params** (optional): Granular parameters to capture per entry point, each as
31+
a `(name, type)` pair where `type` is one of `ATTR_VALUE`, `ATTR_EXISTS`, `ATTR_CALL`,
32+
`KWARG_VALUE`, `KWARG_EXISTS` (from `TelemetryParamType`).
33+
- **workspace_root** (optional, default: current directory): Absolute path to the
34+
`sagemaker-python-sdk` repository root that contains the `sagemaker-core` module.
35+
36+
**Constraints for parameter acquisition:**
37+
- If all required parameters are already provided, You MUST proceed to the Steps
38+
- If any required parameters are missing, You MUST ask for them before proceeding
39+
- When asking for parameters, You MUST request all parameters in a single prompt
40+
- When asking for parameters, You MUST use the exact parameter names as defined
41+
42+
## Steps
43+
44+
### 1. Locate the feature and its public entry points
45+
Open `target_path` under `workspace_root` and determine the set of public entry points to
46+
instrument. A public entry point is a user-facing method or module-level function that a
47+
customer calls directly.
48+
49+
**Constraints:**
50+
- You MUST resolve `target_path` relative to `workspace_root` and read the file(s) before deciding anything, because instrumenting the wrong functions produces misleading telemetry.
51+
- If `entry_points` is provided, You MUST instrument exactly those and no others.
52+
- If `entry_points` is not provided, You MUST select only public entry points and You MUST NOT select names beginning with an underscore (private/internal helpers) because decorating internal helpers inflates and double-counts usage when public methods call them.
53+
- You MUST exclude `__init__`, simple property getters/setters, and trivial pass-through wrappers, because they do not represent a feature invocation.
54+
- You MUST produce an explicit list of fully-qualified targets in the form `Class.method` or `module.function` and confirm it is non-empty before advancing.
55+
56+
### 2. Resolve the Feature enum value
57+
Decide which `Feature` enum member attributes this usage. Use an existing member when one
58+
fits; otherwise create a new one.
59+
60+
**Constraints:**
61+
- You MUST read `sagemaker-core/src/sagemaker/core/telemetry/constants.py` (the `Feature` enum) and the `FEATURE_TO_CODE` dictionary in `sagemaker-core/src/sagemaker/core/telemetry/telemetry_logging.py` before deciding.
62+
- If `feature_name` matches an existing `Feature` member, You MUST reuse it and You MUST NOT create a duplicate, because duplicate members fragment the usage data for one feature across two codes.
63+
- If `feature_name` does not match an existing member, You MUST add it in TWO places using the same next unused integer: (a) the `Feature` enum in `constants.py`, and (b) the `FEATURE_TO_CODE` dictionary in `telemetry_logging.py`.
64+
- You MUST NOT add the new member to only one of those two locations, because the emitter looks up `FEATURE_TO_CODE[str(feature)]` and a missing entry raises `KeyError` that the non-blocking wrapper swallows, so telemetry silently never emits.
65+
- You MUST NOT reuse or renumber an existing integer code, because codes are a stable contract consumed by downstream telemetry analysis.
66+
- Validation: You MUST confirm the chosen `Feature` member exists in both the enum and `FEATURE_TO_CODE` (for a new feature, that both edits are present with matching integers) before advancing.
67+
68+
### 3. Apply the `@_telemetry_emitter` decorator to each entry point
69+
Add the required import(s) and decorate every target from Step 1.
70+
71+
**Constraints:**
72+
- You MUST import the decorator with `from sagemaker.core.telemetry import Feature, _telemetry_emitter`.
73+
- If `telemetry_params` is provided, You MUST additionally import `from sagemaker.core.telemetry.telemetry_logging import TelemetryParamType`.
74+
- You MUST decorate each target as `@_telemetry_emitter(Feature.<FEATURE_NAME>, "<func_name>")`, where `<func_name>` is `"<Class>.<method>"` for methods and `"<function>"` or `"<module>.<function>"` for module-level functions, matching the existing convention in `sagemaker-mlops/src/sagemaker/mlops/feature_store/` and `sagemaker-train/src/sagemaker/ai_registry/`.
75+
- If `telemetry_params` is provided, You MUST pass it as the `telemetry_params=` keyword argument using the `(name, TelemetryParamType.<TYPE>)` tuple form.
76+
- You MUST place the `@_telemetry_emitter` decorator as the outermost decorator unless another decorator must wrap it, because ordering changes which call is measured.
77+
- You MUST NOT change the decorated function's signature, arguments, return value, or logic, because this SOP adds observability only and must not alter feature behavior.
78+
- You MUST NOT add any code that can raise outside the decorator, and You MUST NOT catch or re-raise telemetry errors yourself, because the emitter already guarantees telemetry never blocks the SDK call and must return the underlying result unchanged.
79+
- You MUST NOT decorate the same function more than once, because duplicate emitters double-count usage.
80+
- Validation: You MUST confirm every target now carries exactly one `@_telemetry_emitter` decorator and the imports resolve.
81+
82+
### 4. Add or extend unit tests
83+
Create or update unit tests that assert the telemetry decorator is applied and, when
84+
`telemetry_params` is used, that the expected parameters are emitted.
85+
86+
**Constraints:**
87+
- You MUST mirror the patterns in `sagemaker-core/tests/unit/telemetry/test_granular_telemetry.py` (use `unittest.mock` `Mock`/`patch`, assert on the emitted `&x-...` query parameters and the `Feature` used).
88+
- You MUST place tests in the same module's `tests/unit/` telemetry test location as the code you changed.
89+
- If you created a new `Feature` in Step 2, You MUST add an assertion that the new member is present in both the `Feature` enum and `FEATURE_TO_CODE`, because that mapping is the silent-failure point.
90+
- You MUST NOT weaken or delete existing telemetry assertions to make new tests pass, because that erases regression coverage.
91+
- Validation: the new/updated tests MUST exist and MUST be runnable by the module's test runner.
92+
93+
### 5. Build and verify the module
94+
Run the affected module's build and tests to confirm the change compiles, the tests pass,
95+
and telemetry does not block the instrumented calls.
96+
97+
**Constraints:**
98+
- You MUST run the module's unit tests (for `sagemaker-core`, `python -m pytest sagemaker-core/tests/unit/telemetry/`; for a partner module, the equivalent unit-test target) and You MUST NOT proceed while any telemetry-related test fails, because shipping unverified telemetry can emit incorrect usage data.
99+
- You MUST confirm the instrumented entry points still return their original results when invoked in the tests, because a broken non-blocking guarantee would surface as an altered or missing return value.
100+
- If verification fails, You MUST diagnose and fix the root cause and re-run, and You MUST NOT mark the SOP complete with failing tests, because a green telemetry test suite is the completion criterion for this SOP.
101+
- Validation: the build/test run MUST complete with the telemetry tests green.
102+
103+
### 6. Summarize the changes
104+
Produce a concise summary of what was instrumented for the requesting partner team.
105+
106+
**Constraints:**
107+
- You MUST list each instrumented entry point with its `Feature` and `func_name`, whether a new `Feature` was created (with its integer code), any `telemetry_params` added, and the files changed.
108+
- You MUST state the test command that was run and its result.
109+
- You MUST NOT run `git commit`, `git push`, or create a code review unless the user explicitly asks, because publishing changes is the author's decision.
110+
111+
## Examples
112+
113+
### Example 1: Instrument an existing feature with an existing Feature
114+
**Input:**
115+
- target_path: `sagemaker-mlops/src/sagemaker/mlops/feature_store/feature_utils.py`
116+
- feature_name: `FEATURE_STORE`
117+
118+
**Expected Behavior:**
119+
The agent detects the public functions (e.g., `ingest_dataframe`, `get_feature_group_as_dataframe`),
120+
decorates each with `@_telemetry_emitter(Feature.FEATURE_STORE, "<function>")`, reuses the
121+
existing `FEATURE_STORE` member, adds/extends telemetry unit tests, runs them green, and
122+
summarizes the changes.
123+
124+
### Example 2: New feature with granular parameters
125+
**Input:**
126+
- target_path: `sagemaker-train/src/sagemaker/train/my_feature/client.py`
127+
- feature_name: `MY_FEATURE`
128+
- entry_points: ["MyClient.run", "MyClient.submit"]
129+
- telemetry_params: [("instance_type", "KWARG_VALUE"), ("networking", "ATTR_EXISTS")]
130+
131+
**Expected Behavior:**
132+
The agent adds `Feature.MY_FEATURE = 22` to `constants.py` and `str(Feature.MY_FEATURE): 22`
133+
to `FEATURE_TO_CODE`, decorates `MyClient.run` and `MyClient.submit` with the granular
134+
`telemetry_params`, adds unit tests asserting `x-instanceType` and `x-hasNetworking` are
135+
emitted plus the new-member/mapping assertions, runs the tests green, and summarizes.
136+
137+
## Troubleshooting
138+
139+
### Telemetry never emits at runtime
140+
If the decorator is applied but no telemetry appears:
141+
- Confirm the `Feature` member exists in BOTH the `Feature` enum and `FEATURE_TO_CODE` with the same integer; a missing `FEATURE_TO_CODE` entry raises a swallowed `KeyError`.
142+
- Confirm a `sagemaker_session` is resolvable in the call path, because the emitter only fires when a session is available.
143+
144+
### ImportError on `_telemetry_emitter` or `TelemetryParamType`
145+
- Import `Feature` and `_telemetry_emitter` from `sagemaker.core.telemetry`.
146+
- Import `TelemetryParamType` from `sagemaker.core.telemetry.telemetry_logging` (it is not re-exported from the package root).
147+
148+
### Tests fail with an unexpected return value
149+
- Verify no logic was changed and the `@_telemetry_emitter` decorator is the only addition; the emitter must return the wrapped function's result unchanged.
150+
151+
### Cannot determine which functions to instrument
152+
- Ask the partner team for the `entry_points`, or instrument only the documented public API surface and exclude underscore-prefixed helpers.

‎sagemaker-core/src/sagemaker/core/common_utils.py‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2203,19 +2203,24 @@ def camel_to_snake(camel_case_string: str) -> str:
22032203

22042204

22052205
def walk_and_apply_json(
2206-
json_obj: Dict[Any, Any], apply, stop_keys: Optional[List[str]] = ["metrics"]
2206+
json_obj: Dict[Any, Any],
2207+
apply,
2208+
stop_keys: Optional[List[str]] = ["metrics", "environment_variables"],
22072209
) -> Dict[Any, Any]:
22082210
"""Recursively walks a json object and applies a given function to the keys.
22092211
22102212
stop_keys (Optional[list[str]]): List of field keys that should stop the application function.
22112213
Any children of these keys will not have the application function applied to them.
2214+
A key stops the walk if either its original or its converted form is in stop_keys, so
2215+
the same list works for camel_to_snake and snake_to_upper_camel passes. Environment
2216+
variable names are user facing values stored as keys and must never be converted.
22122217
"""
22132218

22142219
def _walk_and_apply_json(json_obj, new):
22152220
if isinstance(json_obj, dict) and isinstance(new, dict):
22162221
for key, value in json_obj.items():
22172222
new_key = apply(key)
2218-
if (stop_keys and new_key not in stop_keys) or stop_keys is None:
2223+
if stop_keys is None or (key not in stop_keys and new_key not in stop_keys):
22192224
if isinstance(value, dict):
22202225
new[new_key] = {}
22212226
_walk_and_apply_json(value, new=new[new_key])

‎sagemaker-core/src/sagemaker/core/image_uri_config/llama-cpp-arm64.json‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,13 @@
77
},
88
"versions": {
99
"1": {
10+
"container_version": {
11+
"cpu": "cpu-v1"
12+
},
13+
"processor_in_tag": false,
14+
"processors": [
15+
"cpu"
16+
],
1017
"registries": {
1118
"af-south-1": "626614931356",
1219
"ap-east-1": "871362719292",
@@ -48,9 +55,16 @@
4855
"us-west-2": "763104351884"
4956
},
5057
"repository": "llama-cpp-arm64",
51-
"tag_prefix": "server-sagemaker-cpu-v1"
58+
"tag_prefix": "server-sagemaker"
5259
},
5360
"1.0": {
61+
"container_version": {
62+
"cpu": "cpu-v1.0"
63+
},
64+
"processor_in_tag": false,
65+
"processors": [
66+
"cpu"
67+
],
5468
"registries": {
5569
"af-south-1": "626614931356",
5670
"ap-east-1": "871362719292",
@@ -92,7 +106,7 @@
92106
"us-west-2": "763104351884"
93107
},
94108
"repository": "llama-cpp-arm64",
95-
"tag_prefix": "server-sagemaker-cpu-v1.0"
109+
"tag_prefix": "server-sagemaker"
96110
}
97111
}
98112
}

‎sagemaker-core/src/sagemaker/core/jumpstart/hub/parser_utils.py‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,19 +34,24 @@ def snake_to_upper_camel(snake_case_string: str) -> str:
3434

3535

3636
def walk_and_apply_json(
37-
json_obj: Dict[Any, Any], apply, stop_keys: Optional[List[str]] = ["metrics"]
37+
json_obj: Dict[Any, Any],
38+
apply,
39+
stop_keys: Optional[List[str]] = ["metrics", "environment_variables"],
3840
) -> Dict[Any, Any]:
3941
"""Recursively walks a json object and applies a given function to the keys.
4042
4143
stop_keys (Optional[list[str]]): List of field keys that should stop the application function.
4244
Any children of these keys will not have the application function applied to them.
45+
A key stops the walk if either its original or its converted form is in stop_keys, so
46+
the same list works for camel_to_snake and snake_to_upper_camel passes. Environment
47+
variable names are user facing values stored as keys and must never be converted.
4348
"""
4449

4550
def _walk_and_apply_json(json_obj, new):
4651
if isinstance(json_obj, dict) and isinstance(new, dict):
4752
for key, value in json_obj.items():
4853
new_key = apply(key)
49-
if (stop_keys and new_key not in stop_keys) or stop_keys is None:
54+
if stop_keys is None or (key not in stop_keys and new_key not in stop_keys):
5055
if isinstance(value, dict):
5156
new[new_key] = {}
5257
_walk_and_apply_json(value, new=new[new_key])

0 commit comments

Comments
 (0)