Repository navigation
chore(core): sync the bundled SageMaker service model with botocore - #6453
Conversation
9501922 to
6f21b06
Compare
There was a problem hiding this comment.
🤖 AI code review
This PR is a whole-model botocore sync of sagemaker-core: it refreshes service-2.json, resource_plan.csv, and the generated resources.py, shapes.py, config_schema.py, and shape_dag.py. Most of the diff is faithfully regenerated and additive (new shapes defined before use, new MLflow/IdC/observability/HyperPod fields, NotebookInstance maintenance states, etc.), consistent with prior sync PRs. However, the newly generated TrainingSession resource is functionally broken because its identifier (TrainingSessionName) lives inside the nested TrainingSessionDetail describe output rather than at the top level, so the generated accessor code is inconsistent and will raise at runtime.
Reviewed commit 6f21b06. Automated review; verify before acting.
| training_session_detail: The detailed information about the training session. | ||
|
|
||
| """ | ||
|
|
There was a problem hiding this comment.
🔴 High: The generated TrainingSession resource is not usable as emitted. DescribeTrainingSessionResponse wraps everything in TrainingSessionDetail, so the only class attribute is training_session_detail — there is no training_session_name or training_session_status attribute on the resource. As a result:
stop()builds{"TrainingSessionName": self.training_session_name}, butself.training_session_namedoes not exist on the model (it lives underself.training_session_detail.training_session_name), so callingstop()raisesAttributeError.refresh(self, training_session_name)anddelete(self, training_session_name)take the identifier as a required positional argument, butwait_for_status()andwait_for_delete()callself.refresh()with no arguments (as every other resource's wait methods do), which raisesTypeError: refresh() missing 1 required positional argument.get_name()looks for atraining_session_nameattribute and will fail to find it, logging an error and returningNone.
This is the generator's handling of a resource whose identifier is nested in a detail object; the fix likely belongs in codegen (e.g. a REFRESH_IDENTIFIER_FALLBACKS-style mapping or chained-attribute resolution to pull training_session_name from training_session_detail) rather than hand-editing the output. Please verify the regenerated TrainingSession resource against the new model before relying on it; as generated it cannot complete a stop/wait/refresh cycle.
6f21b06 to
06227b8
Compare
Refreshes sagemaker-core's codegen input from the public botocore model and regenerates the layers built from it. The model file is the upstream bytes verbatim; the generated layers are the output of ``python -m sagemaker.core.tools.codegen``. Model: 403 -> 409 operations, 3005 -> 3084 shapes. What the refresh brings in -------------------------- * Six new operations: AttachClusterNodeNetworkInterface, and the five training-session operations (Create, Describe, List, Stop, Delete). * 79 new shapes, of which 19 belong to the training-session operations and 60 to other parts of the API. * 66 existing shapes whose definitions have moved upstream, spanning AIBenchmark, AIRecommendation, App and Cluster among others. * NotebookInstance gains the PendingMaintenance and InMaintenance states. * MlflowApp gains kms_key_id in the configuration schema. * A TrainingSession resource, derived from the standard CRUD naming: create / get / get_all, and delete / refresh / stop / wait_for_delete / wait_for_status. No caller is switched over to any newly generated resource here. This change only brings the generated layer back in line with the model. One codegen change, for a resource whose identifier is nested ------------------------------------------------------------- DescribeTrainingSession returns everything inside a single TrainingSessionDetail member, so TrainingSessionName is not hoisted onto the resource class. Without the identifier as a class attribute, stop() raised AttributeError, refresh() and delete() took it as a required positional while wait_for_status() and wait_for_delete() called refresh() with none, and get_name() returned None. resources_codegen.py already keeps a list of resources whose identifier is absent from their describe output (ImageVersion, Workteam, Workforce, SubscribedWorkteam, HubContent) and injects it as a class attribute for exactly this reason. TrainingSession is added to that list. The nested value stays reachable at training_session_detail.training_session_name. The general fix would be to unwrap a single-member describe response when collecting class attributes, the way resources_extractor already does when building status chains. That touches generation for every resource and is left alone here. Notes for anyone regenerating ----------------------------- Two things about the generator are worth knowing, since both produce a diff that looks plausible but is wrong: * ``codegen`` needs two passes to converge. ``generate_resources()`` reads the existing config_schema.py to decide which resources get a ``populate_inputs_decorator``, and the schema is regenerated after it, so a single pass emits new resources without one. Those resources then silently ignore sagemaker_config defaults. A third pass is a no-op, so two is enough. * ``resources_extractor`` writes resource_plan.csv to the current working directory, so running the unit tests from sagemaker-core/ overwrites the tracked file with a partial plan. Regenerate before diffing, not after running tests. Separately, codegen does not emit the module docstrings at the top of config_schema.py or shape_dag.py. Neither file is excluded from flake8 or docstyle, so dropping them fails the codestyle gate. Both are preserved here.
06227b8 to
837c52c
Compare
There was a problem hiding this comment.
🤖 AI code review
This PR syncs sagemaker-core's bundled SageMaker service model with upstream botocore and regenerates the layers built from it (service-2.json, resource_plan.csv, resources.py, shapes.py, shape_dag.py, config_schema.py), plus one small hand-written change to the generator (resources_codegen.py) adding TrainingSession to the list of resources whose identifier must be injected as a class attribute.
I spot-checked the new TrainingSession resource end to end: the injected training_session_name identifier is used consistently by create/get/refresh/delete/stop, wait_for_status/wait_for_delete read the nested training_session_detail.training_session_status, and all referenced new shapes (TrainingSessionDetail, LoraConfig, TrainingSessionModelConfig/OutputConfig, BillableTokenUsage, etc.) are defined in shapes.py and shape_dag.py. The resources_codegen.py change follows the existing ImageVersion/Workteam/HubContent pattern. Additive parameters (included_data, kms_key_id, idc_config, training_plan_arns, new wait_for_status literals) are backward-compatible, and I saw no required-positional reordering of the kind flagged in prior syncs. No concrete issues found; consistent with precedent that pure codegen syncs need no additional unit tests (Source: #6285).
Reviewed commit 837c52c. Automated review; verify before acting.
Issue #, if available: N/A
Description of changes:
Refreshes
sagemaker-core's codegen input from the public botocore model andregenerates the layers built from it. The model file is the upstream bytes
verbatim, and the generated layers are the output of
python -m sagemaker.core.tools.codegen.What the refresh brings in
This is a whole-model sync, so the diff is not limited to any one feature.
Operations (6 new)
AttachClusterNodeNetworkInterfaceCreateTrainingSession,DescribeTrainingSession,ListTrainingSessions,StopTrainingSession,DeleteTrainingSessionShapes (79 new, 66 changed)
are elsewhere in the API.
AIBenchmark*,AIRecommendation*,App*andCluster*, among others.NotebookInstancegains thePendingMaintenanceandInMaintenancestates.Resources and configuration schema
TrainingSessionresource, derived from the standard CRUD naming:create/get/get_all, anddelete/refresh/stop/wait_for_delete/wait_for_status.MlflowAppgainskms_key_idin the configuration schema.No caller is switched over to any newly generated resource in this PR. This
change only brings the generated layer back in line with the model.
One codegen change, for a resource whose identifier is nested
DescribeTrainingSessionreturns everything inside a singleTrainingSessionDetailmember, soTrainingSessionNameis not hoisted onto theresource class. Without the identifier as a class attribute the generated
resource could not complete a lifecycle:
stop()raisedAttributeError,refresh()anddelete()took the identifier as a required positional whilewait_for_status()/wait_for_delete()calledrefresh()with none, andget_name()returnedNone.resources_codegen.pyalready keeps a list of resources whose identifier isabsent from their describe output —
ImageVersion,Workteam,Workforce,SubscribedWorkteam,HubContent— and injects it as a class attribute forexactly this reason.
TrainingSessionis added to that list. The nested valuestays reachable at
training_session_detail.training_session_name.After the change, with the boto client stubbed:
The more general fix would be to unwrap a single-member describe response when
collecting class attributes, the way
resources_extractoralready does whenbuilding status chains. That changes generation for every resource, so it is
left alone here and worth considering separately.
Notes for anyone regenerating
Two properties of the generator are worth writing down, because both produce a
diff that looks plausible but is wrong:
codegenneeds two passes to converge.generate_resources()reads theexisting
config_schema.pyto decide which resources get apopulate_inputs_decorator, and the schema is regenerated after it. Asingle pass therefore emits new resources without the decorator, and those
resources silently ignore
sagemaker_configdefaults. A third pass is ano-op, so two passes are enough. Both passes are reflected here.
resources_extractorwritesresource_plan.csvto the current workingdirectory. Running the unit tests from
sagemaker-core/overwrites thetracked file with a partial plan, so regenerate before diffing rather than
after running tests.
Separately, codegen does not emit the module docstrings at the top of
config_schema.pyorshape_dag.py. Neither file is excluded fromflake8ordocstyle, so dropping them fails the codestyle gate. Both are preserved here.Testing
sagemaker-coreunit tests on this branch and on unmodified master: both14 failed, 3922 passed, 41 skipped, with identical failing test ids, so theregeneration introduces no new failures. The pre-existing failures include
tests/unit/tools/test_api_coverage.py::test_api_coverage, which fails onmaster as well.
pydocstyleonsrc/sagemaker: 0 violations, matching master.blackat the pinned version: the only file flagged istests/unit/local/test_entities.py, flagged identically on master.Not included
No
VERSIONorCHANGELOG.mdchanges.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.