Fix source_dir in FrameworkProcessor (#6047)

* fix(iam): scope repo-level ECR actions to prevent false deny in preflight validation

Split ecr_policy statements in training, serving, and hyperpod role types
so that only ecr:GetAuthorizationToken (account-level) remains under
Resource: "*". The repository-level actions (BatchGetImage,
GetDownloadUrlForLayer, BatchCheckLayerAvailability) are now scoped to
arn:aws:ecr:*:*:repository/*, which excludes them from
_get_smoke_test_actions. This prevents SimulatePrincipalPolicy from
returning implicitDeny for roles that correctly scope ECR permissions to
specific repo ARNs (least privilege), fixing the regression that blocked
deploys/training/pipelines for those customers.

* fix: support s3 uri in FrameworkProcessor

* add local dependencies to source_dir

* fix: make dependencies optional in FrameworkProcessor code-packaging helpers

_package_code and _pack_and_upload_code declared dependencies as a
required positional arg, but the unit tests (and any direct callers)
invoke them without it, causing:

    TypeError: _package_code() missing 1 required positional argument: 'dependencies'

Move dependencies to the end of both signatures with a None default so
it is optional, matching the public run() signature. Also apply black
formatting and remove unused imports/vars in test_processing.py.

* fix: mount install_requirements.py for S3 source_dir processing jobs

When source_dir is an S3 URI, /opt/ml/processing/input/code maps to the
user's (possibly read-only) S3 location, which does not contain the
managed install_requirements.py helper. The helper was uploaded to a
separate managed prefix that was never mounted into the container, yet
the generated runproc script hardcoded

    python3 /opt/ml/processing/input/code/install_requirements.py

so any S3 source_dir bundle containing a requirements.txt failed at
runtime with a missing-file error.

Mount the managed helper via a dedicated 'aux' ProcessingInput at
/opt/ml/processing/input/aux (never writing into the user's bucket) and
parameterize the generated scripts to reference install_requirements.py
from the correct directory: /input/code for a local source_dir
(unchanged) and /input/aux for an S3 source_dir.

Update test_pack_and_upload_code_with_s3_source_dir_creates_code_input to
assert the aux mount is created (it previously locked in the broken
layout) and add regression tests verifying the generated scripts point at
the mounted helper path.

Also use a context manager for the helper file read.

---------

Co-authored-by: Mohamed Zeidan <81834882+mohamedzeidan2021@users.noreply.github.com>
Co-authored-by: Mohamed Zeidan <zeidmo@amazon.com>

Author: 3 people

sagemaker-core/src/sagemaker/core/modules/configs.py

@@ -91,7 +91,10 @@ class SourceCode(BaseConfig):
 
     Parameters:
         source_dir (Optional[str]):
-            The local directory containing the source code to be used in the training job container.
+            The local directory, S3 URI, or path to a tar.gz file stored locally or in S3
+            that contains the source code to be used in the training job container.
+            When an S3 URI is provided, the source code is used directly from S3
+            without local packaging.
         requirements (Optional[str]):
             The path within ``source_dir`` to a ``requirements.txt`` file. If specified, the listed
             requirements will be installed in the training job container.