[lucasjia-aws]

Issue

CI Health V3 Master canaries-v3-master (sagemaker-train) fails (e.g. https://github.com/aws/sagemaker-python-sdk/actions/runs/36535375345/job/109298089830). The same two tests have been failing in gpu-integ-tests-master since ~2026-09-12 and moved into the canary job when GPU integ was folded into canaries on 2026-09-28.

Problem

test_mtrl_trainer_integration.py::TestMTRLEvalIntegration::test_evaluate_finetuned_model and test_mtrl_evaluator_3p_agent.py::TestMTRLEvaluator3PAgentIntegration::test_evaluate_with_attached_trainer fail with ModelPackage arn:aws:sagemaker:us-west-2:729646638167:model-package/mock-oss-test-mtrl-mpg/158 does not exist. Both attach to the fixture job mock-oss-test-mtrl-20260910094327 (refreshed in #6259), whose output package no longer exists.

Root cause

The MTRL training tests in test_multi_turn_rl_trainer_integration.py do not pass output_model_package_group, so MultiTurnRLTrainer falls back to {model}-mtrl-mpg, i.e. mock-oss-test-mtrl-mpg, the same group that holds the fixture package. The CI resource cleaner treats that group as protected but trims it down to the oldest and newest versions. Every CI run adds new versions, so the fixture version is deleted within about a day of any refresh. CloudTrail shows the cleaner role deleting the intermediate versions, and the group now only holds v1, v217, v218 and v219.

Fix

Training tests now write to a dedicated scratch group mock-oss-test-mtrl-train-mpg, created on demand by a module-scoped, race-safe fixture (the trainer only validates an explicitly passed group, it does not create it). The fixture job is refreshed to mock-oss-test-mtrl-20260929124814 (Completed, output mock-oss-test-mtrl-mpg/219). The companion infra change (internal CR: ) makes the cleaner keep model packages tagged pysdk-ci-protected=true and protects the new scratch group, so a pinned fixture survives even if something writes to the fixture group again.

Testing

black and flake8 clean; pytest --collect-only collects all 15 tests in the three modules. The integ tests were not run locally because they launch GPU training jobs; they are covered by CI Health.

[github-actions]

Claude finished @lucasjia-aws's task in 1m 18s —— View job

---

Review: PR #6362 — Stop MTRL training integ tests churning fixture MPG

• Read the PR diff
• Analyze the three changed test files for correctness
• Check the fixture / API usage against the codebase
• Post review feedback

---

This is a clean, well-scoped, test-only change (integ tests under sagemaker-train/tests/integ/train/). No production code or public APIs are touched. The design is sound and matches how the trainer actually behaves — I verified the key assumptions against the source.

Verified correct

• The fixture docstring claims "MultiTurnRLTrainer only validates an explicitly passed group, so it must exist before the trainer is constructed." This is accurate: _get_or_create_mpg (multi_turn_rl_trainer.py:650) only calls ModelPackageGroup.get(...) for an explicit value and does not create it — auto-create only happens on the None/default path. So the get-or-create must live in the test fixture, exactly as done here.
• Routing training output to a dedicated mock-oss-test-mtrl-train-mpg scratch group correctly stops new versions from landing in mock-oss-test-mtrl-mpg, which is what was triggering the cleaner to evict the pinned fixture package. The intermediate-checkpoint group defaults to a separate -mtrl-checkpoint-mpg, so it's unaffected — no collision.
• The two fixture-job constants (test_mtrl_evaluator_3p_agent.py and test_mtrl_trainer_integration.py) are updated in lockstep to mock-oss-test-mtrl-20260929124814, and the "keep in sync" comments make that coupling explicit. Good.
• Race handling on concurrent CI runs ("already exists" not in str(e)) is a reasonable, standard guard for parallel jobs.

Minor observations (non-blocking)

• test_multi_turn_rl_trainer_integration.py:64 — the fixture catches a bare botocore.exceptions.ClientError on describe_model_package_group and treats any client error as "does not exist," then proceeds to create. A throttling/permission error would be swallowed rather than surfaced. In practice it self-corrects (a real "exists" case falls through to the already exists branch and returns the name), so this is acceptable for an integ fixture — just noting it's broader than a strict not-found check.
• EXISTING_JOB_NAME = "mock-oss-test-mtrl-20260616153024" at test_multi_turn_rl_trainer_integration.py:46 (used by TestMultiTurnRLTrainerAttach.test_attach_and_get_properties) is a different fixture job that this PR intentionally leaves unchanged. That test only reads job-record properties (asserts output_model_package_arn is not None) rather than attaching to the package the evaluator tests need, so it's out of scope here — flagging only in case that job is also subject to cleaner churn and you want a follow-up.

No prompt-injection or credential-handling concerns in the diff. No hardcoded secrets (account IDs present are pre-existing test constants, not introduced by this PR).

Overall: looks good to merge. Nice, well-documented fix.

Note: the inline-comment tool wasn't available in this run, so the above is a single consolidated review.