[hippi345]

Issue #, if available: #6493

Description of changes:

Fixes #6493. FrameworkProcessor._package_code() always used job_name in the sourcedir S3 key, so pipelines never got cache hits for jobs with source_dir. It now uses _pipeline_config.pipeline_name and _pipeline_config.code_hash when they're set, matching ScriptProcessor._upload_code(). Added the unit test test_package_code_with_pipeline_config, which fails on current master (the S3 key contains test-job) and passes with the fix. The 5 local-mode failures in test_processing.py also fail on unmodified master because the test environment has no AWS region set.

Note: prepared with AI assistance and reviewed by the author.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[Nimisha987]

The new hash-based s3_payload URI no longer contains sourcedir.tar.gz. However, _pack_and_upload_code() derives the runproc.sh and install_requirements.py URIs using s3_payload.replace("sourcedir.tar.gz", ...). When the pipeline code hash is used, both replacements return the original URI, causing both helper uploads to target the same S3 object as the source archive.

Could we preserve the sourcedir.tar.gz filename under the hash-specific prefix and add a regression test covering _pack_and_upload_code() with a pipeline code hash? This would ensure the source archive and helper scripts are uploaded to distinct S3 locations.

[hippi345]

Thanks, that was a real collision. _pack_and_upload_code() still derives helper URIs with s3_payload.replace("sourcedir.tar.gz", ...), so the pipeline hash key now keeps that filename: {pipeline_name}/source/{code_hash}/sourcedir.tar.gz. Added test_pack_and_upload_code_with_pipeline_config to assert the hash prefix, distinct helper uploads, and that two hashes produce different keys.

[github-actions]

🤖 AI code review

This PR fixes FrameworkProcessor._package_code() to build a content-addressed S3 key using _pipeline_config.pipeline_name and _pipeline_config.code_hash when a pipeline config is present, enabling ProcessingStep cache hits for jobs with source_dir (issue #6493). The implementation mirrors the existing ScriptProcessor._upload_code() pattern, uses the same local import of _pipeline_config, and correctly preserves the sourcedir.tar.gz filename so the downstream s3_payload.replace("sourcedir.tar.gz", ...) logic that derives runproc.sh/install_requirements.py URIs still works. The two added unit tests cover both the S3-key content and helper-script non-collision across distinct hashes, and existing tests cover the standalone (non-pipeline) path. The change looks correct and well-tested; no issues found.

_{Reviewed commit 5cab44b. Automated review; verify before acting.}