From 8773c7467984f3c493c6a38522a70aa4e17dc3fc Mon Sep 17 00:00:00 2001 From: Harold Sun Date: Thu, 17 Sep 2026 17:59:47 +0000 Subject: [PATCH 1/2] ci: publish the version-named layer parameter from the pipeline MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The second half of #843. That change removed the version-named SSM pointer from the layer stacks, because a parameter whose Name contains the package version is replaced when the version changes and — retained — collides with its own orphan when a version is re-used, which wedged every merge to main for two months. The pointer itself is still wanted, so it is now written after deployment with `aws ssm put-parameter --overwrite`, which is idempotent and cannot collide. Two scripts: publish-layer-parameter.sh reads the layer ARN from the stack's own output and writes /lambda-web-adapter/layer//. It refuses a version containing a slash before making any AWS call, since such a name would fall outside the granted path and could address a `latest` pointer. configure-layer-parameter-iam.sh grants the PipelineExecutionRole ssm:PutParameter on that path. Run once per role, with administrator credentials in the role's account. This is a prerequisite: SAM-managed pipeline roles do not have the permission, and until the bootstrap is run the publish steps fail with AccessDenied. 69 roles are in scope — 32 gamma, 32 prod, 2 China gamma, 2 China prod, 1 beta, all distinct. Three things about the grant are deliberate: * Region-agnostic. `put-role-policy` replaces the whole named policy, so a per-region document meant a second invocation for another region silently revoked the first. The Resource wildcards the region instead, so one run per role covers every region and re-running is idempotent. * No region argument. With the region wildcarded there is nothing for the partition parsed from the role ARN to disagree with; an `aws` role paired with `cn-north-1` previously produced a policy matching nothing while reporting success. * An explicit Deny on `/latest`. The Allow has to wildcard the version segment, which also covers the `latest` pointer CloudFormation owns — and which the e2e fixture resolves at deploy time. Without the Deny, a mistyped version could overwrite stack-managed state out of band, where nobody would look for drift. In the workflows, both architectures deploy before either pointer is written. Ordered the other way — publish x86, deploy arm64 — a missed bootstrap left the region with a new x86 layer and no arm64 one, failed deploy-prod, skipped publish-to-public-ecr, and stopped the release after prod had been partially mutated. Ordered this way the same failure leaves both layers published and the pointers stale. Each stack name is now defined once per job and consumed by both the deploy and the publish step, rather than written twice in ten places with nothing asserting the two agree. Exercised: the publish script's argument, architecture and slash-in-version guards; the IAM script's ARN validation against a non-ARN, a short account id, a user ARN and a nested role path; the rendered policy document; and both workflows parse. --- .../scripts/configure-layer-parameter-iam.sh | 76 ++++++++++++++ .github/scripts/publish-layer-parameter.sh | 67 +++++++++++++ .github/workflows/merge.yaml | 26 ++++- .github/workflows/release.yaml | 98 +++++++++++++++++-- template-arm64.yaml | 7 +- template-x86_64.yaml | 12 +-- 6 files changed, 264 insertions(+), 22 deletions(-) create mode 100755 .github/scripts/configure-layer-parameter-iam.sh create mode 100755 .github/scripts/publish-layer-parameter.sh diff --git a/.github/scripts/configure-layer-parameter-iam.sh b/.github/scripts/configure-layer-parameter-iam.sh new file mode 100755 index 00000000..a2fd062e --- /dev/null +++ b/.github/scripts/configure-layer-parameter-iam.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# +# Grants a PipelineExecutionRole permission to publish the version-named SSM pointers that +# publish-layer-parameter.sh writes. Run once per role, with administrator credentials in +# that role's account; AWS_PROFILE and the standard AWS CLI credential variables are +# honored. +# +# Usage: +# configure-layer-parameter-iam.sh +# +# Deliberately region-agnostic. An earlier draft took a region and embedded it in the +# Resource ARN, which had two problems: `put-role-policy` replaces the whole named policy, +# so running it a second time for another region silently revoked the first one; and the +# partition came from the role ARN while the region came from argv, so a mismatched pair +# (an `aws` role with `cn-north-1`) produced a policy that matched nothing while reporting +# success. Wildcarding the region removes both — one invocation per role covers every +# region, and re-running it is idempotent. +set -euo pipefail +export AWS_PAGER="" + +ROLE_ARN="${1:?pipeline execution role ARN required}" + +IFS=: read -r ARN_PREFIX PARTITION SERVICE ARN_REGION ACCOUNT_ID RESOURCE <<<"$ROLE_ARN" +if [[ "$ARN_PREFIX" != "arn" || "$SERVICE" != "iam" || -n "$ARN_REGION" || + ! "$ACCOUNT_ID" =~ ^[0-9]{12}$ || "$RESOURCE" != role/* ]]; then + echo "Invalid IAM role ARN: $ROLE_ARN" >&2 + exit 2 +fi + +ROLE_PATH="${RESOURCE#role/}" +ROLE_NAME="${ROLE_PATH##*/}" + +CALLER_ACCOUNT="$(aws sts get-caller-identity --query Account --output text)" +if [[ "$CALLER_ACCOUNT" != "$ACCOUNT_ID" ]]; then + echo "AWS credentials are for account $CALLER_ACCOUNT, but $ROLE_ARN is in $ACCOUNT_ID" >&2 + exit 1 +fi + +# The Deny is not redundant. The Allow has to wildcard the version segment — the whole +# point is writing a name that contains the package version — and that wildcard also +# covers `/latest`, which CloudFormation owns (LambdaAdapterLayerX86Parameter, and the +# value the e2e fixture resolves at deploy time). Without the Deny, a mistyped version or +# a future bug in the publish step could overwrite stack-managed state out of band, where +# nobody would think to look for drift. +POLICY_DOCUMENT="$( + jq -cn \ + --arg x86 "arn:${PARTITION}:ssm:*:${ACCOUNT_ID}:parameter/lambda-web-adapter/layer/x86_64/*" \ + --arg arm "arn:${PARTITION}:ssm:*:${ACCOUNT_ID}:parameter/lambda-web-adapter/layer/arm64/*" \ + --arg x86latest "arn:${PARTITION}:ssm:*:${ACCOUNT_ID}:parameter/lambda-web-adapter/layer/x86_64/latest" \ + --arg armlatest "arn:${PARTITION}:ssm:*:${ACCOUNT_ID}:parameter/lambda-web-adapter/layer/arm64/latest" \ + '{ + Version: "2012-10-17", + Statement: [ + { + Sid: "PublishVersionedLayerParameters", + Effect: "Allow", + Action: "ssm:PutParameter", + Resource: [$x86, $arm] + }, + { + Sid: "DenyCloudFormationOwnedLatestPointers", + Effect: "Deny", + Action: "ssm:PutParameter", + Resource: [$x86latest, $armlatest] + } + ] + }' +)" + +aws iam put-role-policy \ + --role-name "$ROLE_NAME" \ + --policy-name LambdaWebAdapterVersionedLayerParameters \ + --policy-document "$POLICY_DOCUMENT" + +echo "Granted ssm:PutParameter on /lambda-web-adapter/layer/{x86_64,arm64}/* to $ROLE_ARN" +echo "(all regions in partition $PARTITION; /latest explicitly denied)" diff --git a/.github/scripts/publish-layer-parameter.sh b/.github/scripts/publish-layer-parameter.sh new file mode 100755 index 00000000..14156b81 --- /dev/null +++ b/.github/scripts/publish-layer-parameter.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# +# Publishes the version-named SSM pointer after CloudFormation has deployed a layer. +# The parameter deliberately lives outside the stack: retaining a +# parameter whose Name contains the package version made CloudFormation collide with +# the retained resource whenever a version was re-used. +# +# Usage: +# publish-layer-parameter.sh +set -euo pipefail +export AWS_PAGER="" + +STACK_NAME="${1:?stack name required}" +ARCHITECTURE="${2:?architecture required}" +VERSION="${3:?version required}" +REGION="${4:?region required}" + +# A version containing a slash would write outside the path the grant covers — and, worse, +# could name a `latest` pointer. Fail before the API call rather than on an opaque +# AccessDenied. +if [[ "$VERSION" != "${VERSION//\//}" || -z "$VERSION" ]]; then + echo "Refusing to publish a parameter for version '$VERSION'" >&2 + exit 2 +fi + +case "$ARCHITECTURE" in + x86_64) + OUTPUT_KEY="LambdaAdapterLayerX86Arn" + DESCRIPTION_ARCH="X86_64" + ;; + arm64) + OUTPUT_KEY="LambdaAdapterLayerArm64Arn" + DESCRIPTION_ARCH="Arm64" + ;; + *) + echo "Unsupported architecture: $ARCHITECTURE" >&2 + exit 2 + ;; +esac + +LAYER_ARN="$( + aws cloudformation describe-stacks \ + --stack-name "$STACK_NAME" \ + --region "$REGION" \ + --query "Stacks[0].Outputs[?OutputKey=='${OUTPUT_KEY}'].OutputValue | [0]" \ + --output text +)" + +if [[ -z "$LAYER_ARN" || "$LAYER_ARN" == "None" ]]; then + echo "Stack $STACK_NAME has no $OUTPUT_KEY output in $REGION" >&2 + exit 1 +fi + +PARAMETER_NAME="/lambda-web-adapter/layer/${ARCHITECTURE}/${VERSION}" +PARAMETER_VERSION="$( + aws ssm put-parameter \ + --name "$PARAMETER_NAME" \ + --description "Layer ARN for the Lambda Web Adapter ${DESCRIPTION_ARCH} Layer: ${VERSION}" \ + --type String \ + --value "$LAYER_ARN" \ + --overwrite \ + --region "$REGION" \ + --query Version \ + --output text +)" + +echo "Published $PARAMETER_NAME -> $LAYER_ARN (parameter version $PARAMETER_VERSION)" diff --git a/.github/workflows/merge.yaml b/.github/workflows/merge.yaml index 0bfda68b..1f279570 100644 --- a/.github/workflows/merge.yaml +++ b/.github/workflows/merge.yaml @@ -228,13 +228,19 @@ jobs: run: | echo "CARGO_PKG_VERSION=$(cargo metadata --no-deps --format-version=1 | jq -r '.packages[0].version')" >> $GITHUB_ENV + - name: Resolve the layer stack names + run: | + # Once, from BETA_STACK_NAME, so the deploy and the publish below cannot drift. + echo "X86_STACK=${BETA_STACK_NAME}-x86" >> $GITHUB_ENV + echo "ARM64_STACK=${BETA_STACK_NAME}-arm64" >> $GITHUB_ENV + - uses: actions/download-artifact@v4 with: name: packaged-beta-x86_64.yaml - name: Deploy x86_64 layer to beta account run: | - sam deploy --stack-name ${BETA_STACK_NAME}-x86 \ + sam deploy --stack-name ${X86_STACK} \ --template packaged-beta-x86_64.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -250,7 +256,7 @@ jobs: - name: Deploy arm64 layer to beta account run: | - sam deploy --stack-name ${BETA_STACK_NAME}-arm64 \ + sam deploy --stack-name ${ARM64_STACK} \ --template packaged-beta-arm64.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -260,6 +266,22 @@ jobs: --no-fail-on-empty-changeset \ --role-arn ${BETA_CLOUDFORMATION_EXECUTION_ROLE} + # Both deploys complete before either pointer is written, deliberately. These steps + # need ssm:PutParameter, which a SAM-managed PipelineExecutionRole does not have + # until .github/scripts/configure-layer-parameter-iam.sh has been run for it, so + # AccessDenied is the expected failure of a missed bootstrap. Ordered this way, that + # failure leaves both layers published and the version pointers stale; ordered + # between the deploys, it left the region with a new x86 layer and no arm64 one. + - name: Publish versioned x86_64 layer parameter + run: | + .github/scripts/publish-layer-parameter.sh \ + "$X86_STACK" x86_64 "${CARGO_PKG_VERSION}" "${BETA_REGION}" + + - name: Publish versioned arm64 layer parameter + run: | + .github/scripts/publish-layer-parameter.sh \ + "$ARM64_STACK" arm64 "${CARGO_PKG_VERSION}" "${BETA_REGION}" + e2e-test-zip: needs: [deploy-beta] runs-on: ubuntu-24.04 diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index 8ecef22a..636d7929 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -428,6 +428,11 @@ jobs: runs-on: ubuntu-24.04 strategy: matrix: ${{fromJSON(needs.load-matrices.outputs.gamma)}} + env: + # One definition each, used by the deploy and by the publish step + # below, so the two cannot drift apart in a copy-paste. + X86_STACK: lambda-adapter-gamma-x86-${{ matrix.region }} + ARM64_STACK: lambda-adapter-gamma-arm64-${{ matrix.region }} steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v4 @@ -464,7 +469,7 @@ jobs: - name: Deploy x86_64 Layer to all regions in gamma account run: | - sam deploy --stack-name lambda-adapter-gamma-x86-${{ matrix.region }} \ + sam deploy --stack-name ${X86_STACK} \ --template packaged-gamma-x86_64-${{ matrix.region }}.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -474,6 +479,7 @@ jobs: --no-fail-on-empty-changeset \ --role-arn ${{ matrix.cloudformation_execution_role }} + - uses: actions/download-artifact@v4 with: name: packaged-gamma-arm64-${{ matrix.region }}.yaml @@ -481,7 +487,7 @@ jobs: - name: Deploy arm64 Layer to supported regions in gamma account if: ${{ matrix.arm64_supported }} run: | - sam deploy --stack-name lambda-adapter-gamma-arm64-${{ matrix.region }} \ + sam deploy --stack-name ${ARM64_STACK} \ --template packaged-gamma-arm64-${{ matrix.region }}.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -491,12 +497,36 @@ jobs: --no-fail-on-empty-changeset \ --role-arn ${{ matrix.cloudformation_execution_role }} + # Both deploys complete before either pointer is written, deliberately. These steps + # need ssm:PutParameter, which a SAM-managed PipelineExecutionRole does not have + # until .github/scripts/configure-layer-parameter-iam.sh has been run for it, so + # AccessDenied is the expected failure of a missed bootstrap. Ordered this way, that + # failure leaves both layers published and the version pointers stale; ordered + # between the deploys, it left the region with a new x86 layer and no arm64 one. + - name: Publish versioned x86_64 layer parameter + run: | + .github/scripts/publish-layer-parameter.sh \ + "$X86_STACK" \ + x86_64 "${CARGO_PKG_VERSION}" "${{ matrix.region }}" + + - name: Publish versioned arm64 layer parameter + if: ${{ matrix.arm64_supported }} + run: | + .github/scripts/publish-layer-parameter.sh \ + "$ARM64_STACK" \ + arm64 "${CARGO_PKG_VERSION}" "${{ matrix.region }}" + deploy-prod: needs: [load-matrices, deploy-gamma, package-prod] runs-on: ubuntu-24.04 environment: prod strategy: matrix: ${{fromJSON(needs.load-matrices.outputs.prod)}} + env: + # One definition each, used by the deploy and by the publish step + # below, so the two cannot drift apart in a copy-paste. + X86_STACK: lambda-adapter-prod-x86-${{ matrix.region }} + ARM64_STACK: lambda-adapter-prod-arm64-${{ matrix.region }} steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v4 @@ -533,7 +563,7 @@ jobs: - name: Deploy x86_64 Layer to all regions in prod account run: | - sam deploy --stack-name lambda-adapter-prod-x86-${{ matrix.region }} \ + sam deploy --stack-name ${X86_STACK} \ --template packaged-prod-x86_64-${{ matrix.region }}.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -543,6 +573,7 @@ jobs: --no-fail-on-empty-changeset \ --role-arn ${{ matrix.cloudformation_execution_role }} + - uses: actions/download-artifact@v4 with: name: packaged-prod-arm64-${{ matrix.region }}.yaml @@ -550,7 +581,7 @@ jobs: - name: Deploy arm64 Layer to supported regions in prod account if: ${{ matrix.arm64_supported }} run: | - sam deploy --stack-name lambda-adapter-prod-arm64-${{ matrix.region }} \ + sam deploy --stack-name ${ARM64_STACK} \ --template packaged-prod-arm64-${{ matrix.region }}.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -560,11 +591,29 @@ jobs: --no-fail-on-empty-changeset \ --role-arn ${{ matrix.cloudformation_execution_role }} + - name: Publish versioned x86_64 layer parameter + run: | + .github/scripts/publish-layer-parameter.sh \ + "$X86_STACK" \ + x86_64 "${CARGO_PKG_VERSION}" "${{ matrix.region }}" + + - name: Publish versioned arm64 layer parameter + if: ${{ matrix.arm64_supported }} + run: | + .github/scripts/publish-layer-parameter.sh \ + "$ARM64_STACK" \ + arm64 "${CARGO_PKG_VERSION}" "${{ matrix.region }}" + deploy-china-gamma: needs: [load-matrices, package-china-gamma] runs-on: ubuntu-24.04 strategy: matrix: ${{fromJSON(needs.load-matrices.outputs.china-gamma)}} + env: + # One definition each, used by the deploy and by the publish step + # below, so the two cannot drift apart in a copy-paste. + X86_STACK: lambda-adapter-gamma-x86-${{ matrix.region }} + ARM64_STACK: lambda-adapter-gamma-arm64-${{ matrix.region }} steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v4 @@ -603,7 +652,7 @@ jobs: - name: Deploy x86_64 Layer to all regions in china run: | - sam deploy --stack-name lambda-adapter-gamma-x86-${{ matrix.region }} \ + sam deploy --stack-name ${X86_STACK} \ --template packaged-china-gamma-x86_64-${{ matrix.region }}.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -613,6 +662,7 @@ jobs: --no-fail-on-empty-changeset \ --role-arn ${{ matrix.cloudformation_execution_role }} + - uses: actions/download-artifact@v4 with: name: packaged-china-gamma-arm64-${{ matrix.region }}.yaml @@ -620,7 +670,7 @@ jobs: - name: Deploy arm64 Layer to supported china regions if: ${{ matrix.arm64_supported }} run: | - sam deploy --stack-name lambda-adapter-gamma-arm64-${{ matrix.region }} \ + sam deploy --stack-name ${ARM64_STACK} \ --template packaged-china-gamma-arm64-${{ matrix.region }}.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -630,12 +680,30 @@ jobs: --no-fail-on-empty-changeset \ --role-arn ${{ matrix.cloudformation_execution_role }} + - name: Publish versioned x86_64 layer parameter + run: | + .github/scripts/publish-layer-parameter.sh \ + "$X86_STACK" \ + x86_64 "${CARGO_PKG_VERSION}" "${{ matrix.region }}" + + - name: Publish versioned arm64 layer parameter + if: ${{ matrix.arm64_supported }} + run: | + .github/scripts/publish-layer-parameter.sh \ + "$ARM64_STACK" \ + arm64 "${CARGO_PKG_VERSION}" "${{ matrix.region }}" + deploy-china-prod: needs: [load-matrices, deploy-china-gamma, package-china-prod] runs-on: ubuntu-24.04 environment: prod strategy: matrix: ${{fromJSON(needs.load-matrices.outputs.china-prod)}} + env: + # One definition each, used by the deploy and by the publish step + # below, so the two cannot drift apart in a copy-paste. + X86_STACK: lambda-adapter-prod-x86-${{ matrix.region }} + ARM64_STACK: lambda-adapter-prod-arm64-${{ matrix.region }} steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v4 @@ -674,7 +742,7 @@ jobs: - name: Deploy x86_64 Layer to all regions in china run: | - sam deploy --stack-name lambda-adapter-prod-x86-${{ matrix.region }} \ + sam deploy --stack-name ${X86_STACK} \ --template packaged-china-prod-x86_64-${{ matrix.region }}.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -684,6 +752,7 @@ jobs: --no-fail-on-empty-changeset \ --role-arn ${{ matrix.cloudformation_execution_role }} + - uses: actions/download-artifact@v4 with: name: packaged-china-prod-arm64-${{ matrix.region }}.yaml @@ -691,7 +760,7 @@ jobs: - name: Deploy arm64 Layer to supported china regions if: ${{ matrix.arm64_supported }} run: | - sam deploy --stack-name lambda-adapter-prod-arm64-${{ matrix.region }} \ + sam deploy --stack-name ${ARM64_STACK} \ --template packaged-china-prod-arm64-${{ matrix.region }}.yaml \ --parameter-overrides CargoPkgVersion=${CARGO_PKG_VERSION} \ --capabilities CAPABILITY_IAM \ @@ -701,6 +770,19 @@ jobs: --no-fail-on-empty-changeset \ --role-arn ${{ matrix.cloudformation_execution_role }} + - name: Publish versioned x86_64 layer parameter + run: | + .github/scripts/publish-layer-parameter.sh \ + "$X86_STACK" \ + x86_64 "${CARGO_PKG_VERSION}" "${{ matrix.region }}" + + - name: Publish versioned arm64 layer parameter + if: ${{ matrix.arm64_supported }} + run: | + .github/scripts/publish-layer-parameter.sh \ + "$ARM64_STACK" \ + arm64 "${CARGO_PKG_VERSION}" "${{ matrix.region }}" + publish-to-public-ecr: needs: [deploy-prod] runs-on: ubuntu-24.04 diff --git a/template-arm64.yaml b/template-arm64.yaml index 5f73ab07..b0c4830c 100644 --- a/template-arm64.yaml +++ b/template-arm64.yaml @@ -39,10 +39,9 @@ Resources: Type: String Value: !Ref LambdaAdapterLayerArm64 - # A per-version parameter (/lambda-web-adapter/layer/arm64/) used to - # live here. See the matching comment in template-x86_64.yaml for why it was removed: - # a Name change replaces the parameter, and UpdateReplacePolicy: Retain orphaned the - # old name, so re-deploying an already-used version failed with AlreadyExists. + # The version-named parameter (/lambda-web-adapter/layer/arm64/) + # is published after deployment with `aws ssm put-parameter --overwrite`. See the + # matching comment in template-x86_64.yaml for why it cannot live in this stack. Outputs: LambdaAdapterLayerArm64Arn: diff --git a/template-x86_64.yaml b/template-x86_64.yaml index d857dbfc..a8455cd2 100644 --- a/template-x86_64.yaml +++ b/template-x86_64.yaml @@ -39,14 +39,10 @@ Resources: Type: String Value: !Ref LambdaAdapterLayerX86 - # A per-version parameter (/lambda-web-adapter/layer/x86_64/) used - # to live here. It was removed: because CloudFormation replaces a parameter when its - # Name changes, and UpdateReplacePolicy was Retain, every version bump orphaned the - # previous name. Deploying a version whose name had already been orphaned — after the - # v1.1.0 revert, or on any re-release — then failed with AlreadyExists and wedged - # every merge to main. Existing per-version parameters are retained, and the layer's - # Description still records the version, so `aws lambda list-layer-versions` maps a - # version to its layer ARN. + # The version-named parameter (/lambda-web-adapter/layer/x86_64/) + # is published after deployment with `aws ssm put-parameter --overwrite`. It cannot + # live in this stack: a Name change replaces the resource, while retaining old + # versions leaves names that CloudFormation collides with when a version is re-used. Outputs: LambdaAdapterLayerX86Arn: From 676fdf4361656ba08a70a5c599986056821f1e95 Mon Sep 17 00:00:00 2001 From: Harold Sun Date: Thu, 17 Sep 2026 18:13:17 +0000 Subject: [PATCH 2/2] ci: take the SSM grant from the infrastructure package, not a bootstrap script MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The PipelineExecutionRole is managed in LambdaWebAdapterInfrastructureCDK, so the grant belongs there rather than in an imperative script run by hand against 69 roles. lib/infra-template.yaml now carries it in PipelineExecutionRolePermissions, allowing ssm:PutParameter on parameter/lambda-web-adapter/layer/{x86_64,arm64}/* and denying .../latest, which CloudFormation owns. Deletes .github/scripts/configure-layer-parameter-iam.sh. Keeping it would have been worse than redundant: it wrote a separate inline policy under its own name, so the two would have coexisted on every role with no indication which was authoritative, and its region-wildcarded Resource was broader than the per-region scope the infra template can express — one pipeline-resources stack exists per region and its role only deploys in that region. The workflow comments and the publish script's header now say where the permission comes from, so the AccessDenied case points at a region whose pipeline-resources stack has not been updated rather than at a script nobody ran. --- .../scripts/configure-layer-parameter-iam.sh | 76 ------------------- .github/scripts/publish-layer-parameter.sh | 5 ++ .github/workflows/merge.yaml | 9 ++- .github/workflows/release.yaml | 9 ++- 4 files changed, 15 insertions(+), 84 deletions(-) delete mode 100755 .github/scripts/configure-layer-parameter-iam.sh diff --git a/.github/scripts/configure-layer-parameter-iam.sh b/.github/scripts/configure-layer-parameter-iam.sh deleted file mode 100755 index a2fd062e..00000000 --- a/.github/scripts/configure-layer-parameter-iam.sh +++ /dev/null @@ -1,76 +0,0 @@ -#!/usr/bin/env bash -# -# Grants a PipelineExecutionRole permission to publish the version-named SSM pointers that -# publish-layer-parameter.sh writes. Run once per role, with administrator credentials in -# that role's account; AWS_PROFILE and the standard AWS CLI credential variables are -# honored. -# -# Usage: -# configure-layer-parameter-iam.sh -# -# Deliberately region-agnostic. An earlier draft took a region and embedded it in the -# Resource ARN, which had two problems: `put-role-policy` replaces the whole named policy, -# so running it a second time for another region silently revoked the first one; and the -# partition came from the role ARN while the region came from argv, so a mismatched pair -# (an `aws` role with `cn-north-1`) produced a policy that matched nothing while reporting -# success. Wildcarding the region removes both — one invocation per role covers every -# region, and re-running it is idempotent. -set -euo pipefail -export AWS_PAGER="" - -ROLE_ARN="${1:?pipeline execution role ARN required}" - -IFS=: read -r ARN_PREFIX PARTITION SERVICE ARN_REGION ACCOUNT_ID RESOURCE <<<"$ROLE_ARN" -if [[ "$ARN_PREFIX" != "arn" || "$SERVICE" != "iam" || -n "$ARN_REGION" || - ! "$ACCOUNT_ID" =~ ^[0-9]{12}$ || "$RESOURCE" != role/* ]]; then - echo "Invalid IAM role ARN: $ROLE_ARN" >&2 - exit 2 -fi - -ROLE_PATH="${RESOURCE#role/}" -ROLE_NAME="${ROLE_PATH##*/}" - -CALLER_ACCOUNT="$(aws sts get-caller-identity --query Account --output text)" -if [[ "$CALLER_ACCOUNT" != "$ACCOUNT_ID" ]]; then - echo "AWS credentials are for account $CALLER_ACCOUNT, but $ROLE_ARN is in $ACCOUNT_ID" >&2 - exit 1 -fi - -# The Deny is not redundant. The Allow has to wildcard the version segment — the whole -# point is writing a name that contains the package version — and that wildcard also -# covers `/latest`, which CloudFormation owns (LambdaAdapterLayerX86Parameter, and the -# value the e2e fixture resolves at deploy time). Without the Deny, a mistyped version or -# a future bug in the publish step could overwrite stack-managed state out of band, where -# nobody would think to look for drift. -POLICY_DOCUMENT="$( - jq -cn \ - --arg x86 "arn:${PARTITION}:ssm:*:${ACCOUNT_ID}:parameter/lambda-web-adapter/layer/x86_64/*" \ - --arg arm "arn:${PARTITION}:ssm:*:${ACCOUNT_ID}:parameter/lambda-web-adapter/layer/arm64/*" \ - --arg x86latest "arn:${PARTITION}:ssm:*:${ACCOUNT_ID}:parameter/lambda-web-adapter/layer/x86_64/latest" \ - --arg armlatest "arn:${PARTITION}:ssm:*:${ACCOUNT_ID}:parameter/lambda-web-adapter/layer/arm64/latest" \ - '{ - Version: "2012-10-17", - Statement: [ - { - Sid: "PublishVersionedLayerParameters", - Effect: "Allow", - Action: "ssm:PutParameter", - Resource: [$x86, $arm] - }, - { - Sid: "DenyCloudFormationOwnedLatestPointers", - Effect: "Deny", - Action: "ssm:PutParameter", - Resource: [$x86latest, $armlatest] - } - ] - }' -)" - -aws iam put-role-policy \ - --role-name "$ROLE_NAME" \ - --policy-name LambdaWebAdapterVersionedLayerParameters \ - --policy-document "$POLICY_DOCUMENT" - -echo "Granted ssm:PutParameter on /lambda-web-adapter/layer/{x86_64,arm64}/* to $ROLE_ARN" -echo "(all regions in partition $PARTITION; /latest explicitly denied)" diff --git a/.github/scripts/publish-layer-parameter.sh b/.github/scripts/publish-layer-parameter.sh index 14156b81..09952dbf 100755 --- a/.github/scripts/publish-layer-parameter.sh +++ b/.github/scripts/publish-layer-parameter.sh @@ -5,6 +5,11 @@ # parameter whose Name contains the package version made CloudFormation collide with # the retained resource whenever a version was re-used. # +# The PipelineExecutionRole's ssm:PutParameter grant for these names lives in +# LambdaWebAdapterInfrastructureCDK (lib/infra-template.yaml, +# PipelineExecutionRolePermissions), which also denies the `latest` pointer that +# CloudFormation owns. +# # Usage: # publish-layer-parameter.sh set -euo pipefail diff --git a/.github/workflows/merge.yaml b/.github/workflows/merge.yaml index 1f279570..401202e2 100644 --- a/.github/workflows/merge.yaml +++ b/.github/workflows/merge.yaml @@ -267,10 +267,11 @@ jobs: --role-arn ${BETA_CLOUDFORMATION_EXECUTION_ROLE} # Both deploys complete before either pointer is written, deliberately. These steps - # need ssm:PutParameter, which a SAM-managed PipelineExecutionRole does not have - # until .github/scripts/configure-layer-parameter-iam.sh has been run for it, so - # AccessDenied is the expected failure of a missed bootstrap. Ordered this way, that - # failure leaves both layers published and the version pointers stale; ordered + # need ssm:PutParameter, granted to the PipelineExecutionRole by + # LambdaWebAdapterInfrastructureCDK (lib/infra-template.yaml, + # PipelineExecutionRolePermissions) — so a region whose pipeline-resources stack has + # not been updated with that grant fails here with AccessDenied. Ordered this way, + # that failure leaves both layers published and the version pointers stale; ordered # between the deploys, it left the region with a new x86 layer and no arm64 one. - name: Publish versioned x86_64 layer parameter run: | diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index 636d7929..cec6bbf9 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -498,10 +498,11 @@ jobs: --role-arn ${{ matrix.cloudformation_execution_role }} # Both deploys complete before either pointer is written, deliberately. These steps - # need ssm:PutParameter, which a SAM-managed PipelineExecutionRole does not have - # until .github/scripts/configure-layer-parameter-iam.sh has been run for it, so - # AccessDenied is the expected failure of a missed bootstrap. Ordered this way, that - # failure leaves both layers published and the version pointers stale; ordered + # need ssm:PutParameter, granted to the PipelineExecutionRole by + # LambdaWebAdapterInfrastructureCDK (lib/infra-template.yaml, + # PipelineExecutionRolePermissions) — so a region whose pipeline-resources stack has + # not been updated with that grant fails here with AccessDenied. Ordered this way, + # that failure leaves both layers published and the version pointers stale; ordered # between the deploys, it left the region with a new x86 layer and no arm64 one. - name: Publish versioned x86_64 layer parameter run: |