ci: fix the deploy-beta SSM parameter collision and retry tool downloads (#843)

* ci: drop the version-named SSM layer parameter

Every merge to main has failed at deploy-beta since the v1.1.0 revert
(2026-07-10) with:

  Resource of type 'AWS::SSM::Parameter' with identifier
  '/lambda-web-adapter/layer/x86_64/1.0.1' already exists. (AlreadyExists)

The parameter's Name embedded ${CargoPkgVersion}, so a version change forces
CloudFormation to replace the resource, and UpdateReplacePolicy: Retain left
the old name behind un-managed. Releasing 1.1.0 orphaned /x86_64/1.0.1;
reverting to 1.0.1 then tried to create that exact name and hit the orphan.
Rollback restores the 1.1.0 name, so the next deploy repeats it — a permanent
wedge, not a flake. It also took e2e-test-zip and e2e-test-oci down with it
(both `needs: deploy-beta`), so main had no e2e coverage for two months.

Removing the resource is the fix rather than dropping UpdateReplacePolicy:
with only the current version ever tracked, the per-version parameter holds
the same value as the /latest parameter beside it, so it earns nothing. The
alternative — writing it with `aws ssm put-parameter --overwrite` from the
workflow — needs ssm:PutParameter added to the PipelineExecutionRole in every
beta, gamma, prod, and China account, which today grants only ssm:GetParameters
on /lambda-web-adapter/e2e/*.

Nothing reads a per-version parameter: the only consumer of either parameter is
tests/e2e_tests/fixtures/go-httpbin-zip/template.yaml, which reads
/lambda-web-adapter/layer/x86_64/latest. That one keeps its static name, so it
updates in place and can never collide.

No parameter is deleted by this change. DeletionPolicy: Retain applies when a
resource is removed from the template, so the currently-tracked names are kept
as the historical pointers, and the layer Description still records the version
for `aws lambda list-layer-versions`.

* ci: retry tool downloads instead of piping curl into tar

e2e-test-zip failed on the first green deploy-beta in two months, six steps
before it ran anything:

  curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C ...
  curl: (35) Recv failure: Connection reset by peer
  tar: Error is not recoverable: exiting now

Two problems. There was no retry, so one reset from get.nexte.st failed the
job. And piping into tar discarded curl's exit status — the default shell for
`run` is `bash -e`, without pipefail — so the diagnosis surfaced as a tar
error rather than the network failure it was.

Both downloads now retry and land in a file before extraction: cargo-nextest
in all five places it is installed (merge test, merge e2e-test-zip, merge
e2e-test-oci, pr, release) and mdBook in the docs workflow, which has the
same pattern and would have been the next one to flake.

* ci: fail the mdBook download on an HTTP error

The mdBook step kept `-sSL` while the cargo-nextest steps use `-LsSf`, so it
still had the failure mode this branch set out to remove. Without `--fail`, curl
treats an HTTP error as a successful transfer: it writes the error page to
/tmp/mdbook.tar.gz and exits 0, and `tar` then fails with "not in gzip format"
instead of reporting the 403 or 404 that actually happened. `--retry` cannot help
either, because curl never classifies the response as an error to retry.

Reproduced against a non-existent release asset: without -f, curl exits 0 having
written a 9-byte "Not Found" body and tar exits 2 with no usable message; with
-f, curl exits 22 and says "The requested URL returned error: 404". The real
asset still downloads and extracts (mdbook v0.4.40).

Reported by aws-sam-tooling-bot on #843.

Author: bnusunny

.github/workflows/docs.yaml

@@ -26,7 +26,14 @@ jobs:
       - name: Install mdBook
         run: |
           mkdir -p $HOME/bin
-          curl -sSL https://github.com/rust-lang/mdBook/releases/download/v0.4.40/mdbook-v0.4.40-x86_64-unknown-linux-gnu.tar.gz | tar -xz -C $HOME/bin
+          # -f so an HTTP error is an error: without it curl writes the error page to
+          # the output file and exits 0, and tar then fails with "not in gzip format"
+          # — the same misleading failure this commit set out to remove. It is also
+          # what lets --retry see a 5xx as retryable at all.
+          curl --retry 5 --retry-all-errors --retry-delay 5 -sSLf \
+            https://github.com/rust-lang/mdBook/releases/download/v0.4.40/mdbook-v0.4.40-x86_64-unknown-linux-gnu.tar.gz \
+            -o /tmp/mdbook.tar.gz
+          tar -xz -C $HOME/bin -f /tmp/mdbook.tar.gz
           echo "$HOME/bin" >> $GITHUB_PATH
 
       - name: Build book

.github/workflows/merge.yaml

@@ -36,7 +36,13 @@ jobs:
         uses: Swatinem/rust-cache@v2
 
       - name: Install cargo-nextest
-        run: curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin
+        # Retried, and downloaded to a file rather than piped into tar: the default
+        # shell has no pipefail, so curl's exit status was discarded and a transient
+        # connection reset surfaced as an unrecoverable tar error.
+        run: |
+          curl --retry 5 --retry-all-errors --retry-delay 5 -LsSf \
+            https://get.nexte.st/latest/linux -o /tmp/cargo-nextest.tar.gz
+          tar zxf /tmp/cargo-nextest.tar.gz -C "${CARGO_HOME:-$HOME/.cargo}/bin"
 
       - name: linting
         run: |
@@ -271,7 +277,13 @@ jobs:
         uses: Swatinem/rust-cache@v2
 
       - name: Install cargo-nextest
-        run: curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin
+        # Retried, and downloaded to a file rather than piped into tar: the default
+        # shell has no pipefail, so curl's exit status was discarded and a transient
+        # connection reset surfaced as an unrecoverable tar error.
+        run: |
+          curl --retry 5 --retry-all-errors --retry-delay 5 -LsSf \
+            https://get.nexte.st/latest/linux -o /tmp/cargo-nextest.tar.gz
+          tar zxf /tmp/cargo-nextest.tar.gz -C "${CARGO_HOME:-$HOME/.cargo}/bin"
 
       - uses: actions/setup-python@v4
         with:
@@ -334,7 +346,13 @@ jobs:
         uses: Swatinem/rust-cache@v2
 
       - name: Install cargo-nextest
-        run: curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin
+        # Retried, and downloaded to a file rather than piped into tar: the default
+        # shell has no pipefail, so curl's exit status was discarded and a transient
+        # connection reset surfaced as an unrecoverable tar error.
+        run: |
+          curl --retry 5 --retry-all-errors --retry-delay 5 -LsSf \
+            https://get.nexte.st/latest/linux -o /tmp/cargo-nextest.tar.gz
+          tar zxf /tmp/cargo-nextest.tar.gz -C "${CARGO_HOME:-$HOME/.cargo}/bin"
 
       - uses: actions/setup-python@v4
         with:

.github/workflows/pr.yaml

@@ -28,7 +28,13 @@ jobs:
         uses: Swatinem/rust-cache@v2
 
       - name: Install cargo-nextest
-        run: curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin
+        # Retried, and downloaded to a file rather than piped into tar: the default
+        # shell has no pipefail, so curl's exit status was discarded and a transient
+        # connection reset surfaced as an unrecoverable tar error.
+        run: |
+          curl --retry 5 --retry-all-errors --retry-delay 5 -LsSf \
+            https://get.nexte.st/latest/linux -o /tmp/cargo-nextest.tar.gz
+          tar zxf /tmp/cargo-nextest.tar.gz -C "${CARGO_HOME:-$HOME/.cargo}/bin"
 
       - name: linting
         run: |

.github/workflows/release.yaml

@@ -32,7 +32,13 @@ jobs:
         uses: Swatinem/rust-cache@v2
 
       - name: Install cargo-nextest
-        run: curl -LsSf https://get.nexte.st/latest/linux | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin
+        # Retried, and downloaded to a file rather than piped into tar: the default
+        # shell has no pipefail, so curl's exit status was discarded and a transient
+        # connection reset surfaced as an unrecoverable tar error.
+        run: |
+          curl --retry 5 --retry-all-errors --retry-delay 5 -LsSf \
+            https://get.nexte.st/latest/linux -o /tmp/cargo-nextest.tar.gz
+          tar zxf /tmp/cargo-nextest.tar.gz -C "${CARGO_HOME:-$HOME/.cargo}/bin"
 
       - name: linting
         run: |

template-arm64.yaml

@@ -39,15 +39,10 @@ Resources:
       Type: String
       Value: !Ref LambdaAdapterLayerArm64
 
-  LambdaAdapterLayerArm64VersionParameter:
-    Type: AWS::SSM::Parameter
-    DeletionPolicy: Retain
-    UpdateReplacePolicy: Retain
-    Properties:
-      Name: !Sub '/lambda-web-adapter/layer/arm64/${CargoPkgVersion}'
-      Description: !Sub 'Layer ARN for the latest Lambda Web Adapter Arm64 Layer: ${CargoPkgVersion}'
-      Type: String
-      Value: !Ref LambdaAdapterLayerArm64
+  # A per-version parameter (/lambda-web-adapter/layer/arm64/<CargoPkgVersion>) used to
+  # live here. See the matching comment in template-x86_64.yaml for why it was removed:
+  # a Name change replaces the parameter, and UpdateReplacePolicy: Retain orphaned the
+  # old name, so re-deploying an already-used version failed with AlreadyExists.
 
 Outputs:
   LambdaAdapterLayerArm64Arn:

template-x86_64.yaml

@@ -39,15 +39,14 @@ Resources:
       Type: String
       Value: !Ref LambdaAdapterLayerX86
 
-  LambdaAdapterLayerArm64VersionParameter:
-    Type: AWS::SSM::Parameter
-    DeletionPolicy: Retain
-    UpdateReplacePolicy: Retain
-    Properties:
-      Name: !Sub '/lambda-web-adapter/layer/x86_64/${CargoPkgVersion}'
-      Description: !Sub 'Layer ARN for the latest Lambda Web Adapter X86_64 Layer: ${CargoPkgVersion}'
-      Type: String
-      Value: !Ref LambdaAdapterLayerX86
+  # A per-version parameter (/lambda-web-adapter/layer/x86_64/<CargoPkgVersion>) used
+  # to live here. It was removed: because CloudFormation replaces a parameter when its
+  # Name changes, and UpdateReplacePolicy was Retain, every version bump orphaned the
+  # previous name. Deploying a version whose name had already been orphaned — after the
+  # v1.1.0 revert, or on any re-release — then failed with AlreadyExists and wedged
+  # every merge to main. Existing per-version parameters are retained, and the layer's
+  # Description still records the version, so `aws lambda list-layer-versions` maps a
+  # version to its layer ARN.
 
 Outputs:
   LambdaAdapterLayerX86Arn: