Merge sync PRs to prerelease right away and retry failed backports (#2293) (#2295)

* Merge sync PRs to prerelease right away and retry failed backports

Since the v1.10 release merge, every auto-backport to prerelease has failed, and the fixes reached prerelease only through manual backports weeks later.
Most failures were an ordering race rather than real divergence: sync PRs stay open for minutes to hours, and a second PR to main touching the same file is cherry-picked onto a prerelease that does not have the first change yet.
#2141 conflicted on a freeze file because the sync PR of #2138 merged 19 seconds after it.
Failed backports are never retried, so each one then breaks every later PR on the same files (#2227 after #2141, #2258 and #2257 after #2235).

The workflow now merges a clean sync PR immediately and publishes prerelease itself, since a merge made with GITHUB_TOKEN does not fire the push event publish.yml listens to.
A failed backport labels the source PR backport-failed, so the missing changes are listable instead of buried in bot comments.
After each sync merge, and after any manual merge into prerelease, the workflow comments /sync-prerelease on the oldest labeled PR, so a backport that only waited on an earlier change applies on its own.

No concurrency group: GitHub cancels a pending run when another queues, which would drop backports silently when several PRs merge within minutes.
The cost of merging without review is that a main-only PR now needs the no-sync-prerelease label before it merges, where it used to be enough to leave the sync PR unmerged.

* Label sync PRs that fail to merge

A sync PR can also fail to merge when prerelease moves between the cherry-pick and the merge.
That backport now gets the backport-failed label too, instead of being missed by the list and the retry.
The open sync PR is left for a manual merge.

Publishing no longer depends on the label cleanup succeeding after the merge, since nothing else publishes a GITHUB_TOKEN merge.

* Retry the oldest failed backport even past 30 labeled PRs

gh pr list returns at most 30 results by default, newest first, so
with more labeled PRs the sort only saw the newest 30 and the oldest
backport, often the one the others wait on, would never be retried.

(cherry picked from commit 35c61f4)

Co-authored-by: Christophe Dervieux <christophe.dervieux@gmail.com>

Author: github-actions[bot]

.github/workflows/README.md

@@ -33,8 +33,11 @@ Note that technically, <prerelease.quarto.org> is also a deploy preview on Netli
 - `port-to-prerelease.yml` - This workflow is used to sync changes made to main for quarto.org to prerelease branch for prerelease.quarto.org. 
   - It is triggered when a PR is merged in to `main`. It can also be triggered manually by adding a comment `/sync-prerelease` on a merged PR.
   - This workflow uses [`korthout/backport-action`](https://github.com/korthout/backport-action) to create a PR with the changes merged into `main` branch to be synced to `prerelease` branch.
-  - It will also write a new `/deploy-preview` comment in the new PR to trigger the preview deployment from `preview.yml`.
-  - This is possible because it uses a fine-grained PAT token which allows a workflow to trigger another using usual event (GITHUB_TOKEN does not allow that usually). This is configured in repo secrets.
+  - When the cherry-pick applies cleanly, the workflow merges the sync PR right away and publishes prerelease by calling `publish.yml` (a merge made with GITHUB_TOKEN does not trigger the usual push event). Merging immediately keeps prerelease current, so the next PR touching the same files does not conflict against a sync PR still waiting for review.
+  - A PR that must not reach prerelease (e.g. docs for behavior that differs between versions) needs the `no-sync-prerelease` label **before** it is merged into `main`.
+  - When the cherry-pick fails, the source PR gets the `backport-failed` label: `gh pr list --label backport-failed --state merged` lists every change still missing from prerelease. After backporting one by hand, remove its label.
+  - After each sync merge, and after any PR is merged into `prerelease` by hand, the workflow retries the oldest `backport-failed` PR by commenting `/sync-prerelease` on it. A successful retry triggers the next one; the chain stops at the first PR that fails again.
+  - The retry comment uses a fine-grained PAT token, which allows a workflow to trigger another using usual event (GITHUB_TOKEN does not allow that usually). This is configured in repo secrets.
 
 ## Reusable Composite Actions
 

.github/workflows/port-to-prerelease.yml

@@ -5,15 +5,16 @@ on:
   issue_comment:
     types: [created]
 
-permissions:
-  contents: write # so it can comment
-  pull-requests: write # so it can create pull requests
-  checks: write # so it can create a check run
+permissions: {} # each job opts in below
 
 jobs:
   sync-to-prerelease:
     name: Backport pull request
     runs-on: ubuntu-latest
+    permissions:
+      contents: write # so it can comment
+      pull-requests: write # so it can create, merge and label pull requests
+      checks: write # so it can create a check run
     # Don't run on closed unmerged pull requests and only PRs to main,
     # skipping PRs labeled 'no-sync-prerelease' (e.g. release merges that
     # would otherwise try to cherry-pick the whole prerelease history back).
@@ -29,6 +30,8 @@ jobs:
         github.event.comment.user.id != 41898282 &&
         startsWith(github.event.comment.body, '/sync-prerelease')
       )
+    outputs:
+      merged: ${{ steps.merge.outputs.merged }}
     steps:
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
         with:
@@ -55,18 +58,92 @@ jobs:
           label_pattern: ''
           target_branches: prerelease
 
-      - name: Trigger deploy using a comment
+      # Merging right away keeps prerelease up to date before the next PR to
+      # main is cherry-picked; a sync PR left open makes the next backport
+      # touching the same file conflict.
+      - name: Merge the sync pull request
+        id: merge
         if: steps.synced-pr.outputs.was_successful == 'true'
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
         env:
-          CREATED_PULL_NUMBERS: ${{ steps.synced-pr.outputs.created_pull_numbers }}
-        with:
+          GH_TOKEN: ${{ github.token }}
+          SYNC_PR: ${{ steps.synced-pr.outputs.created_pull_numbers }}
+        run: |
+          gh pr merge "$SYNC_PR" --repo "$GITHUB_REPOSITORY" --squash
+          echo "merged=true" >> "$GITHUB_OUTPUT"
+
+      - name: Remove the backport-failed label
+        if: steps.synced-pr.outputs.was_successful == 'true'
+        env:
+          GH_TOKEN: ${{ github.token }}
+          SOURCE_PR: ${{ github.event.pull_request.number || github.event.issue.number }}
+        run: |
+          if gh pr view "$SOURCE_PR" --repo "$GITHUB_REPOSITORY" --json labels --jq '.labels[].name' | grep -qx backport-failed; then
+            gh pr edit "$SOURCE_PR" --repo "$GITHUB_REPOSITORY" --remove-label backport-failed
+          fi
+
+      # Also when the sync PR could not be merged, e.g. prerelease moved and
+      # it now conflicts; it stays open for a manual merge.
+      - name: Label the failed backport
+        if: >
+          !cancelled() && (
+            steps.synced-pr.outputs.was_successful == 'false' ||
+            steps.merge.outcome == 'failure'
+          )
+        env:
+          GH_TOKEN: ${{ github.token }}
+          SOURCE_PR: ${{ github.event.pull_request.number || github.event.issue.number }}
+        run: gh pr edit "$SOURCE_PR" --repo "$GITHUB_REPOSITORY" --add-label backport-failed
+
+  # Merging with GITHUB_TOKEN does not trigger the push event that publishes
+  # prerelease, so publish it explicitly. Publishing the branch tip rather than
+  # the merge commit keeps the latest content when two syncs land together.
+  publish-prerelease:
+    needs: [ sync-to-prerelease ]
+    # Publish even if a later step of the sync job failed after the merge.
+    if: ${{ !cancelled() && needs.sync-to-prerelease.outputs.merged == 'true' }}
+    uses: ./.github/workflows/publish.yml
+    # A called workflow cannot exceed its caller's grant.
+    permissions:
+      contents: read
+      statuses: write
+    with:
+      ref: prerelease
+      prerelease: true
+    secrets:
+      NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
+
+  # A backport that failed because an earlier change was missing from
+  # prerelease can apply once that change lands. After a sync merge, or a
+  # manual merge into prerelease, retry the oldest failed backport. Each
+  # successful retry triggers the next one; the chain stops at the first
+  # backport that fails again.
+  retry-failed-backport:
+    name: Retry the oldest failed backport
+    needs: [ sync-to-prerelease ]
+    runs-on: ubuntu-latest
+    permissions:
+      pull-requests: read # so it can list labeled pull requests
+    if: >
+      !cancelled() && (
+        needs.sync-to-prerelease.outputs.merged == 'true' || (
+          github.event_name == 'pull_request_target' &&
+          github.event.pull_request.merged &&
+          github.event.pull_request.base.ref == 'prerelease'
+        )
+      )
+    steps:
+      - name: Find the oldest PR labeled backport-failed
+        id: oldest
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          pr=$(gh pr list --repo "$GITHUB_REPOSITORY" --base main --state merged --label backport-failed --limit 1000 --json number,mergedAt --jq 'sort_by(.mergedAt) | .[0].number // empty')
+          echo "pr=$pr" >> "$GITHUB_OUTPUT"
+
+      - name: Re-run its backport with a comment
+        if: steps.oldest.outputs.pr != ''
+        env:
           # special fine-grained token for issue comment trigger
-          github-token: ${{ secrets.ISSUE_COMMENT_TRIGGER_PAT }}
-          script: |
-            await github.rest.issues.createComment({
-              owner: context.repo.owner,
-              repo: context.repo.repo,
-              issue_number: process.env.CREATED_PULL_NUMBERS,
-              body: "/deploy-preview",
-            });
+          GH_TOKEN: ${{ secrets.ISSUE_COMMENT_TRIGGER_PAT }}
+          RETRY_PR: ${{ steps.oldest.outputs.pr }}
+        run: gh api "repos/$GITHUB_REPOSITORY/issues/$RETRY_PR/comments" -f body="/sync-prerelease"