ci(repo): retry a failed device spec once and report the results on the pull request

The device job passes `--retries 1 --github-report`. A test that passes only on its retry is reported as flaky
and does not fail the job. The reporter writes the job summary, which replaces the step that copied e2e's
summary there, and on a pull request it posts one comment per platform and keeps it current. For that comment
the job gets `pull-requests: write`. The evidence upload gains `report.json` and `junit.xml`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Author: mikepitre

.claude/skills/verify-clerk-expo/SKILL.md

@@ -255,7 +255,7 @@ For a remote device, `down` ends the runner job and waits for it to finish, and
 - `npm test --prefix .claude/skills/verify-clerk-expo` runs the CLI's unit tests, with no network, key, or device. `npm run typecheck --prefix .claude/skills/verify-clerk-expo` runs `tsc`. The `Verify Skill Tests` job in `.github/workflows/ci.yml` runs both on Linux when a pull request changes the skill, the fixture, or a package the fixture links.
 - `src/core/`, `src/platform/ios/`, `src/platform/android/`, `specs/fixtures.ts`, `testing/`, and every test but `test/host.test.ts`, `test/remote-host.test.ts`, `test/freshness.test.ts`, and `test/native-build.test.ts` are shared with the clerk-ios and clerk-android verification skills. Change them there first, then copy them here, and never edit them here. `doctor`'s `core-drift` check fails when a file under `src/core/` differs from `src/core/MANIFEST`. Do not rewrite the manifest here to make it pass.
 - `src/host.ts`, `src/fixture.ts`, `src/native-build.ts`, and `src/freshness.ts` are this repository's own: the screens, the build of the fixture, the native build that CI keeps between runs, the Metro ports, and the check that Metro serves current JS. `specs/native.ts` holds the per-platform locators for the native views.
-- `.github/workflows/verify-device-specs.yml` runs `up --backend local`, `run --all`, and `down` on a runner for each platform, so the device is on the runner itself and no remote session starts. It runs for a pull request that is not a draft, and a draft gets a notice instead. The runners are GitHub-hosted, `macos-26` and `ubuntu-24.04`, and the repository variables `VERIFY_CI_RUNNER_IOS` and `VERIFY_CI_RUNNER_ANDROID` name other labels. It sets `VERIFY_LOCAL_BUILD=standalone`, which makes a local lease build the Release app with the JS embedded and start no Metro, and `VERIFY_NATIVE_CACHE`, which keeps the native build between runs ([freshness.md](references/freshness.md)). The Platform API key comes from the `MOBILE_VERIFICATION_PLATFORM_API_KEY` repository secret. It uploads each run's `run.json`, video, screenshots, and `e2e` summary and failure pages for three days, and only when the run is sealed with no secret found in it.
+- `.github/workflows/verify-device-specs.yml` runs `up --backend local`, `run --all --retries 1 --github-report`, and `down` on a runner for each platform, so the device is on the runner itself and no remote session starts. A test that fails and then passes on its one retry is `flaky` there and does not fail the job. Each job writes its results to the job summary. On a pull request it also posts one comment per platform and updates that comment on later runs, and for that the job's token may write pull request comments. It runs for a pull request that is not a draft, and a draft gets a notice instead. The runners are GitHub-hosted, `macos-26` and `ubuntu-24.04`, and the repository variables `VERIFY_CI_RUNNER_IOS` and `VERIFY_CI_RUNNER_ANDROID` name other labels. It sets `VERIFY_LOCAL_BUILD=standalone`, which makes a local lease build the Release app with the JS embedded and start no Metro, and `VERIFY_NATIVE_CACHE`, which keeps the native build between runs ([freshness.md](references/freshness.md)). The Platform API key comes from the `MOBILE_VERIFICATION_PLATFORM_API_KEY` repository secret. It uploads each run's `run.json`, video, screenshots, and e2e's `report.json`, `junit.xml`, summary, and failure pages for three days, and only when the run is sealed with no secret found in it.
 - `src/platform/session-device.ts` holds the steps that a remote session runs on its device, `.github/workflows/verify-remote.yml` is the workflow that the session runs in, and `src/core/remote/` is the code on both ends of it. The runner installs this skill with `npm ci` and uses the same `node_modules/.bin/agent-device` as a Mac does.
 - To call `agent-device` yourself, use `node_modules/.bin/agent-device` in the skill directory with `AGENT_DEVICE_STATE_DIR=.claude/skills/verify-clerk-expo/.verify/agent-device`. Each worktree runs its own daemon from there, and `down` stops it. Never print `.verify/agent-device/daemon.json`, because it holds the daemon's auth token.
 - `features/` is the feature map. Start with `features/README.md`, and update a feature file in the same PR as a change to the behavior it maps.

.github/workflows/verify-device-specs.yml

@@ -64,6 +64,7 @@ jobs:
     permissions:
       contents: read
       actions: read
+      pull-requests: write
     strategy:
       fail-fast: false
       matrix:
@@ -244,8 +245,9 @@ jobs:
         timeout-minutes: 30
         env:
           CLERK_PLATFORM_API_KEY: ${{ secrets.MOBILE_VERIFICATION_PLATFORM_API_KEY }}
+          GITHUB_TOKEN: ${{ github.token }}
         run: |
-          "$CLI" run --all --platform "$PLATFORM" --backend local
+          "$CLI" run --all --platform "$PLATFORM" --backend local --retries 1 --github-report
 
       - name: Release the device and delete the Clerk application
         if: always() && steps.install.outcome == 'success'
@@ -275,15 +277,6 @@ jobs:
             fs.appendFileSync(process.env.GITHUB_OUTPUT, "upload=" + upload + "\n");
           '
 
-      - name: Summarize the run
-        if: ${{ !cancelled() && steps.evidence.outputs.upload == 'true' }}
-        run: |
-          for summary in "$SKILL"/.verify/runs/*/e2e/summary.md "$SKILL"/.verify/runs/*/e2e-*/summary.md; do
-            if [ -f "$summary" ]; then
-              cat "$summary" >> "$GITHUB_STEP_SUMMARY"
-            fi
-          done
-
       - name: Upload the evidence
         if: ${{ !cancelled() && steps.evidence.outputs.upload == 'true' }}
         uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
@@ -295,6 +288,8 @@ jobs:
             ${{ env.SKILL }}/.verify/runs/*/run.json
             ${{ env.SKILL }}/.verify/runs/*/video.mp4
             ${{ env.SKILL }}/.verify/runs/*/screenshots
+            ${{ env.SKILL }}/.verify/runs/*/e2e*/report.json
+            ${{ env.SKILL }}/.verify/runs/*/e2e*/junit.xml
             ${{ env.SKILL }}/.verify/runs/*/e2e*/summary.md
             ${{ env.SKILL }}/.verify/runs/*/e2e*/failures
             ${{ env.SKILL }}/.verify/runs/*/e2e*/artifacts