Publish a daily cross-repository Babel milestones page as part of the dashboard site

.github/workflows/dashboard.yaml

@@ -1,15 +1,25 @@
-# Runs the full validation suite against every environment daily, then builds
-# and deploys the dashboard website to GitHub Pages.
+# Runs the full validation suite against every environment daily, refreshes the
+# cross-repository milestones data, then builds and deploys the dashboard
+# website to GitHub Pages.
+#
+# Three jobs, because the two data products have nothing to do with each other
+# and should not be able to take each other down: `validate` is 26 minutes of
+# pytest against six deployments, `milestones` is a handful of GitHub API calls.
+# They run in parallel, each uploading its own finished data file, and `publish`
+# assembles whatever arrived, carries forward the last published copy of
+# anything that did not, and makes the one and only write to gh-pages.
 name: Test dashboard
 
 on:
   schedule:
     - cron: '30 6 * * *'
   workflow_dispatch:
 
+# Read-only by default; only `publish` raises it. The two producer jobs run
+# untrusted input — sheet cells, issue bodies, whatever a deployment returns —
+# and neither has any business being able to push to gh-pages.
 permissions:
-  contents: write   # the deploy action pushes to the gh-pages branch
-  issues: read      # the GitHub issue tests read this repo's issues
+  contents: read
 
 concurrency:
   group: dashboard-deploy
@@ -19,13 +29,25 @@ concurrency:
   # never cancels anything — a cron run must not be able to kill a slower one that
   # is mid-flight, because that is the run that appends the day's history line —
   # and neither does a push, which queues behind whatever is deploying rather than
-  # racing it.
+  # racing it. The group is workflow-level, so all three jobs below live or die
+  # together with the run that started them.
   cancel-in-progress: ${{ github.event_name == 'workflow_dispatch' }}
 
 jobs:
-  test-and-deploy:
+  validate:
     runs-on: ubuntu-latest
     timeout-minutes: 350
+    permissions:
+      contents: read
+      issues: read    # the GitHub issue tests read this repo's issues
+    outputs:
+      # Set the moment the data files exist, which is several steps before this
+      # job may deliberately fail itself over a broken target. `publish` gates
+      # on this rather than on needs.validate.result for exactly that reason:
+      # "one of six targets broke" is an ordinary day here, and it must still
+      # publish the five that worked. Outputs set before a later step fails are
+      # preserved.
+      data-published: ${{ steps.generate-report.outputs.data-published }}
     steps:
       - uses: actions/checkout@v4
 
@@ -102,11 +124,14 @@ jobs:
             esac
           done
           # Recorded, not raised here: the report is still worth generating and
-          # publishing for the targets that did work. The last step turns this
-          # into a failed job, after the deploy.
+          # publishing for the targets that did work. The last step of this job
+          # turns it into a failure, after the data has been handed to `publish`.
           echo "BROKEN_TARGETS=$broken" >> "$GITHUB_ENV"
 
       - name: Fetch the previous run history
+        # For generate_report to append today's line to. Not the same thing as
+        # publish's carry-forward, which covers this job never getting here.
+        #
         # The published file is the source of truth: the deploy action
         # force-pushes gh-pages as a single commit, so the branch is not an
         # append log. The Pages CDN caches for ~10 minutes, so a manual
@@ -118,14 +143,142 @@ jobs:
             || : > old_history.jsonl
 
       - name: Generate the report
+        id: generate-report
         run: |
           uv run python -m src.babel_validation.tools.generate_report \
             --raw-dir raw --targets-ini tests/targets.ini \
             --history-in old_history.jsonl --out-dir website/public/data
+          # Steps run under `bash -e`, so this line is only reached when the
+          # generator exited 0 — it exits nonzero when every target was
+          # unreachable, which is a run with nothing worth publishing.
+          echo "data-published=true" >> "$GITHUB_OUTPUT"
+
+      - name: Upload the dashboard data
+        uses: actions/upload-artifact@v4
+        with:
+          name: dashboard-data
+          path: website/public/data
+          retention-days: 14
+
+      - name: Upload raw outcomes for debugging
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: raw-jsonl
+          path: raw/
+          retention-days: 14
+
+      - name: Fail if any target's run broke
+        # Last, so the data for the targets that worked has already been handed
+        # to `publish`, which is gated on the output above and not on this job's
+        # result. `all_targets_unreachable` in generate_report already stops a
+        # run where *everything* died; this catches the partial case, which
+        # otherwise renders as "environment unreachable" and reads as someone
+        # else's fault. Expect a red `validate` beside a green `publish` on
+        # those days: the site is up to date and a target still needs looking at.
+        if: env.BROKEN_TARGETS != ''
+        run: |
+          echo "These targets produced no usable results:${BROKEN_TARGETS}" >&2
+          echo "See the annotations above for each one's pytest exit code." >&2
+          exit 1
+
+  milestones:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    permissions:
+      contents: read
+      # For this repository's own milestones. The other four in targets.ini's
+      # Repositories list are public repositories in another organization, which
+      # any valid token reads regardless of what is declared here — that is the
+      # whole reason this needs no PAT. See docs/milestones-page.md.
+      issues: read
+    outputs:
+      data-published: ${{ steps.generate-milestones.outputs.data-published }}
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v6
+
+      - name: Generate the milestones data
+        id: generate-milestones
+        env:
+          GITHUB_TOKEN: ${{ github.token }}
+        run: |
+          uv run python -m src.babel_validation.tools.generate_milestones \
+            --targets-ini tests/targets.ini \
+            --output milestones-data/milestones.json
+          echo "data-published=true" >> "$GITHUB_OUTPUT"
+
+      - name: Upload the milestones data
+        uses: actions/upload-artifact@v4
+        with:
+          name: milestones-data
+          path: milestones-data
+          retention-days: 14
+
+  publish:
+    needs: [validate, milestones]
+    # always(), or GitHub skips this the moment either producer fails — and
+    # `validate` failing over one broken target is an ordinary day. The
+    # condition is on the jobs' outputs rather than their results for the same
+    # reason: what matters is whether usable data exists, not whether the job
+    # that made it went on to raise a finding about a deployment.
+    if: always() && (needs.validate.outputs.data-published == 'true' || needs.milestones.outputs.data-published == 'true')
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    permissions:
+      contents: write   # the deploy action pushes to the gh-pages branch
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Download the dashboard data
+        if: needs.validate.outputs.data-published == 'true'
+        continue-on-error: true   # the carry-forward below is the real backstop
+        uses: actions/download-artifact@v4
+        with:
+          name: dashboard-data
+          path: website/public/data
+
+      - name: Download the milestones data
+        if: needs.milestones.outputs.data-published == 'true'
+        continue-on-error: true
+        uses: actions/download-artifact@v4
+        with:
+          name: milestones-data
+          path: website/public/data
+
+      - name: Carry forward the last published copy of anything missing
+        # The deploy below force-pushes gh-pages as a single commit and cleans
+        # by default, so a file missing from website/dist is a file *deleted*
+        # from the live site. One producer failing would therefore take down a
+        # page that has nothing to do with it.
+        #
+        # So anything this run did not produce is refetched from the live site
+        # and republished unchanged. Each page renders its own generated_at, so
+        # a carried-forward file reads as stale rather than as current. If even
+        # the live copy is gone — the first run of a new data file — the page
+        # falls back to its own load-error state, which is why every component
+        # has one; the build does not read these files, so nothing else breaks.
+        run: |
+          mkdir -p website/public/data
+          for name in report.json history.jsonl milestones.json; do
+            path="website/public/data/$name"
+            # An explicit `if`, not `[ -s "$path" ] && continue`: steps run under
+            # `bash -e`, where a bare test that fails is a coin flip on whether
+            # the step aborts. The whole point of this step is to survive things
+            # being missing.
+            if [ -s "$path" ]; then
+              continue
+            fi
+            url="https://translatorsri.github.io/babel-validation/data/$name"
+            echo "::warning title=Carrying forward stale data::$name was not regenerated by this run; refetching the published copy"
+            curl -fsSL -o "$path" "$url" \
+              || echo "::warning title=Nothing to carry forward::$url is unavailable too; $name will be missing from the site"
+          done
 
       # Node 24 for npm 11 — see the note in tests.yaml. Without this the job
-      # gets the runner's default Node and fails at npm ci, four hours into the
-      # run, after every test has already been paid for.
+      # gets the runner's default Node and fails at npm ci.
       - uses: actions/setup-node@v4
         with:
           node-version: 24
@@ -143,22 +296,3 @@ jobs:
         uses: JamesIves/github-pages-deploy-action@v4
         with:
           folder: website/dist
-
-      - name: Fail if any target's run broke
-        # Last, so the report for the targets that worked is already published.
-        # `all_targets_unreachable` in generate_report already stops a run where
-        # *everything* died; this catches the partial case, which otherwise
-        # renders as "environment unreachable" and reads as someone else's fault.
-        if: env.BROKEN_TARGETS != ''
-        run: |
-          echo "These targets produced no usable results:${BROKEN_TARGETS}" >&2
-          echo "See the annotations above for each one's pytest exit code." >&2
-          exit 1
-
-      - name: Upload raw outcomes for debugging
-        if: always()
-        uses: actions/upload-artifact@v4
-        with:
-          name: raw-jsonl
-          path: raw/
-          retention-days: 14

CLAUDE.md

@@ -65,6 +65,19 @@ Beware: the root `.gitignore`'s Python-template `lib/` pattern matches *any* dir
 `lib`, including under `website/src/` — a file there builds locally but never reaches CI.
 Check `git status` shows new frontend files as tracked.
 
+**Exactly one workflow writes `gh-pages`, and it must stay that way.** `dashboard.yaml` deploys
+`website/dist` to the *root* of that branch with `github-pages-deploy-action`, which cleans by
+default — it force-pushes the branch as a single commit. So a second publisher does not coexist
+with it: whichever ran last deletes the other's output, and a `clean-exclude` plus a shared
+`concurrency` group is two things to keep in step forever. This was a real near-miss — the
+milestones page was originally published by its own workflow into `gh-pages:milestones/`, which
+the dashboard's deploy would have silently deleted on the next daily run.
+
+A new page therefore joins the existing pipeline rather than adding a publisher: a generator
+writing one more JSON file into `website/public/data/`, an Astro route mounting a Vue island, and
+a job in `dashboard.yaml` feeding the one deploy. See [docs/milestones-page.md](docs/milestones-page.md)
+for the worked example.
+
 ## Architecture
 
 ### Library (`src/babel_validation/`)
@@ -98,16 +111,31 @@ the rows into `TestRow` dataclasses. Rows marked as not expected to pass are wra
 ### Dashboard Website
 
 - **`website/`** — Astro + Vue site deployed to GitHub Pages
-  (https://translatorsri.github.io/babel-validation/). Three pages under one `Layout.astro`
+  (https://translatorsri.github.io/babel-validation/). Four pages under one `Layout.astro`
   shell (nav bar, cards on a tinted page, `data-bs-theme` dark mode, all custom CSS in
   `src/styles/theme.css`): `/` renders the environment cards, the promotion-drift panel and
   the `/status` matrix from `report.json`; `/results/` renders the tests-by-environment
   matrix behind a sticky filter bar; `/history/` renders `history.jsonl` plus a diff against
-  the previous run. Everything about the report that is not markup — link builders, labels,
-  the interestingness predicate — lives in `src/reportData.js`. Regenerated daily by `.github/workflows/dashboard.yaml`: pytest per target with
-  `--report-jsonl` (a `pytest_runtest_logreport` hook in `tests/conftest.py`), then
-  `src/babel_validation/tools/generate_report.py` aggregates the raw outcomes, fetches each
-  target's `/status`, and writes both data files into `website/public/data/`.
+  the previous run; `/milestones/` renders `milestones.json`. Each is a thin `.astro` route
+  mounting one Vue island with `client:only="vue"`. Everything about the report that is not
+  markup — link builders, labels, the interestingness predicate — lives in `src/reportData.js`.
+- **`.github/workflows/dashboard.yaml`** regenerates all three data files daily and makes the
+  only write to `gh-pages`. Three jobs: `validate` runs pytest per target with `--report-jsonl`
+  (a `pytest_runtest_logreport` hook in `tests/conftest.py`) and then
+  `src/babel_validation/tools/generate_report.py`, which aggregates the raw outcomes and fetches
+  each target's `/status`; `milestones` runs
+  `src/babel_validation/tools/generate_milestones.py` against the GitHub API, in parallel and
+  independently, so the milestones page does not inherit a 26-minute test run's failure modes;
+  `publish` assembles both, carries forward the last published copy of anything this run did not
+  produce, builds and deploys.
+
+  Two things there are load-bearing and easy to undo. **`publish` gates on job outputs, not
+  `needs.<job>.result`**: `validate` deliberately fails itself when a target's run broke, which
+  happens on ordinary days, so a result-based condition would skip a report that generated
+  perfectly well. And **the deploy cleans the branch**, so a data file missing from
+  `website/dist` is a file deleted from the live site — which is why `publish` refetches rather
+  than publishing a gap, and why nothing else may write `gh-pages`. See
+  [docs/milestones-page.md](docs/milestones-page.md).
 - **`scala-validation/`** — Legacy, unmaintained
 
 ## Untrusted Input

README.md

@@ -139,19 +139,26 @@ database sizes, NameRes latency). Because test expectations are pinned to the en
 where a new Babel version lands first, environments are not expected to all be green — the
 dashboard's purpose is to show *which* issues are visible in *which* environment.
 
-The `.github/workflows/dashboard.yaml` workflow regenerates and deploys it daily (or on
-manual dispatch): it runs pytest per target with `--report-jsonl`, turns the raw outcomes
-and `/status` responses into `report.json` and `history.jsonl` with
-`src.babel_validation.tools.generate_report`, and publishes the built site to the
-`gh-pages` branch.
+It also carries a `/milestones/` page: every open milestone across the Babel repositories in
+one chronological list, which GitHub cannot show because milestones do not span repositories.
+
+The `.github/workflows/dashboard.yaml` workflow regenerates and deploys all of it daily (or on
+manual dispatch), in three jobs. `validate` runs pytest per target with `--report-jsonl` and
+turns the raw outcomes and `/status` responses into `report.json` and `history.jsonl` with
+`src.babel_validation.tools.generate_report`. `milestones` reads the GitHub API into
+`milestones.json` with `src.babel_validation.tools.generate_milestones`, in parallel, so a page
+that takes seconds to build does not wait on a 26-minute test run or fail with it. `publish`
+builds the site from whichever data files arrived — refetching the last published copy of any
+that did not, since the deploy replaces the whole branch — and makes the only write to
+`gh-pages`. Nothing else may publish there.
 
 The Vue components' client-side logic (URL round-tripping, filtering, pagination, the
 odd-one-out shading, the drift grouping, and the withholding of blocklist detail) has vitest
 tests in `website/test/`, run by `npm test` and by the Tests workflow.
 
 To work on the site against the data the live dashboard is showing, download the published
-`report.json` and `history.jsonl` instead of generating them (both land in
-`website/public/data/`, which is gitignored):
+`report.json`, `history.jsonl` and `milestones.json` instead of generating them (all three land
+in `website/public/data/`, which is gitignored):
 
 ```shell
 $ cd website && npm install && npm run fetch-data && npm run dev
@@ -167,6 +174,14 @@ $ uv run python -m src.babel_validation.tools.generate_report --raw-dir raw \
 $ cd website && npm install && npm run dev
 ```
 
+The milestones page needs no test run, only a token — `GITHUB_TOKEN` must be set, and reading
+these five public repositories needs no scopes beyond the default:
+
+```shell
+$ GITHUB_TOKEN=$(gh auth token) uv run python -m src.babel_validation.tools.generate_milestones \
+      --targets-ini tests/targets.ini --output website/public/data/milestones.json
+```
+
 ## The Babel Validator in Scala
 
 An initial version of the Babel Validator was written in Scala, but this is no longer being maintained.