Skip to content

Replace the GitHub Pages site with a daily cross-environment test dashboard - #118

Merged
gaurav merged 20 commits into
mainfrom
replace-website
Sep 1, 2026
Merged

Replace the GitHub Pages site with a daily cross-environment test dashboard#118
gaurav merged 20 commits into
mainfrom
replace-website

Conversation

@gaurav

@gaurav gaurav commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Important

Superseded by #120, where development continues. That PR redesigns this dashboard from scratch — a nav bar and page shell, three pages instead of one, a sticky filter bar, and a promotion-drift panel — and it carries every commit below.

This PR is kept open, as a draft, only so that anything worth saving can be checked against it once #120 has merged. The pre-merge push: trigger for the dashboard workflow has moved to #120's branch, so pushes here no longer run or deploy anything. Do not add work to this branch.

The Prefix Comparator has moved into Babel (NCATSTranslator/Babel#889) and Autocomplete is moving into Babel Explorer (TranslatorSRI/babel-explorer#7), so the old site's tools are retired. In their place, https://translatorsri.github.io/babel-validation/ becomes a dashboard showing the full test suite run daily against every environment. Because test expectations are pinned to the environment where a new Babel version lands first, environments are not expected to all be green — the dashboard's purpose is to show which issues are visible in which environment.

What shipped

  • Outcome capture: pytest --report-jsonl PATH writes one JSON record per test (outcome, wasxfail, concise crash message, record_property metadata) from a pytest_runtest_logreport hook — xdist-safe, no new dependencies. The file is truncated per run and its parent directory created, so a local re-run reports the run you just did rather than merging with the last one. The Google Sheet tests record category, source, and query metadata structurally.
  • Report generator (src/babel_validation/tools/generate_report.py): aggregates raw outcomes per test (worst outcome wins across phases and subtests, strict XPASS classified as xpassed), merges each row's metadata across every environment that recorded it, fetches each environment's /status endpoint, and writes report.json (latest run, full detail) plus history.jsonl (one compact summary line per run, for trends). A malformed record, a history line that is not a run object, or a targets.ini section that defines only one of the two service URLs degrades that one value rather than aborting the report.
  • Dashboard website (Astro + Vue, replacing the old pages): a status-by-environment table in deployment order (exp → dev → ci → ci-es → test → prod) with difference highlighting; a paginated tests-by-environment matrix defaulting to "interesting only" rows (failing, unexpectedly passing, or differing across environments); a collapsible filter panel whose state — filters, page, page size and the expanded test — lives in the URL and is restored intact by a shared link, with a "Copy link to this view" button; links out to GitHub issues, direct NodeNorm/NameRes queries, and Babel Explorer; and a trends page over the run history.
  • Daily workflow (.github/workflows/dashboard.yaml): cron + workflow_dispatch; one 45-minute-capped pytest run per target (a failing suite is normal — the report is the artifact), then generate, build, and deploy to gh-pages. The release-triggered deploy workflow is deleted — it would publish a site with stale or missing data.
  • Local development and tests: npm run fetch-data downloads the live site's report.json and history.jsonl into website/public/data/, so frontend work does not need a local suite run first (generating them from scratch is still documented in README.md, "The dashboard website"). npm test runs vitest over the components' URL round-tripping, filtering and pagination — logic the Python suite cannot see — and the Tests workflow runs it alongside the Python unit tests.
  • Deleted: website-vue3-vite/ (never deployed) and the Autocomplete/Prefix Comparator pages and their dependencies.

Security

Everything feeding the report is untrusted (issue bodies, sheet cells, service responses) and the site is public. The generator is the choke point: text is repr-escaped and truncated, /status responses pass a key whitelist, issue ids and source URLs become links only when they match the targets.ini Repositories allowlist, blocklist details are withheld entirely, and the Google Sheet IDs come from environment variables and never appear in the output (guarded by a unit test). The Vue components render report values via {{ }} interpolation only and construct links from validated parts — including the test key from the URL, which is looked up with Object.hasOwn so a crafted ?test= cannot pull a member of Object.prototype into the table.

Notes for review

  • Includes the two sheet-ID commits also cherry-picked into Read the Google Sheet IDs from the environment instead of checking them in #117; whichever merges second will need a trivial rebase.
  • tests/unit/test_generate_report.py covers the generator, including its untrusted-input and degradation paths; website/test/dashboard.test.js covers the components' URL and pagination logic. Both run in CI on every pull request.

Before merge (blocking)

Future work

Everything still open below has been carried into #120's follow-up list — work from that one. This list is kept only as the record of what this PR knew about.

Review history — kept for anyone tracing why a particular line looks the way it does; the durable conclusions are in the code, the tests and CLAUDE.md above.

A code review of the branch found nine issues, fixed in four commits:

  • Shared links did not survive (8029211): Vue registers watchers before created(), so the watchers saw readUrl()'s assignments as user edits — the filters handler reset page to 1 and rewrote the URL, and a link carrying ?page=3 always landed on page 1. The pinned-row lookup also indexed report.results raw, so ?test=constructor pinned an Object.prototype member and blanked the table. The trends page's ../ back-link resolved above <base href="/babel-validation/"> to the site root.
  • Generator robustness (95b12c6): fetch_status called .rstrip() on a URL targets.ini need not define, read_raw_records passed a non-string id to parse_nodeid, and append_history checked only that prior lines were valid JSON, not that they were run objects — each aborted the whole report. Result annotation also ran only for the first environment seen, so a row whose first environment failed during setup (before record_property runs) lost its category and service links even though the others had recorded them.
  • --report-jsonl ergonomics (9de751f): the README's own example died in pytest_configure because the parent directory did not exist, and appending meant a local re-run kept a fixed test red under worst-outcome-wins. CI never noticed either, since it writes one fresh file per target.
  • Workflow trigger (ac880f5): removed, as the blocking box above required.

Two items from the future-work list were then pulled into this PR rather than deferred. A run where no target recorded a single test is now dropped from history on the way in and not written on the way out (b23df3b), which retires the all-zero shakedown row on the live site at the next daily run instead of by hand. And the frontend gained a vitest harness (ab55b78): four of its six tests fail against the components as they stood before the review round above.

Earlier rounds of the branch fixed the xdist option-loading trap, the root .gitignore swallowing website/src/lib/, and node-ID key normalization; those traps are written up in CLAUDE.md.

🤖 Generated with Claude Code

gaurav and others added 20 commits August 26, 2026 18:57
pytest --report-jsonl PATH appends one JSON line per test (nodeid, phase,
outcome, wasxfail, duration, user_properties, concise crash message) from a
pytest_runtest_logreport hook. Only the xdist controller writes: workers
forward their TestReports, so the guard on PYTEST_XDIST_WORKER prevents
duplicate lines without any file locking.

The two Google Sheet test functions now record category, source, source_url,
query_id and query_label via record_property, so the dashboard gets that
metadata structurally instead of regexing failure messages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
src.babel_validation.tools.generate_report aggregates the --report-jsonl
records per test (worst outcome wins across phases and subtests), classifies
them into passed/failed/xfailed/xpassed/skipped/error (strict XPASS included),
splits the target out of parametrize ids (either end, longest name first so
ci-es beats ci), and merges in each target's NodeNorm/NameRes /status.

report.json is published on a public website and everything feeding it is
untrusted, so this module is the choke point: text is repr-escaped and
truncated, issue ids and source URLs only become links when they match the
targets.ini Repositories allowlist, /status responses pass a key whitelist,
and blocklist test details are withheld entirely.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Prefix Comparator has moved into Babel (NCATSTranslator/Babel#889) and
Autocomplete is moving into Babel Explorer (TranslatorSRI/babel-explorer#7),
so both pages are gone, along with the never-deployed website-vue3-vite app
that duplicated the Google Sheet test logic in the browser.

The Astro site now renders data/report.json and data/history.jsonl (produced
by generate_report): per-environment status cards (Babel/NameRes versions,
database sizes, NameRes p95 latency, outcome counts), a tests-by-environment
matrix defaulting to 'interesting only' rows (failing, unexpectedly passing,
or differing across environments) with links out to the Google Sheet row, the
GitHub issue, direct NodeNorm/NameRes queries and Babel Explorer, and a
trends page listing one summary row per daily run.

All report text is rendered via {{ }} interpolation only, and links are
constructed from generator-validated parts, since the report is derived from
untrusted input. Dependencies used only by the deleted tools are dropped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The new dashboard.yaml workflow (daily cron plus workflow_dispatch) runs the
full test suite against every target in targets.ini except localhost — one
sequential pytest run per target, capped at 45 minutes each so a hung
environment cannot sink the job, with a nonzero exit treated as normal since
the report is the artifact — then generates report.json/history.jsonl and
deploys the built site to gh-pages. The previous run's history is fetched
from the live site, because the deploy action force-pushes gh-pages as a
single commit.

The release-triggered deploy workflow is gone: it would publish a site with
stale or missing data, and workflow_dispatch covers manual redeploys.

README.md and CLAUDE.md now describe the dashboard, its generation flow, and
the untrusted-input rules its generator and Vue components must follow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ecking it in

The sheet is shared as 'anyone with the link' because the CSV export is
fetched unauthenticated, so the ID is the capability that grants access.
GoogleSheetTestCases() now resolves it from the BABEL_VALIDATION_SHEET_ID
environment variable (loaded from .env locally; a repository secret of the
same name in the dashboard workflow), fails loudly when it is missing, and
rejects values that do not look like a sheet ID before they reach a URL.
Its __str__ no longer embeds the ID, since that string is interpolated into
assertion messages the dashboard publishes.

Once the sheet is re-shared under a new ID, the old ID in the public Git
history stops working. The leak-guard unit test now plants a fake ID in the
environment and checks the generator cannot surface it in report.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The blocklist sheet logs Red Team offensive terms, so its ID is even more of
a secret than the test-case sheet's. Both sheets now resolve their IDs
through a shared resolve_sheet_id() helper: the environment variable
(BABEL_VALIDATION_SHEET_ID / BABEL_VALIDATION_BLOCKLIST_SHEET_ID, loaded
from .env locally, repository secrets in the dashboard workflow), failing
loudly on a missing or implausible value.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…l, pagination, shareable links

The per-environment status cards become one transposed table with
environments as columns in deployment order (exp, dev, ci, ci-es, test,
prod) and one row per status value, so a version working its way towards
prod reads left to right and the odd environment out is highlighted
(table-warning on cells that differ from the majority). Test-outcome counts
follow as rows in the same table, and adding a status row is now one entry
in STATUS_ROWS. The results matrix and the trends page use the same column
order.

The always-visible 'interesting only' checkbox — one accidental click away
from rendering thousands of rows — moves into a collapsible Filters panel
alongside text search, test-source and has-outcome filters, and the matrix
is paginated (100 rows per page by default, selectable) instead of rendered
in full.

Filter state, page, and the expanded test live in the URL query string, so
the address bar always reproduces the current view; a 'Copy link to this
view' button copies it, and a shared link to a test hidden by the current
filters pins that test at the top rather than showing nothing. Query
parameters are untrusted input: they feed only string filters and key
lookups, never markup or fetch targets.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
workflow_dispatch only works once the file is on the default branch; this
push trigger is for pre-merge test runs and comes out before merge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pytest rejects '=' inside a -k expression, so the documented -k "row=42"
never worked; select the full node ID instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…allows

The Python-template 'lib/' pattern in the root .gitignore matched
website/src/lib/, so deploymentOrder.js was never committed and the CI
build failed with 'Could not resolve' while local builds — with the file
present on disk — passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Without a path argument, pytest-xdist workers do not load tests/conftest.py
early enough to register --target/--report-jsonl, and every worker dies at
argparse ('unrecognized arguments') — which is exactly what happened on the
first dashboard run: six 'successful' pytest invocations, zero results.

The generator now refuses to succeed when every target is unreachable, so a
broken run stops the workflow before the deploy step instead of publishing
an empty dashboard. The report files are still written for debugging.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…st tests'

Node IDs are rootdir-relative: 'pytest tests --target dev' (the workflow's
invocation) yields 'tests/github_issues/...' where 'pytest tests/nodenorm/...'
yields 'nodenorm/...'. The generator's kind checks and the Dashboard's
nodenorm//nameres/ prefix checks assumed the bare form, so the first real
workflow run reported github_issues_ran=false despite 96 issues having run,
and every service link would have been missing. Keys now have the tests/
prefix stripped, verified by regenerating the report from that run's actual
raw artifact.

Also: only github_issues/test_github_issues.py counts towards
github_issues_ran (the github_issues/unit/ parser tests run without a
token), and the workflow deselects '-m unit' — the per-target runs were
repeating the unit suite six times into the report's unattributed bucket.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both cost a broken CI run this session: pytest -n without an explicit tests
path leaves workers ignorant of the conftest options, and the root
.gitignore's Python-template 'lib/' pattern silently swallows
website/src/lib/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The trends page sits under `<base href="/babel-validation/">`, so its
`../` back-link resolved to the site root rather than the dashboard.

Dashboard.vue's watchers are registered before created(), so they saw
readUrl()'s assignments as edits: the filters handler reset page to 1 and
rewrote the URL, and a link with ?page=3 always landed on page 1. Ignore
watcher fire-ups until the initial read has flushed.

Also index the pinned-row lookup with Object.hasOwn: ?test=constructor
otherwise pinned an Object.prototype member and blanked the table. And
drop non-object history lines in Trends.vue, which the template indexes
as run.targets[target].

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fetch_status() called .rstrip() on a URL that targets.ini need not
define, read_raw_records() passed a non-string id to parse_nodeid, and
append_history() only checked that prior lines were valid JSON, not that
they were objects with targets — each aborted the whole report rather
than the one value.

_annotate_result() also ran only for the first target seen, so a row
whose first target failed during setup — before record_property runs —
lost its category and service links even though the other targets had
recorded them. Merge props across every target.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README's own example (--report-jsonl raw/local.jsonl, no mkdir) died
in pytest_configure before collecting anything. Appending was also wrong
for a local re-run: build_results takes the worst outcome per test, so
stale records kept a fixed test red. CI writes one fresh file per target,
so it never noticed either.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It was only there so the job could be exercised before the file reached
the default branch. Left in, every push to the branch would start a
~4.5h run that force-pushes gh-pages, racing the daily cron.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Frontend work needs a report.json and a history.jsonl, and generating
them locally means running the suite against at least one environment
first. The live site already publishes both, so download them into
website/public/data/ instead — the directory the dev server reads, and
one the root .gitignore's data/ rule already covers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The shakedown run died before pytest reported anything, leaving a row of
real /status values and all-zero counts at the top of the live
history.jsonl. Every run copies the prior file forward verbatim, so that
row would have sat in the trends table forever.

A run where no target recorded a single test is a broken run rather than
a data point, so drop it on the way in and decline to write one on the
way out. The live file heals itself on the next daily run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three frontend bugs the last review round found — a shared link
losing its page, a ?test= key reaching Object.prototype, a history line
that is not an object blanking the trends table — were all invisible to
the Python suite, which never loads a component.

`npm test` mounts Dashboard and Trends against a stub report and asserts
what a shared link restores. Four of the six tests fail against the
components as they stood before that review. The Tests workflow runs
them alongside the Python unit tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gaurav gaurav added this to the Babel Validation v1.0 milestone Aug 27, 2026
gaurav added a commit that referenced this pull request Aug 27, 2026
workflow_dispatch only works once the file is on the default branch, so a
pre-merge run has to trigger on push. #118 carried that trigger and is
now kept only for reference, so it moves here — the redesign is what
needs deploying for review.

Only one branch should have it at a time: each run takes about four hours
and force-pushes gh-pages. Remove it before merging, as #118 did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gaurav
gaurav marked this pull request as draft August 27, 2026 22:59
Base automatically changed from fix-open-issues to main August 31, 2026 13:20
gaurav added a commit that referenced this pull request Sep 1, 2026
…lt around promotion drift (#120)

The GitHub Pages site was an earlier, weaker version of what is now
[Babel Explorer](https://github.com/TranslatorSRI/babel-explorer). This
replaces it with a dashboard that runs the validation suite against
every environment daily and publishes the result, organised around the
question the data says actually matters: **where in the promotion
pipeline did this appear?**

Closes #113. Supersedes #118 (left as a draft until this merges) and
#117, which is closed: its four code files were byte-identical to these
and its prose is a subset of what is here.

## What's here

**The old site is gone.** The Prefix Comparator moved into Babel
(NCATSTranslator/Babel#889) and Autocomplete into Babel Explorer
(TranslatorSRI/babel-explorer#7), so both pages are deleted, along with
`website-vue3-vite/` — a never-deployed app that duplicated the Google
Sheet test logic in the browser (44 files, ~10k lines).

**The suite emits machine-readable results.** `pytest --report-jsonl` (a
`pytest_runtest_logreport` hook in `tests/conftest.py`) writes one JSON
line per test phase — raw pytest facts only.
`src/babel_validation/tools/generate_report.py` turns those into
`report.json` and appends one summary line to `history.jsonl`; all
classification into passed/failed/xfailed/xpassed/skipped/error happens
there, where it is unit-testable, rather than in the fixture that
produced it. A raw file is treated as damageable: one unparseable or
malformed record costs that record and nothing else. A result whose node
ID names no environment is counted as unattributed and published as a
count, never as a table row, because the site has no column to render it
in.

**A daily workflow runs it** at 06:30 UTC: the suite against every
environment, then the report, the site build and a deploy to `gh-pages`.
Three things about that loop are worth knowing, because each was a bug
first:

- **The list of environments comes from `read_targets()`**, not from a
second copy in the YAML. When those disagreed, the extra target reached
the site as a permanently unreachable column, sorted after prod, and as
an always-empty position in every `?sig=` signature — which silently
invalidates previously shared links.
- **A failing test run and a broken one are told apart.** Failing tests
are the artifact this workflow exists to publish, so exit 0 or 1 passes
silently — but only when that target actually wrote results, because the
command is `uv run pytest` and uv exits 1 for its own failures too,
before pytest starts. `timeout` firing (124) is annotated as a warning,
because the environment hung and the report already says so; anything
else — a collection error, a usage error, nothing collected — is an
error annotation. Every broken target fails the job in a final step
*after* the deploy, so the targets that did work still publish.
`generate_report` already refused to publish when *every* target was
unreachable; this is the partial case, which is the one that looks fine.
- **Only a manual dispatch cancels a run in progress.** A blanket
`cancel-in-progress` would let a push, or the next day's cron, kill a
scheduled run mid-flight — and that run is what appends the day's line
to `history.jsonl`.

**Three pages, not one.** *Dashboard* (run banner, per-environment
pipeline cards, promotion-drift panel, environment detail matrix),
*Results* (a sticky filter bar over the test matrix, filterable by
category, source and environment, with shareable URL state), and
*History* (what changed since the previous run, then a row per run).

**The design follows the data.** Measured against the 2026-09-01 report:
of **4,584** results, **528** are interesting, and **519 of those differ
across environments** — only 9 fail everywhere. Failures per environment
in promotion order run exp 21 → dev 79 → ci 81 → ci-es 222 → test 80 →
prod 327, and the top outcome patterns are `pass ×5 FAIL` (148), `pass
pass pass FAIL pass FAIL` (100) and `xfail ×5 XPASS` (67). Drift
dominates the file, and it used to be hidden behind a checkbox labelled
"interesting only". The redesign promotes it to structure.

**Bootstrap 5.3 with a small theme layer, not PrimeVue.** A component
library only reaches inside the Vue islands, the matrix is a pivot we
would hand-write regardless, and the default view is ~530 rows, so
virtual scroll buys nothing.

**The report is untrusted input, and is treated as such.** No `v-html`
anywhere; report values render only through `{{ }}`; links are built
from generator-validated parts; blocklist rows withhold their detail in
the expanded-row markup as well as the label; no Google Sheet ID or link
appears anywhere in the output. A facet is as public as a cell, so the
filter dropdowns exclude blocklist rows exactly as the table does. The
URL parameters are treated the same way: they are read before the report
loads, so there is nothing to check them against, and every one of them
goes through `Object.hasOwn` rather than indexing a JSON-parsed object
directly — a rule now in `CLAUDE.md`, because it has been two bugs in
the same file.

**The Google Sheet IDs are out of the repository, and have been
rotated.** They live in `BABEL_VALIDATION_SHEET_ID` and
`BABEL_VALIDATION_BLOCKLIST_SHEET_ID` (`.env` locally, repository
secrets in Actions), resolved through `resolve_sheet_id()`. This matters
more than it looks: the sheets are shared as "anyone with the link"
because the CSV export is unauthenticated, so the ID *is* the credential
— and the old one is checked into `main` and sits in this repository's
public history, where `git rm` cannot reach it. Both sheets have been
restricted, both old IDs now return 401 credential-free on the `gviz`
CSV export and on `/pub?output=csv`, and both secrets hold new IDs as of
2026-09-01 (#126). A new `env.default` documents every variable and what
each one turns off when missing, so `cp env.default .env` is the whole
setup, and `CLAUDE.md` now tells coding agents not to read `.env` back —
everything an agent reads lands in a transcript that is stored, replayed
and pasted into issues.

## What it produces

A run takes **19–36 minutes**, almost all of it the six sequential
per-target pytest invocations; the report, build and deploy together are
about 30 seconds. The most recent run published 4,152 Google Sheet
results, 313 blocklist results and 96 GitHub issue results across six
environments.

Tests: **46 vitest** specs across `results`, `history`, `driftPanel`,
`statusMatrix`, `overview` and `reportData`, run in CI beside the Python
unit tests, which are now **192**.

## What it deliberately does not do

- **No NameRes ES in Test.** Test matches CI — NodeNorm ES against the
Solr-backed NameRes — because NameRes ES is being validated in CI
(`[ci-es]`) first. The Redis-backed `nodenorm.test.transltr.io` was
switched off on 2026-08-31, so there is no Redis NodeNorm left in Test
to compare against; dev, exp and prod still run one.
- **No sparklines on History** until there are enough runs to plot
(~14); marked with a `ponytail:` note in `History.vue`.
- **The six per-target runs stay sequential.** A matrix job would cut
the wall clock from ~26 minutes to roughly 7, but it restructures the
job graph and the `gh-pages` deploy, so it is #122 rather than another
commit here.
- **`DEPLOYMENT_ORDER` stays hardcoded.** Promotion order is a semantic
fact about the pipeline that no config file states, so unlike the
workflow's target list it cannot be derived from `targets.ini`. What it
no longer does is fail quietly: an unrecognised environment used to sort
last, which is indistinguishable from being the end of the pipeline, so
the Dashboard now names it instead.

## Before merging

Nothing outstanding. All four items that were blocking are done: the
temporary `push:` trigger is removed, #117 is resolved, the sheet IDs
and repository secrets are rotated, and the dashboard has been run once
against the new secrets — [run
33469165720](https://github.com/TranslatorSRI/babel-validation/actions/runs/33469165720),
whose published report carries 4,152 sheet rows and 313 blocklist rows,
which is the evidence that both new IDs resolved. A green tick alone
would not have shown that: a nonzero pytest exit is normal here.

One thing to watch on the first run after merge: `workflow_dispatch`
only starts working once `dashboard.yaml` is on the default branch, so
the exit-code triage above has been verified by running the step's
script under `bash -e` against stubbed exits, but has never executed on
a runner. That applies to the empty-results check as well.

## Follow-on work

Filed, on the **Babel Validation v1.0** milestone:

- #122 — run the six per-environment test runs as a matrix job (~26
minutes → ~7).
- #123 — a shared `?sig=` link silently matches nothing once the
environment list changes, because the signature is positional.
- #124 — the root README's example pytest transcripts are stale
(`configfile: pytest.ini`, old tool versions).
- #125 — the run history can be silently truncated: each run rebuilds
`history.jsonl` from the *published* copy fetched over HTTP, so one
failed or CDN-stale fetch publishes a one-line file, and the deploy
replaces the branch rather than appending to it.

🤖 Generated with Claude Code
@gaurav
gaurav merged commit da0b68d into main Sep 1, 2026
1 of 2 checks passed
@gaurav
gaurav deleted the replace-website branch September 1, 2026 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant