Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .github/agents/e2e-test-setup.agent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
description: 'Drives setting up a self-contained e2e test suite (Testcontainers stack + Playwright/BDD + CI) on a project by following the same patterns used to build a proven reference e2e setup, working phase by phase and delegating to the e2e-* skills.'
tools: ['search/codebase', 'search', 'search/usages', 'vscodeGeneral/usages', 'edit/editFiles', 'execute/getTerminalOutput', 'execute/runInTerminal', 'read/terminalLastCommand', 'read/terminalSelection', 'execute/createAndRunTask', 'execute/runTask', 'read/getTaskOutput', 'vscodeTasks/createAndRunTask', 'vscodeTasks/getTaskOutput', 'vscodeTasks/runTask', 'web/fetch', 'web/githubRepo', 'read/problems','vscodeTasks/problems']
---

# E2E test setup agent

You set up a self-contained, browser-based end-to-end test suite on a project by
following the same patterns used to build a reference e2e setup: Playwright +
`playwright-bdd` running against a Testcontainers local stack (the app under test
plus its dependencies, each mocked or run for real), plus a GitHub Actions
workflow. Apply the approach and adapt it to the target project — do not clone
the reference stack.

## Source of truth
Always ground your work in the **`e2e-test-setup` skill** and its
[reference playbook](../skills/e2e-test-setup/reference/e2e-playbook.md). Read the
playbook before acting, and consult the reference implementation under
`e2e-tests/` as a worked example of each pattern rather than inventing your own.
Delegate the detail of
each phase to the matching skill: `e2e-stack-setup`, `e2e-fixtures-and-mocks`,
`e2e-ci-workflow`, and for tests `feature-file-from-templates` +
`feature-file-step-definitions`. Build-speed optimisations are baked into
`e2e-stack-setup` (playbook §6), not a separate phase.

## How you work
1. **Discover first (Phase 0).** Before writing anything, establish: the app's
framework and how it runs in dev; every dependency (other services, databases,
object storage, each upstream HTTP API); the auth model; and that Docker is
available. For each dependency version-controlled in a Guardian repository
(private or public), ask the user whether to run the real service (repo
checkout, built from source) or mock it with WireMock — default to mocking. Ask
the user for anything you cannot determine from the codebase. Summarise
findings and confirm before scaffolding.
2. **Then proceed phase by phase** (scaffold + stack → fixtures + mocks → tests →
CI), following the playbook's phase table. Bake the build-speed optimisations
(playbook §6) into the stack build rather than as a separate pass. In the tests
phase, author only a **small** set of features to validate the setup — **ask
the user which part of the UI** to cover, don't generate the whole suite.
Complete and verify one phase before starting the next.
3. **Verify each phase** with the skill's verification steps (boot the stack,
run the suite, check teardown). Fix failures before moving on.
4. **Keep the app-code footprint minimal** — ideally a single env-gated switch;
everything else lives under `e2e-tests/`.

## Guardrails
- Never commit real secrets or personal data into fixtures; use synthetic values.
- Pin CI actions to commit SHAs.
- Preserve the playbook's §4 decisions unless the target genuinely differs; when
it does, follow playbook §8 and record what you changed and why.
- Prefer stock images with bind-mounted fixtures over bespoke Dockerfiles.
- Run the app **natively** in dev; containerise only for `test:ci` / CI, and
bind-mount the code, never copy it (playbook §9–§10, guiding principles).
- Keep `dev` / `dev:local` working with **no browser setup** — provide cookies and
routing server-side, not via forced cookies or browser mocks.
- **Don't run the test suite unless asked.** When you do, use `test`: start
`dev:local` if no local stack is up, and **abort if `dev` (remote infra) is
running** — never test against remote infrastructure.
- Confirm before destructive or shared-system actions (pushing, deleting,
editing CI secrets).

## Guardian projects
Each phase skill ends with a **Guardian specifics** section covering pan-domain
auth, dev-nginx, the private `guardian/workflow` datastore and the GitHub App
token — read it as part of that phase.

## Output
At the end of each phase, report what was created/changed (as file links), how
you verified it, and the next phase. Keep prose short.
50 changes: 50 additions & 0 deletions .github/instructions/e2e-stack.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
applyTo: "e2e-tests/setup/**,e2e-tests/images/**"
description: "Conventions for the e2e Testcontainers stack code and its Dockerfiles."
---

# E2E stack conventions

Rules for the local-stack setup code under `e2e-tests/setup/` and the Dockerfiles
under `e2e-tests/images/`. Background and rationale live in the
`e2e-test-setup` skill's [reference playbook](../skills/e2e-test-setup/reference/e2e-playbook.md).

## Dockerfiles
- Base app/backend images on `debian:bookworm-slim` (glibc). Do **not** use
Alpine for images that run a JDK or use `apt-get`.
- Bake **only the toolchain** (via `mise` from a copied `.tool-versions`); never
`COPY` application/backend source into the image — it is bind-mounted at
runtime.
- Keep the build context tiny: build from a temp dir holding just
`.tool-versions` + the Dockerfile.
- Start every Dockerfile with the `# syntax=docker/dockerfile:1` directive.

## Container start code
- Create exactly one `Network` per stack run; stop it and every started
container in the failure path and in `stopLocalStack`.
- Prefer stock images with bind-mounted fixtures/config over bespoke images
(WireMock, LocalStack, Postgres, nginx).
- Run all mocks from the single shared WireMock image via `MOCK_WIREMOCK_CONFIGS`
+ `startMockWiremock`; add a mock by adding a fixture folder and a table entry,
not a Dockerfile.
- Give each service a network alias matching the real upstream hostname so the
app resolves it inside the Docker network without config overrides.
- Every container must have an explicit `Wait` strategy (log message or HTTP
healthcheck) and a sensible `withStartupTimeout`.
- Bind-mount source read-write only where the toolchain writes (`target/`,
`public/build`); mount fixtures read-only.
- Seed datastores from the host after the container is ready; seed the SQL DB
only after the owning service's migrations have created the schema, parents
before FK children.

## Safety
- Never commit real secrets, tokens or personal data into `e2e-tests/fixtures/`
— use synthetic values.
- Keep the production app change footprint minimal (env-gated switches only).

## Guiding principles
- The code lives at the dev-container root; when a Docker image runs the app
(CI / `test:ci`), it **bind-mounts** the code, never copies it in.
- Run the app **natively** in dev; containerise only for CI / `test:ci`.
- Provide anything a browser needs (cookies, routing) **server-side** so `dev` /
`dev:local` need no forced cookies or browser mocks.
39 changes: 39 additions & 0 deletions .github/instructions/feature-files.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
applyTo: "e2e-tests/features/**/*.feature"
description: "Conventions for authoring Cucumber/Gherkin .feature files in workflow-frontend."
---

# Feature File Conventions

These rules apply to every `.feature` file under `e2e-tests/features/`.

## Structure
- Start with a `Feature:` title, followed by 1–3 indented plain-English lines describing intent.
- Use the standard `Background:` for every feature; only the final "opened" line changes to name the relevant page:
```gherkin
Background:
Given the application stack is running
And I am signed in through pan-domain auth
And I have opened <the relevant page>
```
- One `Scenario:` per distinct, observable behaviour or branch (default state, open/close, each action, each data/permission variation).

## Steps
- Write steps as user behaviour (what the user sees or does), not implementation detail.
- Use `Given` for preconditions, `When` for actions, `Then` for expected outcomes, `And` to continue the previous keyword.
- Keep wording consistent across scenarios so step definitions can be reused.

## Evidence comments
- Immediately after each scenario, cite every source file the scenario relies on:
```gherkin
# Evidence: public/path/to/template.html
# Evidence: public/path/to/controller.js
```
- Use real, workspace-relative paths, and only cite files that were actually inspected.

## Coverage
- Every interactive element and state branch in the source template must map to at least one scenario.
- Data-driven branches (feature switches, permissions) need both on/off (present/absent) scenarios.
- Do not pad the file with scenarios for purely decorative elements; note such gaps instead.

See the `feature-file-from-templates` skill for the full authoring workflow, and `feature-file-step-definitions` for wiring scenarios to Playwright tests.
47 changes: 47 additions & 0 deletions .github/prompts/setup-e2e-tests.prompt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
mode: 'agent'
description: 'Kick off setting up an e2e test suite (Testcontainers stack + Playwright/BDD + CI) on this project, following the same patterns used to build a proven reference e2e setup.'
tools: ['codebase', 'search', 'usages', 'editFiles', 'runCommands', 'fetch', 'githubRepo', 'problems']
---

# Set up e2e tests

Set up a self-contained end-to-end test suite on this project by following the
same patterns used to build a reference e2e setup, per the
**`e2e-test-setup` skill** and its
[reference playbook](../skills/e2e-test-setup/reference/e2e-playbook.md). Use the
`e2e-test-setup` agent's phased approach, adapting each pattern to this project's
stack rather than copying the reference stack.

Before scaffolding anything, run **Phase 0 discovery** and confirm findings with
me. Establish:

1. **App runtime** — framework/language and how the app is built and run in dev.
2. **Dependencies** — every service and store the app talks to:
- other services, and whether each is version-controlled in a Guardian
repository (private or public);
- databases (SQL, DynamoDB, …);
- object storage (S3);
- each upstream HTTP API.
For each Guardian-repo dependency, I'll confirm with you whether to run the
real service (checked out and built from source) or mock it with WireMock.
3. **Auth model** — how a request is authenticated (cookie/JWT/OIDC/pan-domain).
4. **Environment** — is Docker available in the dev container and in CI?
5. **Scope** — which of these phases to do now: scaffold+stack, fixtures+mocks,
feature tests, CI workflow. (Build-speed optimisations are baked into the
stack build, not a separate phase.) The feature-tests phase only **validates
the setup** — tell me which part of the UI to extract a small set of features
for, rather than covering the whole app.

Infer as much as possible from the codebase first, then ask me only what's left.
Once I confirm the discovery summary, proceed phase by phase, delegating to the
`e2e-stack-setup`, `e2e-fixtures-and-mocks` and `e2e-ci-workflow` skills, and
verifying each phase before the next.

Follow the standard command set and guiding principles in the playbook (§9–§11):
the app runs natively in dev and is containerised only in CI; provide anything
the browser needs server-side (no forced cookies or browser mocks for dev).

Each phase skill ends with a **Guardian specifics** section (pan-domain auth, the
private `guardian/workflow` datastore, the GitHub App token) — read it as part of
that phase.
111 changes: 111 additions & 0 deletions .github/skills/e2e-ci-workflow/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
---
name: e2e-ci-workflow
description: 'Add a GitHub Actions workflow that runs the Playwright/Testcontainers e2e suite in CI: check out any private Guardian repo (for a dependency you run for real) via a GitHub App token, install the toolchain and the Playwright headless shell only, run the BDD suite, and upload traces on failure. Use when adding e2e tests to CI, creating the GitHub Actions workflow for Playwright, or granting CI access to a private dependency repository.'
argument-hint: '<any private repos for real dependencies, and CI runner constraints>'
---

# E2E CI workflow

Phase 5 of the playbook: run the e2e suite in GitHub Actions. Follow the pattern in
[ci-workflow.md](../e2e-test-setup/reference/ci-workflow.md)
(worked example).
**Read [../e2e-test-setup/reference/e2e-playbook.md](../e2e-test-setup/reference/e2e-playbook.md)
§6–§7 first.**

## Workflow shape
Trigger on `push` to the default branch, `pull_request`, and
`workflow_dispatch`. `permissions: contents: read`. Run on `ubuntu-22.04` with
`defaults.run.working-directory: e2e-tests`.

Steps, in order:

1. **(If you run any dependency for real from a private Guardian repo) mint a
GitHub App token** — default pattern:
```yaml
- uses: actions/create-github-app-token@<pinned-sha> # vX
id: private-repo-token
with:
client-id: ${{ vars.PRIVATE_REPO_APP_CLIENT_ID }}
private-key: ${{ secrets.PRIVATE_REPO_APP_PRIVATE_KEY }}
repositories: <owner>/<private-repo>
```
This is the **recommended default** for checking out a private dependency
repo: create a GitHub App with read access to that repo, store its client id
as a repo **variable** and its private key as a repo **secret**. (Alternatives
— a PAT or a deploy key — are possible but less scoped; prefer the App token.)

2. **Check out the app repo** (`actions/checkout`).

3. **Check out each private dependency repo** into the path the stack expects
(e.g. `e2e-tests/target/<dependency>`) using
`token: ${{ steps.private-repo-token.outputs.token }}`. A **public** Guardian
repo needs no token — check it out with plain `actions/checkout`.

4. **Install the toolchain** — `actions/setup-node` with
`node-version-file: '.tool-versions'`, `cache: yarn`,
`cache-dependency-path: e2e-tests/yarn.lock`.

5. **Install deps** — app deps (root) then e2e deps, both
`--frozen-lockfile`.

6. **Install Playwright headless shell only** —
`yarn playwright install --with-deps chromium --only-shell` (the shell, not
full Chromium — faster).

7. **Run** — `yarn test:ci` (spins up all infra and the app container, then runs
the suite headlessly; equivalent to `bddgen` + starting the stack +
`playwright test`).

8. **Upload artifacts on failure** — `actions/upload-artifact` with the
`target/test-results` path, `if: failure()`, short retention.

## Rules
- **Pin every action to a commit SHA** with a `# vX` comment (a review
requirement and a CodeQL finding).
- Docker must be available on the runner (it is on GitHub-hosted `ubuntu-*`); the
stack builds/starts containers via Testcontainers, no service containers
needed.
- Handle arch differences: dev is often arm64, CI amd64 — pull arch-appropriate
base images, don't pin single-arch digests.
- Report to CI natively: in `playwright.config.ts`, use the `github` reporter
when `process.env.CI`.

## Verify
- The workflow runs on a PR and the suite passes.
- Failure runs upload traces/videos.
- No secret is printed; the App token is scoped to the private dependency repo(s)
only.

## Guardian specifics

See the live workflow at
[ci-workflow.md](../e2e-test-setup/reference/ci-workflow.md).

### GitHub App token for guardian/workflow
The datastore (run for real) lives in the private `guardian/workflow` repo. A
dedicated GitHub App was created with **read** access to it. The workflow reads:
- `vars.WORKFLOW_APP_CLIENT_ID` (repo **variable**) — the App client id.
- `secrets.WORKFLOW_APP_PRIVATE_KEY` (repo **secret**) — the App private key.

and mints a token with `actions/create-github-app-token`, scoped to
`repositories: guardian/workflow`, used to check it out into
`e2e-tests/target/workflow-backend` (where `stackContainers.ts` / `getBackendDir`
expect it).

### Node version
Node is read from `.tool-versions`. The app needs Node ≥ `22.9.0` (a dependency
of the e2e tests is incompatible with `22.5.1`).

### Pinned actions (keep SHAs current)
- `actions/create-github-app-token`
- `actions/checkout`
- `actions/setup-node`
- `actions/upload-artifact`

All pinned to a commit SHA with a `# vX` comment (CodeQL / review requirement).

### Runner
`ubuntu-22.04`, `working-directory: e2e-tests`. Docker is available on the
GitHub-hosted runner, so Testcontainers builds and runs the stack directly.
Install the headless shell only: `yarn playwright install --with-deps chromium
--only-shell`.
Loading