@@ -190,17 +190,48 @@ an explicit matrix and runs the runner once per row. Branch E2E calls it when
190190the ` test:e2e ` label is set. It is advisory, so a failure doesn't block
191191merging. To run a suite in another environment on CI, add a row.
192192
193- ** Environments with services.** A suite that needs something besides the
193+ ### Adding a feature suite
194+
195+ A feature suite tests one OpenShell feature and lives in the
196+ ` tests/suites/features ` workspace. ` tests/suites/features/keycloak ` is the
197+ example to copy.
198+
199+ 1 . Add a crate under ` tests/suites/features/<name> ` that depends on
200+ ` openshell-testkit ` , and add it to ` members ` in
201+ ` tests/suites/features/Cargo.toml ` .
202+ 2 . Write the tests in ` tests/*.rs ` as ` #[testkit::test] ` functions.
203+ 3 . Re-lock the workspace: `cargo metadata --manifest-path
204+ tests/suites/features/Cargo.toml`, then
205+ ` uv run tasks/scripts/check_cargo_lockfile_alignment.py --fix ` .
206+ 4 . Add a row for the suite to the matrix in ` .github/workflows/e2e-suites.yml ` .
207+
208+ If the feature only needs the gateway, run the suite in an existing
209+ environment such as ` ubuntu-docker-rootful ` , and skip the rest of this
210+ section.
211+
212+ ** Suites that need a service.** A feature that needs something besides the
194213gateway, such as an OIDC issuer, runs in a tmachine environment that provides
195- it. The environment's setup starts the service (with a restart policy, since
196- setup is cached as a disk) and writes how to reach it as ` OPENSHELL_TEST_* `
197- lines in ` /etc/openshell-test/env.d/<service>.env ` . The runner passes those
198- variables to the tests and rejects names without that prefix. A suite-local
199- fixture reads them with plain environment variables, so you can run the suite
200- against your own deployment by setting them yourself. See the
201- ` ubuntu-docker-rootful-keycloak ` environment in ` tests/tmachine/config.nix ` ,
202- ` tests/tmachine/ansible/playbooks/keycloak.yaml ` and
203- ` tests/suites/features/keycloak ` :
214+ it. The environment and the suite share a contract: a set of
215+ ` OPENSHELL_TEST_<SERVICE>_* ` environment variables that say how to reach the
216+ service.
217+
218+ - ** Environment.** Add an environment to ` tests/tmachine/config.nix ` whose
219+ setup playbook starts the service (with a restart policy, since setup is
220+ cached as a disk) and writes the variables as ` KEY=VALUE ` lines to
221+ ` /etc/openshell-test/env.d/<service>.env ` . Pass files the playbook needs
222+ through the environment's setup ` inputs ` , and add them to the tmachine disk
223+ cache key in ` e2e-suites.yml ` . The runner passes the variables to the tests
224+ and rejects names without the ` OPENSHELL_TEST_ ` prefix.
225+ - ** Fixture.** In the suite's ` src/lib.rs ` , define the variable names as
226+ constants and a ` <Service>Context::new(cx) ` that reads them, waits for the
227+ service, and wraps the testkit ` Context ` . Each test calls it on its first
228+ line. Nextest runs each test in its own process, so the fixture builds its
229+ clients there from the variables. A missing variable fails the test; it
230+ never skips it.
231+ - ** Documentation.** Document the variables in the fixture's module docs, so
232+ anyone can run the suite against their own deployment by setting them.
233+
234+ Run the Keycloak suite with:
204235
205236``` shell
206237nix run .# openshell-test-runner -- \
0 commit comments