The package consists of three parts:
- cli located in
bin/ - test library located in
lib - the reusable GitHub workflows consumer repos call, in
.github/workflows/and.github/actions/
bin/build.tsactor buildingbin/git.tsgit wrappers for getting relevant git commits and changed filesbin/main.tsentrypoingbin/slack.tssending notifications to slackbin/test-report.tsprocessing vitest's test reports
CLI command results use writeJson from bin/output.ts to write newline-terminated JSON to stdout.
Use the CLI logger from bin/logger.ts for diagnostics; every level writes to stderr.
ESLint rejects console.log throughout the codebase and all direct console calls in bin/.
The importable library in lib/ keeps its own diagnostics; CLI log levels apply only to bin/.
The CLI's global --log-level option accepts debug, info (default), warn, error, or silent.
It filters diagnostics without affecting command results. Use logger.debug for detailed troubleshooting,
logger.info for progress, logger.warn for recoverable problems, and logger.error for failures.
lib/extend-expect.ts- custom matchersrun-test-result.ts-RunTestResultclass that's the output ofrunfunction- wrapper around run endpoints:
logLog,getStatistics,getDataset, etc
- wrapper around run endpoints:
- Clone and build
apify-test-toolsrepo:
git clone git@github.com:apify/apify-test-tools.git
cd apify-test-tools
npm i
npm run buildFor testing purposes, we use testing-repo-for-github-actions repo so that we don't mess with the production repos:
git clone git@github.com:apify-store/testing-repo-for-github-actions.gitTo work on the library, run the cli from the root of your testing repo:
cd /path/to/testing-repo-for-github-actions
npx tsx /path/to/apify-test-tools/bin/main.ts --help
npx tsx /path/to/apify-test-tools/bin/main.ts get-commits --target-branch master --source-branch feat/testing-feature-branchYou need to istall the local version of apify-test-tools in your cloned testing-repo-for-github-actions:
npm i -D ../path/to/apify-test-toolsYou need to run npm run build inside apify-test-tools repo everytime you want to test your changes in testing-repo-for-github-actions.
The public_-prefixed workflows are the ones consumer repos call: public_pr-build-test,
public_platform-tests, public_push-build-latest, public_claude, public_review and
public_platform-tests-claude-investigate-and-fix. They live here because they call this package's
CLI, so a change to both is one PR. GitHub only reads workflow files at the top level of
.github/workflows, so they sit next to this repo's own CI, and the prefix is what separates the
two: public_ is the API other repos depend on, _ is internal plumbing, and on_/manual_ are
this repo's own triggers.
A public_ filename is part of the contract — it is baked into every consumer's uses: line, so
renaming one is a breaking change that needs a major tag bump, not a tidy-up.
The tag is not semver, so git-cliff would abort on it. It is skipped only because
apify/actions' cliff.toml sets tag_pattern = "v[0-9]+\." — a release failing with
Semver error: unexpected character means that pattern changed, not that anything here is wrong.
The action's "Nothing to release" guard resolves to this tag and no longer fires, so an empty stable
release fails at npm publish instead of erroring cleanly.
Consumers pin @workflows-v0, not @master. See
Versioning and releases in the README for how the tag and the
npm release relate — the short version:
- the workflows and the CLI ship together on every master push, through the tag; no npm release needed
- the lib ships through a stable npm release
- the CLI runs from
.github/cli/apify-test-tools.mjs, a bundle the pre-commit hook rebuilds withnpm run bundle. If you commit with--no-verify, run it yourself: theCLI bundlecheck fails on a stale bundle, and a stale bundle would hold the tag. npm testcovers the bundle without network (test/bundle/).npm run test:e2eruns it against two dummy Actors in an isolated sandbox account (test/e2e/) and skips itself unlessE2E_TEST_APIFY_TOKEN_DBYPODOIHUZABNQTRis set. CI passes that secret only to theE2E testsjob.
npm run lint and actionlint (via the Code checks workflow) both gate master, so run them before
pushing workflow changes.
public_review is the odd one out: it fetches .github/review-prompt.md over HTTP at run time, because a
reusable workflow runs with the caller's repo checked out and never gets its own. Its prompt-ref
input defaults to workflows-v0 so the instructions come from the same release as the workflow — leaving it
at master would run released workflows against unreleased instructions.