Skip to content

ci: run Echidna on PRs - #320

Merged
0xCardiE merged 5 commits into
masterfrom
ci/echidna-ci
Sep 3, 2026
Merged

ci: run Echidna on PRs#320
0xCardiE merged 5 commits into
masterfrom
ci/echidna-ci

Conversation

@0xCardiE

@0xCardiE 0xCardiE commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a GitHub Actions Echidna workflow that runs the full fuzz campaign on every pull request (same budget as local yarn echidna: testLimit 60000, seqLen 320).
  • One matrix job per harness, 180-minute cap (Echidna --timeout 10200s). Failures upload logs, corpus reproducers, and crytic-export/ as artifacts.
  • The Docker runner (yarn echidna) now accepts ECHIDNA_SEED / ECHIDNA_TIMEOUT and tees logs to echidna/logs/.

Echidna image is pinned to ghcr.io/crytic/echidna/echidna:v2.3.1.

Test plan

  • Local smoke against the pinned image on EchidnaPriceOracleHarness
  • Confirm each harness job on this PR runs the 60k/320 campaign (not the old 4k/80 smoke) and either finishes or hits the 180-minute cap cleanly
  • On a property failure, confirm logs + reproducers are uploaded as artifacts

Catch invariant regressions on every PR with a time-boxed
campaign, and keep overnight reproducers when a property fails.
Keep PR checks to the fast matrix only, so the overnight
job does not show up as a skipped status on every PR.
Contracts rarely deploy outside PRs, so keep a single
time-boxed Echidna job on every pull request.
@0xCardiE 0xCardiE changed the title ci: run Echidna on PRs and overnight ci: run Echidna on PRs Sep 3, 2026
Drop the smoke-test budget so pull requests fuzz with the
same campaign as local yarn echidna.

@darkobas2 darkobas2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Notes, none blocking:

  1. Echidna --timeout 10200s inside a 180-minute job leaves about 10 minutes for checkout, yarn install, hardhat compile --force and the image pull. If that overruns, GitHub kills the job at the cap and the if: failure() upload step is skipped, so you lose exactly the logs you wanted. 9000 would be safer.
  2. ECHIDNA_SEED is added for reproducibility but CI never sets one, so "Reproduce a CI counterexample locally" won't actually reproduce. Echidna prints the seed in its output — worth saying in the README to take it from the uploaded log and pass ECHIDNA_SEED=N.
  3. No corpus cache between runs, so every PR fuzzes from an empty corpus. actions/cache on echidna/corpus/by-contract/ would let coverage compound. Fine to skip if you want an identical budget per PR.

Checked and correct:

  • set +e / PIPESTATUS[0] / set -e captures the fuzzer exit code through the tee.
  • scripts/echidna.sh already runs hardhat compile --force on the host, so --hardhat-ignore-compile finds artifacts inside the container.
  • echidna/echidna.yaml sets format: text, so no TUI on a non-tty runner.
  • pull_request (not pull_request_target) with contents: read — fork PRs get no secrets.

Drop Echidna --timeout to 9000s under the 180-minute job cap,
and document copying Seed from CI logs for local replay.
@0xCardiE

0xCardiE commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Thnx for notes, adjusted per comments.

@0xCardiE
0xCardiE merged commit 2590519 into master Sep 3, 2026
7 checks passed
@0xCardiE
0xCardiE deleted the ci/echidna-ci branch September 3, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants