ci: run Echidna on PRs - #320
Merged
Merged
Conversation
Catch invariant regressions on every PR with a time-boxed campaign, and keep overnight reproducers when a property fails.
Keep PR checks to the fast matrix only, so the overnight job does not show up as a skipped status on every PR.
Contracts rarely deploy outside PRs, so keep a single time-boxed Echidna job on every pull request.
Drop the smoke-test budget so pull requests fuzz with the same campaign as local yarn echidna.
darkobas2
approved these changes
Sep 3, 2026
darkobas2
left a comment
Contributor
There was a problem hiding this comment.
Approving. Notes, none blocking:
- Echidna
--timeout10200s inside a 180-minute job leaves about 10 minutes for checkout,yarn install,hardhat compile --forceand the image pull. If that overruns, GitHub kills the job at the cap and theif: failure()upload step is skipped, so you lose exactly the logs you wanted. 9000 would be safer. ECHIDNA_SEEDis added for reproducibility but CI never sets one, so "Reproduce a CI counterexample locally" won't actually reproduce. Echidna prints the seed in its output — worth saying in the README to take it from the uploaded log and passECHIDNA_SEED=N.- No corpus cache between runs, so every PR fuzzes from an empty corpus.
actions/cacheonechidna/corpus/by-contract/would let coverage compound. Fine to skip if you want an identical budget per PR.
Checked and correct:
set +e/PIPESTATUS[0]/set -ecaptures the fuzzer exit code through thetee.scripts/echidna.shalready runshardhat compile --forceon the host, so--hardhat-ignore-compilefinds artifacts inside the container.echidna/echidna.yamlsetsformat: text, so no TUI on a non-tty runner.pull_request(notpull_request_target) withcontents: read— fork PRs get no secrets.
Drop Echidna --timeout to 9000s under the 180-minute job cap, and document copying Seed from CI logs for local replay.
Contributor
Author
|
Thnx for notes, adjusted per comments. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
yarn echidna:testLimit60000,seqLen320).--timeout10200s). Failures upload logs, corpus reproducers, andcrytic-export/as artifacts.yarn echidna) now acceptsECHIDNA_SEED/ECHIDNA_TIMEOUTand tees logs toechidna/logs/.Echidna image is pinned to
ghcr.io/crytic/echidna/echidna:v2.3.1.Test plan
EchidnaPriceOracleHarness