Skip to content

feat(cli): a budget for each metric, and none that passes unchecked - #325

Merged
nucliweb merged 2 commits into
mainfrom
feat/cli-budgets
Oct 25, 2026
Merged

nucliweb merged 2 commits into
mainfrom
feat/cli-budgets

Conversation

@nucliweb

Copy link
Copy Markdown
Owner

Summary

Phase 1 of the budgets: a budget for each metric the CLI measures, and no budget that passes without being checked. The budgets in the reports and the JSON come in the next PR.

  • --budget-fcp, --budget-ttfb and --budget-inp, next to --budget-lcp and --budget-cls.
  • The interaction workflow measures INP, in a step INP that needs a click or a key press, like interactions and input-latency. No workflow ran the INP snippet before, so an INP budget only worked with --snippet INP.
  • A budget never passes unchecked:
    • A budget of a metric that the run does not measure, such as --budget-lcp with --workflow audit, exits with 2 before the page loads, naming the workflow or snippet that measures it.
    • A budget whose metric was not measured, such as INP without interactions or an LCP that failed, exits with 1 and says why.
    • A value that is not a number of 0 or more exits with 2. --budget-lcp abc was NaN, and passed.
  • stderr names the failed budgets also with --json, whose report goes to stdout.
  • cli/src/budgets.js holds the logic: which snippets measure each metric, the parsing, and the verdict of each budget against the value, or the median with --runs. bin.js uses it for a page and for each combination of a matrix.
  • The GitHub Action takes budget-fcp, budget-ttfb and budget-inp. Its budget-lcp and budget-cls default to auto: 2500 and 0.1 with core-web-vitals, and no budget with the other workflows, which do not measure LCP or CLS. An action with workflow: audit and the defaults keeps passing instead of exiting with 2; an empty value still disables a budget.

cli/README.md, content/CLI.mdx (options, the inputs of the Action, the steps of the interaction workflow and a paragraph on what each budget needs) and the CLI changelog describe it.

Test plan

  • cli/tests/unit/budgets.test.js (local): the five options, values that are not numbers, the snippets that measure each metric checked against their sources, the interaction workflow with INP, the metrics a run does not measure, equal to the budget passes, not measured with the reason of its snippet, the step before a follow-up, the median of the runs, the words of stderr.
  • cli/tests/e2e/budgets.test.js (local, new): INP over budget with a slow click in the interaction workflow exits with 1; without a click, not measured and 1; --budget-lcp with --workflow audit and --budget-fcp fast exit with 2 without a request to the page; FCP and TTFB pass with a loose budget and fail with an impossible one, named on stderr with --json. They failed before the change.
  • github-action.test.js (local): the new inputs reach the CLI, and auto gives 2500 and 0.1 with core-web-vitals and nothing with audit.
  • interaction-workflow.test.js and mcp-workflow.test.js (local): INP among the steps that need input, skipped without a click and measured with one.
  • Lint, check:emoji, CLI unit 339 of 339, CLI e2e 582 of 582, test:site 44 of 44, test:ci 22 of 22 (local).
  • By hand with the CLI (local, a page with a button whose click blocks for 300 ms):
    • --workflow interaction --interact-script click.json --budget-inp 200: exit 1, Budget violation: INP 304ms exceeds budget 200ms.
    • The same without the script: exit 1, INP was not measured (budget 200ms): The interactions have no click or type step; add one with --interact-script.
    • --workflow audit --budget-lcp 2500: exit 2, --budget-lcp needs a run that measures LCP, and the audit workflow does not. Use --workflow core-web-vitals or --snippet LCP.
    • --runs 3 --budget-lcp 2500 --budget-cls 0.1 on the default workflow: exit 0.

--budget-fcp, --budget-ttfb and --budget-inp join --budget-lcp and --budget-cls, and the interaction workflow measures INP in a step that needs a click or a key press. cli/src/budgets.js reads the budgets, knows which snippets measure each metric, and gives each budget a verdict against the value, or the median with --runs. A budget of a metric the run does not measure exits with 2 before the page loads, naming what measures it; one whose metric was not measured exits with 1; a value that is not a number exits with 2, where it passed before. stderr names the failed budgets also with --json. The GitHub Action takes budget-fcp, budget-ttfb and budget-inp, and its budget-lcp and budget-cls default to auto, their values with core-web-vitals and no budget with the workflows that do not measure LCP or CLS.
@vercel

vercel Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
webperf-snippets Ready Ready Preview Oct 25, 2026 11:41pm UTC

The changelog of the CLI keeps the entries of both pull requests under Unreleased, and the page of the CLI keeps the wording of #324, not measured, with INP among the steps that need a click and the exception of a budget that needs their metric.
@nucliweb
nucliweb merged commit 4a281dc into main Oct 25, 2026
4 checks passed
@nucliweb
nucliweb deleted the feat/cli-budgets branch October 25, 2026 23:48

This branch was successfully deployed

1 active deployment
Preview — 77594f25 Deployed Oct 25, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant