Skip to content

Benchmarks: k6 load-test harness and regression check are documented but not implemented #693

Description

@devitway

BENCHMARKS.md and the benchmarks.yml workflow document a k6 load-test harness and a regression check that are not present in the tree:

  • scripts/load-test.sh (referenced for smoke/average/stress/spike/soak/breakpoint scenarios) — does not exist. benchmarks.yml calls scripts/load-test.sh "$SCENARIO" || true, so the load test never runs and the || true masks the missing script.
  • scripts/bench-regression.sh (baseline/compare, ">15% regression" gate) — does not exist.

As a result, the throughput/latency figures (Docker pull p95, npm p95, req/s) and the under-load memory figures (RAM at 100/1000 concurrent) in BENCHMARKS.md were never measured.

Scope

Implement the real harness so those figures become measured:

  • scripts/load-test.sh — k6 scenarios against a running NORA (the request distribution already sketched in BENCHMARKS.md § Throughput).
  • scripts/bench-regression.sh — baseline/compare with a configurable threshold.
  • Wire both into benchmarks.yml and publish the JSON reports as release artifacts.

Until then, the docs and CI have been scoped to only the metrics CI actually measures today (binary size, startup time, idle memory, Criterion micro-benchmarks).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions