Skip to content

One-command benchmark harness #5

Description

@riktar

Make #2/#3 reproducible by third parties: a single entry point (make bench-locomo or memledger-bench locomo) that downloads data, runs the pipeline, and emits the result tables.

  • Pinned dependencies for the harness
  • Records model/backends, policy hash, registry hashes in output
  • Documented in evals/README.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions