Skip to content

chore(benchmark): archive the unreferenced SWE-bench Rust runner - #1173

Merged
Chloe-JY merged 1 commit into
developfrom
chore/dead-tauri-commands-benchmark
Sep 1, 2026
Merged

chore(benchmark): archive the unreferenced SWE-bench Rust runner#1173
Chloe-JY merged 1 commit into
developfrom
chore/dead-tauri-commands-benchmark

Conversation

@Harry19081

Copy link
Copy Markdown
Member

Problem

The 2026-08-16 SWE-bench "Benchmark (Beta)" archival parked the whole frontend
surface — panel, run builder, tab renderer, store atoms, benchmarkApi client,
E2E helpers and specs — but deliberately left the Rust side in place. Its own
.archive/README.md entry recorded the consequence:

src-tauri/src/benchmark/ — the Rust runner/commands still compile and are
registered; they are now unreferenced from the frontend and can be removed in
a backend-only PR

Since then src-tauri/src/benchmark/ (3,542 LOC) has compiled into every build
and registered 13 benchmark_* Tauri commands that nothing can call. A sweep of
the 1,019 entries in src-tauri/src/commands/handler_list.inc against every
non-Rust source file in the repository confirms it: none of the 13 appears in
src/, tests/, or src-tauri/capabilities/, and no Rust caller reaches the
module either — the only inbound edge outside the module was the exit-time
evaluator cleanup in app/lifecycle.rs, which exists solely to reap processes
the (now unreachable) runner would have spawned.

This is the backend-only PR that entry called for.

Solution

git mv src-tauri/src/benchmark/ .archive/src-tauri/src/benchmark/, matching
the existing archival convention: paths mirror their original location, nothing
under .archive/ is compiled or bundled, and restoring is a reverse git mv.

Unwired in place:

  • src-tauri/src/commands/handler_list.inc — dropped the 13
    benchmark::benchmark_* registrations and their section comment
  • src-tauri/src/lib.rs — dropped pub mod benchmark;
  • src-tauri/src/app/builder.rs — dropped benchmark from the use crate::{…}
    scope that handler_list.inc resolves against
  • src-tauri/src/app/lifecycle.rs — dropped the use crate::benchmark; import
    and the exit-time benchmark::terminate_running_evaluators_sync() call

Resulting invariant: the app's IPC surface no longer advertises a benchmark_*
command, and no code path can spawn a SWE-bench evaluator subprocess, so the
exit-time reaper it guarded has nothing left to reap.

.archive/README.md's SWE-bench section is updated so the entry that asked for
this removal now records that it happened, together with the in-place edits
needed to reverse it.

Not the Housekeeper token benchmark (housekeeper_token_benchmark,
rpc.validation.housekeeperTokenBenchmark) — a different feature, untouched and
still registered.

Potential risks

  • Persisted user state: none. The runner wrote its run/batch history under
    the app data dir; those files are simply no longer read. Nothing is deleted
    from disk by this change, so restoring the module restores access to them.
  • Wire/IPC compatibility: removing 13 commands changes the invoke surface. A
    frontend build that still called them would now fail at runtime instead of
    succeeding — but the callers were archived in the same repository three weeks
    ago, so no shipped or in-tree frontend references them. Verified by an
    exhaustive scan, not by grep of a single directory.
  • Exit-time behavior: terminate_running_evaluators_sync() is gone from the
    shutdown path. If any user still has an orphaned evaluator process from a
    build predating this change, it will no longer be reaped at app exit. Those
    processes are Docker/CLI children of a feature that has had no UI entry point
    since 2026-08-16, so the practical exposure is nil.
  • Unverified paths: the archived module is no longer type-checked. Any future
    restore must re-run cargo check rather than assume it still compiles.
  • Rollback: reverse the git mv and revert the four in-place edits (all
    recorded in .archive/README.md), or revert this commit wholesale.

Verification

  • cargo check --workspace --all-targets — passes, 44 crates, no warnings
    introduced.
  • cargo clippy --workspace --all-targets -- -D warnings — passes (exit 0).
  • Registry sweep: parsed all 1,019 entries of handler_list.inc and searched
    every .ts/.tsx/.js/.jsx/.mjs/.cjs/.json/.html/.md/.rs/.toml/.yml/.yaml file
    in the repository (9,690 files, excluding node_modules, target,
    .archive) for each command name. The 13 benchmark_* commands have zero
    hits outside handler_list.inc and the module itself.
  • Checked for indirect invocation: the repository's only non-literal invoke()
    call sites (useEmbeddedWebview, diagnostics/rustBridge) both receive
    command names as constants, so no command is reachable by a constructed
    string. No benchmark_* entry appears in src-tauri/capabilities/default.json.
  • Not run: no UI change, so no screenshots. pnpm typecheck / vitest /
    Playwright were not run — this PR changes no TypeScript, and the frontend
    callers were removed in a prior PR.
  • Pre-commit hook trailer is absent. This was committed from a git worktree
    where .husky/_/husky.sh does not exist (gitignored, so git worktree add
    never creates it) and node_modules is not installed. The hook's
    "Pre-commit hook ran." tamper-evidence trailer is therefore always missing in
    a worktree and means nothing here; the checks it would have run (cargo clippy) were run manually and are reported above. No TypeScript is touched,
    so lint-staged/tsc had nothing to check.

The 2026-08-16 SWE-bench UI archival left `src-tauri/src/benchmark/`
compiled and registered with no frontend caller, and recorded that a
backend-only removal should follow. This is that removal.

Moves the runner (3,542 LOC: SWE-bench dataset/task loading, run and
agent-batch orchestration, Docker evaluation, preflight, retention,
history) to `.archive/src-tauri/src/benchmark/` and unwires it:
13 `benchmark_*` registrations, `pub mod benchmark;`, the handler-list
`use` scope, and the exit-time evaluator-termination call.
@Chloe-JY
Chloe-JY merged commit aa975a0 into develop Sep 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants