Handle stuck pending runs with stale sweep

[suhaibmujahid]

Fixes #6748

[Copilot]

Pull request overview

Adds a reconciliation “stale sweep” path to ensure agent runs don’t remain stuck in pending when the Cloud Run completion event is lost or when old executions are garbage-collected (404 on GetExecution).

Changes:

• Introduces an internal maintenance endpoint (/internal/maintenance/finalize-stale-runs) to periodically finalize old, unfinalized runs via the same finalize_run logic as the event-driven path.
• Treats Cloud Run execution 404s as a terminal “gone” state and allows finalization based on summary.json when the execution record has been deleted.
• Adds targeted tests for execution-status mapping, stale-run sweeping behavior, and finalize behavior when executions are “gone” or missing.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
services/hackbot-api/app/jobs.py	Adds ExecutionStatus.gone and maps Cloud Run GetExecution 404 to gone.
services/hackbot-api/app/routers/runs.py	Avoids asserting on missing execution_name; adjusts terminal-status logic to allow gone with summary.json.
services/hackbot-api/app/routers/maintenance.py	New internal maintenance router implementing the stale-run sweep endpoint.
services/hackbot-api/app/routers/init.py	Exports the new maintenance router.
services/hackbot-api/app/main.py	Includes the maintenance router in the FastAPI app.
services/hackbot-api/tests/test_jobs_execution_status.py	New tests for 404→gone mapping and normal completion mapping.
services/hackbot-api/tests/test_finalize_stale_runs.py	New tests covering sweep behavior (finalized, dry-run, still-running, per-run failure isolation).
services/hackbot-api/tests/test_finalize_run.py	New tests for finalizing from summary.json when execution is gone, and for missing execution_name.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.