Skip to content

feat(release-gate): blocking surface-state consistency check (Spec 081 T4) - #1655

Merged
Dumbris merged 3 commits into
mainfrom
claude/festive-khayyam-d78aed
Oct 17, 2026
Merged

Dumbris merged 3 commits into
mainfrom
claude/festive-khayyam-d78aed

Conversation

@Dumbris

@Dumbris Dumbris commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

Spec 081 T4 (US5, FR-018). The release gate now blocks a tag when the CLI, MCP or Web UI shows a server state different from what the core reports.

What it does

New blocking gate entry invariant/surface-consistency, run by a new job surface-consistency. It replaces the reserved T4 slot.

  1. Boots the same five-type server matrix (stdio/http/sse/docker/oauth). Those cells are still reported by matrix-invariants.
  2. Adds four servers so every admin state is on screen: live (healthy), quarantined, disabled, and broken (the endpoint refuses connections, so it is unhealthy).
  3. Compares each surface with the core's GET /api/v1/servers:
Surface Read path Fields
CLI mcpproxy upstream list -o json --config … enabled, quarantined, connected, connection state, health.level/admin_state/status, tool_count
MCP upstream_servers operation=list (its own storage + manager projection) enabled, quarantined, connection state, health.level/admin_state/status
Web UI rendered cards on /ui/servers, read by e2e/web-ui-sweep/surface-snapshot.mjs enabled, quarantined, tool count, status chip colour and text. The expected values come from the core row using ServerCard.vue's rules.
Tray — recorded as not_compared (T3)

It runs in three phases: settled, live (a server is disabled through REST while the page stays open, which tests the SSE/poll update path) and reload. Each phase retries until the surfaces converge. An attempt counts only if the core state did not change while the surfaces were being read. A divergence that remains after the retries fails the gate with a per-field diff, for example webui gate-cons-quarantined webui.chip_class: core=badge-secondary webui=badge-success.

Negative path: --seed-mismatch surface:server:field=value, or the dry-run-only seed_consistency_mismatch workflow_dispatch input. A tag's workflow_call never reads it, and an audit test enforces that.

Local: ./scripts/test-surface-consistency.sh runs next to scripts/test-api-e2e.sh.

No new dependencies. The reader reuses the sweep's locked @playwright/test.

Review

Cross-reviewed with opencode github-copilot/gpt-6.1-sol (zcode login was down at round 1): round 1 → 10 findings, all fixed in 54adbf1; round 2 → 9/10 confirmed fixed + 1 new, both fixed in 03ad256; round 3 → NO NEW FINDINGS.

Testing

  • Written test-first: comparator tests cover agreement, a stale CLI level, an MCP connection state, a Web UI chip and tool count, missing and extra servers, a capture error, and the seed path. A workflow audit checks that the job blocks, compares the Web UI, is in the verdict job's needs list, and only reads the seed on dispatch.
  • Local live run, all five cells (stdio/http/sse/docker/oauth) + the 4 mixed servers, isolated instance (free port, scratch data dir): all three phases pass. The captured rows show the disable reaching the CLI, MCP and the open Web UI page.
  • Local seeded run (webui:gate-cons-quarantined:webui.chip_class=badge-success): fails in every phase with the diff above.
  • golangci-lint v2 (bare and --build-tags server): 0 issues.
  • Gate dry runs on this branch: clean https://github.com/smart-mcp-proxy/mcpproxy-go/actions/runs/39038591871 · seeded https://github.com/smart-mcp-proxy/mcpproxy-go/actions/runs/39038596517

Related #819 #1030

…1 T4)

New gate entry invariant/surface-consistency (job surface-consistency)
replaces the reserved T4 slot. It boots the five-type server matrix, adds
quarantined, disabled, unhealthy and live servers, and asserts that the
CLI (`upstream list -o json`), MCP `upstream_servers list` and the
rendered Web UI agree with the core's /api/v1/servers on enabled,
quarantined, connection state, health.level/admin_state/status and tool
counts. Three phases: settled, live (disable via REST with the page left
open), reload. A divergence that survives the retries fails the gate with
a per-field diff. The macOS tray is recorded as not-compared until T3.

## Changes
- cmd/release-gate: consistency comparator + `consistency` subcommand
- e2e/web-ui-sweep/surface-snapshot.mjs: long-lived Web UI reader
- internal/gatereport: blocking invariant/surface-consistency entry
- release-qa-gate.yml: surface-consistency job, verdict dependency,
  dry-run-only seed_consistency_mismatch input
- scripts/test-surface-consistency.sh: local runner
- docs, specs/081 tasks.md, roadmap

## Testing
- comparator tests: agreement, stale CLI level, MCP state, Web UI chip
  and tool count, missing/extra servers, capture error, seed
- workflow audit: job blocking, compares Web UI, seed gated to dispatch
- local run (stdio/http/sse/oauth): pass; seeded webui mismatch: fail
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Deploying mcpproxy-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 03ad256
Status: ✅  Deploy successful!
Preview URL: https://4aec9552.mcpproxy-docs.pages.dev
Branch Preview URL: https://claude-festive-khayyam-d78ae.mcpproxy-docs.pages.dev

View logs

…check

Comparator:
- connection state stays compared for disabled/quarantined servers:
  exactly for the CLI, connected-vs-not for MCP
- Web UI expectation mirrors ServerCard's no-health branch and the
  usable=false "Unavailable" fallback
- --seed-mismatch refuses fields the surface is not compared on and
  servers the surface does not show

Live command:
- CLI child drops every MCPPROXY_* override (MCPPROXY_DATA picked a
  socket ahead of the gate's TCP endpoint)
- every capture runs under the phase deadline; late agreement fails
- a timed-out/cancelled Web UI command kills the reader, so a late
  reply is never read as the next command's answer
- fragment written before teardown; teardown bounded to 2 minutes
- stderr buffer is synchronized

Local script: Bash 3.2 empty-array expansion, and a trap that runs the
new `release-gate teardown` so an early failure leaks no processes.
…e local script

Round-2 review: teardown bounded Docker calls only (PID loops and pgrep
take no context), so it now runs in a goroutine abandoned after its
budget — the verdict is already written. The local script's new cleanup
trap tore down an instance kept with --keep-core; it now leaves it up
and prints the teardown command.
@Dumbris
Dumbris enabled auto-merge (squash) October 17, 2026 14:18
@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

📦 Build Artifacts

Workflow Run: View Run
Branch: claude/festive-khayyam-d78aed

Available Artifacts

  • archive-darwin-amd64 (32 MB)
  • archive-darwin-arm64 (28 MB)
  • archive-linux-amd64 (19 MB)
  • archive-linux-arm64 (17 MB)
  • archive-windows-amd64 (31 MB)
  • archive-windows-arm64 (28 MB)
  • frontend-dist-pr (0 MB)
  • installer-dmg-darwin-amd64 (27 MB)
  • installer-dmg-darwin-arm64 (25 MB)
  • smart-mcp-proxymcpproxy-go657IK9.dockerbuild (0 MB)

How to Download

Option 1: GitHub Web UI (easiest)

  1. Go to the workflow run page linked above
  2. Scroll to the bottom "Artifacts" section
  3. Click on the artifact you want to download

Option 2: GitHub CLI

gh run download 39040097571 --repo smart-mcp-proxy/mcpproxy-go

Note: Artifacts expire in 14 days.

@codecov-commenter

codecov-commenter commented Oct 17, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 40.33613% with 355 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/release-gate/consistency_cmd.go 14.28% 313 Missing and 5 partials ⚠️
cmd/release-gate/consistency.go 86.11% 17 Missing and 13 partials ⚠️
cmd/release-gate/main.go 0.00% 7 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Dumbris
Dumbris merged commit cfa48a4 into main Oct 17, 2026
62 of 65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants