feat: gate MCP killswitch rollout

[danielkov]

Summary

• add default-off server-side off, shadow, and enforce modes for authenticated MCP tool-call evaluation
• gate fresh activation and change mutations while preserving completed operation replay and break-glass deactivation
• document fleet readiness, bounded telemetry, incident response, rollback, and exact M2 coverage limits
• cover rollout transitions, management gating, and idempotent replay; targeted Killswitch and MCP tests pass

Closes DNO-987

---

Summary by cubic

Introduces a default-off server-side rollout gate for authenticated MCP tool-call Killswitch evaluation. Previously evaluation ran unconditionally and prescriptions could be activated at any time; activation and edits are now blocked outside enforce mode, while deactivation and completed replays stay available.

Rollout modes

• Evaluation runs only in shadow or enforce; shadow measures telemetry without denying and off mode never queries the evaluator.
• Missing, unavailable, or indeterminate local flags fail safe to off, and the serving path never makes a remote PostHog request.
• Enforce takes precedence when both flags are enabled, and replays of completed operations bypass the gate.

Operational guidance

• Adds rollout and evaluator-incident runbooks covering fleet-readiness checks, shadow thresholds, Datadog monitors, break-glass rules, and rollback criteria.
• Covers only tools/call for an active organization user and canonical organization-owned server on hosted dispatch and private proxy.
• Platform break-glass management now shares the validated lifecycle so list and deactivation work when rollout enforcement is off, and audit records use the actor display name from the authoritative user row.

Closes DNO-987.

^{Written for commit 55e866a. Summary will update on new commits.}

[linear-code]

DNO-987

[changeset-bot]

🦋 Changeset detected

Latest commit: 55e866a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
server	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cubic-dev-ai]

Running ultrareview automatically — High-risk roll: adds default-off shadow/enforce gating for MCP killswitch evaluation across serving and lifecycle paths, modifies shared PostHog flag parsing, and reworks platform break-glass composition; a subtle bug could break MCP tool calls or killswitch management.. I'll post findings when complete.

[cubic-dev-ai]

Ultrareview completed in 22m 28s

All reported issues were addressed

Linked issue analysis

Linked issue: DNO-987: docs: gate and operate the MCP Killswitch rollout

Status	Acceptance criteria	Notes
✅	Default-off off/shadow/enforce evaluation gate, with enforce precedence and no evaluator query in off mode	Rollout mode resolution and checkpoint tests cover off, shadow, and enforce transitions; off mode skips evaluation and shadow does not deny.
✅	Block new prescription activation and changes outside enforce mode while allowing deactivation and completed-operation replay	The lifecycle before-apply hook gates fresh non-deactivation mutations, while replay handling occurs before the hook and deactivation is explicitly allowed.
⚠️	Prevent production enablement until hosted and private evaluators are deployed across the fleet	The rollout runbook defines hosted/private fleet-readiness checks and mixed-version rollback guidance, but the diff does not add an automated deployment or fleet-uniformity enforcement mechanism.
⚠️	Run shadow evaluation and measure bounded latency, database load, matched/unmatched/failure outcomes, and authoritative identity coverage	Shadow evaluation behavior and rollout tests are implemented, and the runbook specifies the required metrics and thresholds; concrete dashboard or telemetry configuration is not shown in the diff.
⚠️	Provide alerts, restricted-cohort progression, break-glass incident handling, and rollback criteria	Runbooks document monitors, cohort progression, break-glass rules, incident response, and rollback criteria, but the issue's alerts are described as requiring external Datadog configuration rather than added here.
✅	Document exact supported coverage and avoid claiming M3 or broader AI/MCP coverage	The rollout runbook explicitly limits coverage to authenticated organization-user tools/call traffic on hosted and private proxy surfaces and lists unsupported identities, resources, and methods.
⚠️	Exercise mixed-version prevention, unavailable behavior, next-call activation, expiry, break-glass access, and rollback	The diff tests off/shadow/enforce transitions, management gating, replay, and lift behavior, while the runbooks describe the remaining operational checks; several requested scenarios are not directly covered by the shown tests.
✅	Avoid a TTL negative cache and require transactional visibility for future summary optimizations	The rollout runbook explicitly prohibits a TTL allow/deny cache and requires future summaries or caches to be updated transactionally with activation or changes.

<sub>Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 3 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>