Drive admin data operations from a typed catalog and stream job events

[Medformatik]

Problem

The admin data-workflow UI and server handlers describe the same privileged operations separately. Forms, validation, help text, and effects can drift, and job details poll every two seconds instead of receiving a resumable event stream.

Current state

• DataWorkflowsPage exposes hard-coded OSM, font, Overpass, link cleanup, Transitous, Overture, search, and transit operations.
• apps/api/src/services/admin-job-handlers.ts maps operation types to fixed server-side effects.
• The separate ops agent has a durable journal, cancellation, progress, logs, and a strong boundary against caller-supplied argv/paths.
• Destructive confirmation exists, but there is no shared declarative operation catalog or SSE job stream.

Proposed solution

Create a typed operation catalog that remains server-authored and generates the admin form contract, validation, descriptions, confirmation policy, redacted execution preview, and handler registration. Add a cursor-based SSE endpoint for job state, progress, and log events with reconnect/backfill and polling fallback.

Acceptance criteria

• A single typed catalog declares operation ID/version, input schema, defaults, help text, risk/confirmation policy, capabilities, and redaction rules.
• The admin forms and API validation derive from that catalog rather than duplicate switch statements.
• Effects remain fixed trusted server code; catalog entries cannot introduce arbitrary commands, paths, environment variables, or URLs.
• Before confirmation, the UI shows a redacted, human-readable effect/parameter preview.
• Catalog/handler drift fails tests or startup validation.
• The job event endpoint uses authenticated SSE with monotonic cursors, heartbeats, reconnect, bounded history, and terminal events.
• Reconnect can backfill missed events without duplication; expired cursors fall back to a snapshot.
• The UI updates status, progress, and logs from SSE and retains a bounded polling fallback.
• Cancellation, authorization, disconnect cleanup, slow consumers, and log redaction have integration tests.
• Metrics cover active streams, reconnects, lag, dropped/backfilled events, and handler duration.
• Existing durable journal/recovery behavior remains intact.

Alternatives considered

Continuing hard-coded forms preserves drift. WebSockets add bidirectional complexity to a primarily server-to-client stream. Allowing operators to compose arbitrary commands would weaken the current security boundary.

Area

Admin UI, app API, ops agent, observability.

[Medformatik]

Implemented in e7285d5 on main.

• One server-authored operation catalog (apps/api/src/services/admin-operation-catalog.ts) now defines input schema, form fields, defaults, risk/confirmation policy, a fixed effect, and a redacted preview per operation. Startup validation and tests fail on catalog/handler drift.
• GET /api/admin/operations, POST .../:id/preview, and POST .../:id/run replace the hand-kept data action endpoint; the admin forms render from the served contract and show the server preview before confirming.
• Job details stream over SSE (GET /api/admin/jobs/:id/events) with per-job cursors, Last-Event-ID backfill, snapshot fallback for expired cursors, heartbeats, terminal events, slow-consumer disconnects, and a polling fallback in the UI. GET /api/admin/jobs/stream-metrics exposes stream and handler-duration counters.
• Out of scope, unchanged: backup and bulk-service jobs keep their routes; data-manager jobs keep polling.