feat: add data export management API - #5792
Conversation
🦋 Changeset detectedLatest commit: a76aa6c The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Running ultrareview automatically — 'Adds a new tenant-scoped data export API with authorization, write-only encrypted secrets, row-lock concurrency, and audit events across 80 files and generated SDKs — a bug in project scoping, secret handling, or route lifecycle could leak data or credentials across tenants.'. I'll post findings when complete. |
There was a problem hiding this comment.
Ultrareview completed in 15m 37s
All reported issues were addressed across 80 files
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 24 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
aae73d2 to
a414ea0
Compare
This comment has been minimized.
This comment has been minimized.
9043ebb to
deb74bb
Compare
This comment has been minimized.
This comment has been minimized.
|
Flaky ClickHouse startup; rerun passed. |
There was a problem hiding this comment.
All reported issues were addressed across 54 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
Builds on the database invariant from #5859.
Summary
dataExportsGoa management service, SQLc repository, generated Go/TypeScript SDK surfaces, and genericlistDestinations,createDestination,updateDestination, anddeleteDestinationoperations alongside route CRUD.destination_typediscriminator and transport-specific configuration. OTEL configuration remains typed underotel; handlers dispatch CRUD by type and aggregate per-table list results in stable order so future SIEM and S3 destinations can live in their own tables without flattening transport-specific fields into one model.org:readfor reads andorg:adminfor mutations while retaining the selected-project header and project-pinned persistence. Every repository query is pinned by both organization and project.otel_destination_id; future destination types can add sibling slots on the same route row.sensitive_dataegress policy, defaulting missing database values toexclude. This PR does not implement payload sanitization or define the eventual field-removal contract.Motivation
Migration PR #5859 establishes one non-deleted route row per project and data source while retaining nullable typed destination slots. This PR supplies the organization-authorized application boundary needed to manage that schema safely without making the management API OTEL-specific. Transport-specific configuration and persistence remain isolated, allowing SIEM, S3, and other destination types to use separate tables while sharing one stable destination and routing API.