Skip to content

refactor: route metadata lifecycle operations through the Metastore trait - #146

Merged
VibeSWE merged 3 commits into
mainfrom
refactor/metastore-metadata-boundary
Nov 11, 2026
Merged

VibeSWE merged 3 commits into
mainfrom
refactor/metastore-metadata-boundary

Conversation

@VibeSWE

@VibeSWE VibeSWE commented Nov 11, 2026 •

Copy link
Copy Markdown

Problem

Metastore (src/metastore/metastore_traits.rs) is meant to be the door to Parseable's metadata, but several call sites read or wrote metadata objects directly through ObjectStorage. That makes the trait an incomplete seam: a second metastore backend would still leave some metadata in the bucket.

Change

A behaviour-preserving refactor. Nine call sites now go through the trait: the health check, the startup latency probe, stream existence and deleting checks, stream and date-partition deletion, and tenant discovery. Seven trait methods are added, each implemented for the object-store backend by moving the existing logic:

async fn check_health(&self, tenant_id: &Option<String>) -> Result<(), MetastoreError>;
async fn read_latency(&self, tenant_id: &Option<String>) -> std::time::Duration;
async fn stream_exists(&self, stream_name: &str, tenant_id: &Option<String>) -> Result<bool, MetastoreError>;
async fn is_stream_deleting(&self, stream_name: &str, tenant_id: &Option<String>) -> Result<bool, MetastoreError>;
async fn delete_stream(&self, stream_name: &str, tenant_id: &Option<String>) -> Result<(), MetastoreError>;
async fn delete_stream_partition(&self, stream_name: &str, date: &str, tenant_id: &Option<String>) -> Result<(), MetastoreError>;
async fn list_tenant_candidates(&self) -> Result<Vec<String>, MetastoreError>;

The same objects are touched, in the same order, with the same errors. No conditional writes, caching or race fixes are introduced.

A guard test fails when new code writes or prefix-deletes a metadata path directly on object storage outside the approved adapters. It is a text-level check: it does not catch alternate APIs, macros, or changes inside the exempt adapters.

What stays on object storage

Parquet I/O, first/latest event times derived from directory names, legacy partition discovery and physical occupancy checks. These are data-plane operations. Staging and hot-tier state stay node-local. The full classification (33 metadata operation groups, 10 data-plane, 4 node-local, 1 unclear) is in docs/metastore-boundary-audit.md.

One side effect to know about

Stream and retention deletes now run on the metastore's storage client, which has its own request limiter, where they used to run on the data client. Keys, operations and errors are unchanged; only the request budget they draw from moved.

Not fixed here

The unconditional read-modify-write updates of stream.json, manifests, schemas, RBAC, alert state, filters and the node registry are unchanged and tracked in #141.

Tests

Library suite passes (1,379 passed, 4 ignored). Local-filesystem fixtures check exact bytes, discovery, deletion errors, and tenant and date isolation. An independent review compared every reroute against main and found the object-store behaviour equivalent.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

VibeSWE and others added 2 commits November 11, 2026 13:34
The floating quay.io/parseablehq/quest:main image now expects 202 from
stream deletion. This server still deletes synchronously and returns 200,
so TestSmokeRbacBasic and TestSmokeRoles fail on main and on every pull
request. The previous image digest is no longer available from the
registry, so build Quest from the last source commit before that change.

The compose files take the image and pull policy from QUEST_IMAGE and
QUEST_PULL_POLICY and keep their previous defaults.

Refs #151

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@VibeSWE
VibeSWE merged commit 87dab9a into main Nov 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant