Skip to content

Align /v0/manifest with the proposed Bee listing API (prefix/delimiter/pagination) before the shape hardens #61

Description

@petfold

What exists

GET /v0/manifest/{addr} walks a Mantaray manifest server-side and returns its entries as JSON (crates/ant-gateway/src/retrieval.rs:2667-2694). Bee has no equivalent, so ant is the only Swarm node that can answer ls without the client embedding a Mantaray codec. Some good properties already there:

  • the listing keys off an immutable content-addressed root, so it's cached with immutable headers (set_immutable_cache_headers) — correct and free
  • ManifestEntryInfo carries path, reference, metadata, and size from the data root chunk's BMT span, with None distinguishing metadata-only from unfetchable (crates/ant-control/src/command.rs:816-828)
  • /v0/ namespacing keeps the bee Tier-A surface byte-for-byte compatible, which is the right instinct

The problem

There's a pending Bee API proposal for exactly this endpoint — ethersphere/bee#5535 (labelled triaged) with a draft read-path implementation in ethersphere/bee#5537 — specifying GET /manifest/{address}/{prefix} with S3-ListObjectsV2-style semantics. ant's shape is narrower:

bee#5535 / #5537 ant /v0/manifest/{addr}
{reference}/{prefix} address only — always full manifest from root
delimiter → shallow listing + commonPrefixes absent
limit / aftertruncated, nextMarker absent
sizes opt-in size always attempted
explicit partial-retrievability semantics not expressed in the response

The handler signature is (State(handle), Path(addr)) with no Query extractor, and ControlCommand::ListBzz { reference, bypass_cache, ack } (command.rs:148-152) takes no bound — so enumeration is unbounded by construction, not merely unexposed.

Two practical consequences:

  1. A large manifest returns as one JSON body. A 100k-entry dataset is enumerated in full, held in memory, and serialized in one response. Pagination in the proposal exists specifically to prevent this, with a node-enforced hard cap.
  2. Listing one directory costs the whole manifest. Without delimiter/prefix, ls somedir/ means transferring every path and filtering client-side — most of the round-trip saving the endpoint exists to provide, given away again.

Also, sizes is opt-in upstream because it costs one extra chunk read per entry; ant pays that on every listing, which is the wrong default for a plain ls.

Suggestion

Align /v0/manifest with the #5535 shape — prefix, delimiter, limit, after, sizes, returning commonPrefixes / truncated / nextMarker — while the endpoint is still young and /v0/-namespaced.

The upside is bigger than tidiness. Right now ant is a divergent extension: if the Bee API lands as proposed, clients face two incompatible listing APIs and ant's own is the odd one out. Aligned, ant becomes a reference implementation of a pending Bee API — which is a considerably stronger argument in that thread than a design sketch, and gives the Bee maintainers semantics and tests to copy rather than debate. The endpoint stays under /v0/ either way until Bee ships.

Concretely I'd adopt it from swarmfs, which does ls/find as a client-side trie walk today and would capability-detect the endpoint instead.

Caveats

  • I don't know whether /v0/manifest is deliberately minimal — if it exists to serve one internal consumer (gateway autoindex?) and a fuller API is out of scope, that's a fine answer, and this is then really a request for a note saying so.
  • Disclosure: I filed the Bee issue and PR referenced above, so I'm not a neutral party on the shape. The divergence concern stands regardless of whose shape wins — I'd rather ant and Bee agree than that Bee copy mine.
  • I read the HTTP and control layers, not the node-side walk. If a cap exists further down in the ListBzz handler, point 1 is weaker than stated.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions