The v2 statement store persists a statement only if the local node is among the K closest (by XOR distance in DHT key space) to the statement's topic. This is a question about the whole network, but each node answers it from a local list of statement-store peers it happens to know.
That list is inherently incomplete. A peer counts only after we connect to it and learn via Identify that it supports the statement protocol — DHT discovery alone yields bare PeerIds. The random-walk cost of confirming peers scales with the size of the whole DHT, not with the number of statement-store nodes, and there is no iterative lookup that would converge to the true K closest.
In practice a node knows only a fraction of the statement-store nodes, and nobody can measure how large that fraction is. Both failure modes follow directly from it:
- Over-claiming: a node that doesn't see its closer neighbors believes it is among the K closest, so far more than K nodes store each topic. Wastes disk, recoverable.
- Under-claiming: a node that wrongly judges itself not among the K closest never persists the statement — it is held in memory, propagated once, and dropped. Data loss.
Estimate: may work at 1–2k statement-store nodes, unknown at 10k, will not work at 100k+ (IPFS-like scale).
Possible directions
- Accept the passive view for the first cut and let knowledge grow as connections happen.
- RFC-0059 (suggested in #11933 (comment)): statement-store nodes register as DHT providers under a capability key, so
get_providers returns the full list via iterative lookup, and additionally join a secondary DHT of statement-store nodes only — where "K closest to topic" is answered by a regular Kademlia lookup instead of a local list. Addresses the root cause.
Full analysis: DHT Topology construction
The v2 statement store persists a statement only if the local node is among the K closest (by XOR distance in DHT key space) to the statement's topic. This is a question about the whole network, but each node answers it from a local list of statement-store peers it happens to know.
That list is inherently incomplete. A peer counts only after we connect to it and learn via Identify that it supports the statement protocol — DHT discovery alone yields bare
PeerIds. The random-walk cost of confirming peers scales with the size of the whole DHT, not with the number of statement-store nodes, and there is no iterative lookup that would converge to the true K closest.In practice a node knows only a fraction of the statement-store nodes, and nobody can measure how large that fraction is. Both failure modes follow directly from it:
Estimate: may work at 1–2k statement-store nodes, unknown at 10k, will not work at 100k+ (IPFS-like scale).
Possible directions
get_providersreturns the full list via iterative lookup, and additionally join a secondary DHT of statement-store nodes only — where "K closest to topic" is answered by a regular Kademlia lookup instead of a local list. Addresses the root cause.Full analysis: DHT Topology construction