Fix D31: find a subject by its own key, not by a guessed depth

`pool::subject_topic` hashes the subject key and nothing else. `carry_share`
uses it for every subject it carries, so two peers no longer have to agree about
how big the pool is in order to meet. Buckets keep dividing unknown subjects
among volunteers, which is the job D21 justified them for.

Proven by mutation, and the signature matters more than the pass:
`peers_that_disagree_about_pool_size_still_meet` forces one peer to carry as if
the pool were tiny and the other as if it were large. With the subject topic
they sync in 3 seconds. Without it, `carrier events: []` — sixty seconds and not
one sync session. The peers never discover each other, so in the field this bug
is silence rather than an error, which is why it has passed as flakiness.

The old bucket topic is carried alongside during the transition: moving a
subject's topic fails silently rather than loudly, and the live pool is a phone
driving an insulin pump. Marked for deletion once nothing uses it.

While both are carried, `broadcast` returns 2 — one operation to two
rendezvous, not two sends. Noted at the seam and in D31, because a doubling in
a diagnostic is what this project keeps misreading.

diaswarm-net 47 passed, 0 failed, from cargo's own exit code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: alex-poor

crates/diaswarm-net/src/pool.rs

@@ -131,6 +131,35 @@ fn bucket_score(peer_id: &str, bucket: u64) -> [u8; 32] {
     h.finalize().into()
 }
 
+/// Where a subject's operations are synced: a topic derived from the subject
+/// alone.
+///
+/// **NO DEPTH, NOTHING TO DISAGREE ABOUT — WHICH IS THE ENTIRE POINT.**
+/// [`bucket_topic`] hashes in a `depth` that every peer estimates from its own
+/// view of the pool size, so two peers that have discovered different numbers
+/// of peers derive unrelated topics and never meet. Observed at three peers:
+/// publisher `(2,1)`, relay `(2,1)`, follower `(3,2)`. A granted reader that
+/// guesses wrong cannot find its own subject, which is a hard failure of the
+/// thing the project is for. See [D31](../../docs/decisions.md).
+///
+/// **THIS IS P2PANDA'S TOPIC MODEL USED PLAINLY**: a 32-byte identifier for a
+/// set of data. It takes that half of the model and not the other — upstream
+/// assumes the identifier is a *secret* and protects it with PSI, which suits a
+/// private document shared by a known group. Here the data is public ciphertext
+/// by design (D1), holding is not reading, and a topic anybody can compute is
+/// how a volunteer knows what to carry. Public ciphertext wants public
+/// rendezvous.
+///
+/// Buckets keep the job [D21](../../docs/decisions.md) justified them for:
+/// dividing *unknown* subjects among volunteers. Finding a subject you were
+/// granted is not that job.
+pub fn subject_topic(subject: &str) -> [u8; 32] {
+    let mut h = Sha256::new();
+    h.update(b"diaswarm-subject-v1");
+    h.update(subject.to_ascii_lowercase().as_bytes());
+    h.finalize().into()
+}
+
 /// The gossip topic for a bucket, as 32 bytes.
 ///
 /// Depth is mixed in so that a bucket at depth 6 and its child at depth 7 are

crates/diaswarm-net/src/share.rs

@@ -103,8 +103,32 @@ pub async fn carry_share(
 
     let before = replicator.carried().len();
     for subject in &wanted {
-        let topic = crate::pool::bucket_topic(depth, crate::pool::bucket_of(subject, depth));
-        let _ = replicator.carry(topic, subject).await;
+        // **THE RENDEZVOUS, AND IT HAS NO PARAMETERS.** A subject you were
+        // granted is known by key; finding it is not a sharding problem. See
+        // [D31](../../docs/decisions.md) and `pool::subject_topic`.
+        let _ = replicator.carry(crate::pool::subject_topic(subject), subject).await;
+
+        // **AND THE OLD BUCKET TOPIC, FOR AS LONG AS ANYTHING USES IT.**
+        // Changing where a subject is synced is not a wire-format change that
+        // fails loudly — two peers on different topics simply never meet, which
+        // looks exactly like the network being quiet. The live pool is a phone
+        // driving an insulin pump and a follower somebody reads, so the new
+        // build joins both and nothing goes dark on upgrade.
+        //
+        // DELETE THIS once no peer on the old topic remains. It costs one extra
+        // subscription per carried subject, and it carries the depth bug with
+        // it: if this peer guesses a different depth from its neighbour, the
+        // legacy topic misses and the subject topic above still meets.
+        //
+        // **AND IT DOUBLES `pushed`, WHICH IS NOT A FAULT.**
+        // `Replicator::broadcast` returns the number of *topics* it published
+        // on, so while both are carried a phone's `shadow agrees — … pushed 2`
+        // is one operation sent to two rendezvous, not two sends of one
+        // operation. It drops back to 1 when this block goes. Saying so here
+        // because a doubling in a diagnostic is exactly the kind of thing this
+        // project has spent days misreading.
+        let legacy = crate::pool::bucket_topic(depth, crate::pool::bucket_of(subject, depth));
+        let _ = replicator.carry(legacy, subject).await;
     }
     let after = replicator.carried();
 

crates/diaswarm-net/tests/keys_pool.rs

@@ -195,3 +195,77 @@ async fn a_stranger_carries_a_keys_subject_it_was_never_introduced_to() {
          one copy that anyone can find and D15 still does not hold"
     );
 }
+
+/// PEERS THAT DISAGREE ABOUT THE POOL SIZE STILL MEET.
+///
+/// **THE BUG D31 RECORDS, WRITTEN AS A TEST.** `bucket_topic` hashes in a
+/// `depth` that each peer estimates from its own view of how many peers exist.
+/// Two peers that have discovered different numbers derive unrelated topics and
+/// never meet — observed live at three peers: publisher `(2,1)`, relay `(2,1)`,
+/// follower `(3,2)`. A granted reader that guesses wrong cannot find its own
+/// subject, and the symptom is silence.
+///
+/// Here the disagreement is forced rather than waited for: the publisher
+/// carries as if the pool were tiny and the follower as if it were large. On
+/// the bucket topic they would share nothing. On `subject_topic` there is no
+/// depth to disagree about, so they meet.
+#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
+async fn peers_that_disagree_about_pool_size_still_meet() {
+    let net = network_id("keys-depth-disagreement");
+    let rng = Rng::default();
+    let days = 2i64;
+
+    let subject_key = SigningKey::generate();
+    let author = subject_key.verifying_key();
+    let subject_hex = author.to_hex();
+    let mut subject = Vault::open(tmp("dd-subject"), OFFSET, &subject_key).unwrap();
+    let (mgr, _bundle) = Vault::key_bundle(&rng).unwrap();
+
+    let subject_store = SqliteStoreBuilder::memory().build().await.unwrap();
+    let create = subject.create(mgr).unwrap();
+    wire::publish_control(&subject_store, &subject_key, &create).await.unwrap();
+    for e in 0..days {
+        let segment = subject.seal(26_000 + e, &day(26_000 + e, 100.0)).unwrap();
+        wire::publish(&subject_store, &subject_key, &segment).await.unwrap();
+    }
+
+    let pub_swarm = Swarm::join_network(tmp("dd-pub"), SigningKey::generate(), net).await.unwrap();
+    let (pe, pg) = pub_swarm.parts();
+    let publishing = KeysReplicator::keys(subject_store.clone(), pe, pg).await.unwrap();
+
+    let car_store = SqliteStoreBuilder::memory().build().await.unwrap();
+    let car_swarm = Swarm::join_network(tmp("dd-car"), SigningKey::generate(), net).await.unwrap();
+    let (ce, cg) = car_swarm.parts();
+    let carrying = KeysReplicator::keys(car_store.clone(), ce, cg).await.unwrap();
+
+    // ---- the disagreement, made explicit -------------------------------
+    //
+    // Depth 1 is "I have seen two peers"; depth 9 is "I have seen hundreds".
+    // These are the topics each side WOULD have used, and they share nothing.
+    let as_if_small = pool::bucket_topic(1, pool::bucket_of(&subject_hex, 1));
+    let as_if_large = pool::bucket_topic(9, pool::bucket_of(&subject_hex, 9));
+    assert_ne!(as_if_small, as_if_large, "the premise of this test is a real disagreement");
+    publishing.carry(as_if_small, &subject_hex).await.unwrap();
+    carrying.carry(as_if_large, &subject_hex).await.unwrap();
+
+    // ---- and the topic that does not care ------------------------------
+    let meeting = pool::subject_topic(&subject_hex);
+    publishing.carry(meeting, &subject_hex).await.unwrap();
+    carrying.carry(meeting, &subject_hex).await.unwrap();
+
+    let mut held = 0u32;
+    for _ in 0..60 {
+        tokio::time::sleep(Duration::from_secs(1)).await;
+        held = segments_held(&car_store, &author).await;
+        if held as i64 >= days {
+            break;
+        }
+    }
+    assert_eq!(
+        held as i64, days,
+        "two peers that disagreed about the pool size did not meet on the subject \
+         topic either, so the depth is not the only thing keeping them apart \
+         (carrier events: {:?})",
+        carrying.events()
+    );
+}

docs/decisions.md

@@ -936,8 +936,40 @@ every peer's estimate to land in the same bucket of `depth_for`, and there are
 more ways to disagree at 21 bits than at 1. The test's assertion now prints all
 three depths on failure, so the next occurrence names its own cause.
 
-*Blocks:* honest claims about pool behaviour above a handful of peers, and it is
-a live intermittent defect below that.
+#### ✅ FIXED 2026-09-14 — `pool::subject_topic`, and the old topic kept beside it
+
+`carry_share` now carries every subject at `subject_topic(subject)` — a hash of
+the subject key, no depth, nothing to disagree about. Buckets keep the job
+[D21](#d21--replication-is-p2panda-log-sync-over-the-pools-own-topics)
+justified them for: dividing *unknown* subjects among volunteers.
+
+**Proven by mutation, and the failure signature is the part worth keeping.**
+`peers_that_disagree_about_pool_size_still_meet` forces the disagreement rather
+than waiting for it — one peer carries as if the pool were tiny (depth 1), the
+other as if it were large (depth 9). With the subject topic they sync in **3
+seconds**. With it removed:
+
+```
+carrier events: []      — sixty seconds, no sync sessions at all
+```
+
+Not a failed session, not a retry: the peers never discover each other. **In the
+field this bug is silence, not errors**, which is exactly why it has been read
+as flakiness for as long as it has existed.
+
+**The old bucket topic is carried as well, for now.** Moving where a subject
+syncs is not a wire-format change that fails loudly — peers on different topics
+simply never meet. The live pool is a phone driving an insulin pump and a
+follower somebody reads, so the new build joins both. The block is marked for
+deletion once no peer remains on the old topic.
+
+⚠️ **While both are carried, `pushed` reads 2 rather than 1.**
+`Replicator::broadcast` returns the number of *topics* it published on, so
+`shadow agrees — … pushed 2` is one operation sent to two rendezvous, not two
+sends. It returns to 1 when the legacy block goes.
+
+*Still open:* the divergence in §2 is recorded, not resolved — PSI remains inert
+here and must not be counted as protection.
 
 ### D30 · If you read it, you carry it — *ayni*, as an invariant