Stop the sorted-page cost guard measuring a cache, and give it a clock-free twin

`Build with Java 17 in MacOS` failed on this guard in 5 of the last 12 runs, and
only ever on macOS. Two causes, both in the test.

**It was warmed into meaninglessness.** `timeOf` ran the query once to warm and
then timed three more. But re-querying a collection that was just written
measures a cache, not a decode: over these same 2000 fat rows a blocking sort
costs 353 ms, then 83 ms, then 0.8 ms. By the timed runs, the decode this guard
exists to detect had already been cached away, and what was left was one to five
milliseconds of JIT and scheduling — which reads 0.6x on an idle laptop and 4.9x
on a loaded runner. Each half is now measured once on a collection no query has
touched, and sampled over several fresh ones with the quickest kept, because
contention can only ever add time.

**It compared a fat collection against a lean one**, assuming the index walk
cancelled between them. It does not: the fat collection's index lives in a file
two orders of magnitude larger, so walking it costs more I/O for reasons that
have nothing to do with decoding. Measured cold, a fat sorted page cost 4.6x a
lean one *with the optimisation present and working* — a floor sitting above the
3x threshold, which is the failure CI kept reporting. Both halves are now the
same collection shape sorted two ways, indexed against unindexed, so row count,
document size, file and returned row all cancel and only the discarded-row
decode is left. Measured 4.5x on the worst run of a laptop under load and 45x on
the best, against a 2x threshold; a regression does not narrow that ratio, it
removes it, because the two halves become one code path.

**And a second test with no clock in it at all.** A cost guard's failure mode is
not going red, it is going green while measuring nothing, and on a shared runner
a timing threshold can only ever be a compromise.
`testSortedPageIsPlannedFromTheIndex` asserts the planner sets
`sortIndexDescriptor` for a limited sort on an indexed field and leaves it unset
for an unindexed one. That cannot be made to flake.

Both were confirmed to fail on a planted regression rather than pass vacuously,
and they catch different ones: disabling the execution path fails the cost guard
and not the plan test, disabling the planner fails the plan test. Test-only; no
library code is touched. mvstore module green at 5386 tests.

Author: anidotnet

nitrite-mvstore-adapter/src/test/java/org/dizitart/no2/integration/collection/CollectionSortedFindCostTest.java

@@ -18,6 +18,8 @@
 package org.dizitart.no2.integration.collection;
 
 import org.dizitart.no2.Nitrite;
+import org.dizitart.no2.collection.DocumentCursor;
+import org.dizitart.no2.collection.FindPlan;
 import org.dizitart.no2.collection.Document;
 import org.dizitart.no2.collection.FindOptions;
 import org.dizitart.no2.collection.NitriteCollection;
@@ -35,6 +37,8 @@
 import static org.dizitart.no2.filters.Filter.ALL;
 import static org.dizitart.no2.integration.TestUtil.deleteDb;
 import static org.dizitart.no2.integration.TestUtil.getRandomTempDbFile;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
 import static org.junit.Assert.assertTrue;
 
 /**
@@ -46,15 +50,33 @@
  * gap grows with document size, not just document count. Taking the sort keys from the index
  * removes the decode: over 2000 rows carrying a 150-element array the sorted page went from
  * ~115 ms to ~2 ms.
+ * <p>
+ * <b>Both halves of every comparison here are the same collection shape sorted two different
+ * ways</b>, never a fat collection against a lean one. That earlier framing assumed the index
+ * walk cancelled between the halves, and it does not: the fat collection's index lives in a
+ * file two orders of magnitude larger, so walking it costs more I/O for reasons that have
+ * nothing to do with decoding. Measured cold on macOS, a fat sorted page cost 4.6x a lean one
+ * <i>with the optimisation present and working</i> - through a 3x threshold, which is why this
+ * guard failed on that runner and nowhere else.
  *
  * @author Anindya Chatterjee
  */
 public class CollectionSortedFindCostTest {
     private static final int ROWS = 2000;
     private static final int PAYLOAD = 150;
 
+    /**
+     * Fresh collections per sample, because the cost under test is only visible on a cold one.
+     * Re-querying a collection that was just written measures a cache: over these same 2000 fat
+     * rows a blocking sort cost 353 ms, then 83 ms, then 0.8 ms. The old guard warmed once and
+     * timed the three runs after it, so it was comparing two numbers from which the decode -
+     * the whole subject of the test - had already been cached away.
+     */
+    private static final int SAMPLES = 3;
+
     private final String fileName = getRandomTempDbFile();
     private Nitrite db;
+    private int collectionSeq;
 
     @Before
     public void setUp() {
@@ -72,61 +94,113 @@ public void tearDown() {
     }
 
     /**
-     * The same query, the same row count, the same index - only the size of the documents
-     * differs. A sorted page that decodes every row pays for the payload of every row, so the
-     * fat collection costs many times the lean one. A sorted page that decodes only the row it
-     * returns costs about the same either way.
+     * The regression detector, and it has no clock in it.
+     * <p>
+     * A sorted page that can be answered from an index says so in its {@link FindPlan}: the
+     * planner sets {@code sortIndexDescriptor} when the query has no filter, one sort field, a
+     * limit, and a simple index on exactly that field. When it is set the page reads its order
+     * from the index and fetches only the rows it returns; when it is not, the blocking sort
+     * decodes every stored document. Asserting on the plan tests that decision directly, and
+     * cannot be made to flake by a loaded machine - which the cost guard below can.
+     */
+    @Test
+    public void testSortedPageIsPlannedFromTheIndex() {
+        NitriteCollection collection = fatCollection("planned");
+
+        DocumentCursor fromIndex = collection.find(ALL,
+            FindOptions.orderBy("seq", SortOrder.Descending).limit(1));
+        assertNotNull("a limited sort on an indexed field was not planned from the index, so it "
+                + "is decoding every stored document to order rows it discards",
+            fromIndex.getFindPlan().getSortIndexDescriptor());
+
+        // The control: the same query shape on a field with no index cannot use that path, so a
+        // non-null descriptor above would prove nothing on its own.
+        DocumentCursor blocking = collection.find(ALL,
+            FindOptions.orderBy("unindexed", SortOrder.Descending).limit(1));
+        assertNull("a sort on an unindexed field claims an index to read its order from",
+            blocking.getFindPlan().getSortIndexDescriptor());
+    }
+
+    /**
+     * The same query, over the same rows, ordered by an indexed field and by an unindexed one.
+     * Only the second can be answered from an index; the first is what this optimisation
+     * exists to make cheap. Everything else - row count, document size, the file, the single
+     * row returned - is identical between the halves, so what is left is the decode of the
+     * {@link #ROWS} rows neither query should have returned.
      * <p>
-     * Both halves walk an index of identical size and shape, so that cost cancels; what does
-     * not cancel is the payload of the rows each one decodes. The page is deliberately one row
-     * rather than twenty: returning a fat document legitimately costs more than returning a
-     * lean one, and that difference is the floor of this ratio, so the fewer rows the page
-     * returns the more of the ratio is the {@link #ROWS} rows it should never have touched.
+     * <b>Cold, and the best of {@link #SAMPLES}.</b> Each half is measured once on a collection
+     * no query has touched, because a second look at the same collection measures a cache
+     * rather than a decode. That makes any single sample noisy, so each half is sampled on
+     * several fresh collections and the quickest is kept: contention can only ever add time, so
+     * the quickest run is the one least contaminated by the machine.
      * <p>
-     * Deliberately not "sorted page vs. unsorted page", whose halves do not share the index
-     * walk, nor "sorted page vs. full drain", whose halves share nothing at all. Both halves
-     * here are measured back to back, under whatever load the machine is under.
+     * The threshold is 2x against a measured 4.5x on the worst run of a laptop under load and
+     * 45x on the best. A regression does not narrow this ratio - it removes it, because the two
+     * halves become the same code path and the ratio becomes 1.
      */
     @Test
-    public void testSortedPageCostDoesNotFollowDocumentSize() {
-        double lean = sortedPageCost("lean", 0);
-        double fat = sortedPageCost("fat", PAYLOAD);
+    public void testSortedPageDoesNotDecodeTheRowsItDiscards() {
+        warmTheQueryPath();
+
+        double indexed = Double.MAX_VALUE;
+        double blocking = Double.MAX_VALUE;
+        for (int sample = 0; sample < SAMPLES; sample++) {
+            // Alternated so that any drift over the run lands on both halves alike.
+            blocking = Math.min(blocking, coldSortedPageCost("unindexed"));
+            indexed = Math.min(indexed, coldSortedPageCost("seq"));
+        }
 
-        assertTrue("a sorted page over fat documents took " + fat + "ms against " + lean
-                + "ms over lean ones, same row count - it is still decoding rows it discards",
-            fat < lean * 3);
+        assertTrue("a sorted page over an indexed field took " + indexed + "ms against "
+                + blocking + "ms over an unindexed one, same rows and same documents - it is "
+                + "still decoding the rows it discards", blocking > indexed * 2);
     }
 
-    private double sortedPageCost(String name, int payloadSize) {
-        NitriteCollection collection = db.getCollection(name);
+    /**
+     * JIT only. Run on its own collection so the measured ones stay cache-cold.
+     */
+    private void warmTheQueryPath() {
+        NitriteCollection collection = fatCollection("warm");
+        for (int i = 0; i < 3; i++) {
+            drain(collection, "seq");
+            drain(collection, "unindexed");
+        }
+    }
+
+    private double coldSortedPageCost(String sortField) {
+        NitriteCollection collection = fatCollection("cost");
+        long start = System.nanoTime();
+        drain(collection, sortField);
+        return (System.nanoTime() - start) / 1e6;
+    }
+
+    private static void drain(NitriteCollection collection, String sortField) {
+        FindOptions page = FindOptions.orderBy(sortField, SortOrder.Descending).limit(1);
+        for (Document ignored : collection.find(ALL, page)) {
+            // force the fetch
+        }
+    }
+
+    /**
+     * {@link #ROWS} rows carrying a {@link #PAYLOAD}-element array, except the one the page
+     * returns. Descending on either sort field returns an end of the range, and giving that row
+     * a payload would put the cost of decoding it into a measurement that is supposed to
+     * contain only rows that were never returned.
+     */
+    private NitriteCollection fatCollection(String prefix) {
+        NitriteCollection collection = db.getCollection(prefix + "-" + collectionSeq++);
         collection.createIndex(IndexOptions.indexOptions(IndexType.NON_UNIQUE), "seq");
 
         for (int i = 0; i < ROWS; i++) {
-            Document document = Document.createDocument("seq", i);
-            if (payloadSize > 0) {
-                List<Document> payload = new ArrayList<>(payloadSize);
-                for (int w = 0; w < payloadSize; w++) {
+            Document document = Document.createDocument("seq", i).put("unindexed", ROWS - i);
+            if (i != 0 && i != ROWS - 1) {
+                List<Document> payload = new ArrayList<>(PAYLOAD);
+                for (int w = 0; w < PAYLOAD; w++) {
                     payload.add(Document.createDocument("text", "word" + w).put("start", w * 300));
                 }
                 document.put("payload", payload);
             }
             collection.insert(document);
         }
-
-        FindOptions page = FindOptions.orderBy("seq", SortOrder.Descending).limit(1);
-        return timeOf(() -> {
-            for (Document ignored : collection.find(ALL, page)) {
-                // force the fetch
-            }
-        });
-    }
-
-    private static double timeOf(Runnable task) {
-        task.run(); // warm
-        long start = System.nanoTime();
-        for (int i = 0; i < 3; i++) {
-            task.run();
-        }
-        return (System.nanoTime() - start) / 3e6;
+        return collection;
     }
 }