ERS: the generator builds the leaves of its reduced hash tree once, dropping them when data or a previous chain is added, and no longer builds a set of data groups nothing reads.

Author: dghgit

docs/releasenotes.md

@@ -26,6 +26,7 @@ Date: 2026, TBD
 - The BCFKS key store derived its scrypt keys with the block size r in place of the parallelization parameter p, while writing the p the caller configured out to the store: BcFKSKeyStoreSpi passed getBlockSize() to SCrypt.generate for both arguments and never read the encoded parallelization parameter at all, so every store whose ScryptConfig gave a p other than its r encoded parameters that do not derive its own keys. The store was self-consistent - BC read back what BC wrote - but a conformant RFC 7914 reader computed a different key and so failed the integrity check and the store decryption, and BC could not open such a store written by anyone else. Derivation now follows RFC 7914. A store written by 1.86 or earlier is still read: the integrity check is retried under the old convention, and where a signature check leaves no MAC to settle it the store decryption is retried instead, in both cases reporting the failure of the encoded parameters rather than of the fallback. The write side is governed by org.bouncycastle.bcfks.scrypt_p_eq_r, default true, which writes p equal to r whatever the ScryptConfig asked for: the two conventions then agree, so a store written here is both RFC 7914 correct and readable by 1.86 and earlier. Clearing the property honours the configured p, which those releases cannot read unless p already equals r; the default is intended to become false in a later release, once enough of the installed base is writing parameters that describe themselves. Loading with a BCFKSLoadStoreParameter carrying a ScryptConfig accepts an encoded p equal to either the configured p or the block size, so a store round trips under the configuration that wrote it whichever way the property was set; every other parameter is compared as before. The parallelization parameter is now bounded alongside the cost parameter before the derivation, as the PKCS#8 and PKCS#12 scrypt paths already bound it.
 - Building an evidence record was cubic in the number of data objects: SortedHashList and SortedIndexedHashList held their hashes in a LinkedList and found each insertion point by walking it with get(index), so a single add() was quadratic in the position it inserted at and building a list of n hashes cubic, and both lists sit on the generation path - the reduced hash tree of ERSArchiveTimeStampGenerator, the Merkle tree of BinaryTreeRootCalculator.computeRootHash(), and the hash list of every ERSDataGroup. Each now collects its hashes and sorts them once, in toList(); the sort is stable and the old insertion placed a hash after the last one comparing equal to it, which is where a stable sort puts it, so the order of the leaves and every root hash are unchanged. getFirst() answers with a scan rather than a sort and toList() sorts a copy, so neither accessor disturbs what has been added. Generating a time-stamp request over 8,000 data objects goes from about 210 seconds to under a tenth of a second, and 100,000 objects, which the old code could not reach in any practical time, takes about 0.2 seconds (github #2456).
 - An ERSDataGroup recomputed its hash on every request rather than taking it from the cache ERSCachingData exists to provide: the group overrode getHash(), which left the calculateHash() the cache calls unreachable - and wrong, as it copied the member hashes with a loop bounded by the size of the empty list it was copying into, so it would have digested nothing had anything reached it. The computation is back in calculateHash() and the override is gone, so a group's hash is computed once per digest algorithm and previous-chain hash, as every other ERSData's is. The value itself is unchanged.
+- ERSArchiveTimeStampGenerator rebuilt its reduced hash tree from the data objects on every call, so the usual generateTimeStampRequest() followed by generateArchiveTimeStamp() or generateArchiveTimeStamps() built it twice. The leaves are now built once and dropped when data or a previous chain is added. A Set of the data groups it had been given, which nothing ever read, has gone with it.
 
 ### 2.1.3 Additional Features and Functionality
 

pkix/src/main/java/org/bouncycastle/tsp/ers/ERSArchiveTimeStampGenerator.java

@@ -4,9 +4,7 @@
 import java.io.OutputStream;
 import java.math.BigInteger;
 import java.util.ArrayList;
-import java.util.HashSet;
 import java.util.List;
-import java.util.Set;
 
 import org.bouncycastle.asn1.x509.AlgorithmIdentifier;
 import org.bouncycastle.asn1.ASN1Encoding;
@@ -36,6 +34,7 @@ public class ERSArchiveTimeStampGenerator
 
     private ERSRootNodeCalculator rootNodeCalculator = new BinaryTreeRootCalculator();
     private byte[] previousChainHash;
+    private IndexedPartialHashtree[] partialHashtrees;
 
     public ERSArchiveTimeStampGenerator(DigestCalculator digCalc)
     {
@@ -50,6 +49,7 @@ public ERSArchiveTimeStampGenerator(DigestCalculator digCalc)
     public void addData(ERSData dataObject)
     {
         dataObjects.add(dataObject);
+        partialHashtrees = null;
     }
 
     /**
@@ -60,6 +60,7 @@ public void addData(ERSData dataObject)
     public void addAllData(List<ERSData> dataObjects)
     {
         this.dataObjects.addAll(dataObjects);
+        partialHashtrees = null;
     }
 
     void addPreviousChains(ArchiveTimeStampSequence archiveTimeStampSequence)
@@ -71,6 +72,7 @@ void addPreviousChains(ArchiveTimeStampSequence archiveTimeStampSequence)
         digOut.close();
 
         this.previousChainHash = digCalc.getDigest();
+        this.partialHashtrees = null;
     }
 
     /**
@@ -232,20 +234,18 @@ public List<ERSArchiveTimeStamp> generateArchiveTimeStamps(TimeStampResponse tsp
         return atss;
     }
 
+    // the leaves depend only on the data added and the previous chain hash, so they are built
+    // once and dropped by the methods that change either.
     private IndexedPartialHashtree[] getPartialHashtrees()
     {
-        List<IndexedHash> hashes = ERSUtil.buildIndexedHashList(digCalc, dataObjects, previousChainHash);
-        IndexedPartialHashtree[] trees = new IndexedPartialHashtree[hashes.size()];
-
-        Set<ERSDataGroup> dataGroupSet = new HashSet<ERSDataGroup>();
-        for (int i = 0; i != dataObjects.size(); i++)
+        if (partialHashtrees != null)
         {
-            if (dataObjects.get(i) instanceof ERSDataGroup)
-            {
-                dataGroupSet.add((ERSDataGroup)dataObjects.get(i));
-            }
+            return partialHashtrees;
         }
 
+        List<IndexedHash> hashes = ERSUtil.buildIndexedHashList(digCalc, dataObjects, previousChainHash);
+        IndexedPartialHashtree[] trees = new IndexedPartialHashtree[hashes.size()];
+
         // replace groups
         for (int i = 0; i != hashes.size(); i++)
         {
@@ -265,6 +265,8 @@ private IndexedPartialHashtree[] getPartialHashtrees()
             }
         }
 
+        partialHashtrees = trees;
+
         return trees;
     }