Grain-128AEAD: write the second output segment of a chunked decryption after the first rather than over it, incorporating github PR #2447.

Author: Arpan0995

CONTRIBUTORS.md

@@ -542,7 +542,7 @@ We also wish to acknowledge financial and collaborative support from [CISCO](htt
 - jmeeder \<https://github.com/jmeeder\> - reporting that RFC 4998 evidence-record generation rejected time stamps from an authority naming the digest with NULL parameters where BC names it with them absent, both of which RFC 5754 requires a receiver to accept (issue #2379).
 - rimuln \<https://github.com/rimuln\> - diagnosis and fix for PKCS12 getCertificateAlias returning the alias of an unrelated certificate, tracing it to the alias and certificate enumerations of the keystore's certs table diverging in order once keys() enumerated a copy (issue #2384, PR #2385).
 - Yu Bao \<yubao&#064;paypal.com\> - reporting an API gap, on behalf of the PayPal Cyber Security Team, that the high-level OpenPGP message API (OpenPGPMessageProcessor / OpenPGPMessageInputStream) gave a caller no way to bound how far a compressed data packet expands, where the low-level PGPCompressedData it wraps has carried a bounded getDataStream(long) overload all along, and that OpenPGPPolicy exposed no equivalent property to set. Suggesting a protocol whitelist for CRL Distribution Point fetching, which is now the org.bouncycastle.x509.CRLDP_protocols property, and suggesting that an OCSP response was read up to the length the responder declared for itself, now capped by org.bouncycastle.ocsp.max_response_size, and suggesting bounds on the OpenPGP ASCII armor headers, now capped by org.bouncycastle.openpgp.max_armor_header_length and org.bouncycastle.openpgp.max_armor_headers.
-- Arpan Sharma \<https://github.com/Arpan0995\> - initial audit of BCPQC provider consistency starting with HQC, which led to the exposure of a number of issues in the JCA provider service interfaces for other BCPQC algorithms. In-depth auditing of PQC signature algorithms in the provider leading to the correction of a number of JCA API compliance issues. Initial implementation of the guard that lets a signature context be set on the composite ML-DSA services before initSign / initVerify, the composite counterpart of the base-engine fix for issue #2396 (issue #2412). Audit of the LMS / HSS stateful private key decoder, establishing that the level count, the one-time index and the persisted tree cache were all accepted without validation (issue #2414); that issue was also the inspiration for the further investigation which found the HSS and XMSS^MT decoders accepting a private key whose declared index disagreed with the traversal state stored beside it, allowing a one-time key to be used twice. Audit of the high-level OpenPGP API (org.bouncycastle.openpgp, org.bouncycastle.openpgp.api) and the bcpg packet layer beneath it, spanning certificate evaluation, message verification, decryption and signature subpacket parsing, which found certificates being used without the certifications RFC 9580 requires of them, malformed but signed content reaching ordinary reader code as unchecked exceptions, a truncated message being read as a clean end of message, and unverified plaintext being released ahead of the integrity check (issues #2417, #2424, #2426). Audit of the recipient side of the CMS RFC 9629 KEMRecipientInfo path, which found three sender-controlled fields leaving methods declared to throw CMSException as unchecked exceptions and the kekLength never being confirmed against the wrap algorithm, and which led to the same translation being added to the neighbouring AuthEnvelopedData constructor and the two streaming parsers (issue #2422). Audit of RSA-PSS parameter handling in the provider, which found Signature.setParameter reporting a rejected trailer field as an unchecked exception and leaving the rejected spec half-applied, so that the parameters reported for a signature were not the ones it had been made with (issue #2421). Audit of the PKCS#12 key store entry-setting boundary, which found setKeyEntry reporting a certificate whose public key the provider cannot resolve as an unchecked exception rather than the KeyStoreException it declares, and which led to the same guard being applied to setCertificateEntry and to the RFC 9579 PBMAC1 store, and to the rejected entry no longer being left behind in the store (issue #2419). Further auditing of the verification and expiration paths of the high-level OpenPGP API, which found a configured algorithm policy not being applied to inline message verification, an expired primary key still offering its subkeys, and a data signature being reported valid past its own expiration time. CMS SignedData and PKIX certification path test coverage for the eighteen Composite ML-DSA parameter sets, which had been reachable through the CMS and cert-path signature algorithm finders with no test exercising either (PR #2437).
+- Arpan Sharma \<https://github.com/Arpan0995\> - initial audit of BCPQC provider consistency starting with HQC, which led to the exposure of a number of issues in the JCA provider service interfaces for other BCPQC algorithms. In-depth auditing of PQC signature algorithms in the provider leading to the correction of a number of JCA API compliance issues. Initial implementation of the guard that lets a signature context be set on the composite ML-DSA services before initSign / initVerify, the composite counterpart of the base-engine fix for issue #2396 (issue #2412). Audit of the LMS / HSS stateful private key decoder, establishing that the level count, the one-time index and the persisted tree cache were all accepted without validation (issue #2414); that issue was also the inspiration for the further investigation which found the HSS and XMSS^MT decoders accepting a private key whose declared index disagreed with the traversal state stored beside it, allowing a one-time key to be used twice. Audit of the high-level OpenPGP API (org.bouncycastle.openpgp, org.bouncycastle.openpgp.api) and the bcpg packet layer beneath it, spanning certificate evaluation, message verification, decryption and signature subpacket parsing, which found certificates being used without the certifications RFC 9580 requires of them, malformed but signed content reaching ordinary reader code as unchecked exceptions, a truncated message being read as a clean end of message, and unverified plaintext being released ahead of the integrity check (issues #2417, #2424, #2426). Audit of the recipient side of the CMS RFC 9629 KEMRecipientInfo path, which found three sender-controlled fields leaving methods declared to throw CMSException as unchecked exceptions and the kekLength never being confirmed against the wrap algorithm, and which led to the same translation being added to the neighbouring AuthEnvelopedData constructor and the two streaming parsers (issue #2422). Audit of RSA-PSS parameter handling in the provider, which found Signature.setParameter reporting a rejected trailer field as an unchecked exception and leaving the rejected spec half-applied, so that the parameters reported for a signature were not the ones it had been made with (issue #2421). Audit of the PKCS#12 key store entry-setting boundary, which found setKeyEntry reporting a certificate whose public key the provider cannot resolve as an unchecked exception rather than the KeyStoreException it declares, and which led to the same guard being applied to setCertificateEntry and to the RFC 9579 PBMAC1 store, and to the rejected entry no longer being left behind in the store (issue #2419). Further auditing of the verification and expiration paths of the high-level OpenPGP API, which found a configured algorithm policy not being applied to inline message verification, an expired primary key still offering its subkeys, and a data signature being reported valid past its own expiration time. CMS SignedData and PKIX certification path test coverage for the eighteen Composite ML-DSA parameter sets, which had been reachable through the CMS and cert-path signature algorithm finders with no test exercising either (PR #2437). Identification and fix of the output offset in the AEAD stream cipher data operator that made Grain-128AEAD return corrupted plaintext from a decryption driven in chunks, with the streamed decryption coverage that had been missing (PR #2447).
 - Flowdalic \<https://github.com/Flowdalic\> - initial implementation of an AnimalSniffer-based Android API-level compatibility check for the Gradle build (PR #336).
 - hannesa2 \<https://github.com/hannesa2\> - initial Dependabot configuration for the Gradle and GitHub Actions ecosystems (PR #883).
 - vladhuma \<https://github.com/vladhuma\> - initial implementation of server-side OCSP stapling for the BCJSSE provider, on behalf of Thales Group (PR #1740).

core/src/main/java/org/bouncycastle/crypto/engines/AEADBaseEngine.java

@@ -767,7 +767,7 @@ public int processBytes(byte[] input, int inOff, int len, byte[] output, int out
                 if (available > 0)
                 {
                     this.len = available;
-                    processBufferDecrypt(input, inOff, output, outOff);
+                    processBufferDecrypt(input, inOff, output, outOff + rlt);
                     rlt += available;
                     len -= available;
                     inOff += available;

core/src/test/java/org/bouncycastle/crypto/test/Grain128AEADTest.java

@@ -36,13 +36,16 @@ public AEADCipher createInstance()
             }
         });
         CipherTest.checkAEADCipherMultipleBlocks(this, 1024, 7, 100, 128, 12, new Grain128AEADEngine());
-
+        // PARTLEN has to exceed the 8 byte tag length for a single processBytes() call to release
+        // both buffered tag bytes and bytes taken straight from the input.
+        CipherTest.checkAEADCipherMultipleBlocks(this, 1024, 19, 100, 128, 12, new Grain128AEADEngine());
 
         CipherTest.checkAEADParemeter(this, 16, 12, 8, 20, new Grain128AEADEngine());
 
         testSplitUpdate();
         testExceptions();
         testLongAEAD();
+        testStreamedDecryption();
     }
 
 
@@ -141,6 +144,57 @@ private void testLongAEAD()
         }
     }
 
+    private void testStreamedDecryption()
+        throws InvalidCipherTextException
+    {
+        byte[] key = Hex.decode("000102030405060708090A0B0C0D0E0F");
+        byte[] nonce = Hex.decode("000102030405060708090A0B");
+        byte[] ad = Hex.decode("000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E");
+        byte[] pt = new byte[64];
+        for (int i = 0; i != pt.length; i++)
+        {
+            pt[i] = (byte)i;
+        }
+
+        ParametersWithIV params = new ParametersWithIV(new KeyParameter(key), nonce);
+        Grain128AEADEngine grain = new Grain128AEADEngine();
+
+        grain.init(true, params);
+        grain.processAADBytes(ad, 0, ad.length);
+        byte[] ct = new byte[grain.getOutputSize(pt.length)];
+        int ctLen = grain.processBytes(pt, 0, pt.length, ct, 0);
+        ctLen += grain.doFinal(ct, ctLen);
+        isEquals("cipher text length", ct.length, ctLen);
+
+        grain.init(false, params);
+        grain.processAADBytes(ad, 0, ad.length);
+        byte[] oneShot = new byte[grain.getOutputSize(ctLen)];
+        int oneShotLen = grain.processBytes(ct, 0, ctLen, oneShot, 0);
+        oneShotLen += grain.doFinal(oneShot, oneShotLen);
+        isEquals("one shot plain text length", pt.length, oneShotLen);
+        isTrue("one shot decryption", Arrays.areEqual(pt, oneShot));
+
+        // A processBytes() call that releases buffered tag bytes and bytes taken straight from the
+        // input writes its output in two segments. Split the cipher text at every point so that the
+        // second chunk exceeds the 8 byte tag length and both segments are written by one call.
+        // doFinal() throws when the tag does not verify, so reaching the comparisons means the tag
+        // was accepted and only the plain text placement is under test.
+        for (int split = 1; split != ctLen; split++)
+        {
+            grain.init(false, params);
+            grain.processAADBytes(ad, 0, ad.length);
+
+            byte[] streamed = new byte[grain.getOutputSize(ctLen)];
+            int len = grain.processBytes(ct, 0, split, streamed, 0);
+            len += grain.processBytes(ct, split, ctLen - split, streamed, len);
+            len += grain.doFinal(streamed, len);
+
+            isEquals("streamed plain text length, split " + split, pt.length, len);
+            isTrue("streamed decryption does not match one shot, split " + split, Arrays.areEqual(oneShot, streamed));
+            isTrue("streamed decryption does not match plain text, split " + split, Arrays.areEqual(pt, streamed));
+        }
+    }
+
     private void testExceptions()
         throws InvalidCipherTextException
     {

docs/releasenotes.md

@@ -27,6 +27,7 @@ Date: 2026, TBD
 - Building an evidence record was cubic in the number of data objects: SortedHashList and SortedIndexedHashList held their hashes in a LinkedList and found each insertion point by walking it with get(index), so a single add() was quadratic in the position it inserted at and building a list of n hashes cubic, and both lists sit on the generation path - the reduced hash tree of ERSArchiveTimeStampGenerator, the Merkle tree of BinaryTreeRootCalculator.computeRootHash(), and the hash list of every ERSDataGroup. Each now collects its hashes and sorts them once, in toList(); the sort is stable and the old insertion placed a hash after the last one comparing equal to it, which is where a stable sort puts it, so the order of the leaves and every root hash are unchanged. getFirst() answers with a scan rather than a sort and toList() sorts a copy, so neither accessor disturbs what has been added. Generating a time-stamp request over 8,000 data objects goes from about 210 seconds to under a tenth of a second, and 100,000 objects, which the old code could not reach in any practical time, takes about 0.2 seconds (github #2456).
 - An ERSDataGroup recomputed its hash on every request rather than taking it from the cache ERSCachingData exists to provide: the group overrode getHash(), which left the calculateHash() the cache calls unreachable - and wrong, as it copied the member hashes with a loop bounded by the size of the empty list it was copying into, so it would have digested nothing had anything reached it. The computation is back in calculateHash() and the override is gone, so a group's hash is computed once per digest algorithm and previous-chain hash, as every other ERSData's is. The value itself is unchanged.
 - ERSArchiveTimeStampGenerator rebuilt its reduced hash tree from the data objects on every call, so the usual generateTimeStampRequest() followed by generateArchiveTimeStamp() or generateArchiveTimeStamps() built it twice. The leaves are now built once and dropped when data or a previous chain is added. A Set of the data groups it had been given, which nothing ever read, has gone with it.
+- Grain-128AEAD returned corrupted plaintext from a decryption driven in chunks. The stream cipher data operator splits a processBytes() call that spans the buffered authentication tag into two output segments, the bytes released from the tag buffer and then the bytes taken straight from the caller's input, and wrote the second segment at the caller's output offset instead of after the first, so the second segment overwrote the head of the first and the tail of the reported output was never written at all. The call still returned the full byte count, and because the engine's state update depends on the input and the keystream rather than on where the output lands, the tag still verified: the wrong plaintext came back with no error raised. Any chunk after the first that carried more than the 8 byte tag length was affected. Grain-128AEAD is the only engine that uses this operator, and one shot decryption, the encryption path and every other AEAD engine were unaffected. The second segment is now written at the advanced offset, matching the equivalent step of the general decryption path (github PR #2447).
 
 ### 2.1.3 Additional Features and Functionality