CMS: pass the user keying material to HKDF as the salt as well as in the ECC-CMS-SharedInfo entityUInfo for the RFC 8418 key agreement schemes, as sec. 2.2 requires, relates to github #2454.

Author: dghgit

docs/claude/conventions.md

@@ -8,6 +8,32 @@
 - Tests pass `-Dbc.test.data.home=<core/src/test/data>` for fixture lookups.
 - The `:test` task runs each test class in its own JVM (`forkEvery = 1`).
 
+## Key agreement tests run with and without a UKM
+
+Any test of a CMS or JCA key agreement scheme should exercise it **both with and without a user
+keying material value**, wherever the scheme admits one. The two are different derivations, not one
+derivation with an extra field: a UKM typically feeds the KDF's info *and*, for HKDF-based schemes,
+its salt, so a test that only ever omits the UKM leaves the more complex of the two paths uncovered.
+
+RFC 8418 sec. 2.2 is the worked example (github #2454). Its KEK is
+
+```
+if ukm is provided, then salt = ukm, else salt is not provided
+PRK = HKDF-Extract(salt, K)
+KEK = HKDF-Expand(PRK, DER(ECC-CMS-SharedInfo), SizeInOctets(KEK))
+```
+
+so the UKM has two roles. BC placed it in the ECC-CMS-SharedInfo `entityUInfo` but never passed it
+as the HKDF salt, and 1.86 shipped that way because `testRFC8418X25519AndX448` round-tripped all six
+curve/scheme combinations **with no UKM set**. Nothing was wrong with the no-UKM path, so nothing
+failed.
+
+Note also that a self round trip cannot see this class of defect at all - BC agreed with itself
+either way. Where a spec states the derivation, derive the key independently in the test and
+unwrap/verify with it, and assert that the *wrong* derivation does **not** work
+(`NewEnvelopedDataTest.checkRFC8418Kek` is the model). A known-answer check against the spec's own
+recipe is what makes the UKM half meaningful.
+
 ## X.509 ASN.1 changes — check the RFC first
 
 Anything under `core/src/main/java/org/bouncycastle/asn1/x509/` is a wire-format ASN.1 type from a specific PKI RFC. Before changing or extending one of these classes (parsing rules, structural constraints, defaults, error messages thrown for malformed input), verify the proposed behaviour against the authoritative RFC:

docs/releasenotes.md

@@ -33,6 +33,7 @@ Date: 2026, TBD
 - A CMS message whose EncryptedContentInfo named the RFC 9709 key derivation but carried no readable content-encryption AlgorithmIdentifier in its parameters was reported as a NullPointerException, or as an IllegalArgumentException from the ASN.1 decoder, out of methods declared to throw CMSException, RecipientInformation.getContent() among them. The four places that resolve the wrapper - the CEK derivation, the content cipher selection, the key-size check, and the recipient's allowed-algorithm and tag-size checks - now share one resolver, which reports an absent or unreadable inner algorithm as a CMSException.
 - The two YubiKey OpenPGP smart-card decryptor factories zeroized the user PIN array the KeyPassphraseProvider handed them rather than a copy of it, in a finally block after each private-key operation. Both providers BC ships return the application's own array by reference - DefaultKeyPassphraseProvider hands back the char[] it has cached for the key, and the provider inside OpenPGPApi.editKey returns its argument - so the first card operation destroyed the caller's PIN and the next private-key operation presented an all-zero PIN to the card, which the card refuses at the cost of a PIN retry. The PIN is now fetched as a clone the card operation owns - in OpenPGPSmartCard.requireUserPin, where the smart-card restructuring of this release put the fetch the two factories used to make - and KeyPassphraseProvider.getKeyPassword records that the array it returns stays owned by the provider (github PR #2444).
 - The CRMF PKIPublicationInfo structure accepted, and could be built with, publication information RFC 4211 sec. 6.3 forbids: pubInfos MUST NOT be present if the action is dontPublish, and the field is SEQUENCE SIZE (1..MAX), so a present one is never empty. Both contradictions are now rejected with an IllegalArgumentException, on parsing and on construction from an array of SinglePubInfo. An absent pubInfos with the pleasePublish action, which is how the RFC spells "don't care", is unaffected, and no constructor in the library could produce either rejected form.
+- The CMS RFC 8418 key agreement schemes (dhSinglePass-stdDH-hkdf-sha256/384/512, used with X25519 and X448) derived the key-encryption key with the user keying material in the entityUInfo of the ECC-CMS-SharedInfo but never as the HKDF salt, where RFC 8418 sec. 2.2 requires both - its recipe is salt = ukm, PRK = HKDF-Extract(salt, K), KEK = HKDF-Expand(PRK, DER(ECC-CMS-SharedInfo), SizeInOctets(KEK)). A message carrying a ukm therefore did not interoperate with a conforming implementation in either direction. The ukm is now passed as the salt as well, on both the generating and the receiving side, for those three schemes. A message with a ukm written by 1.86, the only release with RFC 8418 support, is not readable by this release and vice versa; messages without a ukm, and the X9.63-KDF key agreement schemes, are unaffected. The round-trip test now covers both the ukm and no-ukm cases for all six curve and scheme combinations, and checks the key-encryption key against the RFC's own recipe rather than only against BC itself (github #2454).
 
 ### 2.1.3 Additional Features and Functionality
 

pkix/src/main/java/org/bouncycastle/cms/AbstractKeyAgreeRecipient.java

@@ -11,6 +11,11 @@ protected boolean isEC(ASN1ObjectIdentifier algorithmOID)
         return CMSUtils.isEC(algorithmOID);
     }
 
+    protected boolean isHKDF(ASN1ObjectIdentifier algorithmOID)
+    {
+        return CMSUtils.isHKDF(algorithmOID);
+    }
+
     protected boolean isMQV(ASN1ObjectIdentifier algorithmOID)
     {
         return CMSUtils.isMQV(algorithmOID);

pkix/src/main/java/org/bouncycastle/cms/CMSUtils.java

@@ -70,6 +70,7 @@ class CMSUtils
     private static final Set desAlgs = new HashSet();
     private static final Set mqvAlgs = new HashSet();
     private static final Set ecAlgs = new HashSet();
+    private static final Set hkdfAlgs = new HashSet();
     private static final Set gostAlgs = new HashSet();
 
     static
@@ -100,6 +101,10 @@ class CMSUtils
         ecAlgs.add(PKCSObjectIdentifiers.dhSinglePass_stdDH_hkdf_sha384_scheme);
         ecAlgs.add(PKCSObjectIdentifiers.dhSinglePass_stdDH_hkdf_sha512_scheme);
 
+        hkdfAlgs.add(PKCSObjectIdentifiers.dhSinglePass_stdDH_hkdf_sha256_scheme);
+        hkdfAlgs.add(PKCSObjectIdentifiers.dhSinglePass_stdDH_hkdf_sha384_scheme);
+        hkdfAlgs.add(PKCSObjectIdentifiers.dhSinglePass_stdDH_hkdf_sha512_scheme);
+
         // BSI TR-03111 ECKA-EG with X9.63 KDF. Structurally identical to
         // dhSinglePass_stdDH_*kdf_scheme (ECDH + X9.63 KDF + RFC 5753
         // ECC-CMS-SharedInfo per BSI TR-03109-3 / ICAO 9303-11); dispatch
@@ -130,6 +135,11 @@ static boolean isEC(ASN1ObjectIdentifier algorithm)
         return ecAlgs.contains(algorithm);
     }
 
+    static boolean isHKDF(ASN1ObjectIdentifier algorithm)
+    {
+        return hkdfAlgs.contains(algorithm);
+    }
+
     static boolean isGOST(ASN1ObjectIdentifier algorithm)
     {
         return gostAlgs.contains(algorithm);

pkix/src/main/java/org/bouncycastle/cms/KeyAgreeRecipientInfoGenerator.java

@@ -67,6 +67,11 @@ protected boolean isEC(ASN1ObjectIdentifier algorithmOID)
         return CMSUtils.isEC(algorithmOID);
     }
 
+    protected boolean isHKDF(ASN1ObjectIdentifier algorithmOID)
+    {
+        return CMSUtils.isHKDF(algorithmOID);
+    }
+
     protected boolean isMQV(ASN1ObjectIdentifier algorithmOID)
     {
         return CMSUtils.isMQV(algorithmOID);

pkix/src/main/java/org/bouncycastle/cms/jcajce/JceKeyAgreeRecipient.java

@@ -242,7 +242,16 @@ private SecretKey calculateAgreedWrapKey(AlgorithmIdentifier keyEncAlg, Algorith
                 {
                     ukmKeyingMaterial = kmGen.generateKDFMaterial(wrapAlg, keySizeProvider.getKeySize(wrapAlg), null);
                 }
-                userKeyingMaterialSpec = new UserKeyingMaterialSpec(ukmKeyingMaterial);
+                // RFC 8418 sec. 2.2: for the HKDF schemes a ukm is both the entityUInfo of the
+                // ECC-CMS-SharedInfo and the HKDF salt
+                if (userKeyingMaterial != null && isHKDF(keyEncAlg.getAlgorithm()))
+                {
+                    userKeyingMaterialSpec = new UserKeyingMaterialSpec(ukmKeyingMaterial, userKeyingMaterial.getOctets());
+                }
+                else
+                {
+                    userKeyingMaterialSpec = new UserKeyingMaterialSpec(ukmKeyingMaterial);
+                }
             }
             else if (isRFC2631(keyEncAlg.getAlgorithm()))
             {

pkix/src/main/java/org/bouncycastle/cms/jcajce/JceKeyAgreeRecipientInfoGenerator.java

@@ -180,7 +180,16 @@ else if (isEC(keyAgreementOID))
                     byte[] ukmKeyingMaterial = ecc_cms_Generator.generateKDFMaterial(keyEncryptionAlgorithm,
                         keySizeProvider.getKeySize(keyEncryptionOID), userKeyingMaterial);
 
-                    agreementParamSpec = new UserKeyingMaterialSpec(ukmKeyingMaterial);
+                    // RFC 8418 sec. 2.2: for the HKDF schemes a ukm is both the entityUInfo of the
+                    // ECC-CMS-SharedInfo and the HKDF salt
+                    if (userKeyingMaterial != null && isHKDF(keyAgreementOID))
+                    {
+                        agreementParamSpec = new UserKeyingMaterialSpec(ukmKeyingMaterial, userKeyingMaterial);
+                    }
+                    else
+                    {
+                        agreementParamSpec = new UserKeyingMaterialSpec(ukmKeyingMaterial);
+                    }
                 }
                 else if (isRFC2631(keyAgreementOID))
                 {

pkix/src/test/java/org/bouncycastle/cms/test/NewEnvelopedDataTest.java

@@ -7,12 +7,14 @@
 import java.security.AlgorithmParameters;
 import java.security.GeneralSecurityException;
 import java.security.Key;
+import java.security.InvalidKeyException;
 import java.security.KeyFactory;
 import java.security.KeyPair;
 import java.security.KeyPairGenerator;
 import java.security.NoSuchAlgorithmException;
 import java.security.NoSuchProviderException;
 import java.security.PrivateKey;
+import java.security.PublicKey;
 import java.security.SecureRandom;
 import java.security.Security;
 import java.security.cert.CertificateEncodingException;
@@ -31,6 +33,8 @@
 import javax.crypto.SecretKey;
 import javax.crypto.spec.OAEPParameterSpec;
 import javax.crypto.spec.PSource;
+import javax.crypto.Cipher;
+import javax.crypto.KeyAgreement;
 import javax.crypto.spec.SecretKeySpec;
 
 import junit.framework.Assert;
@@ -60,7 +64,9 @@
 import org.bouncycastle.asn1.cms.EncryptedContentInfo;
 import org.bouncycastle.asn1.cms.EnvelopedData;
 import org.bouncycastle.asn1.cms.GCMParameters;
+import org.bouncycastle.asn1.cms.KeyAgreeRecipientInfo;
 import org.bouncycastle.asn1.cms.OtherRecipientInfo;
+import org.bouncycastle.asn1.cms.RecipientEncryptedKey;
 import org.bouncycastle.asn1.cms.RecipientInfo;
 import org.bouncycastle.asn1.cryptopro.CryptoProObjectIdentifiers;
 import org.bouncycastle.asn1.gm.GMObjectIdentifiers;
@@ -72,6 +78,7 @@
 import org.bouncycastle.asn1.pkcs.RC2CBCParameter;
 import org.bouncycastle.asn1.rosstandart.RosstandartObjectIdentifiers;
 import org.bouncycastle.asn1.x500.X500Name;
+import org.bouncycastle.asn1.cms.ecc.ECCCMSSharedInfo;
 import org.bouncycastle.asn1.x509.AlgorithmIdentifier;
 import org.bouncycastle.asn1.x509.Extension;
 import org.bouncycastle.asn1.x509.SubjectPublicKeyInfo;
@@ -130,6 +137,13 @@
 import org.bouncycastle.operator.OutputEncryptor;
 import org.bouncycastle.operator.jcajce.JcaAlgorithmParametersConverter;
 import org.bouncycastle.pqc.jcajce.provider.BouncyCastlePQCProvider;
+import org.bouncycastle.crypto.Digest;
+import org.bouncycastle.crypto.digests.SHA256Digest;
+import org.bouncycastle.crypto.digests.SHA384Digest;
+import org.bouncycastle.crypto.digests.SHA512Digest;
+import org.bouncycastle.crypto.generators.HKDFBytesGenerator;
+import org.bouncycastle.crypto.params.HKDFParameters;
+import org.bouncycastle.util.Pack;
 import org.bouncycastle.util.Strings;
 import org.bouncycastle.util.encoders.Base64;
 import org.bouncycastle.util.encoders.Hex;
@@ -4412,15 +4426,26 @@ private void passwordUTF8Test(String algorithm, PasswordRecipient.PRF prf)
     public void testRFC8418X25519AndX448()
         throws Exception
     {
-        doRFC8418Round("X25519", CMSAlgorithm.ECDH_HKDF_SHA256);
-        doRFC8418Round("X25519", CMSAlgorithm.ECDH_HKDF_SHA384);
-        doRFC8418Round("X25519", CMSAlgorithm.ECDH_HKDF_SHA512);
-        doRFC8418Round("X448",   CMSAlgorithm.ECDH_HKDF_SHA256);
-        doRFC8418Round("X448",   CMSAlgorithm.ECDH_HKDF_SHA384);
-        doRFC8418Round("X448",   CMSAlgorithm.ECDH_HKDF_SHA512);
+        doRFC8418Round("X25519", CMSAlgorithm.ECDH_HKDF_SHA256, null);
+        doRFC8418Round("X25519", CMSAlgorithm.ECDH_HKDF_SHA384, null);
+        doRFC8418Round("X25519", CMSAlgorithm.ECDH_HKDF_SHA512, null);
+        doRFC8418Round("X448",   CMSAlgorithm.ECDH_HKDF_SHA256, null);
+        doRFC8418Round("X448",   CMSAlgorithm.ECDH_HKDF_SHA384, null);
+        doRFC8418Round("X448",   CMSAlgorithm.ECDH_HKDF_SHA512, null);
+
+        // with a ukm the KEK derivation is also checked against the RFC's own recipe: a round trip
+        // against ourselves cannot tell whether the ukm reached the HKDF salt
+        byte[] ukm = Hex.decode("cafed1ea0529");
+
+        doRFC8418Round("X25519", CMSAlgorithm.ECDH_HKDF_SHA256, ukm);
+        doRFC8418Round("X25519", CMSAlgorithm.ECDH_HKDF_SHA384, ukm);
+        doRFC8418Round("X25519", CMSAlgorithm.ECDH_HKDF_SHA512, ukm);
+        doRFC8418Round("X448",   CMSAlgorithm.ECDH_HKDF_SHA256, ukm);
+        doRFC8418Round("X448",   CMSAlgorithm.ECDH_HKDF_SHA384, ukm);
+        doRFC8418Round("X448",   CMSAlgorithm.ECDH_HKDF_SHA512, ukm);
     }
 
-    private void doRFC8418Round(String curve, ASN1ObjectIdentifier kaOid)
+    private void doRFC8418Round(String curve, ASN1ObjectIdentifier kaOid, byte[] ukm)
         throws Exception
     {
         byte[] data = Hex.decode("504b492d4320434d5320456e76656c6f706564446174612053616d706c65");
@@ -4431,11 +4456,19 @@ private void doRFC8418Round(String curve, ASN1ObjectIdentifier kaOid)
 
         byte[] reciKeyId = new byte[]{1, 2, 3, 4, 5};
 
+        JceKeyAgreeRecipientInfoGenerator recGen =
+            new JceKeyAgreeRecipientInfoGenerator(kaOid, origKP.getPrivate(), origKP.getPublic(), CMSAlgorithm.AES128_WRAP);
+
+        recGen.addRecipient(reciKeyId, reciKP.getPublic());
+        recGen.setProvider(BC);
+
+        if (ukm != null)
+        {
+            recGen.setUserKeyingMaterial(ukm);
+        }
+
         CMSEnvelopedDataGenerator edGen = new CMSEnvelopedDataGenerator();
-        edGen.addRecipientInfoGenerator(
-            new JceKeyAgreeRecipientInfoGenerator(kaOid, origKP.getPrivate(), origKP.getPublic(), CMSAlgorithm.AES128_WRAP)
-                .addRecipient(reciKeyId, reciKP.getPublic())
-                .setProvider(BC));
+        edGen.addRecipientInfoGenerator(recGen);
 
         CMSEnvelopedData ed = edGen.generate(
             new CMSProcessableByteArray(data),
@@ -4451,6 +4484,101 @@ private void doRFC8418Round(String curve, ASN1ObjectIdentifier kaOid)
         byte[] recData = recipient.getContent(
             new JceKeyAgreeEnvelopedRecipient(reciKP.getPrivate()).setProvider(BC));
         assertTrue(curve + "/" + kaOid + " mismatch", Arrays.equals(data, recData));
+
+        if (ukm != null)
+        {
+            checkRFC8418Kek(ed, curve, kaOid, reciKP.getPrivate(), origKP.getPublic(), ukm);
+        }
+    }
+
+    /**
+     * RFC 8418 sec. 2.2 fixes the KEK as
+     * <pre>
+     *     if ukm is provided, then salt = ukm, else salt is not provided
+     *     PRK = HKDF-Extract(salt, K)
+     *     KEK = HKDF-Expand(PRK, DER(ECC-CMS-SharedInfo), SizeInOctets(KEK))
+     * </pre>
+     * so a ukm is both the entityUInfo of the ECC-CMS-SharedInfo and the HKDF salt. This derives
+     * the KEK independently of the CMS code and unwraps the message's encryptedKey with it, and
+     * checks that the derivation leaving the salt out - which is what 1.86 produced - no longer
+     * agrees with the message.
+     */
+    private void checkRFC8418Kek(CMSEnvelopedData ed, String curve, ASN1ObjectIdentifier kaOid,
+        PrivateKey reciPrivKey, PublicKey origPubKey, byte[] ukm)
+        throws Exception
+    {
+        EnvelopedData env = EnvelopedData.getInstance(ContentInfo.getInstance(ed.getEncoded()).getContent());
+        KeyAgreeRecipientInfo kari = KeyAgreeRecipientInfo.getInstance(
+            RecipientInfo.getInstance(env.getRecipientInfos().getObjectAt(0)).getInfo());
+
+        assertTrue("ukm not carried in the message", Arrays.equals(ukm, kari.getUserKeyingMaterial().getOctets()));
+
+        AlgorithmIdentifier wrapAlg = AlgorithmIdentifier.getInstance(kari.getKeyEncryptionAlgorithm().getParameters());
+        byte[] encryptedKey = RecipientEncryptedKey.getInstance(
+            kari.getRecipientEncryptedKeys().getObjectAt(0)).getEncryptedKey().getOctets();
+
+        // K, the raw shared secret
+        KeyAgreement agreement = KeyAgreement.getInstance(curve, BC);
+
+        agreement.init(reciPrivKey);
+        agreement.doPhase(origPubKey, true);
+
+        byte[] k = agreement.generateSecret();
+
+        // info, DER(ECC-CMS-SharedInfo) with the ukm as entityUInfo and the KEK size as suppPubInfo
+        byte[] sharedInfo = new ECCCMSSharedInfo(wrapAlg, ukm, Pack.intToBigEndian(128)).getEncoded(ASN1Encoding.DER);
+
+        String label = curve + "/" + kaOid;
+
+        assertTrue(label + ": the KEK derived with salt = ukm does not open the message",
+            unwrapsWith(deriveRFC8418Kek(kaOid, k, ukm, sharedInfo), encryptedKey));
+        assertFalse(label + ": the KEK derived with no salt still opens the message",
+            unwrapsWith(deriveRFC8418Kek(kaOid, k, null, sharedInfo), encryptedKey));
+    }
+
+    private byte[] deriveRFC8418Kek(ASN1ObjectIdentifier kaOid, byte[] secret, byte[] salt, byte[] sharedInfo)
+    {
+        Digest digest;
+
+        if (kaOid.equals(CMSAlgorithm.ECDH_HKDF_SHA256))
+        {
+            digest = new SHA256Digest();
+        }
+        else if (kaOid.equals(CMSAlgorithm.ECDH_HKDF_SHA384))
+        {
+            digest = new SHA384Digest();
+        }
+        else
+        {
+            digest = new SHA512Digest();
+        }
+
+        HKDFBytesGenerator kdf = new HKDFBytesGenerator(digest);
+
+        kdf.init(new HKDFParameters(secret, salt, sharedInfo));
+
+        byte[] kek = new byte[16];
+
+        kdf.generateBytes(kek, 0, kek.length);
+
+        return kek;
+    }
+
+    private boolean unwrapsWith(byte[] kek, byte[] encryptedKey)
+        throws Exception
+    {
+        Cipher cipher = Cipher.getInstance("AESWrap", BC);
+
+        cipher.init(Cipher.UNWRAP_MODE, new SecretKeySpec(kek, "AES"));
+
+        try
+        {
+            return cipher.unwrap(encryptedKey, "AES", Cipher.SECRET_KEY) != null;
+        }
+        catch (InvalidKeyException e)
+        {
+            return false;
+        }
     }
 
     private void verifyECKeyAgreeVectors(PrivateKey privKey, String wrapAlg, byte[] message)