Repository navigation
SPKI and PKCS8 for ML-KEM and ML-DSA tests #184
Description
Filing this to remember to make a note of it. Unless someone beats me to it, I'll probably put something together, at least for the seed-bases ones, as part of other work.
(I won't bother with expandedKey one as those just generally are bad ideas. The IETF fumbled the ball at the last minute and we ended up with three different PKCS8 encodings, two of which should never be used.)
Reacted by Daniel McCarney
Activity
Add SPKI and PKCS8 inputs for ML-DSA
Add SPKI and PKCS8 inputs for ML-DSA
tarcieri commentedon May 6, 2026
Was this addressed by #186?
cpu commentedon May 6, 2026
I thought there were still gaps for ML-KEM
Add SPKI and PKCS8 inputs for ML-DSA
Add SPKI and PKCS8 inputs for ML-DSA
Add SPKI and PKCS8 inputs for ML-DSA
Arpan0995 commentedon Jul 31, 2026
Is the ML-KEM side of this still open? As of today none of the ML-KEM files carry a DER field:
mlkem_*_encaps_test,mlkem_*_keygen_seed_test,mlkem_*_semi_expanded_decaps_testandmlkem_*_testhave nopublicKeyDerorprivateKeyPkcs8equivalent, so the gap @cpu mentioned looks like it is still there. #186 covered ML-DSA only.If nobody has it in flight I would be glad to put it together. One structural question first, because ML-KEM does not fit the #186 shape directly.
In the ML-DSA files the key lives on the test group, so #186 could add
publicKeyDerandprivateKeyPkcs8once per group. In the ML-KEM filesek,dkandseedare per test case, and there are 1701 of them across the twelve files: encaps is 261/265/269 for the three parameter sets, the main file and keygen_seed are 193 and 100 each, decaps is 9. So a per-case DER field is a much bigger change than the ML-DSA one was. Three ways to go:ekDeranddkPkcs8per test case, consistent with where the raw keys already are, at the cost of a large size increase across the existing files.keygen_seed, which is the file whose subject is the key material anyway, and leave encaps/decaps untouched.I would lean towards 2, but this is your call on file layout rather than mine.
Two details worth settling at the same time:
The ML-KEM seed is 64 bytes (
d || z), not 32 as for ML-DSA, so "the seed form" means something different here and the schema description should probably say so explicitly rather than reuse the ML-DSA wording.On the private key CHOICE: I agree the seed form is the one to build vectors around, but the interop situation is worth knowing before deciding, because it is not what I expected. Building all three forms by hand for ML-KEM-512 and importing them on JDK 24, into SunJCE and BouncyCastle 1.85:
seed [0]expandedKeybothML-DSA-44 behaves identically against SUN. So today the seed form, the one this issue is about and the one the drafts prefer, is the form the JDK will not parse, and
expandedKeyis the only one both accept.That is not an artifact of how I built the DER. Asking each provider what it emits for a freshly generated key, and decoding the CHOICE tag inside
privateKey: SunJCE emitsexpandedKey(1660 bytes for ML-KEM-512), BouncyCastle emitsboth(1730). Round-tripping those native encodings, a BouncyCastle-generated ML-KEM private key fails to import into SunJCE, while a SunJCE-generated one imports into BouncyCastle fine. ML-DSA-44 shows the same asymmetry between SUN and BouncyCastle. Neither JDK provider emits or accepts the seed form at all.That is not an argument against seed-form vectors, and I am not suggesting Wycheproof bless the other encodings. It does suggest a vector set covering seed alone would pass on BouncyCastle and fail on a stock JDK for reasons that have nothing to do with the implementation being wrong, so it may be worth deciding deliberately whether the file should carry more than one form, or note which is expected to be parseable where. Happy to be told that is out of scope.
If the shape above is agreed I will generate the vectors and open a PR.
davidben commentedon Jul 31, 2026
That table is missing some other entries. Other implementations only take
seed. That's where implementations have generally aligned.Unfortunately, this is the kind of mess that resulted from one particular implementation gumming up the standards process. To clear out this mess, it is actively desirable for Wycheproof to encourage
seedin its test vectors. The standards process failed, so we're down to implementation pressures to get folks into an interoperable state. I.e. non-seedimplementations failing test vectors is a feature, not a bug. Additionally,bothandexpandedKeyadd extra dimensions of testing complexity because you have to worry about values that are not self-consistent. (Part of whyseedis just a better format.)Regarding SunJCE, that table doesn't match what https://bugs.openjdk.org/browse/JDK-8349163 says. Perhaps you tested against an older version?
Arpan0995 commentedon Aug 3, 2026
You are right, and thank you for the pointer. I tested on Corretto 24.0.2, which predates this. The implementation is JDK-8347938, fixed in 27 on 2026-02-03, and it has since been backported: 26.0.2 (JDK-8381386), 25.0.4 (JDK-8381844), and 21.0.13 and 17.0.21 on the Oracle side (JDK-8385394, JDK-8385800). It makes all three CHOICE forms decodable and switches the default encoding to
seed. My table describes a JDK that has already been superseded, so please disregard it.Re-running the same probe on Temurin 25.0.4+7 (LTS) and again on 27-ea+32, with BouncyCastle 1.85. Both give identical results:
seed [0]expandedKeybothseedbothML-DSA-44 private keys are 54 bytes from the JDK against 2626 from BouncyCastle, and ML-KEM-512 is 86 against 1730. Native encodings round-trip in both directions on both JDKs.
You are also right that the table was missing entries. It covered the two Java providers and nothing else, so it was a poor guide to where implementations in general have landed. I did not measure anything outside the JVM, and I take your word that other implementations accept
seedalone.So this is already in a shipping LTS release, and on current versions the JDK emits
seedby default, which puts the Java side on the path you describe. The BouncyCastle column still holds: 1.85 is the current release, it decodes all three forms, and it still emitsbothby default. If the aim is implementation pressure towardseed, that is where it would land, and a seed-only vector set does that job. I withdraw the suggestion that the file should carry more than one form.On the point about self-consistency in
bothandexpandedKey: that had not occurred to me and it is a good argument on its own. Seed-only avoids an entire class of vector where the two halves disagree.That leaves the layout question, which I do not think the above affects. ML-KEM keys are per test case rather than per test group, so the #186 pattern does not transfer directly. Options 1 to 3 in my earlier comment still stand, and I still lean toward adding the DER forms only in
keygen_seed. Happy to build it that way if you agree, or to leave it with you if you would rather fold it into the other work you mentioned.cpu commentedon Aug 6, 2026
How big is the size increase in practice?
Arpan0995 commentedon Aug 11, 2026
@cpu I measured it against the current files. The twelve ML-KEM files are 8.3 MB together. Adding
ekDerwhereverekis present and a seed-formdkPkcs8wherever theseedis present, then reserialising:encaps_testtest(main)keygen_seed_testsemi_expanded_decaps_testAcross all twelve that is +4.1 MB, so 8.3 MB becomes about 12.4 MB, roughly +50%.
Almost all of that is
ekDer, not the private key. The seed-formdkPkcs8is 86 bytes (172 hex chars) for a valid 64-byte seed, the same for every parameter set, so it is noise.ekDeris the SPKI wrapping ofek, which is 822, 1206 and 1590 bytes for a well-formed key at 512, 768 and 1024, so it roughly doubles the largest field already in each case. That is whyencaps, which is mostlyekand carries one in every case, grows the most.For contrast, option 2 (both fields in
keygen_seedonly) is +0.79 MB, about +9.5% of the corpus, roughly a fifth of option 1.One wrinkle: the two fields are not addable to every case, so option 1 would not be uniform. The
encapsfiles are public-key only, andsemi_expanded_decapscarriesdkbut noseed, so both getekDeralone (those files have no seed, and it cannot be recovered from the expandeddk). The invalid cases in the main file are the reverse, aseedbut noek(the wrong-length private key cases), so they getdkPkcs8but notekDer. I added each field only where its raw input is already present, wrapping it as-is for the invalid-length cases, which is what the totals above reflect.cpu commentedon Aug 12, 2026
Open to input from others but that doesn't seem too bad to me overall 🤷
Arpan0995 commentedon Aug 18, 2026
Thanks. If no one has started on this, I am glad to put the PR together.
One thing to lock before I generate: the layout. Option 1 (per-case) adds ekDer and the seed-form dkPkcs8 wherever the raw key is already present across the twelve ML-KEM files; option 2 keeps it to the keygen_seed files. @cpu is fine with option 1's size, and I lean the same way since it exercises the DER paths everywhere the raw key already appears, but I will follow whichever you and @davidben prefer.
Scope as discussed: ekDer (SPKI) plus the seed-form dkPkcs8, no expandedKey PKCS8 per @davidben's note, and each field added only where its raw input is already present.
If option 1 works and no one else is on it, I will regenerate the twelve ML-KEM files and the generator and open a PR against this issue in a few days.