Update mlkem-native to v1.1.0

[mkannwischer]

mlkem-native v1.1.0 just landed: https://github.com/pq-code-package/mlkem-native/releases/tag/v1.1.0.

This PR updates to the tagged upstream version. In addition, I also removed the constant-time passes: mlkem-native is using explicit declassifications using VALGRIND_MAKE_MEM_DEFINED instead.

• Resolves Update mlkem-native when next release comes #2365

Below are some highlights of this release (not all of which are relevant to the liboqs integration):

mlkem-native v1.1.0 marks the completion of the verification of all x86_64 and AArch64 assembly and the introduction of
SOUNDNESS.md documenting the scope, assumptions and risks of the verification work. It also introduces
various configuration options enabling the customization of mlkem-native for different application contexts. Finally,
new backends for RISC-V RVV and Armv8.1-M MVE have been added.

See the full change log here: pq-code-package/mlkem-native@v1.0.0...v1.1.0

What's New

Security

• Fix missing zeroization of intermediate polynomial vector pkpv in mlk_indcpa_keypair_derand() and mlk_indcpa_enc(). (#1328)
• Fix missing zeroization of pk and sk buffers on keypair generation failure (e.g. OOM during the pairwise consistency test). (#1559)
• Fix a 4-byte buffer overread in x86_64 rejection sampling assembly. The overread was within the stack frame and the excess bytes were not acted on, but the read itself exceeded the nominal buffer bounds. Found while working on the corresponding memory-safety proof. (#1615)
• Make the value barrier volatile to prevent compilers from optimizing it away, strengthening the constant-time countermeasure. This is a purely preventative measure; no insecure compilations of the previous value barrier have been noted. (#1342)
• Mark the stack as non-executable in all assembly files via .note.GNU-stack section markers. (#1340)

Assurance

• Assembly verification: All x86_64 and AArch64 assembly is verified to be functionally correct, memory-safe and
  free of secret-dependent timing, in HOL Light.
• SOUNDNESS.md: New document mapping out what is proved, what is assumed, and where the gaps and risks
  lie. (#1582)

Performance

• AArch64: Re-optimized arithmetic backend for Neoverse-N1 using SLOTHY. (#1088)
• x86_64: AVX2 assembly for polyvec_basemul (#1097), SSE4.1 rejection sampling (#1136), conversion of compression/decompression from intrinsics to assembly (#1543, #1545), and replacement of the Keccak-f1600 x4 C intrinsics with formally verified AVX2 assembly from s2n-bignum (#1576).
• RISC-V RVV: Native backend for rv64gcv targets using the RISC-V Vector Extension 1.0, providing vectorized NTT,
  inverse NTT, polynomial arithmetic, and rejection sampling. NTT and invNTT are for VLEN >= 256, with automatic
  fallback to C for VLEN=128. Other functions are VLEN agnostic. (#1037)
• Armv8.1-M MVE: Experimental native backend for Cortex-M55 and similar targets, including MVE Keccak-f1600 x4 and baremetal build support for the MPS3 AN547 platform. (#1220, #1518, #1524)

Configuration / API

• MLK_CONFIG_CUSTOM_ALLOC_FREE: Custom allocation/deallocation for large internal structures, for systems with limited stack space. (#1389)
• MLK_CONFIG_CONTEXT_PARAMETER: Add opaque context parameter to top-level API, passed through to custom alloc/free
  routines enabled via MLK_CONFIG_CUSTOM_ALLOC_FREE. Useful for applications without global allocator context. (#1467)
• MLK_CONFIG_NO_RANDOMIZED_API: Build only the deterministic (_derand) API. (#1185)
• MLK_CONFIG_SERIAL_FIPS202_ONLY: Disable 4x-batched FIPS-202, allowing use of a simpler serial-only FIPS-202 backend. (#1231)
• Runtime backend dispatch based on a custom CPU capabilities function. (#1152)
• randombytes() may now return an error code, which is propagated through the KEM API. (#1331)
• mlk_kem_check_pk() / mlk_kem_check_sk() added to the public API for FIPS 203 modulus and hash checks. (#1216)
• C++ compatibility for mlkem_native.h. (#1465)
• MLK_CONFIG_CUSTOM_MEMCPY / MLK_CONFIG_CUSTOM_MEMSET: Custom replacements for memcpy and memset. (#1105)

Testing

• Wycheproof test suite for ML-KEM test vector validation. (#1588)
• Unit test framework for internal functions with native backend consistency checks. (#1188)
• Allocation failure testing, RNG failure testing, stack usage measurement, and unaligned buffer testing.
• Baremetal testing on AVR (16-bit) and AArch64-virt (no MMU).

[dstebila]

@bhess Should this have any changes to the CBOM since there's an update in the upstream implementation?

[bhess]

LGTM, thanks for the update @mkannwischer.

@bhess Should this require any CBOM changes due to the upstream update?

The current CBOM does not track the upstream dependency (to be handled in #2048), so since there are no new optimizations or algorithm changes, no CBOM update is needed.

[abhi-dev-engg]

Hello,

I wanted to ask if there is a specific reason why MLK_CONFIG_EXTERNAL_API_QUALIFIER is not defined.
I encountered an issue while trying to use the check_pk and check_sk APIs in test code (for ACVP) since they are marked as external. However, the symbols do not appear to be resolvable during linking. Inspecting liboqs.dylib shows the following:

00000000000f7354 t _PQCP_MLKEM_NATIVE_MLKEM512_C_check_pk
00000000000f7538 t _PQCP_MLKEM_NATIVE_MLKEM512_C_check_sk

As these are marked with lowercase t, they appear to have local (non-exported) visibility.

I also noticed that MLD_CONFIG_EXTERNAL_API_QUALIFIER seems to be defined, which made me wonder if a similar qualifier was intentionally omitted for ML-KEM ?

[mkannwischer]

Hello,

I wanted to ask if there is a specific reason why MLK_CONFIG_EXTERNAL_API_QUALIFIER is not defined. I encountered an issue while trying to use the check_pk and check_sk APIs in test code (for ACVP) since they are marked as external. However, the symbols do not appear to be resolvable during linking. Inspecting liboqs.dylib shows the following:

00000000000f7354 t _PQCP_MLKEM_NATIVE_MLKEM512_C_check_pk 00000000000f7538 t _PQCP_MLKEM_NATIVE_MLKEM512_C_check_sk

As these are marked with lowercase t, they appear to have local (non-exported) visibility.

I also noticed that MLD_CONFIG_EXTERNAL_API_QUALIFIER seems to be defined, which made me wonder if a similar qualifier was intentionally omitted for ML-KEM ?

This is more of an oversight - it seems it wasn't strictly needed before.
I have opened a PR to fix it: pq-code-package/mlkem-native#1635

Can you add a patch to the config in your ACVP PR, so we can get this merged first?

[abhi-dev-engg]

Can you add a patch to the config in your ACVP PR, so we can get this merged first?

Sure, I can add the changes as patch in my PR, post this PR merge.