Add experimental rustls backend that uses the ossl bindings#454
Open
simo5 wants to merge 16 commits into
Open
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new rustls-ossl crate that implements a rustls CryptoProvider using the project's OpenSSL bindings. It significantly expands the ossl wrapper to support FIPS enforcement, post-quantum algorithms (ML-DSA, ML-KEM, SLH-DSA), and TLS-specific cryptographic operations. Feedback highlights several critical issues, including a memory leak in the MAC availability check, incorrect parsing of SubjectPublicKeyInfo (SPKI) structures during key loading and signature verification, and potential performance and robustness concerns in the KDF and key exchange implementations.
simo5
force-pushed
the
rustls
branch
3 times, most recently
from
8461dc8 to
c2f472a
Compare
Member
Author
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a new rustls-ossl crate, which provides a Rustls crypto provider backed by OpenSSL bindings (ossl). It also adds support for ChaCha20 and ChaCha20-Poly1305 ciphers, in-place cipher updates, TLS1 PRF derivation, and various helper functions for checking algorithm availability. The review comments identify several critical and high-severity issues, including a potential integer underflow/DoS vulnerability in TLS 1.2 decryption, incorrect parsing of SPKI public keys in signature verification, a logical typo and potential panic in HKDF info expansion, and OpenSSL error queue pollution. Additionally, opportunities for performance optimization (avoiding heap allocations in HMAC) and code cleanup (removing redundant ? operators) were highlighted.
Signed-off-by: Simo Sorce <simo@redhat.com>
Introduces the `rustls-ossl` workspace crate to enable using OpenSSL as the cryptographic backend for `rustls`. The `ossl` crate bindings are updated to expose OpenSSL's RAND functions (`RAND_bytes` and `RAND_priv_bytes`), which are then utilized by the new crate to implement `rustls::crypto::SecureRandom`. Additionally, this includes a stub for `KeyProvider` and updates to formatting targets in the Makefile. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
This adds the ability to load DER-encoded private keys in the rustls OpenSSL provider by implementing the `KeyProvider` trait, enabling signing operations. To support this, `EvpPkey::from_der`, `get_type`, and `spki_der` methods were added to the `ossl` crate to handle DER parsing (`d2i_AutoPrivateKey_ex`) and public key extraction (`i2d_PUBKEY`). Additionally, an `OsslSigningKey` struct was created to map OpenSSL key types to rustls signature algorithms, and a lazy global `OsslContext` was introduced to manage the required library context. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
Add `OsslSigner` to implement the `rustls::sign::Signer` trait, enabling message signing via OpenSSL. The `choose_scheme` method is implemented in `OsslSigningKey` to negotiate and instantiate the correct signer based on the key type and offered schemes. To support the immutable requirements of the `Signer` trait, `EvpPkey` and `OsslSignature` operations were updated to accept `&self` instead of `&mut self`. Additionally, `Clone` was implemented for `EvpPkey` using `EVP_PKEY_up_ref` to handle key references safely. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
Implement the `SignatureVerificationAlgorithm` trait for various signature schemes (ML-DSA, EdDSA, ECDSA, RSA) using the OpenSSL backend. The supported algorithms are dynamically detected based on their availability in the OpenSSL context and wired into the default `CryptoProvider` to enable certificate and handshake signature validation. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
simo5
force-pushed
the
rustls
branch
2 times, most recently
from
8990af9 to
16a11f7
Compare
Add `hash` and `hmac` modules to the rustls provider to support rustls cryptographic operations using OpenSSL primitives. To satisfy the rustls traits, which require state duplication (e.g., `Context::fork` and reusing HMAC keys), `Clone` is implemented for `EvpMdCtx`, `OsslDigest`, `EvpMacCtx`, `OsslMac`, and `OsslSecret` in the base `ossl` crate by wrapping OpenSSL's context duplication functions. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
Adds in-place cipher updates and TLS 1 PRF derivation to the base OpenSSL wrapper. For the rustls integration, it implements AES-GCM AEAD, HKDF, and TLS 1.2 PRF to assemble complete TLS 1.2 (via a feature flag) and TLS 1.3 cipher suites. This provides the necessary symmetric encryption, decryption, and key derivation capabilities required to secure TLS traffic. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
This introduces `keyex.rs` with implementations of `SupportedKxGroup` and `ActiveKeyExchange` for ECDH (X25519, SECP curves) and ML-KEM. The `kx_groups` field is now appropriately populated in the default `CryptoProvider`. To support extracting key shares during key exchange, `EvpPkey::export_public` is added. `EvpPkey::available` is also introduced to conditionally register supported key exchange groups based on the OpenSSL context. Furthermore, key parameter serialization and deserialization logic in `pkey.rs` is refactored into type-specific structures to simplify and organize `EvpPkey` operations. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
Signed-off-by: Simo Sorce <simo@redhat.com>
Introduces integration tests that spawn an OpenSSL `s_server` instance with various key types (RSA, ECDSA, EdDSA) and connect to it using a rustls client configured with the `rustls-ossl` crypto provider. A custom certificate verifier is added to allow pinning dynamically generated self-signed certificates. This ensures the provider can successfully perform real-world TLS handshakes and signature verification across multiple supported algorithms. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
This introduces FIPS provider loading and enforcement capabilities to the OpenSSL context and adds dynamic availability checks for digests, MACs, and public keys. Additionally, it implements the `fips()` method across all rustls crypto provider components (Hash, HMAC, KeyProvider, AEADs, etc.). This integration ensures that the rustls OpenSSL backend can properly report FIPS compliance and restrict operations to approved algorithms when the FIPS provider is active. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
This adds the ChaCha20 stream cipher and the ChaCha20-Poly1305 AEAD cipher to the supported encryption algorithms, integrating Poly1305 into the existing AEAD setup flow. Supporting these algorithms expands the library's cryptographic capabilities, providing developers with a modern, highly secure, and performant alternative to AES. Comprehensive test vectors for encryption, decryption, and authentication tags are included to ensure correctness. Signed-off-by: Simo Sorce <simo@redhat.com>
This adds `CHACHA20_POLY1305` to the supported AEAD algorithms, enabling the corresponding cipher suites for TLS 1.2 and TLS 1.3. It also implements the specific IV and nonce construction logic required for ChaCha20-Poly1305 in TLS 1.2, which does not use explicit nonces like AES-GCM. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
Add support for the `secp521r1` (P-521) key exchange group and the `Ed448` signature scheme to broaden the supported cryptographic capabilities. Update the basic integration tests to verify functionality across multiple protocol versions (TLS 1.2, TLS 1.3) and cipher suites (AES-128, AES-256, ChaCha20) rather than relying on default negotiations. Additionally, the test client now validates the connection by downloading and verifying a specific file, ensuring accurate data transfer instead of just asserting a successful handshake. Assisted-by: Gemini <gemini@google.com> Signed-off-by: Simo Sorce <simo@redhat.com>
Signed-off-by: Simo Sorce <simo@redhat.com>
EVP_MD_CTX_dup(0 is not available until 3.2.0, so raise the minimum requirement for rustls Signed-off-by: Simo Sorce <simo@redhat.com>
Jakuje
reviewed
Jun 1, 2026
| }) | ||
| } | ||
|
|
||
| #[cfg(ossl_mlkem)] |
Contributor
There was a problem hiding this comment.
Suggested change
| #[cfg(ossl_mlkem)] | |
| #[cfg(ossl_mldsa)] |
|
|
||
| use crate::osslctx; | ||
|
|
||
| /* whis will need to grop up to 1024 bytes if we ever add support for FFDH groups |
Contributor
There was a problem hiding this comment.
Suggested change
| /* whis will need to grop up to 1024 bytes if we ever add support for FFDH groups | |
| /* whis will need to grow up to 1024 bytes if we ever add support for FFDH groups |
| }, | ||
| hkdf_provider: &kdf::HKDF_SHA256, | ||
| aead_alg: &cipher::CHACHA20_POLY1305, | ||
| quic: None, /* TODO */ |
Contributor
There was a problem hiding this comment.
TODO in this PR or next one?
| None | ||
| } | ||
|
|
||
| /* This is only for TLS1.2 which will never get ML-DSA or any PQ algorithm */ |
Contributor
There was a problem hiding this comment.
nit:
Suggested change
| /* This is only for TLS1.2 which will never get ML-DSA or any PQ algorithm */ | |
| /* This is only for TLS1.2 which will never get ML-DSA nor any PQ algorithm */ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add a somewhat complete implementation for a cryptography backend usable with rustls.
Missing support for ChaCha20Poly1305, but otherwise can deal with TLS1.2 and TLS1.3 using AES GCM
This compiles with rustls 0.23.40 but not the latest development branch which reworks a bunch of details of the CrytoProvider traits.
Checklist
Reviewer's checklist: