Skip to content

Possibly helpful TODO list (with assistance from a Claude plugin)? #15034

Description

@clin1234

Free-Threading Analysis Report

Generated from https://github.com/devdanzin/ft-review-toolkit

Extension: cryptography (Rust/pyo3, 105+ Rust source files)


Migration Status

Status: DECLARED — structural migration complete, runtime gaps remain

Timeline:

PyPI classifier declared: Free Threading :: 3 - Stable
Total ft-related commits: ~33 over 2 years


Executive Summary

  • Readiness: Close — structural migration done, but 3 runtime races need fixing before the 3 - Stable classifier is accurate
  • RACE findings: 3 — confirmed or highly likely data races under concurrent access
  • UNSAFE findings: 2 — operations unsafe without GIL serialization
  • PROTECT findings: 1 — shared state needing synchronization
  • MIGRATE findings: 2 — structural improvements needed

Findings by Priority

RACE Findings (fix immediately) — 3

# Finding File:Line Severity Agents
1 Aliased raw buffer slices in CffiBuf/CffiMutBuf — use-after-free if backing bytearray is mutated or resized by another thread src/rust/src/buf.rs:119–186 CRITICAL shared-state, lock-discipline, atomic-finder, unsafe-api
2 Shared EVP_CIPHER_CTX copied concurrently on six frozen AEAD pyclasses — EVP_CIPHER_CTX_copy not documented thread-safe for concurrent reads from same source src/rust/src/backend/aead.rs:55–63,182,245 HIGH unsafe-api, stw-advisor
3 EVP_default_properties_enable_fips(NULL) writes process-global OpenSSL FIPS flag with no Rust-level guard — concurrent is_fips_enabled() reads race it src/rust/cryptography-openssl/src/fips.rs:47–54 MEDIUM shared-state, lock-discipline, atomic-finder

UNSAFE Findings (fix before declaring free-threading support) — 2

# Finding File:Line Severity
4 EvpCipherAead::key (Py<PyAny>) is re-extracted via CffiBuf in &self fallback paths — cryptographic key material may be read while another thread mutates the source Python object src/rust/src/backend/aead.rs:57,188,251 HIGH
5 unsafe impl Sync for Hmac/Cmac in OpenSSL wrapper layer — assertion was vacuously true under GIL; concurrent .copy() calls now pass the same HMAC_CTX/CMAC_CTX pointer to HMAC_CTX_copy without OpenSSL-documented safety src/rust/cryptography-openssl/src/hmac.rs:21, cmac.rs:22 MEDIUM

PROTECT Findings (add synchronization) — 1

# Finding File:Line Severity
6 OSSL_get_max_threads + OSSL_set_max_threads(NULL) non-atomic read-modify-write in module init — mitigated by Python's import lock today but structurally racy src/rust/src/lib.rs:308–318 LOW

MIGRATE Findings (structural changes) — 2

# Finding File:Line Severity
7 GIL release incomplete: DSA sign/verify, DH exchange, and RSA recover_data_from_signature still hold GIL during OpenSSL computation — sibling operations were released in #14370 src/rust/src/backend/dsa.rs:69,170, dh.rs:155, rsa.rs:512 MEDIUM
8 import_exception! macro creates process-global type-object statics — may be unsound under sub-interpreters; migrate to LazyPyImport (the pattern already used everywhere else in this repo) src/rust/src/exceptions.rs:30–41 LOW

SAFE Patterns (confirmed safe)

  • PyOnceLock-backed import caches (types.rs, ~100 statics): Thread-safe by PyO3's design; write-once, read-many after initialization
  • LazyLock<HashMap> statics (pkcs7.rs, ocsp.rs, sign.rs): Immutable after construction, LazyLock guarantees single initialization
  • cipher_registry.rs REGISTRY: PyOnceLock: Correct pattern, thread-safe
  • KeepAlive<T> (cryptography-keepalive/src/lib.rs): UnsafeCell makes it !Sync by construction; compiler enforces non-sharing; all uses are stack-local
  • unsafe impl Sync for AeadCtx (aead.rs): Correct — mutating methods are &mut self-gated; concurrent &self is read-only; BoringSSL/AWS-LC documents seal/open thread-safe on distinct contexts
  • pyo3 per-instance PyRwLock on non-frozen pyclasses (Hash, HmacContext, XOFHash): &mut self update/finalize are serialized by pyo3's borrow checker; concurrent call raises RuntimeError, not UB
  • VerificationCertificate::public_key OnceLock (cryptography-x509-verification/src/ops.rs:28–35): TOCTOU is benign (redundant compute only, idempotent result)
  • Module init OSSL_set_max_threads: Protected by Python's import lock in practice

Recommendations

Immediate (RACE + UNSAFE — Findings 1–5)

1. Fix Finding 1 — copy Python buffer data on extraction

In buf.rs, replace slice::from_raw_parts with an owned copy for the free-threading path. The existing Py_3_11 path via PyBuffer::get acquires the buffer protocol lock — verify it is held for the full slice lifetime. For CffiMutBuf, enforce that the output buffer object is distinct from all input objects (Python identity check before extraction).

2. Fix Finding 2 — protect EvpCipherAead base contexts with Mutex

Wrap base_encryption_ctx and base_decryption_ctx in std::sync::Mutex<CipherCtx> in aead.rs. Lock only for the ctx.copy(...) call, not for the entire encrypt/decrypt operation. The six frozen AEAD pyclasses will then be safe for concurrent use.

use std::sync::Mutex;

pub(crate) struct EvpCipherAead {
    cipher: AeadCipher,
    key: pyo3::Py<pyo3::PyAny>,
    base_encryption_ctx: Mutex<openssl::cipher_ctx::CipherCtx>,
    base_decryption_ctx: Mutex<openssl::cipher_ctx::CipherCtx>,
    tag_len: usize,
    tag_first: bool,
    is_ccm: bool,
}

3. Fix Finding 3 — guard enable_fips with OnceLock

In fips.rs, wrap EVP_default_properties_enable_fips in static FIPS_ONCE: OnceLock<()> to make enabling idempotent and race-free. Document that enable_fips must be called before spawning threads for correctness of in-flight operations.

static FIPS_ENABLED: std::sync::OnceLock<()> = std::sync::OnceLock::new();

pub fn enable() -> OpenSSLResult<()> {
    FIPS_ENABLED.get_or_try_init(|| {
        unsafe {
            cvt(ffi::EVP_default_properties_enable_fips(ptr::null_mut(), 1))?;
        }
        Ok(())
    })
    .map(|_| ())
}

4. Fix Finding 4 — extract key bytes once at construction, not per-call

In EvpCipherAead, replace the key: Py<PyAny> field with key_bytes: Zeroizing<Vec<u8>> (copy at construction via extract::<&[u8]>()?.to_owned()). Remove the self.key.extract::<CffiBuf>() calls in the CCM/fallback paths.

use zeroize::Zeroizing;

pub(crate) struct EvpCipherAead {
    key_bytes: Zeroizing<Vec<u8>>,  // owned copy, zeroed on drop
    // ...
}

5. Fix Finding 5 — audit or remove unsafe impl Sync on Hmac/Cmac

Either verify against OpenSSL documentation that HMAC_CTX_copy is safe for concurrent reads from the same source (and add a comment citing the source), or remove unsafe impl Sync and rely on pyo3's per-object borrow check for all Hmac/Cmac operations.

Short-term (PROTECT — Finding 6)

6. Fix Finding 6 — wrap OSSL_set_max_threads in OnceLock

Replace the bare get+set in lib.rs:308–318 with a static OSSL_INIT: OnceLock<()> so initialization runs exactly once even if import-lock semantics change.

static OSSL_MAX_THREADS_INIT: std::sync::OnceLock<()> = std::sync::OnceLock::new();
OSSL_MAX_THREADS_INIT.get_or_init(|| unsafe {
    let current = openssl_sys::OSSL_get_max_threads(ptr::null_mut());
    openssl_sys::OSSL_set_max_threads(ptr::null_mut(), max(available, current));
});

Longer-term (MIGRATE — Findings 7–8)

7. Fix Finding 7 — complete GIL-release coverage

Apply py.detach() to DsaPrivateKey::sign (dsa.rs:69), DsaPublicKey::verify (dsa.rs:170), DHPrivateKey::exchange (dh.rs:155), and RsaPublicKey::recover_data_from_signature (rsa.rs:512). The pattern is already established in ec.rs and rsa.rs (for covered operations). Also unify RSA sign's probe call into its py.detach() block (rsa.rs:317).

8. Fix Finding 8 — migrate import_exception! to LazyPyImport

Replace the 12 pyo3::import_exception! statics in exceptions.rs with LazyPyImport (the same mechanism used throughout types.rs), scoping exception type objects per-interpreter for correctness under sub-interpreters.


Notes

  • Free-threaded interpreter on this system: python3t.exe (python3t -m pip for package management)
  • To generate a TSan stress test: /ft-review-toolkit:explore . stress-test, then run the generated script with python3t tsan_stress_<name>.py
  • For a phased migration plan: /ft-review-toolkit:plan

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions