RFC: Opt-in native_enum mode — expose #[pyclass] enums as real enum.Enum subclasses

[wakita181009]

Opt-in native_enum mode for #[pyclass] enums?

Follow-up to #2887.

Context

#[pyclass] enum fails isinstance(x, enum.Enum) and is missing most of the enum protocol (__iter__, __members__, name, value, MyEnum(0), MyEnum["X"], …), so pydantic / FastAPI / SQLAlchemy / match-case need hand-written shims.

#2887 raised that true enum.Enum subclassing is blocked by #991 (Enum isn't in the C-API). The C-API isn't needed — enum.Enum("Name", [(name, value), …]) is a pure-Python functional constructor that returns a fully metaclass-blessed class. Calling it via PyO3 sidesteps #991.

Prior art

• pybind11 v3 does exactly this in py::native_enum + .finalize() (native_enum_data.h). No metaclass reimplementation — getters/setters only buffer name/value pairs before the functional call.
• Inspired by that, I built pyenum (wakita181009/pyenum) as a separate crate on PyO3 0.28: #[derive(PyEnum)] produces a real enum.Enum subclass (class cached in PyOnceLock<Py<PyType>>, auto-derives IntoPyObject / FromPyObject, supports Enum / IntEnum / StrEnum / Flag / IntFlag, pickle + match-case tested).

pyenum can stay external — I'm not assuming it needs upstreaming. Opening this Discussion to ask whether PyO3 wants a native mode.

Sketch

#[pyclass(native_enum = "Enum")]   // or "IntEnum" | "StrEnum" | "Flag" | "IntFlag"
pub enum Status { Draft = 0, Published = 1, Archived = 2 }

First access → enum.<Base>("Status", [...], module=..., qualname=...) via functional API, cached in PyOnceLock<Py<PyType>>. Unit variants only; tuple/struct/generic rejected at compile time.

Open questions

1. Opt-in vs default. @Tpt floated making this the default for simple enums. I lean opt-in for v1 (silent semantic shift otherwise).
2. Migration. Current #[pyclass] enum behavior kept under the bare attribute; new behavior strictly behind native_enum = "…". Deprecate over time or not?
3. Pickle / module / qualname. Auto-populate from enclosing #[pymodule], explicit override available.
4. Interaction with eq / eq_int / hash / frozen. Mostly redundant under EnumMeta. Warn, ignore, or error?
5. _missing_ / custom __new__. Out of scope for v1?

Perf

pyenum targets: first construction < 2 ms (≤32 variants) / < 20 ms (≤1024), steady-state conversion < 1 µs (cache hit = PyOnceLock load + attr lookup). I'll post measured numbers as a follow-up to address @Tpt's "no significant perf hit" bar.

Asks

• does the functional-API approach change your "add methods to exposed enums" preference in enums generated by Pyo3 don't actually subclass Python's Enum #2887 — or is a separate crate the right home for this case?
• if native support is on the table, is native_enum = "…" the right surface?
• Any blockers that affect either direction (in-PyO3 or external crate).

If there's interest I can split into (1) small RFC PR for the attribute + scope, (2) implementation PR with cache + derive changes + trybuild/pytest + benchmarks. If the answer is "keep it external," that's fine too — pyenum continues on its own.

[Tpt]

Thank you!

My 2cents:

does the functional-API approach change your "add methods to exposed enums" preference in #2887

My (personal!) feeling is that it depends on performances. If the functional-API is nearly as fast (or faster?) then it looks to me like a nice way to go. A benchmark would be very useful here.

Any blockers that affect either direction (in-PyO3 or external crate).

Something I overlooked in my previous answer, sorry: enum as supported by PyO3 support #[pymethods] block. It would be great to make them work to with "native enums". A possibility is maybe to create an explicit class that extends Enum or one of its subclass (I have not check it's actually possible, take it with a grain of salt). We just merged subclassing support for abi3 on Python 3.12+ so it might work fairly well. Experimenting in this area might be interesting.

In any case I do not have the final call on this, so take my answer accordingly.

[wakita181009]

Benchmark Complete: Cache Strategy Comparison

I measured multiple caching strategies to understand the performance implications of different approaches to exposing Rust enums as Python enum.Enum.

Benchmark Results

Strategy	to_python (Rust→Python)	roundtrip	Python Protocol
PyO3 #[pyclass] enum	58 ns	69 ns	❌ Not enum.Enum
pyenum (full cache)	59 ns	65 ns	✅ enum.Enum subclass
Python enum class only cached	108 ns	233 ns	✅ enum.Enum subclass
No cache	26,600 ns	26,300 ns	✅ enum.Enum subclass

Hardware: Apple Silicon M-series, PyO3 0.28.3, Python 3.11

Critical Finding: Why Caching Python enum.Enum is Essential

Without caching Python enum.Enum, the conversion overhead becomes unacceptable.

When we don't cache the Python enum.Enum class, we must build the class from scratch every time we need to convert a Rust enum value to Python. This results in a massive performance penalty of ~26 µs per conversion — approximately 400× slower than the cached approach.

The minimum requirement is to cache the Python enum.Enum class once per interpreter. With just this basic caching (the "class only cached" strategy in the table), we achieve reasonable but suboptimal performance:

• to_python: 108 ns (vs 58 ns for #[pyclass])
• roundtrip: 233 ns (vs 69 ns for #[pyclass])

This translates to roughly 2× overhead for single conversions and 3.5× overhead for round-trips, caused by the need to call getattr on the Python class for every member access.

Only with full application-level caching (as pyenum does) can we achieve performance equivalent to #[pyclass] while maintaining full enum.Enum compatibility.

By caching not just the Python class but also each member as a Py<PyAny>, we eliminate Python-side getattr calls entirely. This brings the performance to parity with #[pyclass]:

• to_python: 59 ns (same as #[pyclass])
• roundtrip: 65 ns (actually faster than #[pyclass]'s 69 ns)

Key Takeaways

1. PyO3's #[pyclass] is fast but incomplete

• ✅ Fast: Uses native Rust enum → Python class mapping
• ❌ Not a Python enum.Enum: isinstance(x, enum.Enum) returns False
• ❌ No IntEnum/StrEnum/Flag support
• ❌ Breaks match/case, pydantic, FastAPI expectations

2. Caching Python enum.Enum is non-negotiable for acceptable performance

• Uncached: 26,600 ns — impractical for any real application
• Class-only cached: 108-233 ns — usable but significant overhead
• Full member caching required for #[pyclass]-equivalent speed

3. pyenum achieves the best of both worlds

• True enum.Enum subclass with full Python protocol support
• Performance matching or exceeding #[pyclass] (~60ns)
• Zero-cost via derive macro (#[derive(PyEnum)])

Implication for __pyo3_as_enum__ Protocol

For a potential __pyo3_as_enum__ protocol, which caching strategy should be adopted?

• Class-only cache: Simpler implementation, but ~2× slower than #[pyclass]
• Full member cache: Optimal performance, matching #[pyclass] speed

Should Python enum.Enum compatibility aim for parity with #[pyclass] performance, or accept the trade-off for simpler implementation? pybind11 (C++) demonstrates class-only caching is sufficient for most use cases
Reference implementation: https://github.com/wakita181009/pyenum/blob/main/crates/pyenum/benches/comparison.rs