Releases: benoitc/erlang-python
Release list
v1.7.1
v1.7.0
Added
-
Shared Router Architecture for Event Loops
- Single
py_event_routerprocess handles all event loops (both shared and isolated) - Timer and FD messages include loop identity for correct dispatch
- Eliminates need for per-loop router processes
- Handle-based Python C API using PyCapsule for loop references
- Single
-
Isolated Event Loops - Create isolated event loops with
ErlangEventLoop(isolated=True)- Default (
isolated=False): uses the shared global loop managed by Erlang - Isolated (
isolated=True): creates a dedicated loop with its own pending queue - Full asyncio support (timers, FD operations) for both modes
- Useful for multi-threaded Python applications where each thread needs its own loop
- See
docs/asyncio.mdfor usage and architecture details
- Default (
v1.6.1
Fixed
- ASGI headers now correctly use bytes instead of str - Fixed ASGI spec compliance issue where headers were being converted to Python
strobjects instead ofbytes. The ASGI specification requires headers to belist[tuple[bytes, bytes]]. This was causing authentication failures and form parsing issues with frameworks like Starlette and FastAPI, which search for headers using bytes keys (e.g.,b"content-type").- Added explicit header handling in
asgi_scope_from_map()to bypass generic conversion - Headers are now correctly converted using
PyBytes_FromStringAndSize() - Supports both list
[name, value]and tuple{name, value}header formats from Erlang - Fixes GitHub issue #1
- Added explicit header handling in
v1.6.0
Added
-
Python Logging Integration - Forward Python's
loggingmodule to Erlang'sloggerpy:configure_logging/0,1- Setup Python logging to forward to Erlangerlang.ErlangHandler- Python logging handler that sends to Erlangerlang.setup_logging(level, format)- Configure logging from Python- Fire-and-forget architecture using
enif_send()for non-blocking messaging - Level filtering at NIF level for performance
- Thread-safe - works from any Python thread
-
Distributed Tracing - Collect trace spans from Python code
py:enable_tracing/0,py:disable_tracing/0- Enable/disable span collectionpy:get_traces/0- Retrieve collected spanserlang.Span(name, **attrs)- Context manager for creating spanserlang.trace(name)- Decorator for tracing functions- Automatic parent/child span linking via thread-local storage
-
New Erlang modules:
py_logger,py_tracer
Performance
-
Type conversion optimizations - Faster Python ↔ Erlang marshalling
- Use
enif_is_identicalfor atom comparison instead ofstrcmp - Stack allocate small tuples/maps (≤16 elements) to avoid heap allocation
- Use
enif_make_map_from_arraysfor O(n) map building vs O(n²) puts
- Use
-
Fire-and-forget NIF architecture - Log and trace calls never block Python execution
Fixed
-
Python 3.12+ event loop thread isolation - Fixed asyncio timeouts on Python 3.12+
ErlangEventLoopnow only used for main thread; worker threads getSelectorEventLoop- Per-call
ErlNifEnvfor thread-safe timer scheduling in free-threaded mode - Fail-fast error handling in
erlang_loop.pyinstead of silent hangs - Added
gil_acquire()/gil_release()helpers to avoid GIL double-acquisition
-
Intermittent test failures on free-threaded Python - Added startup synchronization
v1.5.0
Added
-
py_asgimodule - Optimized ASGI request handling with:- Pre-interned Python string keys (15+ ASGI scope keys)
- Cached constant values (http type, HTTP versions, methods, schemes)
- Thread-local response pooling (16 slots per thread, 4KB initial buffer)
- Direct NIF path bypassing generic py:call()
- ~60-80% throughput improvement over py:call()
- Configurable runner module via
runneroption - Sub-interpreter and free-threading (Python 3.13+) support
-
py_wsgimodule - Optimized WSGI request handling with:- Pre-interned WSGI environ keys
- Direct NIF path for marshalling
- ~60-80% throughput improvement over py:call()
- Sub-interpreter and free-threading support
-
Web frameworks documentation - New documentation at
docs/web-frameworks.md
v1.4.0 - Erlang-native asyncio event loop
Added
-
Erlang-native asyncio event loop - Custom asyncio event loop backed by Erlang's scheduler
ErlangEventLoopclass inpriv/erlang_loop.py- Sub-millisecond latency via Erlang's
enif_select(vs 10ms polling) - Zero CPU usage when idle - no busy-waiting or polling overhead
- Full GIL release during waits for better concurrency
- Native Erlang scheduler integration for I/O events
- Event loop policy via
get_event_loop_policy()
-
TCP support for asyncio event loop
create_connection()- TCP client connectionscreate_server()- TCP server with accept loop_ErlangSocketTransport- Non-blocking socket transport with write buffering_ErlangServer- TCP server withserve_forever()support
-
UDP/datagram support for asyncio event loop
create_datagram_endpoint()- Create UDP endpoints with full parameter support_ErlangDatagramTransport- Datagram transport implementation- Parameters:
local_addr,remote_addr,reuse_address,reuse_port,allow_broadcast DatagramProtocolcallbacks:datagram_received(),error_received()- Support for both connected and unconnected UDP
-
Asyncio event loop documentation
- New documentation:
docs/asyncio.md
- New documentation:
Performance
- Event loop optimizations
- Fixed
run_until_completecallback removal bug - Cached
ast.literal_evallookup at module initialization - O(1) timer cancellation via handle-to-callback_id reverse map
- Detach pending queue under mutex, build Erlang terms outside lock
- O(1) duplicate event detection using hash set
- Added
PERF_BUILDcmake option for aggressive optimizations (-O3, LTO, -march=native)
- Fixed
Full Changelog: https://github.com/benoitc/erlang-python/blob/main/CHANGELOG.md
1.3.2
Fixed
- torch/PyTorch introspection compatibility - Fixed
AttributeError: 'erlang.Function' object has no attribute 'endswith'when importing torch or sentence_transformers in contexts where erlang_python callbacks are registered.- Root cause: torch does dynamic introspection during import, iterating through Python's namespace and calling
.endswith()on objects. Theerlangmodule's__getattr__was returningErlangFunctionwrappers for any attribute access. - Solution: Added C-side callback name registry. Now
__getattr__only returnsErlangFunctionwrappers for actually registered callbacks. Unregistered attributes raiseAttributeError(normal Python behavior). - New test:
test_callback_name_registryinpy_reentrant_SUITE.erl
- Root cause: torch does dynamic introspection during import, iterating through Python's namespace and calling
1.3.1
v1.3.0
Added
- Asyncio Support - New
erlang.async_call()for asyncio-compatible callbacksawait erlang.async_call('func', arg1, arg2)- Call Erlang from async Python code- Integrates with asyncio event loop via
add_reader() - No exceptions raised for control flow (unlike
erlang.call()) - Releases dirty NIF thread while waiting (non-blocking)
- Works with FastAPI, Starlette, aiohttp, and other ASGI frameworks
- Supports concurrent calls via
asyncio.gather()
Fixed
- Flag-based callback detection in replay path - Fixed SuspensionRequired exceptions leaking when ASGI middleware catches and re-raises exceptions
Changed
- C code optimizations and refactoring
- Thread safety fixes: Used
pthread_oncefor async callback initialization - Timeout handling: Added
read_with_timeout()andread_length_prefixed_data()helpers - Code deduplication: Merged suspended state creation functions, extracted helpers
- Performance: Optimized list conversion using
enif_make_list_cell()
- Thread safety fixes: Used
v1.2.0
Added
-
Context Affinity - Bind Erlang processes to dedicated Python workers for state persistence
py:bind()/py:unbind()- Bind current process to a worker, preserving Python statepy:bind(new)- Create explicit context handles for multiple contexts per processpy:with_context(Fun)- Scoped helper with automatic bind/unbind- Context-aware functions:
py:ctx_call/4-6,py:ctx_eval/2-4,py:ctx_exec/2 - Automatic cleanup via process monitors when bound processes die
- O(1) ETS-based binding lookup for minimal overhead
- New test suite:
test/py_context_SUITE.erl
-
Python Thread Support - Any spawned Python thread can now call
erlang.call()without blocking- Supports
threading.Thread,concurrent.futures.ThreadPoolExecutor, and any other Python threads - Each spawned thread lazily acquires a dedicated "thread worker" channel
- One lightweight Erlang process per Python thread handles callbacks
- Automatic cleanup when Python thread exits via
pthread_key_tdestructor - New module:
py_thread_handler.erl- Coordinator and per-thread handlers - New C file:
py_thread_worker.c- Thread worker pool management - New test suite:
test/py_thread_callback_SUITE.erl - New documentation:
docs/threading.md- Threading support guide
- Supports
-
Reentrant Callbacks - Python→Erlang→Python callback chains without deadlocks
- Exception-based suspension mechanism interrupts Python execution cleanly
- Callbacks execute in separate processes to prevent worker pool exhaustion
- Supports arbitrarily deep nesting (tested up to 10+ levels)
- Transparent to users -
erlang.call()works the same, just without deadlocks - New test suite:
test/py_reentrant_SUITE.erl - New examples:
examples/reentrant_demo.erlandexamples/reentrant_demo.py
Changed
- Callback handlers now spawn separate processes for execution, allowing workers
to remain available for nestedpy:eval/py:calloperations - Modular C code structure - Split monolithic
py_nif.c(4,335 lines) into
logical modules for better maintainability:py_nif.h- Shared header with types, macros, and declarationspy_convert.c- Bidirectional type conversion (Python ↔ Erlang)py_exec.c- Python execution engine and GIL managementpy_callback.c- Erlang callback support and asyncio integration- Uses
#includeapproach for single compilation unit (no build changes needed)
Fixed
- Multiple sequential erlang.call() - Fixed infinite loop when Python code makes
multiple sequentialerlang.call()invocations in the same function. The replay
mechanism now falls back to blocking pipe behavior for subsequent calls after the
first suspension, preventing the infinite replay loop. - Memory safety in C NIF - Fixed memory leaks and added NULL checks
nif_async_worker_new: msg_env now freed on pipe/thread creation failuremulti_executor_stop: shutdown requests now properly freed after joincreate_suspended_state: binary allocations cleaned up on failure paths- Added NULL checks on all
enif_alloc_resourceandenif_alloc_envcalls
- Dialyzer warnings - Added
{suspended, ...}return type to NIF specs for
worker_call,worker_eval, andresume_callbackfunctions - Dead code removal - Cleaned up unused code discovered during code review:
- Removed
execute_direct()function inpy_exec.c(duplicated inline logic) - Removed unused
reffield fromasync_pending_tstruct inpy_nif.h - Removed
worker_recv/2frompy_nif.erl(declared but never implemented in C)
- Removed
Documentation
- Doxygen-style C documentation - Added documentation to all C source files:
- Architecture overview with execution mode diagrams
- Type mapping tables for conversions
- GIL management patterns and best practices
- Suspension/resume flow diagrams for callbacks
- Function-level
@param,@return,@pre,@warning,@seeannotations