Skip to content

Register Boost.Python converters for NumPy 2.0 scalar types#1131

Open
dwpaley wants to merge 4 commits into
masterfrom
fix/numpy2-scalar-converters
Open

Register Boost.Python converters for NumPy 2.0 scalar types#1131
dwpaley wants to merge 4 commits into
masterfrom
fix/numpy2-scalar-converters

Conversation

@dwpaley

@dwpaley dwpaley commented Feb 18, 2026

Copy link
Copy Markdown
Contributor

Workaround for boostorg/python#511

NumPy 2.0 scalar types (np.float32, np.int32, np.int64, etc.) no longer subclass Python's built-in float/int, so Boost.Python's rvalue converters fail to convert them. Register custom from-Python converters that use the PyNumber_Float/PyNumber_Long protocols instead.

Fixes #1084

@bkpoon

bkpoon commented Feb 20, 2026

Copy link
Copy Markdown
Member

/azp run FULL

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

Workaround for boostorg/python#511

NumPy 2.0 scalar types (np.float32, np.int32, np.int64, etc.) no longer
subclass Python's built-in float/int, so Boost.Python's rvalue converters
fail to convert them. Register custom from-Python converters that use the
PyNumber_Float/PyNumber_Long protocols instead.

Fixes #1084
@bkpoon bkpoon force-pushed the fix/numpy2-scalar-converters branch from 6d0f610 to 2cdaf5b Compare February 23, 2026 18:05

@bkpoon bkpoon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Integer overflow detection

The current PyLong_AsLong + static_cast<CppType> pattern silently truncates values that don't fit the target type (e.g. np.int64(2147483648) assigned to flex.int silently stores a wrong value). PyLong_AsLong does not set an error on overflow — it truncates to C long range.

Fix: Use PyLong_AsLongLong to get the full value, then explicitly range-check against std::numeric_limits<CppType> before casting. This raises OverflowError for out-of-range values instead of silently corrupting data.

Three suggested changes:

  1. Add #include <limits> for std::numeric_limits
  2. Replace the integer converter body with overflow-safe version
  3. Add overflow boundary tests

Comment thread scitbx/array_family/boost_python/numpy_bridge.cpp
Comment thread scitbx/array_family/boost_python/numpy_bridge.cpp Outdated
Comment thread scitbx/array_family/boost_python/tst_numpy_scalar_conversions.py
Comment thread scitbx/array_family/boost_python/tst_numpy_scalar_conversions.py
dwpaley and others added 3 commits May 22, 2026 13:05
Co-authored-by: Billy K. Poon <bkpoon@lbl.gov>
Co-authored-by: Billy K. Poon <bkpoon@lbl.gov>
Co-authored-by: Billy K. Poon <bkpoon@lbl.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

+= operator fails for flex arrays when adding numpy-typed values

2 participants