Skip to content

Commit ff65528

Browse files
soswowsoswow
committed
everything to take from
1 parent 75b4586 commit ff65528

29 files changed

Lines changed: 5820 additions & 36 deletions

‎docs/dev/PR_split_current_branch.md‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Suggested PR split: current uncommitted work (`more-nanobind-migration`)
2+
3+
Maps the branch’s uncommitted work into **a few** reviewable merges. Adjust if
4+
your maintainer wants one nanobind mega-PR instead.
5+
6+
---
7+
8+
## 1. Inventory (what the branch touches)
9+
10+
### Modified (tracked)
11+
12+
| Area | Paths |
13+
|------|--------|
14+
| Pybind buffer `TypeDesc` | `src/python/py_oiio.cpp` |
15+
| Nanobind core + module glue | `src/python-nanobind/py_oiio.cpp`, `src/python-nanobind/py_oiio.h` |
16+
| Nanobind build | `src/python-nanobind/CMakeLists.txt` |
17+
| Nanobind migration note | `src/python-nanobind/MIGRATION_STATUS.md` |
18+
| CTest / Python paths | `src/cmake/testing.cmake` |
19+
| Libutil tests | `src/libutil/CMakeLists.txt` |
20+
| Shared tests + refs | `testsuite/python-imagebuf/…`, `testsuite/python-imagespec/…`, `testsuite/python-paramlist/…` |
21+
22+
### Untracked (new)
23+
24+
| Area | Paths |
25+
|------|--------|
26+
| C++ unit test (PEP 3118 `=` / format mirror) | `src/libutil/python_array_format_test.cpp` |
27+
| Coverage helper script | `src/build-scripts/report_nanobind_python_coverage.bash` |
28+
| New nanobind translation units | `src/python-nanobind/py_{argparse,deepdata,imagebuf,imagebufalgo,imageinput,imageoutput}.cpp` |
29+
| New testsuites | `testsuite/python-oiio/`, `testsuite/python-argparse-nanobind/` |
30+
| Fixture | `testsuite/python-imageoutput/multipart.exr` |
31+
32+
---
33+
34+
## 2. Build constraint (nanobind)
35+
36+
`PyOpenImageIONanobind` is **one** CMake target: `nanobind_srcs` lists every
37+
`py_*.cpp`, and `py_oiio.cpp` calls every `declare_*`. So **all new nanobind
38+
`.cpp` files and glue normally land in the same PR** unless you restack the
39+
branch so each merge trims `nanobind_srcs` / init (high effort).
40+
41+
---
42+
43+
## 3. Clamped PR sequence (recommended: **3 PRs**)
44+
45+
### PR 1 — Pybind only: buffer format → `TypeDesc`
46+
47+
**Files:** `src/python/py_oiio.cpp` only.
48+
49+
**Contents:** `typedesc_from_python_array_code` parity (e.g. leading `=`, `q` /
50+
`Q`, formatting).
51+
52+
**Why first:** Tiny, isolated, easy revert; aligns pybind with what nanobind
53+
already does before you land the rest of the migration. (Impact on typical
54+
NumPy paths is small; this is mostly hygiene / edge-case correctness.)
55+
56+
**Tests:** Optional local `ctest` for existing pybind Python tests; no new
57+
tests required.
58+
59+
---
60+
61+
### PR 2 — Nanobind migration (one “feature” merge)
62+
63+
**Single merge** for everything that makes the nanobind extension complete and
64+
CI-meaningful:
65+
66+
- All **new / changed** under `src/python-nanobind/` (including every new
67+
`py_*.cpp`, `py_oiio.cpp` / `py_oiio.h`, `CMakeLists.txt`, `MIGRATION_STATUS.md`)
68+
- **`src/cmake/testing.cmake`** — nanobind test lists, `python-oiio` /
69+
`python-argparse-nanobind`, image I/O / IBA groups, etc., as on your branch
70+
- **Tests + refs** that validate nanobind against pybind for this work:
71+
- `testsuite/python-imagebuf/…` (e.g. HALF `get_pixels` parity)
72+
- `testsuite/python-imagespec/…`, `testsuite/python-paramlist/…`
73+
- `testsuite/python-argparse-nanobind/` (full tree)
74+
- `testsuite/python-imageoutput/multipart.exr` if required
75+
76+
**Why one PR:** Matches the **single shared library** layout; avoids half-wired
77+
`nanobind_srcs` between merges. Use **commits inside the PR** (per file or per
78+
subsystem) if reviewers want smaller logical chunks without separate merges.
79+
80+
---
81+
82+
### PR 3 — Optional “extras” (drop or fold into PR 2)
83+
84+
Bundle non-core or skippable items so they do not block the migration:
85+
86+
- `src/libutil/python_array_format_test.cpp` + `src/libutil/CMakeLists.txt`
87+
(`unit_python_array_format`) — C++ mirror for `=` / PEP 3118 edge cases
88+
- `testsuite/python-oiio/` + `src/cmake/testing.cmake` entries for it **if**
89+
you did not already merge those lines in **PR 2**
90+
- `src/build-scripts/report_nanobind_python_coverage.bash`
91+
92+
If you prefer **fewer files to track**, fold all of **PR 3** into **PR 2** and
93+
delete this third bucket.
94+
95+
---
96+
97+
## 4. Aggressive option: **2 PRs**
98+
99+
1. **PR 1** — same as §3 PR 1 (`src/python/py_oiio.cpp` only).
100+
2. **PR 2** — **everything else** from the inventory (full nanobind migration +
101+
all tests + `testing.cmake` + libutil test + `python-oiio` + coverage script).
102+
103+
Same review cost as “one big branch,” but **bisect** can still isolate pybind
104+
buffer fixes.
105+
106+
---
107+
108+
## 5. What not to mix in without an explicit goal
109+
110+
| Item | Suggestion |
111+
|------|------------|
112+
| Public `OpenImageIO.typedesc_from_python_array_code(...)` | Avoid unless you want a **stable** Python API + stubs. |
113+
| Shared header in `libutil` + bindings | Separate refactor PR. |
114+
| Unrelated `testing.cmake` / plugin toggles | Keep migration PR focused. |
115+
116+
---
117+
118+
## 6. Checklist
119+
120+
- [ ] `./compile.sh` (or CI-equivalent) for backends you enable.
121+
- [ ] `ctest` for every testsuite you register (`python-*`, `*.nanobind`).
122+
- [ ] Regenerate `ref/out.txt` only for tests you touch.
123+
124+
This file is **developer guidance**; trim or delete before upstreaming if
125+
`docs/dev/` should stay minimal.

‎src/build-scripts/report_nanobind_python_coverage.bash‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
#!/usr/bin/env bash
2+
3+
# Copyright Contributors to the OpenImageIO project.
4+
# SPDX-License-Identifier: Apache-2.0
5+
#
6+
# Summarize gcov line coverage for PyOpenImageIONanobind (src/python-nanobind/py_*.cpp).
7+
#
8+
# Prerequisites:
9+
# - Configure with -DCODECOV=ON (see src/cmake/compiler.cmake).
10+
# - Build PyOpenImageIONanobind.
11+
# - Run tests that load the nanobind module (e.g. ctest -R '\.nanobind').
12+
#
13+
# Usage:
14+
# ./src/build-scripts/report_nanobind_python_coverage.bash /path/to/build
15+
#
16+
# Requires llvm-cov (Apple Xcode toolchain has llvm-cov; Homebrew llvm also works).
17+
18+
set -euo pipefail
19+
20+
if [[ $# -lt 1 ]]; then
21+
echo "Usage: $0 /path/to/cmake/build" >&2
22+
exit 2
23+
fi
24+
25+
BUILD_ROOT="$(cd "$1" && pwd)"
26+
OBJDIR="${BUILD_ROOT}/src/python-nanobind/CMakeFiles/PyOpenImageIONanobind.dir"
27+
SRCROOT="$(cd "$(dirname "$0")/../.." && pwd)/src/python-nanobind"
28+
29+
LLVM_COV=""
30+
if command -v llvm-cov >/dev/null 2>&1; then
31+
LLVM_COV="$(command -v llvm-cov)"
32+
elif command -v xcrun >/dev/null 2>&1; then
33+
LLVM_COV="xcrun llvm-cov"
34+
else
35+
echo "llvm-cov not found in PATH" >&2
36+
exit 1
37+
fi
38+
39+
if [[ ! -d "$OBJDIR" ]]; then
40+
echo "Object directory not found: $OBJDIR" >&2
41+
exit 1
42+
fi
43+
44+
echo "# Nanobind py_*.cpp coverage (llvm-cov gcov summary)"
45+
echo "# OBJDIR=$OBJDIR"
46+
echo "# SRCROOT=$SRCROOT"
47+
echo
48+
49+
shopt -s nullglob
50+
for gcno in "${OBJDIR}"/py_*.cpp.gcno; do
51+
base="$(basename "$gcno" .gcno)"
52+
echo "--- ${base}"
53+
( cd "$OBJDIR" && ${LLVM_COV} gcov -b -s "$SRCROOT" "$gcno" ) 2>&1 \
54+
| grep -F -A 6 "File '${SRCROOT}/${base}'" | head -8 || \
55+
( cd "$OBJDIR" && ${LLVM_COV} gcov -b -s "$SRCROOT" "$gcno" ) 2>&1 \
56+
| grep -F -A 6 "File '${base}'" | head -8 || true
57+
done
58+
59+
echo
60+
echo "Annotated sources (*.gcov) are written under:"
61+
echo " $OBJDIR"

‎src/cmake/testing.cmake‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -263,7 +263,14 @@ macro (oiio_add_all_tests)
263263
python-imagespec
264264
python-paramlist
265265
python-roi
266-
python-typedesc)
266+
python-typedesc
267+
python-oiio
268+
python-imagebuf
269+
python-argparse-nanobind)
270+
set (nanobind_python_tests_need_images
271+
python-imageinput
272+
python-imageoutput
273+
python-imagebufalgo)
267274
set (nanobind_python_test_suffix ".nanobind")
268275
if (OIIO_BUILD_PYTHON_PYBIND11)
269276
oiio_add_tests (
@@ -278,6 +285,7 @@ macro (oiio_add_all_tests)
278285
python-roi
279286
python-texturesys
280287
python-typedesc
288+
python-oiio
281289
filters
282290
ENVIRONMENT "${_pybind_tests_pythonpath}"
283291
)
@@ -297,6 +305,12 @@ macro (oiio_add_all_tests)
297305
SUFFIX ${nanobind_python_test_suffix}
298306
ENVIRONMENT "${_nanobind_tests_pythonpath}"
299307
)
308+
oiio_add_tests (
309+
${nanobind_python_tests_need_images}
310+
SUFFIX ${nanobind_python_test_suffix}
311+
IMAGEDIR oiio-images
312+
ENVIRONMENT "${_nanobind_tests_pythonpath}"
313+
)
300314
endif ()
301315
endif ()
302316

‎src/libutil/CMakeLists.txt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -237,6 +237,11 @@ if (OIIO_BUILD_TESTS AND BUILD_TESTING)
237237
${OPENIMAGEIO_OPENEXR_TARGETS})
238238
add_test (unit_typedesc ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/typedesc_test)
239239

240+
fancy_add_executable (NAME python_array_format_test SRC python_array_format_test.cpp
241+
NO_INSTALL FOLDER "Unit Tests"
242+
LINK_LIBRARIES OpenImageIO_Util)
243+
add_test (unit_python_array_format ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/python_array_format_test)
244+
240245
fancy_add_executable (NAME type_traits_test SRC type_traits_test.cpp
241246
NO_INSTALL FOLDER "Unit Tests"
242247
LINK_LIBRARIES OpenImageIO_Util

‎src/libutil/python_array_format_test.cpp‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
// Copyright Contributors to the OpenImageIO project.
2+
// SPDX-License-Identifier: Apache-2.0
3+
// https://github.com/AcademySoftwareFoundation/OpenImageIO
4+
5+
// Regression for PEP 3118 format mapping used by Python bindings
6+
// (typedesc_from_python_array_code in src/python/py_oiio.cpp and
7+
// src/python-nanobind/py_oiio.cpp). Not linked against the bindings
8+
// (no Python.h here); this file must be kept in sync with those two copies.
9+
10+
#include <OpenImageIO/typedesc.h>
11+
#include <OpenImageIO/unittest.h>
12+
13+
using namespace OIIO;
14+
15+
namespace {
16+
17+
TypeDesc
18+
typedesc_from_python_array_code(string_view code)
19+
{
20+
if (code.size() >= 2 && code[0] == '=') {
21+
code.remove_prefix(1);
22+
}
23+
24+
TypeDesc t(code);
25+
if (!t.is_unknown()) {
26+
return t;
27+
}
28+
29+
if (code == "b" || code == "c") {
30+
return TypeDesc::INT8;
31+
}
32+
if (code == "B") {
33+
return TypeDesc::UINT8;
34+
}
35+
if (code == "h") {
36+
return TypeDesc::INT16;
37+
}
38+
if (code == "H") {
39+
return TypeDesc::UINT16;
40+
}
41+
if (code == "i") {
42+
return TypeDesc::INT;
43+
}
44+
if (code == "I") {
45+
return TypeDesc::UINT;
46+
}
47+
if (code == "l") {
48+
return TypeDesc::INT64;
49+
}
50+
if (code == "L") {
51+
return TypeDesc::UINT64;
52+
}
53+
if (code == "q") {
54+
return TypeDesc::INT64;
55+
}
56+
if (code == "Q") {
57+
return TypeDesc::UINT64;
58+
}
59+
if (code == "f") {
60+
return TypeDesc::FLOAT;
61+
}
62+
if (code == "d") {
63+
return TypeDesc::DOUBLE;
64+
}
65+
if (code == "float16" || code == "e") {
66+
return TypeDesc::HALF;
67+
}
68+
return TypeDesc::UNKNOWN;
69+
}
70+
71+
} // namespace
72+
73+
74+
int
75+
main(int /*argc*/, char* /*argv*/[])
76+
{
77+
OIIO_CHECK_EQUAL(typedesc_from_python_array_code("f"), TypeFloat);
78+
OIIO_CHECK_EQUAL(typedesc_from_python_array_code("=f"), TypeFloat);
79+
OIIO_CHECK_EQUAL(typedesc_from_python_array_code("=d"),
80+
TypeDesc(TypeDesc::DOUBLE));
81+
OIIO_CHECK_EQUAL(typedesc_from_python_array_code("=B"), TypeUInt8);
82+
83+
return unit_test_failures;
84+
}

‎src/python-nanobind/CMakeLists.txt‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,13 @@ set (nanobind_srcs
77
py_paramvalue.cpp
88
py_roi.cpp
99
py_imagespec.cpp
10-
py_typedesc.cpp)
10+
py_typedesc.cpp
11+
py_deepdata.cpp
12+
py_imageinput.cpp
13+
py_imageoutput.cpp
14+
py_imagebuf.cpp
15+
py_imagebufalgo.cpp
16+
py_argparse.cpp)
1117

1218
set (nanobind_build_package_dir ${CMAKE_BINARY_DIR}/lib/python/nanobind/OpenImageIO)
1319
file (MAKE_DIRECTORY ${nanobind_build_package_dir})

‎src/python-nanobind/MIGRATION_STATUS.md‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,26 @@ Generated from the binding sources. The nanobind extension is `PyOpenImageIONano
1010
| `py_typedesc.cpp` | <code>TypeDesc</code><br>Enums: <code>BASETYPE</code>, <code>AGGREGATE</code>, <code>VECSEMANTICS</code><br>Module <code>Type*</code> constants |
1111
| `py_imagespec.cpp` | <code>ImageSpec</code> (bound methods/properties, typed <code>attribute</code> / buffer paths via shared helpers). |
1212
| `py_paramvalue.cpp` | <code>ParamValue</code>, <code>ParamValueList</code><br>Enum: <code>Interp</code> |
13+
| `py_deepdata.cpp` | <code>DeepData</code> (same surface as pybind <code>declare_deepdata</code>) |
14+
| `py_imageinput.cpp` | <code>ImageInput</code> (factory methods return owned instances via <code>take_ownership</code>; matches pybind behavior). |
15+
| `py_imageoutput.cpp` | <code>ImageOutput</code> |
16+
| `py_imagebufalgo.cpp` | <code>ImageBufAlgo</code> namespace (dummy holder), <code>PixelStats</code>, <code>CompareResults</code>, IBA helpers — ported from <code>src/python/py_imagebufalgo.cpp</code>. |
17+
18+
## Nanobind-only API (new vs pybind)
19+
20+
These types are **not** mirrored under `src/python/` (pybind11). They wrap C++ APIs that were previously unavailable from Python.
21+
22+
| Source file | Python / C++ API |
23+
| --- | --- |
24+
| `py_argparse.cpp` | <code>ArgParse</code>, <code>ArgParseArg</code> — Python bindings for <code>OpenImageIO/argparse.h</code> (OIIO’s CLI-style argument parser; unrelated to the Python standard library’s <code>argparse</code> module). |
1325

1426
## Migrated — partial (gaps or intentional deltas vs pybind)
1527

1628
| Source file | Migrated (vs pybind) | Missing or divergent (vs pybind) |
1729
| --- | --- | --- |
18-
| `py_oiio.cpp` (`_OpenImageIO` module) | <ul><li><code>attribute</code> (one-arg and typed)</li><li><code>get_int_attribute</code></li><li><code>get_float_attribute</code></li><li><code>get_string_attribute</code></li><li><code>getattribute</code></li><li><code>__version__</code></li></ul> | <ul><li><code>geterror</code></li><li><code>get_bytes_attribute</code></li><li>Module <code>set_colorspace</code> (helper taking <code>ImageSpec</code> — the instance method is on <code>ImageSpec</code> in nanobind)</li><li><code>set_colorspace_rec709_gamma</code></li><li><code>equivalent_colorspace</code></li><li><code>is_imageio_format_name</code></li><li><code>AutoStride</code></li><li><code>openimageio_version</code>, <code>VERSION</code>, <code>VERSION_STRING</code>, <code>VERSION_MAJOR</code>, <code>VERSION_MINOR</code>, <code>VERSION_PATCH</code>, <code>INTRO_STRING</code></li><li>Optional: stack traces when <code>OPENIMAGEIO_DEBUG_PYTHON</code> is set (<code>Sysutil</code>)</li><li><code>make_pyobject</code>: no pybind-style <code>debugfmt</code> when the type is unhandled (returns default quietly)</li></ul> |
30+
| `py_oiio.cpp` (`_OpenImageIO` module) | <ul><li><code>attribute</code> (one-arg and typed)</li><li><code>get_int_attribute</code></li><li><code>get_float_attribute</code></li><li><code>get_string_attribute</code></li><li><code>getattribute</code></li><li><code>geterror</code></li><li><code>is_imageio_format_name</code> (wrapper taking <code>std::string</code> for Python <code>str</code>)</li><li><code>__version__</code></li></ul> | <ul><li><code>get_bytes_attribute</code></li><li>Module <code>set_colorspace</code> (helper taking <code>ImageSpec</code> — the instance method is on <code>ImageSpec</code> in nanobind)</li><li><code>set_colorspace_rec709_gamma</code></li><li><code>equivalent_colorspace</code></li><li><code>AutoStride</code></li><li><code>openimageio_version</code>, <code>VERSION</code>, <code>VERSION_STRING</code>, <code>VERSION_MAJOR</code>, <code>VERSION_MINOR</code>, <code>VERSION_PATCH</code>, <code>INTRO_STRING</code></li><li>Optional: stack traces when <code>OPENIMAGEIO_DEBUG_PYTHON</code> is set (<code>Sysutil</code>)</li><li><code>make_pyobject</code>: no pybind-style <code>debugfmt</code> when the type is unhandled (returns default quietly)</li></ul> |
1931
| `__init__.py` (package) | Env / DLL path setup, <code>from ._OpenImageIO import *</code>, version docstring. | <strong>TODO:</strong> Python CLI entry-point trampolines when the install layout matches the full wheel. |
32+
| `py_imagebuf.cpp` | <ul><li><code>ImageBuf</code> (constructors, I/O, metadata, pixels, deep, <code>_repr_png_</code>, shared buffer helpers in <code>py_oiio</code>)</li><li><code>write(ImageOutput&amp;)</code></li></ul> | <ul><li><code>get_pixels(HALF)</code> vs pybind: regression-tested by <code>test_half_getpixels()</code> in <code>testsuite/python-imagebuf/src/test_imagebuf.py</code> (<code>python-imagebuf</code> / <code>python-imagebuf.nanobind</code>).</li></ul> |
2033

2134
---
2235

@@ -26,14 +39,9 @@ These exist only under `src/python/` today; there are **no** corresponding `py_*
2639

2740
| Source file | Python / C++ API |
2841
| --- | --- |
29-
| `py_imageinput.cpp` | <ul><li><code>ImageInput</code></li><li>open, read, formats, …</li></ul> |
30-
| `py_imageoutput.cpp` | <code>ImageOutput</code> |
31-
| `py_imagebuf.cpp` | <code>ImageBuf</code> |
32-
| `py_imagebufalgo.cpp` | <ul><li><code>ImageBufAlgo</code> (namespace)</li><li><code>PixelStats</code></li><li><code>CompareResults</code></li><li>Exposed <code>IBA_*</code> helpers</li></ul> |
3342
| `py_texturesys.cpp` | <ul><li><code>Wrap</code></li><li><code>MipMode</code></li><li><code>InterpMode</code></li><li><code>TextureOpt</code></li><li><code>TextureSystem</code></li></ul> |
3443
| `py_imagecache.cpp` | <code>ImageCache</code> (wrapped) |
3544
| `py_colorconfig.cpp` | <code>ColorConfig</code> |
36-
| `py_deepdata.cpp` | <code>DeepData</code> |
3745

3846
---
3947

0 commit comments

Comments
 (0)