Skip to content

Commit ae187ca

Browse files
wiredfoolhugovk
andauthored
Apply suggestions from code review
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
1 parent cca80e2 commit ae187ca

5 files changed

Lines changed: 46 additions & 61 deletions

File tree

.github/workflows/test-windows.yml

Lines changed: 3 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -35,17 +35,11 @@ jobs:
3535
strategy:
3636
fail-fast: false
3737
matrix:
38-
python-version: [ "3.10", "3.11", "3.12", "3.13" ]
38+
python-version: ["pypy3.11", "pypy3.10", "3.10", "3.11", "3.12", "3.13", "3.14"]
3939
architecture: ["x64"]
4040
os: ["windows-latest"]
41-
pyarrow: ["true"]
4241
include:
4342
# Test the oldest Python on 32-bit
44-
- { python-version: "3.9", architecture: "x86", os: "windows-2019", pyarrow: "false" }
45-
# test the non-pyarrow capable ones
46-
- { python-version: "3.14", architecture: "x64", os: "windows-latest", pyarrow: "false" }
47-
- { python-version: "pypy3.11", architecture: "x64", os: "windows-latest", pyarrow: "false" }
48-
- { python-version: "pypy3.10", architecture: "x64", os: "windows-latest", pyarrow: "false" }
4943
timeout-minutes: 30
5044

5145
name: Python ${{ matrix.python-version }} (${{ matrix.architecture }})
@@ -92,10 +86,9 @@ jobs:
9286
run: |
9387
python3 -m pip install PyQt6
9488
95-
- name: Install PyArrow Test Dependency
96-
if: "matrix.pyarrow == 'true'"
89+
- name: Install PyArrow dependency
9790
run: |
98-
python3 -m pip install --only-binary=:all: pyarrow
91+
python3 -m pip install --only-binary=:all: pyarrow || true
9992
10093
- name: Install dependencies
10194
id: install

Tests/test_arrow.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,7 @@
44

55
from PIL import Image
66

7-
from .helper import (
8-
hopper,
9-
)
7+
from .helper import hopper
108

119

1210
@pytest.mark.parametrize(

docs/reference/arrow_support.rst

Lines changed: 32 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -4,84 +4,85 @@
44
Arrow Support
55
=============
66

7-
Arrow is an in memory data exchange format that is the spritual
8-
successor to the numpy array interface. It provides for zero copy
9-
access to columnar data, which in our case is Image data.
7+
`Arrow <https://arrow.apache.org/>`__
8+
is an in-memory data exchange format that is the spiritual
9+
successor to the NumPy array interface. It provides for zero-copy
10+
access to columnar data, which in our case is ``Image`` data.
1011

11-
The goal with Arrow is to provide native zero-copy interop with any
12-
arrow provider or consumer in the Python ecosystem.
12+
The goal with Arrow is to provide native zero-copy interoperability
13+
with any Arrow provider or consumer in the Python ecosystem.
1314

14-
.. warning:: Zero-copy does not mean zero allocation -- The internal
15+
.. warning:: Zero-copy does not mean zero allocation -- the internal
1516
memory layout of Pillow images contains an allocation for row
1617
pointers, so there is a non-zero, but significantly smaller than a
17-
full copy memory cost to reading an arrow image.
18+
full-copy memory cost to reading an Arrow image.
1819

1920

2021
Data Formats
2122
============
2223

23-
Pillow currently supports exporting arrow images in all modes
24+
Pillow currently supports exporting Arrow images in all modes
2425
**except** for ``BGR;15``, ``BGR;16`` and ``BGR;24``. This is due to
25-
line length packing in these modes making for non-continuous memory.
26+
line-length packing in these modes making for non-continuous memory.
2627

27-
For single band images, the exported array is width*height elements,
28-
with each pixel corresponding to the appropriate arrow type.
28+
For single-band images, the exported array is width*height elements,
29+
with each pixel corresponding to the appropriate Arrow type.
2930

30-
For multiband images, the exported array is width*height fixed length
31-
4 element arrays of uint8. This is memory compatible with the raw
32-
image storage of 4 bytes per pixel.
31+
For multiband images, the exported array is width*height fixed-length
32+
four-element arrays of uint8. This is memory compatible with the raw
33+
image storage of four bytes per pixel.
3334

34-
Mode ``1`` images are exported as 1 uint8 byte/pixel, as this is
35+
Mode ``1`` images are exported as one uint8 byte/pixel, as this is
3536
consistent with the internal storage.
3637

3738
Pillow will accept, but not produce, one other format. For any
38-
multichannel image with 32 bit storage per pixel, Pillow will accept
39+
multichannel image with 32-bit storage per pixel, Pillow will accept
3940
an array of width*height int32 elements, which will then be
40-
interpreted using the mode specific interpretation of the bytes.
41+
interpreted using the mode-specific interpretation of the bytes.
4142

42-
The image mode must match the arrow band format when reading single
43-
channel images
43+
The image mode must match the Arrow band format when reading single
44+
channel images.
4445

4546
Memory Allocator
4647
================
4748

4849
Pillow's default memory allocator, the :ref:`block_allocator`,
49-
allocates up to a 16MB block for images by default. Larger images
50+
allocates up to a 16 MB block for images by default. Larger images
5051
overflow into additional blocks. Arrow requires a single continuous
5152
memory allocation, so images allocated in multiple blocks cannot be
52-
exported in the arrow format.
53+
exported in the Arrow format.
5354

5455
To enable the single block allocator::
5556

5657
from PIL import Image
5758
Image.core.set_use_block_allocator(1)
5859

59-
Note that this is a global setting, not a per image setting.
60+
Note that this is a global setting, not a per-image setting.
6061

6162
Unsupported Features
6263
====================
6364

64-
* Table/Dataframe protocol. We currently support a single array.
65+
* Table/dataframe protocol. We support a single array.
6566
* Null markers, producing or consuming. Null values are inferred from
6667
the mode. e.g. RGB images are stored in the first three bytes of
67-
each 32 bit pixel, and the last byte is an implied null.
68-
* Schema Negotiation. There is an optional schema for the requested
69-
datatype in the arrow source interface. We currently ignore that
68+
each 32-bit pixel, and the last byte is an implied null.
69+
* Schema negotiation. There is an optional schema for the requested
70+
datatype in the Arrow source interface. We ignore that
7071
parameter.
71-
* Array Metadata.
72+
* Array metadata.
7273

7374
Internal Details
7475
================
7576

7677
Python Arrow C interface:
7778
https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html
7879

79-
The memory that is exported from the arrow interface is shared -- not
80+
The memory that is exported from the Arrow interface is shared -- not
8081
copied, so the lifetime of the memory allocation is no longer strictly
81-
tied to the life of the python object.
82+
tied to the life of the Python object.
8283

8384
The core imaging struct now has a refcount associated with it, and the
84-
lifetime of the core image struct is now divorced from the python
85+
lifetime of the core image struct is now divorced from the Python
8586
image object. Creating an arrow reference to the image increments the
8687
refcount, and the imaging struct is only released when the refcount
87-
reaches 0.
88+
reaches zero.

src/PIL/Image.py

Lines changed: 7 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -3210,16 +3210,9 @@ class SupportsArrowArrayInterface(Protocol):
32103210
data interface.
32113211
"""
32123212

3213-
# Sorry, no types for you until pre-commit.ci stops changing stringy
3214-
# PyCapsule type to an unimportable value type, which then fails lint.
3215-
# PyCapsules are not importable, and only available in the C-API layer.
3216-
# def __arrow_c_array__(
3217-
# self, requested_schema: 'PyCapsule' = None
3218-
# ) -> tuple['PyCapsule', 'PyCapsule']:
3219-
# raise NotImplementedError()
3220-
3221-
# old not typed definition.
3222-
def __arrow_c_array__(self, requested_schema=None):
3213+
def __arrow_c_array__(
3214+
self, requested_schema: "PyCapsule" = None # type: ignore[name-defined] # noqa: F821, UP037
3215+
) -> tuple["PyCapsule", "PyCapsule"]: # type: ignore[name-defined] # noqa: F821, UP037
32233216
raise NotImplementedError()
32243217

32253218

@@ -3312,7 +3305,7 @@ def fromarray(obj: SupportsArrayInterface, mode: str | None = None) -> Image:
33123305

33133306

33143307
def fromarrow(obj: SupportsArrowArrayInterface, mode, size) -> Image:
3315-
"""Creates an image with zero copy shared memory from an object exporting
3308+
"""Creates an image with zero-copy shared memory from an object exporting
33163309
the arrow_c_array interface protocol::
33173310
33183311
from PIL import Image
@@ -3323,7 +3316,7 @@ def fromarrow(obj: SupportsArrowArrayInterface, mode, size) -> Image:
33233316
If the data representation of the ``obj`` is not compatible with
33243317
Pillow internal storage, a ValueError is raised.
33253318
3326-
Pillow images can also be converted to arrow objects::
3319+
Pillow images can also be converted to Arrow objects::
33273320
33283321
from PIL import Image
33293322
import pyarrow as pa
@@ -3339,13 +3332,13 @@ def fromarrow(obj: SupportsArrowArrayInterface, mode, size) -> Image:
33393332
:param size: Image size. This must match the storage of the arrow object.
33403333
:returns: An Image Object
33413334
3342-
Note that according to the arrow spec, both the producer and the
3335+
Note that according to the Arrow spec, both the producer and the
33433336
consumer should consider the exported array to be immutable, as
33443337
unsynchronized updates will potentially cause inconsistent data.
33453338
33463339
See: :ref:`arrow-support` for more detailed information
33473340
3348-
.. versionadded:: 11.2
3341+
.. versionadded:: 11.2.0
33493342
33503343
"""
33513344
if not hasattr(obj, "__arrow_c_array__"):

src/_imaging.c

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -240,14 +240,14 @@ ArrowError(int err) {
240240
return ImagingError_MemoryError();
241241
}
242242
if (err == IMAGING_ARROW_INCOMPATIBLE_MODE) {
243-
return ImagingError_ValueError("Incompatible Pillow mode for Arrow Array");
243+
return ImagingError_ValueError("Incompatible Pillow mode for Arrow array");
244244
}
245245
if (err == IMAGING_ARROW_MEMORY_LAYOUT) {
246246
return ImagingError_ValueError(
247247
"Image is in multiple array blocks, use imaging_new_block for zero copy"
248248
);
249249
}
250-
return ImagingError_ValueError("Unknown Error");
250+
return ImagingError_ValueError("Unknown error");
251251
}
252252

253253
void
@@ -313,7 +313,7 @@ _new_arrow(PyObject *self, PyObject *args) {
313313
PyImagingNew(ImagingNewArrow(mode, xsize, ysize, schema_capsule, array_capsule)
314314
);
315315
if (!ret) {
316-
return ImagingError_ValueError("Invalid arrow array mode or size mismatch");
316+
return ImagingError_ValueError("Invalid Arrow array mode or size mismatch");
317317
}
318318
return ret;
319319
}

0 commit comments

Comments
 (0)