Consider adopting an image-format-per-plugin model

[bwoodsend]

Pillow's current model of all image formats in one package means that Pillow's footprint will inevitably grow boundlessly. The 10.1.0 Linux wheels were ~3.5MB, 10.2.0 went up to 4.5MB then 11.3.0 jumped all the way up to ~6.5MB for AVIF support (and these are compressed sizes – the installed footprint is worse). For anyone not using the new image formats and where a small footprint is coveted (serverless lambdas, small end user applications, services shipped as Docker images), every new addition is also a large regression.

From a security perspective, a new unneeded format is also a choice between an increased surface area of attack or squatting on old versions of Pillow – both of which are obviously bad. (Being able to limit formats using Image.open(untrusted_content, formats=["JPEG", "PNG"]) helps but it's not really enough that I'd feel comfortable ignoring a CVE in libavif.)

I'm aware that pillow can be built without support for some formats by building from source or just by deleting bits of the package post-install but neither are appealing options and they both get much worse if you want all the files on a server to come from say RPMs for better efficiency and security auditing.

I was using the pillow-avif-plugin in one of my projects for a while before it got merged into pillow. I thought it was a really good model. Anyone who didn't need AVIF support didn't have to pay for its large footprint and anyone who did only had to add one import pillow_avif and an extra item to their dependencies to get it. The integration felt very seamless.

Would you consider adopting a policy whereby all new and/or large and/or niche and/or complex formats are permanently kept as separately distributed pillow_$format_plugin wheels rather than merged into pillow itself?

[aclark4life]

@bwoodsend I don't think we'd rule it out but I'd definitely want to be careful with the implementation. In fact there is a discussion of adding a framework trove classifier in #8892 in case that helps.

[radarhere]

I think the author of pillow-avif-plugin always considered it to be a temporary measure until Pillow integrated support for the format, and people rejoiced when it was combined into Pillow. When we didn't include it in the wheels immediately, there was pushback.

You would also need to convince the users in #4247 that they should just use an existing plugin.

I'm just saying that others have a different perspective.

From a prior discussion, #8869 (comment)

I'm keeping an eye on PEP-771, which would provide a nice path forward that would allow size-conscious users the ability to opt-out of features they don't need (not only AVIF).

[hugovk]

New plugins do add considerable weight to wheels. We now have nearly 100 wheels weighing 550 MB, which is likely to increase. We fairly recently used up our PyPI quota and had to request an increase.

For new plugins, I think we should encourage separate plugins rather than adding them to Pillow core. Especially for heavy plugins which aren't needed across the board.

For example, here's a project that's removed the large AVIF files that they don't need: randovania/randovania#8732.

Things which could help:

1. Allow some popular plugins to be maintained under the https://github.com/python-pillow organisation.

   • This could be similar to how pytest hosts plugins.
   • The original maintainers and domain experts can continue looking after their plugin, and the core Pillow team would also be able to lend a hand with maintenance.
   • This would give some sort of "official seal of approval" that I think many want.

2. We could add plugins as "extras" in pyproject.toml, so for example, you'd pip install pillow[jpegxl] that'd also install the jxlpy package.

   • Possibly limit to those plugins hosted under python-pillow

3. If this works well, consider "debundling" some heavy core plugins such as AVIF.

   • I think we could keep a lot of the current core plugins, definitely the extremely widespread ones such as PNG, JPEG and so on, and a lot of the small ones.

4. Separately, we could consider deprecating some of the obscure core plugins. There are definitely a few added during the early PIL days, where it was really hard to find sample images when adding tests decades later! These should be long deprecations, which could be dropped if we get feedback of real use.