Perform QA for Improvements to the ImageAPI in version 5.1.6

[patrick-schweiger]

Description

A backport of some improvements/fixes in the ImageAPI which is impacting a customer in version 5.1.6 and leading to outages. These fixes might help with their ongoing outages. The improvements are untested and should be run and tested somewhere to make sure the image stuff still works.
#36210

Acceptance Criteria

• Successfully pass QA and ensure image functionality is not broken

Priority

Medium

Additional Context

This should be completed as part of the maintenance team sprint beginning on Monday, June 22. Estimate at 3 or 5 pointer.

[dsilvam]

This one needs to be merged first #36210
@dsolistorres @gortiz-dotcms

[dsolistorres]

🔴 QA Failed — image processing is broken out-of-the-box

Verified PR #36210 (backport-image-improvements-5.1.6 @ f2ca3c6a) on a native-arm64 build of the branch + ES 6.1.3_for_5.1.6 + Postgres 13, demo starter. With the default configuration, every server-side image transformation fails.

Symptom

Any request through the image filter exporter returns HTTP 404, while raw file delivery still works:

GET /contentAsset/image/<id>/fileAsset/filter/Resize/resize_w/300   -> 404
GET /contentAsset/raw-data/<id>/fileAsset                            -> 200 (unaffected)

Backend log on every image request:

WARN  filter.ImageFilter: provider == null!
ERROR servlets.BinaryExporterServlet: [BinaryContentExporterException] An error occurred when accessing
'/image/<id>/fileAsset/filter/Resize/resize_w/300': provider == null!

This affects all filters (resize, thumbnail, crop, jpeg, png, gif, pdf) because the exporter resolves the engine on every request.

Root cause

ImageFilterApiImpl.deregisterProviders() deregisters a hard-coded WebP reader SPI but never null-checks the lookup result:

// ImageFilterApiImpl.java
private String[] providersToIgnore = Config.getStringArrayProperty(
    "IMAGE_READER_SPIS_TO_DEREGISTER",
    new String[]{"net.sf.javavp8decoder.imageio.WebPImageReaderSpi"});   // <-- not present in 5.1.6's deps
...
ImageReaderSpi provider = lookupProviderByName(registry, providerClazz); // returns null (ClassNotFoundException)
registry.deregisterServiceProvider(provider);                            // deregister(null) throws IllegalArgumentException

The backport bumps TwelveMonkeys to 3.7.0 and bundles imageio-webp-3.7.0.jar, whose WebP SPI is com.twelvemonkeys.imageio.plugins.webp.WebPImageReaderSpi. The deregister default still names the old net.sf.javavp8decoder... class, which is absent from the 5.1.6 classpath → Class.forName fails → lookupProviderByName returns null → deregisterServiceProvider(null) throws. The exporter calls imageFilterAPI().resolveFilters(...) → ImageEngine.resolve() → the ImageFilterApiImpl constructor on every request, so the failure is total.

This is an incomplete backport: the deregistration code came from newer dotCMS where net.sf.javavp8decoder was the actual WebP dependency. 5.1.6 uses TwelveMonkeys instead, so the target class doesn't exist.

Confirmed fix

Pointing the config at the actually-registered class makes every filter work again:

Config	…/filter/Resize/resize_w/300
Default (ships broken)	❌ HTTP 404, provider == null!
IMAGE_READER_SPIS_TO_DEREGISTER=com.twelvemonkeys.imageio.plugins.webp.WebPImageReaderSpi	✅ HTTP 200, PNG 300×243

With the corrected config, the full pipeline passes — Resize, Thumbnail (150×100), Crop (200×150), Resize+Jpeg chain, GIF resize, default Png, and the PDF render path (#23384) handled a 3-page / 2500² PDF under a 512 MB heap with no OOM.

Recommended fix (code): add a null guard in deregisterProviders() — this is a latent NPE on any branch where the named SPI isn't registered:

ImageReaderSpi provider = lookupProviderByName(registry, providerClazz);
if (provider != null) {
    registry.deregisterServiceProvider(provider);
}

Optionally also correct the default class name for 5.1.6's dependency set. Operators can mitigate immediately without a rebuild by setting IMAGE_READER_SPIS_TO_DEREGISTER (env DOT_IMAGE_READER_SPIS_TO_DEREGISTER) to the TwelveMonkeys class above, or to empty.

[dsolistorres]

✅ QA Passed — fix applied and verified

The image-processing regression reported above has been fixed and the full filter pipeline re-verified end-to-end on a native build of the branch (ES 6.1.3_for_5.1.6 + Postgres 13, demo starter).

Fix

Commit 117b7f5e34 on backport-image-improvements-5.1.6 (PR #36210) — a null guard in ImageFilterApiImpl.deregisterProviders():

ImageReaderSpi provider = lookupProviderByName(registry, providerClazz);
if (provider != null) {
    registry.deregisterServiceProvider(provider);
}

Why the guard is needed here but not in evergreen

deregisterProviders() removes the net.sf.javavp8decoder.imageio.WebPImageReaderSpi WebP reader so ImageIO doesn't pick that (inferior) decoder. The code is identical on main and has no null check there either — the difference is the dependency set, not the code:

• evergreen (main): the net.sf.javavp8decoder library is on the classpath (main's own code references net.sf.javavp8decoder.imageio.WebPImageReader directly), so the default SPI class exists and is registered → lookupProviderByName returns a real provider → deregisterServiceProvider(provider) succeeds. The null path never happens. (main also has the preferred native libvips engine; the JVM engine is the fallback.)
• 5.1.6.k backport: javavp8decoder was not backported. Instead the backport adds TwelveMonkeys imageio-webp 3.7.0, whose WebP reader is a different class (com.twelvemonkeys.imageio.plugins.webp.WebPImageReaderSpi). So the hard-coded net.sf.javavp8decoder... SPI is absent → lookupProviderByName returns null → deregisterServiceProvider(null) throws IllegalArgumentException, which broke every image request through ImageFilterExporter (HTTP 404, provider == null!).

On 5.1.6.k there is simply no javavp8decoder reader to deregister (and none needs deregistering — TwelveMonkeys supplies WebP). The null guard makes deregisterProviders() tolerate "the SPI I was told to remove isn't here." (Strictly, the missing guard is a latent bug on main too; it just never fires because main's classpath matches the hard-coded default.)

Verification — every filter changed by the backport

Fixed build, default config, no config workaround. All returned HTTP 200 with correct output; zero provider == null / OOM:

Filter	Test	Result
Resize	resize_w/300	300×243 PNG ✅
Scale (→Resize)	scale_w/300	300×243 PNG ✅
Crop	crop_w/200/crop_h/150	200×150 PNG ✅
Thumbnail	thumbnail_w/150/thumbnail_h/100	150×100 PNG ✅
Jpeg	jpeg_q/70	JPEG ✅
Png	Png	2000×1299 PNG ✅
Gif	Gif	GIF89a ✅
Grayscale	Grayscale	PNG ✅
Exposure	exposure_exp/1.2	PNG ✅
Gamma	gamma_g/1.5	PNG ✅
Hsb	hsb_h/0.5/hsb_s/1.0/hsb_b/1.0	PNG ✅
Flip	flip_flip/true	PNG ✅
Rotate	rotate_a/90	1100×1355 (dims swapped) ✅
Resize,Jpeg (chain)	resize_w/400/jpeg_q/70	JPEG ✅
Resize on GIF (new ResizeGifImageFilter)	resize_w/64	64×224 ✅
PDF (#23384 OOM fix)	filter/PDF	612×792 PNG ✅
PDF,Resize (chain)	PDF,Resize/resize_w/200	200×258 PNG ✅
PDF page/dpi	pdf_page/2/pdf_dpi/150	1275×1649 PNG ✅

Notes: SubSampleImageFilter (new) is exercised internally by ResizeImageFilter on large downscales. PDFImageFilter (#23384) was also load-tested in isolation — a 3-page / 2500² PDF rendered under a capped 512 MB heap with no OOM.

Verdict: PASS. Recommend merging PR #36210 with commit 117b7f5e34 included.