Fix uint32_t overflow of row offsets in avifImageYUVAnyToRGBAnySlow

In the slow YUV-to-RGB fallback in src/reformat.c, the Y/U/V/A row
pointers were computed with 'j * yRowBytes' style multiplications where
both operands are uint32_t. For images whose plane size exceeds 4 GiB
the product wraps around and the loop reads from a different row of the
same buffer, producing silently corrupted output.

Cast the row index to size_t before the multiplication so the offset is
computed in size_t arithmetic, matching what is already done for the
RGB destination pointers a few lines below and the same pattern applied
in c79a400.

Author: rootvector2

CHANGELOG.md

@@ -23,6 +23,8 @@ The changes are relative to the previous release, unless the baseline is specifi
 * Update svt.cmd/svt.sh/LocalSvt.cmake: v4.1.0
 * Fix decoding layered image with multiple scaled alpha layers
 * Fix NaN bypass of AVIF_CLAMP in gain map tone mapping (use fminf/fmaxf)
+* Fix uint32_t overflow in row offset arithmetic in
+  avifImageYUVAnyToRGBAnySlow().
 * avifenc: reject mismatched --depth for Y4M input
 * Use libaom AOMD_SET_FRAME_SIZE_LIMIT if available
 

src/reformat.c

@@ -686,10 +686,10 @@ static avifResult avifImageYUVAnyToRGBAnySlow(const avifImage * image,
     for (uint32_t j = 0; j < image->height; ++j) {
         // uvJ is used only when yuvHasColor is true.
         const uint32_t uvJ = yuvHasColor ? (j >> state->yuv.formatInfo.chromaShiftY) : 0;
-        const uint8_t * ptrY8 = &yPlane[j * yRowBytes];
-        const uint8_t * ptrU8 = uPlane ? &uPlane[(uvJ * uRowBytes)] : NULL;
-        const uint8_t * ptrV8 = vPlane ? &vPlane[(uvJ * vRowBytes)] : NULL;
-        const uint8_t * ptrA8 = aPlane ? &aPlane[j * aRowBytes] : NULL;
+        const uint8_t * ptrY8 = &yPlane[(size_t)j * yRowBytes];
+        const uint8_t * ptrU8 = uPlane ? &uPlane[((size_t)uvJ * uRowBytes)] : NULL;
+        const uint8_t * ptrV8 = vPlane ? &vPlane[((size_t)uvJ * vRowBytes)] : NULL;
+        const uint8_t * ptrA8 = aPlane ? &aPlane[(size_t)j * aRowBytes] : NULL;
         const uint16_t * ptrY16 = (const uint16_t *)ptrY8;
         const uint16_t * ptrU16 = (const uint16_t *)ptrU8;
         const uint16_t * ptrV16 = (const uint16_t *)ptrV8;