Skip to content

fix(ios+android): crop/resize/rotate correctness (5 bugs)#117

Open
rayabelcode wants to merge 2 commits intomrousavy:mainfrom
rayabelcode:fix/ios-crop-resize-orientation
Open

fix(ios+android): crop/resize/rotate correctness (5 bugs)#117
rayabelcode wants to merge 2 commits intomrousavy:mainfrom
rayabelcode:fix/ios-crop-resize-orientation

Conversation

@rayabelcode
Copy link
Copy Markdown

@rayabelcode rayabelcode commented Apr 17, 2026

Summary

Fixes five related bugs that make Image.crop(), Image.resize(), Image.rotate(), and ImageFactory.createBlankImage() produce incorrect output. They compound - chaining crop + rotate + resize (a common flow, including vision-camera v5's advertised Photo.toImageAsync() -> crop -> resize -> saveToTemporaryFileAsync()) gets the wrong region, the wrong orientation, pixel-inflated files, AND clipped rotation output.

Bugs 1-4 are iOS-only. Bug 5 affects both iOS and Android.

Commits in this PR:

  1. fix(ios): correct crop rect size, orientation, and renderer scale - bugs 1-4
  2. fix: rotate slow path clipped content by keeping canvas at input size - bug 5 (both platforms)

The bugs

All iOS fixes are in packages/react-native-nitro-image/ios/Public/NativeImage.swift (plus one in HybridImageFactory.swift). The Android fix is in packages/react-native-nitro-image/android/src/main/java/com/margelo/nitro/image/HybridImage.kt.

1. crop - rect size used full image dims instead of the crop size (iOS)

let targetWidth = endX - startX          // computed...
let targetHeight = endY - startY         // ...but never used
...
let targetRect = CGRect(origin: CGPoint(x: startX, y: startY),
                        size: CGSize(width: uiImage.size.width,     // BUG
                                     height: uiImage.size.height))  // BUG

CGImage.cropping(to:) intersects the rect with the image bounds, so a full-image-sized rect returned "image minus origin offset" rather than a crop of (targetWidth, targetHeight).

2. crop - orientation / scale mismatch between rect and cgImage (iOS)

Two linked issues:

  • When uiImage.imageOrientation != .up (common for images from vision-camera v5's Photo.toImageAsync(), which constructs UIImage(cgImage:, scale: 1, orientation: uiOrientation)), uiImage.size is display-space but uiImage.cgImage is sensor-space. The caller's (startX, startY, endX, endY) rect is in display-space, so cgImage.cropping(to:) landed in the wrong region of the bitmap.
  • UIImage(cgImage: croppedCgImage) defaulted the output to .up, dropping the source's orientation and surfacing raw sensor-oriented pixels to callers who expected display-space output.

3. resize - device-scale-factor pixel inflation (iOS)

let renderer = UIGraphicsImageRenderer(size: targetSize)   // no format

UIGraphicsImageRenderer(size:) defaults to UIScreen.main.scale (2 on standard iPhones, 3 on Pro). So resize(2000, 1500) on a 3x device produced a 6000x4500-pixel bitmap. uiImage.size still returned (2000, 1500) in points, so JS saw correct Image.width/Image.height - but saveToTemporaryFileAsync / toEncodedImageData encode at the CGImage's actual pixel dimensions, so saved files were 3x too large in each dimension.

4. Same default-scale issue in rotate() slow path and createBlankImage (iOS)

Both use UIGraphicsImageRenderer(size:) / UIGraphicsImageRendererFormat() without setting scale.

5. rotate() slow path - output canvas sized to input, clips rotated content (iOS + Android)

Both platforms' slow-path rotate used the input's dimensions as the output buffer size:

  • iOS (NativeImage.swift): UIGraphicsImageRenderer(size: uiImage.size, ...) - canvas stays at input size.
  • Android (HybridImage.kt): createBitmap(bitmap.width, bitmap.height) - destination bitmap stays at input size.

When degrees is not a multiple of 180 (e.g., a portrait 1500x2000 image rotated -90 degrees), the rotated content wants a 2000x1500 canvas but gets a 1500x2000 one. cgImage.cropping(to:) / Canvas.drawBitmap clip the extents: the caller receives an image at the input dimensions with half the rotated content missing. On the JS side, Image.width / Image.height still report the (unchanged) input dims, so the bug is invisible from metadata - but the visible content is wrong.

Symptom observed in a production vision-camera v5 app: "image is cropped to the right size but rotated 90 degrees so you only see part of it" - exactly the canvas-clipping fingerprint.

The fixes

crop - normalize input, use actual crop size, wrap output with explicit scale+orientation (iOS)

// Normalize so cgImage and our display-space rect agree.
let normalized: UIImage
if uiImage.imageOrientation == .up && uiImage.scale == 1 {
    normalized = uiImage
} else {
    let format = UIGraphicsImageRendererFormat()
    format.scale = 1
    let renderer = UIGraphicsImageRenderer(size: uiImage.size, format: format)
    normalized = renderer.image { _ in uiImage.draw(at: .zero) }
}
guard let cgImage = normalized.cgImage else { ... }

let targetRect = CGRect(x: startX, y: startY, width: targetWidth, height: targetHeight)
guard let croppedCgImage = cgImage.cropping(to: targetRect) else { ... }

let croppedUiImage = UIImage(cgImage: croppedCgImage, scale: 1, orientation: .up)

resize / rotate slow path / createBlankImage - force scale=1 (iOS)

let format = UIGraphicsImageRendererFormat()
format.scale = 1
let renderer = UIGraphicsImageRenderer(size: targetSize, format: format)

rotate slow path - size canvas to rotated bounding box (iOS + Android)

Compute the rotated bounding box via |cos|*w + |sin|*h and |sin|*w + |cos|*h. Size the renderer / destination bitmap to that. Translate so the rotated content lands centered on the output canvas.

iOS:

let radians = degrees * .pi / 180
let width = uiImage.size.width
let height = uiImage.size.height
let outputWidth = abs(cos(radians)) * width + abs(sin(radians)) * height
let outputHeight = abs(sin(radians)) * width + abs(cos(radians)) * height
let outputSize = CGSize(width: outputWidth, height: outputHeight)
let renderer = UIGraphicsImageRenderer(size: outputSize, format: format)
// ...translate to outputSize/2, rotate, draw input centered on origin

Android:

val radians = Math.toRadians(degrees)
val outputW = (abs(cos(radians)) * srcW + abs(sin(radians)) * srcH).toInt()
val outputH = (abs(sin(radians)) * srcW + abs(cos(radians)) * srcH).toInt()
val matrix = Matrix()
matrix.setRotate(degrees.toFloat(), source.width / 2f, source.height / 2f)
matrix.postTranslate((outputW - source.width) / 2f, (outputH - source.height) / 2f)
val destination = createBitmap(outputW, outputH)

Works for any angle (the cos/sin formula gives a tight bounding box for arbitrary rotations), but the bug is most visible at 90/270 where the canvas transposes from the content.

Reproduction

// Any source image works; vision-camera v5 is the most direct path to hitting these.
const photo = await photoOutput.capturePhoto({}, {});
const image = await photo.toImageAsync();
console.log(image.width, image.height);  // iPhone 15 Pro: 3024, 4032 (portrait)

const cropped = await image.cropAsync(100, 200, 1225, 1700);
// Before: ~2924x3832 image, landscape-sensor-oriented (bugs 1 + 2)
// After:  ~1125x1500 image of the requested region, portrait-oriented

const resized = await cropped.resizeAsync(2000, 1500);
const path = await resized.saveToTemporaryFileAsync('jpg', 40);
// Before (3x device): 6000x4500-pixel JPEG (bug 3)
// After:              2000x1500-pixel JPEG

const rotated = await cropped.rotateAsync(-90);
// Before: result.width/height still (1125, 1500) - input dims - but rotated
//         content is clipped to that canvas (bug 5)
// After:  result.width/height swap to (1500, 1125) - a true landscape
//         bounding box with complete content

Test plan

  • Patch applied to node_modules/react-native-nitro-image in an Expo SDK 55 app (vision-camera v5.0.1, nitro-modules 0.35.x)
  • Full capture -> crop -> resize -> save pipeline exercised on iPhone 15 Pro (3x scale)
    • Cropped region matches the requested rectangle
    • Cropped image is in portrait orientation (matches display-space input)
    • Saved JPEG dimensions match requested pixel count (no 3x inflation)
  • Landscape-toggle rotation (crop then rotate -90 then resize then save) verified on iPhone 15 Pro and Android - output JPEG is a complete landscape card, not a clipped fragment
  • Android untouched on bugs 1-4; Android crop/resize/createBlankImage behavior preserved
  • Unit tests in this repo - I did not find existing Swift/Kotlin tests to extend; happy to add some if there's a preferred pattern

Four related bugs in iOS caused Image.crop() / Image.resize() (and,
by extension, Image.rotate() slow path and ImageFactory.createBlankImage)
to produce incorrect output. They surface most visibly through the
vision-camera v5 flow Photo.toImageAsync() -> crop -> resize ->
saveToTemporaryFileAsync(), but repro from any source image.

ios/Public/NativeImage.swift:

1. crop: the targetRect used CGSize(uiImage.size.width, uiImage.size.height)
   instead of the computed (targetWidth, targetHeight). CGImage.cropping
   intersects the rect with the image bounds, so callers got "image minus
   origin offset" instead of an actual crop of the requested region.

2. crop: the pre-crop UIImage could have non-.up imageOrientation and/or
   scale != 1 (both common for the Photo.toImageAsync() bridge, which
   constructs UIImage(cgImage:, scale: 1, orientation: uiOrientation)).
   uiImage.size is display-space while cgImage is sensor-space, so the
   caller's rect landed in the wrong region of cgImage. UIImage(cgImage:)
   also defaulted the output to .up orientation, dropping the source's
   orientation and producing sensor-oriented pixels.

   Fix: if orientation != .up or scale != 1, draw through a scale=1
   UIGraphicsImageRenderer to bake both into pixels before cropping,
   then wrap the cropped CGImage with explicit scale=1 and .up so
   downstream consumers see consistent state.

3. resize + rotate (slow path): UIGraphicsImageRenderer(size:) without
   a UIGraphicsImageRendererFormat defaults to UIScreen.main.scale.
   On 2x/3x devices "resize to 2000x1500" produced 4000x3000 or
   6000x4500-pixel output, while uiImage.size still reported
   (2000, 1500) in points. JS callers saw correct Image.width/height
   but the saved JPEG was 2-3x too large in each dimension.

   Fix: set UIGraphicsImageRendererFormat().scale = 1 before constructing
   the renderer.

ios/HybridImageFactory.swift:

4. createBlankImage had the same default-scale issue; applied the same
   scale = 1 fix.

Android (HybridImage.kt) is unchanged -- its Bitmap-based path has no
point/pixel distinction and was already correct.

Verified on an iPhone 15 Pro (3x screen scale) via a vision-camera v5
photo-scanning app. Before: cropped region was wrong, orientation
dropped to landscape sensor-space, saved files 3x larger than requested.
After: crop produces the requested region in portrait orientation, and
saved files match the requested pixel dimensions.
@rayabelcode rayabelcode force-pushed the fix/ios-crop-resize-orientation branch from 64b9896 to 43a7280 Compare April 17, 2026 15:06
The rotate slow path on iOS (NativeImage.swift) and Android
(HybridImage.kt) sized the output canvas to the input image's
dimensions. When degrees is not a multiple of 180 (e.g., a 90 degree
rotation of a portrait image into a landscape), the rotated content
extends beyond the canvas and gets clipped -- the caller sees an image
at the input dimensions with half the rotated content missing.

Fix: compute the rotated bounding box as
  outputW = abs(cos(r)) * w + abs(sin(r)) * h
  outputH = abs(sin(r)) * w + abs(cos(r)) * h

Size the renderer / destination bitmap to that, then translate so the
rotated content lands centered on the output canvas.

Verified end-to-end on iPhone 15 Pro (3x) and a Pixel-class Android
device via a vision-camera v5 photo-scanning app: rotating a portrait
card 90 degrees now produces a correctly sized landscape JPEG with
complete content, rather than a portrait JPEG with a clipped landscape
rectangle inside.

Works for any angle (the cos/sin formula gives a tight bounding box for
arbitrary rotations), but the bug is most visible at 90/270 where the
canvas is transposed from the content.
@rayabelcode rayabelcode changed the title fix(ios): correct crop rect size, orientation, and renderer scale fix(ios+android): crop/resize/rotate correctness (5 bugs) Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant