Skip to content

#5294: widen Math.abs to long in BytesRaw.shift for Integer.MIN_VALUE#5406

Closed
gemshrine wants to merge 1 commit into
objectionary:masterfrom
gemshrine:fix/issue-5294-shift-min-value
Closed

#5294: widen Math.abs to long in BytesRaw.shift for Integer.MIN_VALUE#5406
gemshrine wants to merge 1 commit into
objectionary:masterfrom
gemshrine:fix/issue-5294-shift-min-value

Conversation

@gemshrine

Copy link
Copy Markdown
Contributor

Summary

Closes #5294. BytesRaw.shift(int) computed Math.abs(bits) % Byte.SIZE and Math.abs(bits) / Byte.SIZE for mod and offset. Math.abs(Integer.MIN_VALUE) == Integer.MIN_VALUE (the classic Java sign-bit overflow), so for bits == Integer.MIN_VALUE both derived values came out negative — offset = -268435456. Since bits < 0, the negative-offset value went straight into shiftLeft, where source = index + offset is deeply negative and doesn't satisfy source >= bytes.length, so the else branch executed bytes[source] with a negative index and threw an uncaught ArrayIndexOutOfBoundsException rather than the domain-level org.eolang.error an EO caller expects.

Fix

Widen the absolute-value computation to long:

-        final int mod = Math.abs(bits) % Byte.SIZE;
-        final int offset = Math.abs(bits) / Byte.SIZE;
+        final long positive = Math.abs((long) bits);
+        final int mod = (int) (positive % Byte.SIZE);
+        final int offset = (int) (positive / Byte.SIZE);

Math.abs((long) Integer.MIN_VALUE) is 2147483648L — the positive representation the original code intended. positive % 8 = 0 and positive / 8 = 268435456, both int-safe. The rest of the method is untouched.

The offset then dominates any real byte-array length, so shiftLeft's source >= bytes.length guard fires on every index and the result is all-zero bytes — the semantically correct answer for a shift bigger than the array width. Integer.MAX_VALUE had the same mirror behaviour (huge positive offset via Math.abs there was fine, but the shift result was accidentally correct only because bits >= 0 picked shiftRight); with the widening it stays correct explicitly.

Tests

Added two @Test methods next to the existing shift coverage in BytesOfTest:

  • shiftsByIntegerMinValueYieldsZeroWithoutCrashingBytesOf(0xFFFFFFFFL).shift(Integer.MIN_VALUE).asNumber(Long.class) returns 0L instead of raising AIOOBE. This is the exact reproducer path from the issue.
  • shiftsByIntegerMaxValueYieldsZero — mirror on the positive side, guarding the shiftRight branch's large-offset behaviour.

The existing checksShift and checksShifts parameterised cases are untouched — their bits values fit in an int after Math.abs, so the arithmetic path they exercise is byte-for-byte identical before and after the widening.

Reachability from EO

Cited in the issue: bytes.eo defines right x.neg > [x] > left, and i32.min-value.neg is itself (two's-complement identity), so an EO caller of <bytes>.left <i32-min> lands directly in the buggy path. After this fix that call returns all-zero bytes without a JVM-level exception.

…er.MIN_VALUE

Math.abs(Integer.MIN_VALUE) returns Integer.MIN_VALUE itself,
because 2147483648 has no positive int representation. That made
mod and offset in shift(int) come out negative, so the shiftLeft
branch indexed into bytes[] with a deeply negative source and
raised an uncaught ArrayIndexOutOfBoundsException instead of a
domain error. Reachable from EO via bytes.left -2147483648 (i32
min value, whose neg is itself).

Casting bits to long before Math.abs makes the positive amount
2147483648L, so mod becomes 0 and offset becomes 268435456: both
non-negative and large enough that shiftLeft (and shiftRight for
Integer.MAX_VALUE, which had the mirror large-offset case) takes
the source-out-of-range branch and returns all-zero bytes, which
is the correct wrap for a shift bigger than the array width.

Added two targeted tests on BytesOf(long): shift(Integer.MIN_VALUE)
and shift(Integer.MAX_VALUE) both yield 0L instead of crashing.
Existing checksShift/checksShifts parameterized cases are untouched
by the widening because their bits fit int and the arithmetic path
is unchanged.
@sonarqubecloud

Copy link
Copy Markdown

@gemshrine

Copy link
Copy Markdown
Contributor Author

Closing as duplicate of #5300, which already applies the same Math.abs(bits % Byte.SIZE) shape and is under active work by @someshk1703 — didn't spot it before opening this one. Sorry for the noise.

@gemshrine gemshrine closed this Jul 11, 2026
@gemshrine
gemshrine deleted the fix/issue-5294-shift-min-value branch July 11, 2026 18:46
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Performance Analysis

All benchmarks are within the acceptable range. No critical degradation detected (threshold is 100%). Please refer to the detailed report for more information.

Click to see the detailed report
Test Base Score PR Score Change % Change Unit Mode
benchmarks.XmirBench.xmirToEO 13585.131 13348.486 -236.645 -1.74% ms/op Average Time

✅ Performance gain: benchmarks.XmirBench.xmirToEO is faster by 236.645 ms/op (1.74%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BytesRaw.shift results in ArrayIndexOutOfBoundsException for Integer.MIN_VALUE shift value due to Math.abs overflow

1 participant