PPC64: clampDoubleToUint8 — P9 branchless via xsmaxjdp + isel

The previous clampDoubleToUint8 had two mispredictable branches (≤0/NaN
and ≥255), each with a small body that jumped to a shared exit. Hot
on Uint8ClampedArray store paths.

POWER9 added xsmaxjdp/xsminjdp which use Java/JS semantics (ISA v3.0B
§7.6.1.7): any NaN is treated as "less than any number that is not a
NaN". So xsmaxjdp(input, 0) collapses {NaN, -Inf, ≤ 0} all to 0 in a
single instruction — the entire "≤ 0 or NaN → 0" branch dance
disappears.

After the max, fctid (round-to-nearest-even per FPSCR default —
matches ECMA Uint8ClampedArray's round-half-to-even) saturates
out-of-int64 values to INT64_MAX. The remaining upper clamp
(output > 255 → 255) is one cmpdi + isel.

POWER9 path (7 insns, no branches):
  zeroDouble  fpscratch
  xsmaxjdp    fpscratch, input, fpscratch  ; max(input, 0); NaN→0
  fctid       fpscratch, fpscratch
  mfvsrd      output, fpscratch
  li          max255, 255
  cmpdi       output, 255
  isel        output, max255, output, GreaterThan

POWER8 path: unchanged (xsmaxjdp unavailable; fctid maps NaN to
INT64_MAX which would clamp to 255 instead of the spec-required 0,
so we keep the explicit NaN-filtering branches).

Verified end-to-end:
- Real P9 jit-test --jitflags=none: 13715 PASS / 0 FAIL (default)
- Real P9 jit-test --jitflags=none MOZ_PPC64_FORCE_POWER8=1: 13715 / 0
- Real P9 jstests default: PASS
- Real P9 jstests MOZ_PPC64_FORCE_POWER8=1: PASS
- Sim MOZ_PPC64_FORCE_POWER9=1 jit-test: 13651 / 0
- Sim MOZ_PPC64_FORCE_POWER10=1 jit-test: 13651 / 0
- Sim MOZ_PPC64_FORCE_POWER8=1 jit-test: 13651 / 0

PLAN.md item mozilla-firefox#23 (POWER9 fast path; POWER8 fallback unchanged).

Author: runlevel5

js/src/jit/ppc64/MacroAssembler-ppc64.cpp

@@ -388,10 +388,34 @@ void MacroAssemblerPPC64Compat::handleFailureWithHandlerTail(
 }
 
 void MacroAssembler::clampDoubleToUint8(FloatRegister input, Register output) {
-  Label positive, below255, done;
   ScratchDoubleScope fpscratch(asMasm());
 
-  // <= 0 or NaN --> 0
+  if (HasPOWER9()) {
+    // P9 xsmaxjdp uses Java/JS semantics (ISA v3.0B §7.6.1.7): any NaN
+    // is treated as "less than any number that is not a NaN", so
+    // xsmaxjdp(input, 0) collapses {NaN, -Inf, ≤ 0} to 0 in one insn —
+    // the "≤ 0 or NaN → 0" branch dance disappears.
+    //
+    // After the max, fctid (round-to-nearest-even per FPSCR default,
+    // matches ECMA Uint8ClampedArray's round-half-to-even) saturates
+    // out-of-int64 values to INT64_MAX. Remaining upper clamp
+    // (output > 255 → 255) is one cmpdi + isel.
+    zeroDouble(fpscratch);
+    as_xsmaxjdp(fpscratch, input, fpscratch);
+    as_fctid(fpscratch, fpscratch);
+    as_mfvsrd(output, fpscratch);
+    UseScratchRegisterScope temps(asMasm());
+    Register max255 = temps.Acquire();
+    xs_li(max255, 255);
+    as_cmpdi(output, 255);
+    as_isel(output, max255, output, GreaterThan);
+    return;
+  }
+
+  // POWER8 fallback: xsmaxjdp is unavailable, so filter NaN explicitly
+  // before fctid. Per Power ISA, fctid maps NaN to INT64_MAX, which
+  // would clamp to 255 instead of the spec-required 0.
+  Label positive, below255, done;
   zeroDouble(fpscratch);
   branchDouble(DoubleGreaterThan, input, fpscratch, &positive);
   {
@@ -401,7 +425,6 @@ void MacroAssembler::clampDoubleToUint8(FloatRegister input, Register output) {
 
   bind(&positive);
 
-  // >= 255 --> 255
   loadConstantDouble(255.0, fpscratch);
   branchDouble(DoubleLessThan, input, fpscratch, &below255);
   {
@@ -411,12 +434,8 @@ void MacroAssembler::clampDoubleToUint8(FloatRegister input, Register output) {
 
   bind(&below255);
 
-  // Round to nearest even and convert to int64.
-  // PPC64's frin rounds ties away from zero, not to even. Use fctid
-  // which rounds according to FPSCR (default: round-to-nearest-even).
   as_fctid(fpscratch, input);
   as_mfvsrd(output, fpscratch);
-
   bind(&done);
 }