Skip to content

Lower segment-view VarHandle accesses in UnsafeFastPath in JDK 21+ #24199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vijaysun-omr merged 1 commit into from
Jun 24, 2026
+106 −6
Merged

Lower segment-view VarHandle accesses in UnsafeFastPath in JDK 21+ #24199

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion runtime/compiler/codegen/J9RecognizedMethodsEnum.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -694,7 +694,7 @@ FirstJ9Method = LastOMRMethod + 1,
java_lang_invoke_VarHandleX_FieldStaticReadOnlyOrReadWrite_method,

java_lang_invoke_VarHandleByteArrayAsX_ArrayHandle_method,
java_lang_invoke_VarHandleByteArrayAsX_ByteBufferHandle_method,
java_lang_invoke_VarHandleByteArrayAsX_ByteBufferHandle_method, java_lang_invoke_VarHandleSegmentAsX_method,

jdk_internal_foreign_layout_ValueLayouts_AbstractValueLayout_accessHandle,
jdk_internal_foreign_AbstractMemorySegmentImpl_reinterpret, java_lang_foreign_MemorySegment_method,
Expand Down
6 changes: 6 additions & 0 deletions runtime/compiler/env/j9method.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4540,6 +4540,11 @@ void TR_ResolvedJ9Method::construct()
&& !strncmp(className, "java/lang/invoke/VarHandleByteArrayAsShorts$ByteBufferHandle",
60))) {
setRecognizedMethodInfo(TR::java_lang_invoke_VarHandleByteArrayAsX_ByteBufferHandle_method);
} else if (classNameLen >= 39 && classNameLen <= 42
&& !strncmp(className, "java/lang/invoke/VarHandleSegmentAs", 35)) {
// The memory segment view VarHandle operation classes
// VarHandleSegmentAsBytes/Chars/Shorts/Ints/Longs/Floats/Doubles
setRecognizedMethodInfo(TR::java_lang_invoke_VarHandleSegmentAsX_method);
}
} else if ((classNameLen == 31) && !strncmp(className, "java/lang/foreign/MemorySegment", 31)) {
if (nameLen >= 3 && (!strncmp(name, "get", 3) || !strncmp(name, "set", 3)))
Expand Down Expand Up @@ -5784,6 +5789,7 @@ bool TR_J9MethodBase::isVarHandleOperationMethod(TR::RecognizedMethod rm)
case TR::java_lang_invoke_VarHandleX_FieldStaticReadOnlyOrReadWrite_method:
case TR::java_lang_invoke_VarHandleByteArrayAsX_ArrayHandle_method:
case TR::java_lang_invoke_VarHandleByteArrayAsX_ByteBufferHandle_method:
case TR::java_lang_invoke_VarHandleSegmentAsX_method:
#else
case TR::java_lang_invoke_ArrayVarHandle_ArrayVarHandleOperations_OpMethod:
case TR::java_lang_invoke_StaticFieldVarHandle_StaticFieldVarHandleOperations_OpMethod:
Expand Down
104 changes: 99 additions & 5 deletions runtime/compiler/optimizer/UnsafeFastPath.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,26 @@
//
static bool isVarHandleOperationMethod(TR::RecognizedMethod rm)
{
return TR_J9MethodBase::isVarHandleOperationMethod(rm)
&& rm != TR::java_lang_invoke_VarHandleByteArrayAsX_ByteBufferHandle_method;
if (rm == TR::java_lang_invoke_VarHandleByteArrayAsX_ByteBufferHandle_method)
return false;

#if defined(J9VM_OPT_OPENJDK_METHODHANDLE)
// The memory segment view operation methods perform the access on a base that is
// either null (NativeMemorySegmentImpl, raw native address) or a primitive array
// (HeapMemorySegmentImpl). Both shapes can be lowered to the same aladd(base, offset)
// load/store only while arrays are contiguous and on-heap, so give up when arraylets
// or off-heap allocation are possible.
if (rm == TR::java_lang_invoke_VarHandleSegmentAsX_method) {
if (TR::Compiler->om.canGenerateArraylets())
return false;
#if defined(J9VM_GC_SPARSE_HEAP_ALLOCATION)
if (TR::Compiler->om.isOffHeapAllocationEnabled())

@vijaysun-omr vijaysun-omr Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there something simple that can be done to handle the off heap case here ? @zl-wang @rmnattas

@zl-wang zl-wang Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from reading the comment, the two cases (either native or byte-array object) can be distinguished by the fact that base == null, such that off-heap can handle (fastPath) them in a similar way as in gencon, except loading up dataAddr for byte-array off-heap. I am pretty sure we already did this for off-heap static vs. instance-field.

@rmnattas rmnattas Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with MemorySegments and its nuances, but we fastPath unsafe accesses by inserting runtime check for base == null and isArray to insert load dataAddr if needed. Will look up the code and helpers for that as they might be useful here too.

@zl-wang zl-wang Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for this one, you even don't need to test isArray, since base!=NULL means isArray already

@nbhuiyan nbhuiyan Jun 24, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your insights @rmnattas @zl-wang. I will investigate and implement the off heap support in the near future, and that can be part of a different PR. I might have to reach out to you for additional details for this.

@vijaysun-omr, I think we could add the off-heap support in a separate PR in the future, as this PR forms part of a bigger piece of work that has been thoroughly tested end-to-end, and any major change to this PR will delay the final (and the most impactful) piece of MemorySegment optimization.

@vijaysun-omr vijaysun-omr Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I will approve now and you can add the off heap support separately.

@rmnattas rmnattas Jun 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's the createUnsafeGetWithOffset helper but its much more inclusive than what it seem you need here, (it checks type signature and interface possibility).
Maybe you can point me to the code that does the transformation as some helpers already support OffHeap.

return false;
#endif /* J9VM_GC_SPARSE_HEAP_ALLOCATION */
}
#endif /* J9VM_OPT_OPENJDK_METHODHANDLE */

return TR_J9MethodBase::isVarHandleOperationMethod(rm);
}

static TR::RecognizedMethod getVarHandleAccessMethodFromInlinedCallStack(TR::Compilation *comp, TR::Node *node)
Expand Down Expand Up @@ -182,6 +200,7 @@ static bool isVarHandleOperationMethodOnArray(TR::RecognizedMethod rm)

case TR::java_lang_invoke_VarHandleX_Array_method:
case TR::java_lang_invoke_VarHandleByteArrayAsX_ArrayHandle_method:
case TR::java_lang_invoke_VarHandleSegmentAsX_method:
#else
case TR::java_lang_invoke_ArrayVarHandle_ArrayVarHandleOperations_OpMethod:
case TR::java_lang_invoke_ByteArrayViewVarHandle_ByteArrayViewVarHandleOperations_OpMethod:
Expand Down Expand Up @@ -370,6 +389,63 @@ bool TR_UnsafeFastPath::tryTransformUnsafeAtomicCallInVarHandleAccessMethod(TR::
return true;
}

// The Unsafe.getXUnaligned(Object, long) / putXUnaligned(Object, long, X) wrappers are
// plain Java methods whose bodies re-assemble the value byte- or short-wise when the
// offset may be misaligned. On targets that support misaligned accesses they can be
// lowered to a single load/store just like the corresponding aligned natives, but they
// are not part of isUnsafeGetPutWithObjectArg(), so they get their own helpers here.
static bool isUnalignedUnsafeGetPut(TR::RecognizedMethod rm)
{
switch (rm) {
case TR::jdk_internal_misc_Unsafe_getCharUnaligned:
case TR::jdk_internal_misc_Unsafe_getShortUnaligned:
case TR::jdk_internal_misc_Unsafe_getIntUnaligned:
case TR::jdk_internal_misc_Unsafe_getLongUnaligned:
case TR::jdk_internal_misc_Unsafe_putCharUnaligned:
case TR::jdk_internal_misc_Unsafe_putShortUnaligned:
case TR::jdk_internal_misc_Unsafe_putIntUnaligned:
case TR::jdk_internal_misc_Unsafe_putLongUnaligned:
return true;
default:
return false;
}
return false;
}

static bool isUnalignedUnsafePut(TR::RecognizedMethod rm)
{
switch (rm) {
case TR::jdk_internal_misc_Unsafe_putCharUnaligned:
case TR::jdk_internal_misc_Unsafe_putShortUnaligned:
case TR::jdk_internal_misc_Unsafe_putIntUnaligned:
case TR::jdk_internal_misc_Unsafe_putLongUnaligned:
return true;
default:
return false;
}
return false;
}

static TR::DataType dataTypeForUnalignedUnsafeGetPut(TR::RecognizedMethod rm)
{
switch (rm) {
case TR::jdk_internal_misc_Unsafe_getCharUnaligned:
case TR::jdk_internal_misc_Unsafe_getShortUnaligned:
case TR::jdk_internal_misc_Unsafe_putCharUnaligned:
case TR::jdk_internal_misc_Unsafe_putShortUnaligned:
return TR::Int16;
case TR::jdk_internal_misc_Unsafe_getIntUnaligned:
case TR::jdk_internal_misc_Unsafe_putIntUnaligned:
return TR::Int32;
case TR::jdk_internal_misc_Unsafe_getLongUnaligned:
case TR::jdk_internal_misc_Unsafe_putLongUnaligned:
return TR::Int64;
default:
TR_ASSERT(false, "Method is not an unaligned unsafe get/put\n");
}
return TR::NoType;
}

static bool needUnsignedConversion(TR::RecognizedMethod methodToReduce)
{
switch (methodToReduce) {
Expand All @@ -378,6 +454,7 @@ static bool needUnsignedConversion(TR::RecognizedMethod methodToReduce)
case TR::com_ibm_jit_JITHelpers_getCharFromArrayVolatile:
case TR::java_lang_StringUTF16_getChar:
case TR::java_lang_StringUTF16_putChar:
case TR::jdk_internal_misc_Unsafe_getCharUnaligned:
return true;

default:
Expand Down Expand Up @@ -751,18 +828,26 @@ int32_t TR_UnsafeFastPath::perform()
if (recognizedVarHandleOpMethod != TR::unknownMethod)
callerMethod = recognizedVarHandleOpMethod;
TR::RecognizedMethod calleeMethod = symbol->getRecognizedMethod();
if (isKnownUnsafeCaller(callerMethod) && TR_J9MethodBase::isUnsafeGetPutWithObjectArg(calleeMethod)) {
// The single load/store an unaligned wrapper reduces to uses the packed element
// shape, so only callers known to access packed elements (byte array views and
// memory segments) qualify, and only on targets that allow misaligned accesses.
bool isKnownCaller = isKnownUnsafeCaller(callerMethod);
bool isUnalignedCallee = isKnownCaller && isUnalignedUnsafeGetPut(calleeMethod)
&& !comp()->cg()->getSupportsAlignedAccessOnly() && isUnsafeCallerAccessingArrayElement(callerMethod);
if (isKnownCaller && (TR_J9MethodBase::isUnsafeGetPutWithObjectArg(calleeMethod) || isUnalignedCallee)) {
if (isUnsafeCallerAccessingStaticField(callerMethod))
isStatic = true;
if (isUnsafeCallerAccessingArrayElement(callerMethod))
isArrayOperation = true;

if (isArrayOperation)
if (isUnalignedCallee)
type = dataTypeForUnalignedUnsafeGetPut(calleeMethod);
else if (isArrayOperation)
type = TR_J9MethodBase::unsafeDataTypeForArray(calleeMethod);
else
type = TR_J9MethodBase::unsafeDataTypeForObject(calleeMethod);

if (TR_J9MethodBase::isUnsafePut(calleeMethod))
if (TR_J9MethodBase::isUnsafePut(calleeMethod) || isUnalignedUnsafePut(calleeMethod))
value = node->getChild(3);

if (TR_J9MethodBase::isVolatileUnsafe(calleeMethod))
Expand Down Expand Up @@ -813,7 +898,16 @@ int32_t TR_UnsafeFastPath::perform()
}

object = node->getChild(objectChild);
#if defined(J9VM_OPT_OPENJDK_METHODHANDLE)
// A MemorySegment access base is null for native segments, so it must not
// be marked non-null. The address computation below is correct either way:
// null base + absolute offset, or array base (non-null) + element offset.
if (callerMethod != TR::java_lang_invoke_VarHandleSegmentAsX_method) {
object->setIsNonNull(true);
}
#else /* J9VM_OPT_OPENJDK_METHODHANDLE */
object->setIsNonNull(true);
#endif /* J9VM_OPT_OPENJDK_METHODHANDLE */
if (isStatic) {
TR::Node *jlClass = object;
TR::Node *j9Class = TR::Node::createWithSymRef(TR::aloadi, 1, 1, jlClass,
Expand Down