Skip to content

Enable MemorySegment field accessors via DirectBufferEx buffer migration #1723

@jfallows

Description

@jfallows

Summary

Enable incremental adoption of MemorySegment VarHandle-based field accessors on the hot path by extending Agrona's buffer hierarchy with DirectBufferEx / MutableDirectBufferEx / AtomicBufferEx interfaces that carry a segment() accessor alongside the existing Unsafe-backed access.

The original goal of replacing UnsafeBuffer with SafeBuffer has evolved. Instead of a full buffer implementation swap, we extend the existing hierarchy so that every buffer in the pipeline exposes its backing MemorySegment via segment() and uses wrapAdjustment() as the index offset into the root segment — enabling zero-allocation segment access on the hot path.

Changes

Buffer type hierarchy

  • DirectBufferEx extends DirectBuffer — adds segment(), wrap(DirectBufferEx), wrap(MemorySegment), getBytes(MemorySegment)
  • MutableDirectBufferEx extends MutableDirectBuffer, DirectBufferEx — adds putBytes(DirectBufferEx), putBytes(MemorySegment)
  • AtomicBufferEx extends AtomicBuffer, MutableDirectBufferEx
  • UnsafeBufferEx extends UnsafeBuffer implements AtomicBufferEx — Unsafe-backed with segment overlay
  • ExpandableArrayBufferEx extends ExpandableArrayBuffer implements MutableDirectBufferEx — heap expandable
  • ExpandableDirectByteBufferEx extends ExpandableDirectByteBuffer implements MutableDirectBufferEx — direct expandable

Zero-allocation hot path

  • UnsafeBufferEx / SafeBuffer wrap methods store root backing segment instead of asSlice() per call
  • wrapAdjustment() used as index offset into the unsliced segment
  • DefaultBufferPool.buffer(slot, offset) uses wrap(DirectBufferEx, offset, length) instead of raw address wrap

Flyweight migration

  • Generated Flyweight.wrap / tryWrap / buffer() use DirectBufferEx
  • Generated Flyweight.Builder.wrap / buffer() use MutableDirectBufferEx
  • Both store segment field via buffer.segment()

Ring buffer path

  • MessageConsumer extends MessageHandlerEx (replaces Agrona MessageHandler)
  • EngineWorker uses streamsBuffer.readEx() delivering MutableDirectBufferEx

Codebase-wide type migration

  • ~400+ files migrated from DirectBuffer / MutableDirectBuffer to Ex variants
  • AtomicBufferAtomicBufferEx where assigned UnsafeBufferEx
  • ExpandableArrayBuffer / ExpandableDirectByteBuffer → Ex variants

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions