Skip to content

Reduce excessive linear scans in lastIndex() hot path #1704

@peter-lawrey

Description

@peter-lawrey

lastIndex() still calls into linear scan logic far more often than expected under concurrent writer/reader load, even
after the recent sequenceForPosition(Long.MAX_VALUE) fast-path work reduced brute-force indexed-anchor fall-throughs.

Example stress output:

LastIndexFastPathStress: cores=32 writers=4 readers=64 delayNs=0 writes=78766646 lastIndexCalls=314356 retries=9
retries/call=0.000029 avgLatency=12459680ns maxLatency=2971.10ms

The retry/fall-through counters suggest the original brute-force path is mostly addressed, but the perf breakdown still
shows many calls through linearScanByPosition, especially the lastSequenceNumber tail-check path. This means
lastIndex() is still doing repeated scan work for what should usually be a cheap “latest written sequence” lookup.

Areas to investigate:

  • Reduce how often lastIndex() / toEnd() reaches linearScanByPosition.
  • Avoid repeated scan work when writePosition and sequence metadata are already consistent.
  • Consider a dedicated lastIndex/tail lookup path instead of flowing through the broader tailer movement logic.
  • Review hot loops for avoidable interface dispatch and improve JIT inlining opportunities:
    • cache repeated wireForIndex(), bytes(), store(), rollCycle() lookups locally;
    • prefer concrete/internal types in hot private methods where API boundaries are not required;
    • split special Long.MAX_VALUE paths from generic position lookup paths where that removes polymorphic calls or
      branchy logic.
  • Keep semantics around partial writes, metadata documents, and recovery unchanged.

Acceptance criteria:

  • Add or keep a stress/regression test that reports scan-source counters.
  • sequenceForPosition brute-force fall-throughs remain near zero under contention.
  • lastIndex() performs substantially fewer linearScanByPosition tail-check calls under the stress test.
  • No public API changes unless explicitly justified.
  • Relevant queue/indexing tests continue to pass.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions