Skip to content

Do not pass source bytes to Loader.java at all and mark newlines for lazy DefNodes#3998

Merged
eregon merged 2 commits into
ruby:mainfrom
eregon:no-source-bytes-for-java
Mar 17, 2026
Merged

Do not pass source bytes to Loader.java at all and mark newlines for lazy DefNodes#3998
eregon merged 2 commits into
ruby:mainfrom
eregon:no-source-bytes-for-java

Conversation

@eregon

@eregon eregon commented Mar 16, 2026

Copy link
Copy Markdown
Member

Follow-up of #3983, implementing a variant of the first option, the difference being we create a new MarkNewlinesVisitor object vs keeping it around.
We also need to keep the Source object around to have access to line offsets, but that's fine as the first commit completely removes the need to pass the source byte[] to the Loader :)
With this, we can use lazy DefNode in TruffleRuby and be fully compatible with eager loading.

From the commits:

Do not pass source bytes to Loader.java at all

  • That way we are sure it's not retained by it.
  • Source#bytes seems to have no usages.
  • Arrays.binarySearch() works just fine if the value is > the last array element.

Mark newlines for lazy DefNodes

@eregon
eregon requested review from enebo and kddnewton March 16, 2026 19:58
eregon added 2 commits March 16, 2026 21:03
* That way we are sure it's not retained by it.
* Source#bytes seems to have no usages.
* Arrays.binarySearch() works just fine if the value is > the last array element.
* By running the MarkNewlinesVisitor on them when they are lazily loaded.
* See ruby#3983.
* Achieves the same nodes as with eager DefNode.
@eregon

eregon commented Mar 16, 2026

Copy link
Copy Markdown
Member Author

Benchmarks in truffleruby/truffleruby#4182 (comment), it's about 35% faster to parse the TruffleRuby core library!

@kddnewton kddnewton left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine to me. @enebo?

@eregon

eregon commented Mar 17, 2026

Copy link
Copy Markdown
Member Author

This is basically fixing #3983 to make it usable, merging to unblock truffleruby/truffleruby#4182 and a chain of PRs on top of it.

If anyone wants to access the sourceBytes they can easily keep it themselves as they are given to Parser.parseAndSerialize(sourceBytes, parsingOptions).

@eregon
eregon merged commit 616930e into ruby:main Mar 17, 2026
71 of 72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants