Skip to content

Remove Prism.lex_ripper#3871

Merged
kddnewton merged 1 commit into
ruby:mainfrom
Earlopain:lex-ripper-plain
Jan 27, 2026
Merged

Remove Prism.lex_ripper#3871
kddnewton merged 1 commit into
ruby:mainfrom
Earlopain:lex-ripper-plain

Conversation

@Earlopain

@Earlopain Earlopain commented Jan 26, 2026

Copy link
Copy Markdown
Collaborator

Since on_sp is emitted, it doesn't do a whole lot anymore.

This leaves one incompatibility for code like "x#$%"

Ripper confuses this for bare interpolation with a global, but $% is not a valid global name. Still,
it emits two string tokens in such a case. It doesn't make sense for prism to work around this bug,
so the affected files are added as excludes.

@Earlopain
Earlopain force-pushed the lex-ripper-plain branch 7 times, most recently from 4a23307 to 1ba9b44 Compare January 26, 2026 11:43
@Earlopain
Earlopain marked this pull request as ready for review January 26, 2026 11:46

@kddnewton kddnewton left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove this API entirely instead of deprecating, I can't imagine anyone is using it.

I think I'd like us to support the splitting though, unless it's super difficult.

@kddnewton

Copy link
Copy Markdown
Collaborator

We need splitting for escaped newlines in heredocs anyway.

@Earlopain

Copy link
Copy Markdown
Collaborator Author

I think we can remove this API entirely instead of deprecating, I can't imagine anyone is using it.

I did a codesearch and didn't find anyone using it. Let's remove 👍

I think I'd like us to support the splitting though

Do you mean for "x#$%"? I created an upstream issue for this at https://bugs.ruby-lang.org/issues/21849. Since it's a single string, I would expect ripper to emit a single token for it like prism does. I don't think it makses sense to replicate the buggy behavior. And since it's only for our tests, excludes seem fine.

If you insist I try to implement it but the value of doing that seems very low to me. Doing it correctly is a bit complex since it depends on where the string appears. Quoted heredocs for example don't allow interpolation so ripper does the correct thing there and that doesn't just apply do dash heredocs.

@kddnewton

Copy link
Copy Markdown
Collaborator

Nah that's okay I don't insist :)

@kddnewton

Copy link
Copy Markdown
Collaborator

But let's do remove lex_ripper

Since `on_sp` is emitted, it doesn't do a whole lot anymore.

This leaves one incompatibility for code like `"x#$%"`

Ripper confuses this for bare interpolation with a global, but `$%` is not a valid global name. Still,
it emits two string tokens in such a case. It doesn't make sense for prism to work around this bug,
so the affected files are added as excludes.

Since the only usage of this method makes sense for testing in prism itself,
the method is removed instead of deprecated.
@Earlopain Earlopain changed the title Deprecate Prism.lex_ripper Remove Prism.lex_ripper Jan 26, 2026
@kddnewton
kddnewton merged commit 89a4643 into ruby:main Jan 27, 2026
118 of 121 checks passed
@Earlopain
Earlopain deleted the lex-ripper-plain branch February 20, 2026 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants