Skip to content

Add APIs for case folding to the standard library#154742

Open
Jules-Bertholet wants to merge 10 commits into
rust-lang:mainfrom
Jules-Bertholet:casefold
Open

Add APIs for case folding to the standard library#154742
Jules-Bertholet wants to merge 10 commits into
rust-lang:mainfrom
Jules-Bertholet:casefold

Conversation

@Jules-Bertholet
Copy link
Copy Markdown
Contributor

@Jules-Bertholet Jules-Bertholet commented Apr 3, 2026 •
edited by rustbot
Loading

View all comments

Libs-api requested these, so here they are.

New public API (gated behind #[feature(casefold)]):

impl char {
    pub fn to_casefold(self) -> ToCasefold;
}

impl str {
    pub fn to_casefold(&self) -> String;
    pub fn eq_ignore_case(&self) -> bool;
}

pub struct ToCasefold { ... }
impl Iterator for ToCasefold { type Item = char; ... }
impl DoubleEndedIterator for ToCasefold { ... }
impl FusedIterator for ToCasefold { }
impl ExactSizeIterator for ToCasefold { ... }
impl fmt::Display for ToCasefold { ... }

Notes

  • This only adds a negligible amount of static data to core::unicode. To accomplish that, we compute the case-folding for most characters as the lowercase of their uppercase; this double mapping adds some complexity to the implementation.
  • No normalization (e.g. NFC) is performed, so visually and semantically equivalent strings can compare unequal.
  • I have not put any effort into optimizing eq_ignore_case(); there may be a more performant implementation.
  • char::eq_ignore_case() is left to future work—it's a potential footgun, so we may want to think more deeply about how to expose and document that API.

@rustbot label T-libs-api A-unicode

@rustbot
Copy link
Copy Markdown
Collaborator

rustbot commented Apr 3, 2026

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

library/core/src/unicode/unicode_data.rs is generated by the src/tools/unicode-table-generator tool.

If you want to modify unicode_data.rs, please modify the tool then regenerate the library source file via ./x run src/tools/unicode-table-generator instead of editing unicode_data.rs manually.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Apr 3, 2026
@rustbot
Copy link
Copy Markdown
Collaborator

rustbot commented Apr 3, 2026

r? @scottmcm

rustbot has assigned @scottmcm.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: @scottmcm, libs
  • @scottmcm, libs expanded to 8 candidates
  • Random selection from Mark-Simulacrum, jhpratt, scottmcm

@rustbot rustbot added A-Unicode Area: Unicode T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Apr 3, 2026
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Jules-Bertholet Jules-Bertholet force-pushed the casefold branch 2 times, most recently from 5b5e617 to bf4ee7c Compare April 3, 2026 13:25
@scottmcm
Copy link
Copy Markdown
Member

scottmcm commented Apr 16, 2026

@rustbot reroll

@rustbot rustbot assigned jhpratt and unassigned scottmcm Apr 16, 2026
@rust-bors

This comment has been minimized.

Sign in to view
@rustbot

This comment has been minimized.

Sign in to view
jhpratt
jhpratt reviewed Apr 18, 2026
View reviewed changes
Comment thread library/alloc/src/str.rs Outdated
Comment thread library/alloc/src/str.rs
Comment thread library/core/src/char/methods.rs Outdated
Comment thread library/core/src/char/methods.rs Outdated
@rustbot

This comment has been minimized.

Sign in to view
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Jules-Bertholet Jules-Bertholet force-pushed the casefold branch 2 times, most recently from dd25c4f to b14b43b Compare April 18, 2026 04:50
@jhpratt
Copy link
Copy Markdown
Member

jhpratt commented Apr 18, 2026

LGTM. r=me once CI passes.

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Jules-Bertholet
Copy link
Copy Markdown
Contributor Author

Jules-Bertholet commented Apr 18, 2026

r? libs-api

@rustbot rustbot assigned the8472 and unassigned jhpratt Apr 18, 2026
@jhpratt
Copy link
Copy Markdown
Member

jhpratt commented Apr 18, 2026

I don't mind, but any particular reason for the reassign? I thought it was good to go.

@Jules-Bertholet
Copy link
Copy Markdown
Contributor Author

Jules-Bertholet commented Apr 18, 2026 •
edited
Loading

The API needs libs-API approval, I believe. They expressed interest in something like this, but there was never an ACP. (I also need to add a tracking issue after I get that)

@clarfonthey clarfonthey added the I-libs-api-nominated Nominated for discussion during a libs-api team meeting. label May 18, 2026
@rustbot rustbot assigned Mark-Simulacrum and unassigned the8472 May 27, 2026
Mark-Simulacrum
Mark-Simulacrum requested changes May 31, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@Mark-Simulacrum Mark-Simulacrum left a comment •
edited by rustbot
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r=me modulo the debug assert comment having some answer (not necessarily code changes)

View changes since this review

Comment thread library/alloc/src/str.rs
Comment thread library/core/src/unicode/unicode_data.rs Outdated

// If, after updating the Unicode data
// to a new Unicode version, the below
// assertion starts to fail in tests,
Copy link
Copy Markdown
Member

@Mark-Simulacrum Mark-Simulacrum May 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And we're confident those tests run with debug-asserts enabled? Why can't this be done in the generator (or as dedicated tests produced by the generator)?

Copy link
Copy Markdown
Contributor Author

@Jules-Bertholet Jules-Bertholet May 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, confident, I checked. I've also now added a bunch of comments to document these assertions.

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 31, 2026
@rustbot
Copy link
Copy Markdown
Collaborator

rustbot commented May 31, 2026

Reminder, once the PR becomes ready for a review, use @rustbot ready.

@Jules-Bertholet
Copy link
Copy Markdown
Contributor Author

Jules-Bertholet commented May 31, 2026

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 31, 2026
@Mark-Simulacrum Mark-Simulacrum removed the I-libs-api-nominated Nominated for discussion during a libs-api team meeting. label Jun 2, 2026
@Mark-Simulacrum
Copy link
Copy Markdown
Member

Mark-Simulacrum commented Jun 2, 2026

@bors r+ rollup

@rust-bors
Copy link
Copy Markdown
Contributor

rust-bors Bot commented Jun 2, 2026

📌 Commit ee7f3dd has been approved by Mark-Simulacrum

It is now in the queue for this repository.

@rust-bors rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 2, 2026
jhpratt added a commit to jhpratt/rust that referenced this pull request Jun 2, 2026
@jhpratt
Rollup merge of rust-lang#154742 - Jules-Bertholet:casefold, r=Mark-S…
435f983 
…imulacrum

Add APIs for case folding to the standard library

[Libs-api requested these](rust-lang#154287 (comment)), so here they are.

New public API (gated behind `#[feature(casefold)]`):

```rust
impl char {
    pub fn to_casefold(self) -> ToCasefold;
}

impl str {
    pub fn to_casefold(&self) -> String;
    pub fn eq_ignore_case(&self) -> bool;
}

pub struct ToCasefold { ... }
impl Iterator for ToCasefold { type Item = char; ... }
impl DoubleEndedIterator for ToCasefold { ... }
impl FusedIterator for ToCasefold { }
impl ExactSizeIterator for ToCasefold { ... }
impl fmt::Display for ToCasefold { ... }
```

## Notes

- This only adds a negligible amount of static data to `core::unicode`. To accomplish that, we compute the case-folding for most characters as the lowercase of their uppercase; this double mapping adds some complexity to the implementation.
- No normalization (e.g. NFC) is performed, so visually and semantically equivalent strings can compare unequal.
- I have not put any effort into optimizing `eq_ignore_case()`; there may be a more performant implementation.
- `char::eq_ignore_case()` is left to future work—it's a potential footgun, so we may want to think more deeply about how to expose and document that API.

@rustbot label T-libs-api A-unicode
@jhpratt jhpratt mentioned this pull request Jun 2, 2026
Closed
rust-bors Bot pushed a commit that referenced this pull request Jun 2, 2026
@bors
Auto merge of #157279 - jhpratt:rollup-KEQkzRP, r=jhpratt
76d4faf 
Rollup of 8 pull requests

Successful merges:

 - #157240 (Enable Enzyme for aarch64-apple-darwin)
 - #157276 (miri subtree update)
 - #154742 (Add APIs for case folding to the standard library)
 - #157130 (Use a `ArrayVec` in `CastTarget`)
 - #157195 (Move feature gating to the new attr parsing infrastructure)
 - #157256 (tests: adapt for LLVM codegen change)
 - #157265 (Update books)
 - #157277 (triagebot.toml: add LawnGnome to libs reviewers)
@jhpratt
Copy link
Copy Markdown
Member

jhpratt commented Jun 2, 2026

@bors r-

#157279 (comment)

@rust-bors rust-bors Bot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 2, 2026
@rust-bors
Copy link
Copy Markdown
Contributor

rust-bors Bot commented Jun 2, 2026

This pull request was unapproved.

This PR was contained in a rollup (#157279), which was unapproved.

View changes since this unapproval

@Jules-Bertholet
Revert "Use ascii::Char in convert_while_ascii"
e6d5e95 
This reverts commit 1ec2ee9,
which unfortunately prevented `convert_while_ascii`
from vectorizing :(

"LLVM gave, and LLVM hath taken away"
@Jules-Bertholet
Copy link
Copy Markdown
Contributor Author

Jules-Bertholet commented Jun 2, 2026

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jhpratt jhpratt jhpratt left review comments

@Mark-Simulacrum Mark-Simulacrum Mark-Simulacrum requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@Mark-Simulacrum Mark-Simulacrum

Labels

A-Unicode Area: Unicode S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@Jules-Bertholet @rustbot @rust-log-analyzer @scottmcm @jhpratt @clarfonthey @nia-e @Mark-Simulacrum @the8472