Skip to content

Return concrete types from LiteMap iter methods#8072

Open
nicopap wants to merge 1 commit into
unicode-org:mainfrom
npapale:litemap-concrete-iter
Open

Return concrete types from LiteMap iter methods#8072
nicopap wants to merge 1 commit into
unicode-org:mainfrom
npapale:litemap-concrete-iter

Conversation

@nicopap

@nicopap nicopap commented Jun 12, 2026

Copy link
Copy Markdown

Return concrete types from LiteMap iter methods

impl Trait is limited to a single trait. But we'd like to expose
iterators that implement more than DoubleEndedIterator (eg: ExactSize,
Fused). This is why we return a concrete type now.

This closes #8054.

Considerations

  • No Copy derive, because that's against API guidelines for Iterators
  • Manual impl of size_hint, to be able to use ExactSizeIterator
    without panicking.
  • Manual impl of nth and fold, as other iterator methods tend to be
    implemented in term of those, and forwarding them to the inner
    iterator might avoid performance penalty of the default impl.

CLA status

CLA ETA: Got back an email confirmation from the Unicode member service 06/07/2026

Changelog

litemap: Make return types of LiteMap iter methods (iter, iter_mut, values, keys) concrete:

  • New types: ValuesIter, KeysIter, implements relevant traits, returned by values and keys respectively.
  • Changed: The return types of the aforementioned methods go from an impl DoubleEndedIterator to concrete types such as S::KeyValueIter, S::KeyValueIterMut or the aforementioned new types.

@nicopap nicopap requested review from Manishearth and sffc as code owners June 12, 2026 10:16
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@Manishearth Manishearth left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good idea. Generally we should do this more to make it easier to compose our iterators.

Waiting on CLA.

@nicopap

nicopap commented Jun 22, 2026

Copy link
Copy Markdown
Author

I'll keep up to date with the CLA application progression. I just submitted it to management.

@sffc sffc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks about right. I don't like how Rust doesn't have a good way to automatically delegate the Iterator trait. You picked 4 fns here, which seem like good ones, but you could have picked more or fewer; there isn't a great way to pick which ones to leave as default.

Comment thread utils/litemap/src/map.rs
Comment thread utils/litemap/src/map.rs
{
/// Produce an ordered iterator over key-value pairs
pub fn iter(&'a self) -> impl DoubleEndedIterator<Item = (&'a K, &'a V)> {
pub fn iter(&'a self) -> S::KeyValueIter {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought: This is fine since S::KeyValueIter is already public and already used in a bunch of bounds.

An alternative would be to add a third iterator wrapper for this.

@nicopap

nicopap commented Jun 30, 2026

Copy link
Copy Markdown
Author

I have the CLA documents signed, I need to scan them and send them to the unicode org email still.

I will implement the suggested changes to the PR, and rebase.

@nicopap nicopap force-pushed the litemap-concrete-iter branch from 54fdeab to 0a00977 Compare July 1, 2026 08:50
Comment thread utils/litemap/src/map.rs Outdated
impl_const_get_with_index_for_integer!(i128);
impl_const_get_with_index_for_integer!(isize);

type MapKv<'a, I, K, V, Out> = core::iter::Map<I, fn((&'a K, &'a V)) -> &'a Out>;

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the biggest worry here for me, I'm not sure how the compiler handles the function pointer. In practice, it will always be the closure used in LiteMap::values and LiteMap::keys passed to iter().map(_), so I don't know if the compiler is smart enough to directly inline the closure.

The alternative is described in the PR description, which might make the exposed type quite difficult to read.

@sffc sffc Jul 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably fine. But, since you're exporting a new type anyway, a cleaner solution would be to remove the core::iter::Map and just do the mapping in the fns you are implementing. The map fn is only a field access.

pub struct KeysIter<'a, K, V, S: StoreIterable<'a, K, V>> {
    iter: S::KeyValueIter
}

impl<'a, K, V, S: StoreIterable<'a, K, V>> Iterator for KeysIter<'a, K, V, S> {
    type Item = &'a K;

    #[inline]
    fn size_hint(&self) -> (usize, Option<usize>) {
        // don't need to map here
        self.iter.size_hint()
    }
    #[inline]
    fn next(&mut self) -> Option<Self::Item> {
        // access the .0 of the value using Option::map
        self.iter.next().map(|val| &val.0)
    }
    #[inline]
    fn fold<B, F>(self, init: B, f: F) -> B
    where
        Self: Sized,
        F: FnMut(B, Self::Item) -> B,
    {
        // here we can apply a map fn
        self.iter.map(|val| val.0).fold(init, f)
    }
    #[inline]
    fn nth(&mut self, n: usize) -> Option<Self::Item> {
        self.iter.nth(n).map(|val| &val.0)
    }
}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, with this, it's also possible to make KeysIter and ValuesIter generic over the iterator itself rather than S: StoreIterable which allows to use derive for the Debug and Clone impls.

@nicopap nicopap force-pushed the litemap-concrete-iter branch from 0a00977 to 5a66aa0 Compare July 1, 2026 09:17
@nicopap

nicopap commented Jul 1, 2026

Copy link
Copy Markdown
Author
  • I added the inline attributes,
  • I submitted the signed corporate CLA to the email mentioned in www.unicode.org.
  • I rebased to follow main.

@Manishearth Manishearth requested a review from sffc July 1, 2026 09:38
@nicopap nicopap force-pushed the litemap-concrete-iter branch 2 times, most recently from 15fca4b to e4d4097 Compare July 2, 2026 08:04
@nicopap

nicopap commented Jul 2, 2026

Copy link
Copy Markdown
Author

Applying the suggestion from @sffc wrt to mapping inside the methods rather than before creating the iterator (I should have thought about that) made the code much cleaner, since KeysIter and ValuesIter are now simply iterator adaptor to get a key/value from an iterator over tuples. We can even use derive for Clone and Debug.

Looking at the code for implementing StoreIterable in vec_impl and slice_impl, I feel like it might be possible to apply the same trick there to avoid the function pointer (although it's def not worth changing the code, as I don't know it's meaningful to avoid it)

CLA ETA: Email sent yesterday, waiting for the reply.

impl Trait is limited to a single trait. But we'd like to expose
iterators that implement more than DoubleEndedIterator (eg: ExactSize,
Fused). This is why we return a concrete type now.

This closes unicode-org#8054.

Considerations
--------------

* No Copy derive, because that's against API guidelines for Iterators
* Manual impl of `size_hint`, to be able to use ExactSizeIterator
  without panicking.
* Manual impl of `nth` and `fold`, as other iterator methods tend to be
  implemented in term of those, and forwarding them to the inner
  iterator **might** avoid performance penalty of the default impl.
@nicopap nicopap force-pushed the litemap-concrete-iter branch from e4d4097 to 4ed57de Compare July 3, 2026 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

litemap: Consider returning concrete types on LiteMap::{iter,iter_mut,keys,values}

4 participants