Rollup merge of rust-lang#149328 - shua:stralloc, r=Amanieu

Add `String<A>` type with custom allocator parameter

This change is part of the `allocator_api` feature set [rust-lang#32838](rust-lang#32838) (also see [wg-allocators roadmap] or [libs-team ACP]).
The previous attempts at adding an allocator parameter to string are at [rust-lang#101551], and [rust-lang#79500] (I think those authors should get much of the credit here, I am re-writing what they worked out in those threads).

## workaround

There is a type inference ambiguity introduced when adding a generic parameter to a type which previously had none, even when that parameter has a default value (more details in [rust-lang#98931]). I've done the same [workaround] as [rust-lang#101551], which is to make `alloc::string::String` a type alias to `String<Global>`, but I've arranged the modules a bit differently to make rebase/merges a bit easier.

This workaround unfortunately changes the type name of the `String` language item, and that would be user-facing in error or diagnostic output. I understand from [this comment](rust-lang#101551 (comment)) that this change is acceptable.

## changes to existing API

Most of the methods on the original `String` have been implemented for the generic version instead. I don't foresee anything more moving from `String<Global>` to `String<A>`, as the remaining methods are all constructors which implicitly use the `Global` allocator.

There are three general types of changes:
1. methods which don't allocate: here we just change `impl Foo for String` to `impl<A: Allocator> Foo for String<A>`
2. converting to/from other allocator generic types like `Vec<u8, A>` and `Box<str, A>`: here we can use the existing allocator from those types.
3. methods which clone from some other type with an allocator: here it's ambiguous whether the end result should be `String<A>`, `String<Global>`, or `String<impl Allocator + Default>`, etc; in general I try to leave these out of this change, but where some analogous change was made to `Vec<T, A>` I follow that.

## new methods

Some methods have been added to `String<A>` which are not strictly necessary to land this change, but are considered helpful enough to users, and close enough to existing precedent in `Vec<T, A>`. Specifically, 4 new constructors (`new_in`, `with_capacity_in`, `try_with_capacity_in`, `from_raw_parts_in`), 1 destructor (`into_raw_parts_with_alloc`), and 1 getter (`allocator`). Technically, the updated `from_utf8_unchecked` should be sufficient for constructing, but we can add some safe constructors so users don't have to sully themselves.

## not implemented

Variants of `from_utf{8,16}*` which internally allocate or use `Cow` have been punted on this PR, maybe a followup would make sense to either rewrite them, or add some `*_in` variant.

[wg-allocators roadmap]: rust-lang/wg-allocators#7
[libs-team ACP]: rust-lang/libs-team#101
[workaround]: rust-lang#79499 (comment)
[rust-lang#101551]: rust-lang#101551
[rust-lang#79500]: rust-lang#79500
[rust-lang#98931]: rust-lang#98931

Author: JonathanBrouwer

RELEASES.md

@@ -303,8 +303,8 @@ Stabilized APIs
 - [`impl PartialEq<String> for PathBuf`](https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#impl-PartialEq%3CString%3E-for-PathBuf)
 - [`impl PartialEq<str> for Path`](https://doc.rust-lang.org/stable/std/path/struct.Path.html#impl-PartialEq%3Cstr%3E-for-Path)
 - [`impl PartialEq<String> for Path`](https://doc.rust-lang.org/stable/std/path/struct.Path.html#impl-PartialEq%3CString%3E-for-Path)
-- [`impl PartialEq<PathBuf> for String`](https://doc.rust-lang.org/stable/std/string/struct.String.html#impl-PartialEq%3CPathBuf%3E-for-String)
-- [`impl PartialEq<Path> for String`](https://doc.rust-lang.org/stable/std/string/struct.String.html#impl-PartialEq%3CPath%3E-for-String)
+- [`impl PartialEq<PathBuf> for String`](https://doc.rust-lang.org/stable/std/string/type.String.html#impl-PartialEq%3CPathBuf%3E-for-String)
+- [`impl PartialEq<Path> for String`](https://doc.rust-lang.org/stable/std/string/type.String.html#impl-PartialEq%3CPath%3E-for-String)
 - [`impl PartialEq<PathBuf> for str`](https://doc.rust-lang.org/stable/std/primitive.str.html#impl-PartialEq%3CPathBuf%3E-for-str)
 - [`impl PartialEq<Path> for str`](https://doc.rust-lang.org/stable/std/primitive.str.html#impl-PartialEq%3CPath%3E-for-str)
 - [`Ipv4Addr::from_octets`](https://doc.rust-lang.org/stable/std/net/struct.Ipv4Addr.html#method.from_octets)
@@ -854,7 +854,7 @@ Stabilized APIs
 - [`<[T]>::split_off_first_mut`](https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_off_first_mut)
 - [`<[T]>::split_off_last`](https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_off_last)
 - [`<[T]>::split_off_last_mut`](https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_off_last_mut)
-- [`String::extend_from_within`](https://doc.rust-lang.org/stable/alloc/string/struct.String.html#method.extend_from_within)
+- [`String::extend_from_within`](https://doc.rust-lang.org/stable/alloc/string/type.String.html#method.extend_from_within)
 - [`os_str::Display`](https://doc.rust-lang.org/stable/std/ffi/os_str/struct.Display.html)
 - [`OsString::display`](https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.display)
 - [`OsStr::display`](https://doc.rust-lang.org/stable/std/ffi/struct.OsStr.html#method.display)
@@ -868,7 +868,7 @@ Stabilized APIs
 - [`impl From<PipeReader> for OwnedFd`](https://doc.rust-lang.org/stable/std/os/fd/struct.OwnedFd.html#impl-From%3CPipeReader%3E-for-OwnedFd)
 - [`impl From<PipeWriter> for OwnedFd`](https://doc.rust-lang.org/stable/std/os/fd/struct.OwnedFd.html#impl-From%3CPipeWriter%3E-for-OwnedFd)
 - [`Box<MaybeUninit<T>>::write`](https://doc.rust-lang.org/stable/std/boxed/struct.Box.html#method.write)
-- [`impl TryFrom<Vec<u8>> for String`](https://doc.rust-lang.org/stable/std/string/struct.String.html#impl-TryFrom%3CVec%3Cu8%3E%3E-for-String)
+- [`impl TryFrom<Vec<u8>> for String`](https://doc.rust-lang.org/stable/std/string/type.String.html#impl-TryFrom%3CVec%3Cu8%3E%3E-for-String)
 - [`<*const T>::offset_from_unsigned`](https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from_unsigned)
 - [`<*const T>::byte_offset_from_unsigned`](https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.byte_offset_from_unsigned)
 - [`<*mut T>::offset_from_unsigned`](https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from_unsigned-1)
@@ -906,14 +906,14 @@ These previously stable APIs are now stable in const contexts:
 - [`char::is_whitespace`](https://doc.rust-lang.org/stable/std/primitive.char.html#method.is_whitespace)
 - [`<[[T; N]]>::as_flattened`](https://doc.rust-lang.org/stable/std/primitive.slice.html#method.as_flattened)
 - [`<[[T; N]]>::as_flattened_mut`](https://doc.rust-lang.org/stable/std/primitive.slice.html#method.as_flattened_mut) 
-- [`String::into_bytes`](https://doc.rust-lang.org/stable/std/string/struct.String.html#method.into_bytes)
-- [`String::as_str`](https://doc.rust-lang.org/stable/std/string/struct.String.html#method.as_str)
-- [`String::capacity`](https://doc.rust-lang.org/stable/std/string/struct.String.html#method.capacity)
-- [`String::as_bytes`](https://doc.rust-lang.org/stable/std/string/struct.String.html#method.as_bytes)
-- [`String::len`](https://doc.rust-lang.org/stable/std/string/struct.String.html#method.len)
-- [`String::is_empty`](https://doc.rust-lang.org/stable/std/string/struct.String.html#method.is_empty)
-- [`String::as_mut_str`](https://doc.rust-lang.org/stable/std/string/struct.String.html#method.as_mut_str)
-- [`String::as_mut_vec`](https://doc.rust-lang.org/stable/std/string/struct.String.html#method.as_mut_vec)
+- [`String::into_bytes`](https://doc.rust-lang.org/stable/std/string/type.String.html#method.into_bytes)
+- [`String::as_str`](https://doc.rust-lang.org/stable/std/string/type.String.html#method.as_str)
+- [`String::capacity`](https://doc.rust-lang.org/stable/std/string/type.String.html#method.capacity)
+- [`String::as_bytes`](https://doc.rust-lang.org/stable/std/string/type.String.html#method.as_bytes)
+- [`String::len`](https://doc.rust-lang.org/stable/std/string/type.String.html#method.len)
+- [`String::is_empty`](https://doc.rust-lang.org/stable/std/string/type.String.html#method.is_empty)
+- [`String::as_mut_str`](https://doc.rust-lang.org/stable/std/string/type.String.html#method.as_mut_str)
+- [`String::as_mut_vec`](https://doc.rust-lang.org/stable/std/string/type.String.html#method.as_mut_vec)
 - [`Vec::as_ptr`](https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.as_ptr)
 - [`Vec::as_slice`](https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.as_slice)
 - [`Vec::capacity`](https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.capacity)
@@ -2965,7 +2965,7 @@ Stabilized APIs
 
 - [`impl<T: Send> Sync for mpsc::Sender<T>`](https://doc.rust-lang.org/stable/std/sync/mpsc/struct.Sender.html#impl-Sync-for-Sender%3CT%3E)
 - [`impl TryFrom<&OsStr> for &str`](https://doc.rust-lang.org/stable/std/primitive.str.html#impl-TryFrom%3C%26'a+OsStr%3E-for-%26'a+str)
-- [`String::leak`](https://doc.rust-lang.org/stable/alloc/string/struct.String.html#method.leak)
+- [`String::leak`](https://doc.rust-lang.org/stable/alloc/string/type.String.html#method.leak)
 
 These APIs are now stable in const contexts:
 
@@ -5024,8 +5024,8 @@ and related tools.
 [`collections::TryReserveError`]: https://doc.rust-lang.org/std/collections/struct.TryReserveError.html
 [`HashMap::try_reserve`]: https://doc.rust-lang.org/std/collections/hash_map/struct.HashMap.html#method.try_reserve
 [`HashSet::try_reserve`]: https://doc.rust-lang.org/std/collections/hash_set/struct.HashSet.html#method.try_reserve
-[`String::try_reserve`]: https://doc.rust-lang.org/alloc/string/struct.String.html#method.try_reserve
-[`String::try_reserve_exact`]: https://doc.rust-lang.org/alloc/string/struct.String.html#method.try_reserve_exact
+[`String::try_reserve`]: https://doc.rust-lang.org/alloc/string/type.String.html#method.try_reserve
+[`String::try_reserve_exact`]: https://doc.rust-lang.org/alloc/string/type.String.html#method.try_reserve_exact
 [`Vec::try_reserve`]: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.try_reserve
 [`Vec::try_reserve_exact`]: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.try_reserve_exact
 [`VecDeque::try_reserve`]: https://doc.rust-lang.org/std/collections/struct.VecDeque.html#method.try_reserve
@@ -5166,7 +5166,7 @@ and related tools.
 [`BufWriter::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.BufWriter.html#method.into_parts
 [`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]: https://github.com/rust-lang/rust/pull/84662
 [`Vec::shrink_to`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.shrink_to
-[`String::shrink_to`]: https://doc.rust-lang.org/stable/std/string/struct.String.html#method.shrink_to
+[`String::shrink_to`]: https://doc.rust-lang.org/stable/std/string/type.String.html#method.shrink_to
 [`OsString::shrink_to`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.shrink_to
 [`PathBuf::shrink_to`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.shrink_to
 [`BinaryHeap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.shrink_to
@@ -9361,7 +9361,7 @@ Compatibility Notes
 [`Iterator::try_for_each`]: https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.try_for_each
 [`NonNull::cast`]: https://doc.rust-lang.org/std/ptr/struct.NonNull.html#method.cast
 [`Option::filter`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.filter
-[`String::replace_range`]: https://doc.rust-lang.org/std/string/struct.String.html#method.replace_range
+[`String::replace_range`]: https://doc.rust-lang.org/std/string/type.String.html#method.replace_range
 [`Take::set_limit`]: https://doc.rust-lang.org/std/io/struct.Take.html#method.set_limit
 [`hint::unreachable_unchecked`]: https://doc.rust-lang.org/std/hint/fn.unreachable_unchecked.html
 [`os::unix::process::parent_id`]: https://doc.rust-lang.org/std/os/unix/process/fn.parent_id.html
@@ -9606,7 +9606,7 @@ Compatibility Notes
 [`process::id`]: https://doc.rust-lang.org/std/process/fn.id.html
 [`slice::rotate_left`]: https://doc.rust-lang.org/std/primitive.slice.html#method.rotate_left
 [`slice::rotate_right`]: https://doc.rust-lang.org/std/primitive.slice.html#method.rotate_right
-[`String::retain`]: https://doc.rust-lang.org/std/string/struct.String.html#method.retain
+[`String::retain`]: https://doc.rust-lang.org/std/string/type.String.html#method.retain
 [cargo/5041]: https://github.com/rust-lang/cargo/pull/5041
 [cargo/5083]: https://github.com/rust-lang/cargo/pull/5083
 
@@ -11057,8 +11057,8 @@ Compatibility Notes
 [`Result::unwrap_or_default`]: https://doc.rust-lang.org/std/result/enum.Result.html#method.unwrap_or_default
 [`SocketAddr::is_ipv4`]: https://doc.rust-lang.org/std/net/enum.SocketAddr.html#method.is_ipv4
 [`SocketAddr::is_ipv6`]: https://doc.rust-lang.org/std/net/enum.SocketAddr.html#method.is_ipv6
-[`String::insert_str`]: https://doc.rust-lang.org/std/string/struct.String.html#method.insert_str
-[`String::split_off`]: https://doc.rust-lang.org/std/string/struct.String.html#method.split_off
+[`String::insert_str`]: https://doc.rust-lang.org/std/string/type.String.html#method.insert_str
+[`String::split_off`]: https://doc.rust-lang.org/std/string/type.String.html#method.split_off
 [`Vec::dedup_by_key`]: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.dedup_by_key
 [`Vec::dedup_by`]: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.dedup_by
 [`VecDeque::resize`]:  https://doc.rust-lang.org/std/collections/vec_deque/struct.VecDeque.html#method.resize
@@ -12972,8 +12972,8 @@ Compatibility Notes
 [`Ipv6Addr::is_unspecified`]: http://doc.rust-lang.org/nightly/std/net/struct.Ipv6Addr.html#method.is_unspecified
 [`Path::strip_prefix`]: http://doc.rust-lang.org/nightly/std/path/struct.Path.html#method.strip_prefix
 [`RandomState::new`]: http://doc.rust-lang.org/nightly/std/collections/hash_map/struct.RandomState.html#method.new
-[`String::as_mut_str`]: http://doc.rust-lang.org/nightly/std/string/struct.String.html#method.as_mut_str
-[`String::as_str`]: http://doc.rust-lang.org/nightly/std/string/struct.String.html#method.as_str
+[`String::as_mut_str`]: http://doc.rust-lang.org/nightly/std/string/type.String.html#method.as_mut_str
+[`String::as_str`]: http://doc.rust-lang.org/nightly/std/string/type.String.html#method.as_str
 [`Vec::as_mut_slice`]: http://doc.rust-lang.org/nightly/std/vec/struct.Vec.html#method.as_mut_slice
 [`Vec::as_slice`]: http://doc.rust-lang.org/nightly/std/vec/struct.Vec.html#method.as_slice
 [`clone_from_slice`]: http://doc.rust-lang.org/nightly/std/primitive.slice.html#method.clone_from_slice
@@ -13159,7 +13159,7 @@ Compatibility Notes
 [`os::unix::fs::DirBuilderExt::mode`]: http://doc.rust-lang.org/nightly/std/os/unix/fs/trait.DirBuilderExt.html#tymethod.mode
 [`os::unix::fs::DirBuilderExt`]: http://doc.rust-lang.org/nightly/std/os/unix/fs/trait.DirBuilderExt.html
 [`string::Drain`]: http://doc.rust-lang.org/nightly/std/string/struct.Drain.html
-[`string::String::drain`]: http://doc.rust-lang.org/nightly/std/string/struct.String.html#method.drain
+[`string::String::drain`]: http://doc.rust-lang.org/nightly/std/string/type.String.html#method.drain
 [`vec::Drain`]: http://doc.rust-lang.org/nightly/std/vec/struct.Drain.html
 [`vec::Vec::drain`]: http://doc.rust-lang.org/nightly/std/vec/struct.Vec.html#method.drain
 [`vec_deque::Drain`]: http://doc.rust-lang.org/nightly/std/collections/vec_deque/struct.Drain.html
@@ -13508,7 +13508,7 @@ Miscellaneous
 [`Rc::make_mut`]: http://doc.rust-lang.org/nightly/alloc/rc/struct.Rc.html#method.make_mut
 [`Rc::try_unwrap`]: http://doc.rust-lang.org/nightly/alloc/rc/struct.Rc.html#method.try_unwrap
 [`Result::expect`]: http://doc.rust-lang.org/nightly/core/result/enum.Result.html#method.expect
-[`String::into_boxed_str`]: http://doc.rust-lang.org/nightly/collections/string/struct.String.html#method.into_boxed_str
+[`String::into_boxed_str`]: http://doc.rust-lang.org/nightly/collections/string/type.String.html#method.into_boxed_str
 [`TcpStream::read_timeout`]: http://doc.rust-lang.org/nightly/std/net/struct.TcpStream.html#method.read_timeout
 [`TcpStream::set_read_timeout`]: http://doc.rust-lang.org/nightly/std/net/struct.TcpStream.html#method.set_read_timeout
 [`TcpStream::write_timeout`]: http://doc.rust-lang.org/nightly/std/net/struct.TcpStream.html#method.write_timeout

compiler/rustc_span/src/symbol.rs

@@ -1972,6 +1972,7 @@ symbols! {
         str_inherent_from_utf8_unchecked_mut,
         strict_provenance_lints,
         string_deref_patterns,
+        string_in_global,
         stringify,
         struct_field_attributes,
         struct_inherit,

library/alloc/src/boxed.rs

@@ -210,6 +210,8 @@ use crate::alloc::{AllocError, Allocator, Global, Layout};
 use crate::raw_vec::RawVec;
 #[cfg(not(no_global_oom_handling))]
 use crate::str::from_boxed_utf8_unchecked;
+#[cfg(not(no_global_oom_handling))]
+use crate::vec::Vec;
 
 /// Conversion related impls for `Box<_>` (`From`, `downcast`, etc)
 mod convert;
@@ -2072,11 +2074,13 @@ impl<T: Clone, A: Allocator + Clone> Clone for Box<[T], A> {
 
 #[cfg(not(no_global_oom_handling))]
 #[stable(feature = "box_slice_clone", since = "1.3.0")]
-impl Clone for Box<str> {
+impl<A: Allocator + Clone> Clone for Box<str, A> {
     fn clone(&self) -> Self {
-        // this makes a copy of the data
-        let buf: Box<[u8]> = self.as_bytes().into();
-        unsafe { from_boxed_utf8_unchecked(buf) }
+        let alloc = Box::allocator(self).clone();
+        let len = self.len();
+        let mut vec: Vec<u8, A> = Vec::with_capacity_in(len, alloc);
+        vec.extend_from_slice(self.as_bytes());
+        unsafe { from_boxed_utf8_unchecked(vec.into_boxed_slice()) }
     }
 }
 

library/alloc/src/str.rs

@@ -7,6 +7,7 @@
 // It's cleaner to just turn off the unused_imports warning than to fix them.
 #![allow(unused_imports)]
 
+use core::alloc::Allocator;
 use core::borrow::{Borrow, BorrowMut};
 use core::iter::FusedIterator;
 use core::mem::MaybeUninit;
@@ -52,7 +53,7 @@ use core::{mem, ptr};
 use crate::borrow::ToOwned;
 use crate::boxed::Box;
 use crate::slice::{Concat, Join, SliceIndex};
-use crate::string::String;
+use crate::string::generic::String;
 use crate::vec::Vec;
 
 /// Note: `str` in `Concat<str>` is not meaningful here.
@@ -186,15 +187,15 @@ where
 }
 
 #[stable(feature = "rust1", since = "1.0.0")]
-impl Borrow<str> for String {
+impl<A: Allocator> Borrow<str> for String<A> {
     #[inline]
     fn borrow(&self) -> &str {
         &self[..]
     }
 }
 
 #[stable(feature = "string_borrow_mut", since = "1.36.0")]
-impl BorrowMut<str> for String {
+impl<A: Allocator> BorrowMut<str> for String<A> {
     #[inline]
     fn borrow_mut(&mut self) -> &mut str {
         &mut self[..]
@@ -234,7 +235,7 @@ impl str {
     #[stable(feature = "str_box_extras", since = "1.20.0")]
     #[must_use = "`self` will be dropped if the result is not used"]
     #[inline]
-    pub fn into_boxed_bytes(self: Box<Self>) -> Box<[u8]> {
+    pub fn into_boxed_bytes<A: Allocator>(self: Box<Self, A>) -> Box<[u8], A> {
         self.into()
     }
 
@@ -496,8 +497,8 @@ impl str {
     #[rustc_allow_incoherent_impl]
     #[must_use = "`self` will be dropped if the result is not used"]
     #[inline]
-    pub fn into_string(self: Box<Self>) -> String {
-        let slice = Box::<[u8]>::from(self);
+    pub fn into_string<A: Allocator>(self: Box<Self, A>) -> String<A> {
+        let slice = Box::<[u8], A>::from(self);
         unsafe { String::from_utf8_unchecked(slice.into_vec()) }
     }
 
@@ -613,8 +614,9 @@ impl str {
 #[stable(feature = "str_box_extras", since = "1.20.0")]
 #[must_use]
 #[inline]
-pub unsafe fn from_boxed_utf8_unchecked(v: Box<[u8]>) -> Box<str> {
-    unsafe { Box::from_raw(Box::into_raw(v) as *mut str) }
+pub unsafe fn from_boxed_utf8_unchecked<A: Allocator>(v: Box<[u8], A>) -> Box<str, A> {
+    let (ptr, alloc) = Box::into_raw_with_allocator(v);
+    unsafe { Box::from_raw_in(ptr as *mut str, alloc) }
 }
 
 /// Converts leading ascii bytes in `s` by calling the `convert` function.