Add (implements "...") component name support (#2453)

* [wasmparser] Add `[implements=<I>]L` component name support

Add parsing, validation, and uniqueness rules for the new
`[implements=<interface>]label` extern name form from the component
model `implements` proposal. An implements name labels an instance
import/export that implements a named interface.

Uniqueness: `[implements=<I>]L` conflicts with bare label `L` and with
`[method]L.L` / `[static]L.L` (the existing l.l edge case), but is
strongly unique from interface names, constructors, and normal
method/static names.

See implements feature:
WebAssembly/component-model#613

* [wit-parser] Add `implements` syntax for named interface imports/exports

Support the `import label: iface;` and `export label: iface;` WIT
syntax, which encodes as `[implements=<I>]label` in the binary format.

This allows importing or exporting the same interface multiple times
under different names.

Changes include:
  - Add `implements: Option<InterfaceId>` field to  `WorldItem::Interface`
  - Parse `NamedPath` variant in the WIT AST with disambiguation against
    fully-qualified `namespace:package/interface` paths
  - Decode `[implements=<I>]label` names in all binary decoding paths
    via a new `decode_world_instance` helper
  - Thread `implements` through world elaboration and ID remapping
  - Add `name_world_key_with_item` for binary encoding

On the API change for `name_world_key_with_item`, I opted to introduce
this new fn that is used only at the few call sites that need it vs updating
the ~50 sites for name_world_key.

* [wit-component] Add implements encoding support to wit-component

Update component encoding to use `name_world_key_with_item` at sites
that produce component-level extern names, so that `implements`
imports
and exports are encoded as `[implements=<I>]L` in the binary format.

Five call sites are changed from `name_world_key` to the
implements-aware variant: import_map key construction, component export
names, ImportedResourceDrop lookups, and both direct and indirect
InterfaceFunc lookups.

* [wit-dylib] add implements to WorldItem::Interface

* [wit-smith] Add implements interface generation

Teach wit-smith to generate `ImplementsInterface` items in worlds,
producing `%label: path;` WIT syntax which encodes as
`[implements=<I>]L` in the component binary. This enables fuzzing of
the implements feature through the existing roundtrip_wit fuzzer.

* Generate nominal interface/type ids in `wit-component`

This commit is a long-overdue change in the componentization process in
`wit-component` to ensure that when a component is generated there are
unique `TypeId` and `InterfaceId` entries corresponding to all
types/interfaces which map to the final component. This is distinct from
the structure of WIT where the same `InterfaceId`, for example, can be
both imported and exported. When generating a component this means that
two distinct interfaces are imported/exported. Previously
`wit-component` has had a lot of very carefully filled out and handled
maps and such to ensure everything works, and this should make things
significantly easier because the possibility over overlap is now
nonexistent.

* Leverage nominal types/interfaces in encoding

This commit builds on the previous commit to simplify internal
structures within the encoding process of a component. Notably there's
no longer any need to have separate maps for imports/exports and instead
everything is located in a single set of maps. Not major changes, but
this is hoped to unlock future changes and additionally pave the way to
simplifying some internals in the future.

* Add `cm-implements` feature

Gate the new parsing behind this feature.

* Shift where `implements` is in the AST

Use a new `WorldKey` instead of `WorldItem`.

* Remove hand-rolled parsing

* Only write blessed files if they change

Helps run-on-file-change tools not infinite loop

* Reimplement how `implements` is represented

In the component model binary format this is no longer packed into
import/export names but is instead an auxiliary piece of metadata on
imports/exports. This required quite a few updates to parsing, AST
structures, etc, throughout. Additionally within `wit-parser` this is
now represented with no AST changes from before. Instead
`WorldKey::Name` is used with a `WorldItem::Interface`, and the only
difference from the previous "anonymous interfaces" is that the
interface pointed to has a name. This is what's used to indicate an
`implements` value is desired.

Initial plumbing within `wit-component` is done, but tests are not fully
passing yet.

* Handle `import a: b` in `generate_nominal_type_ids`

This commit updates the `generate_nominal_type_ids` to handle the new
form of imports that are showing up with the `implements` form. This is
going to be necessary for bindings generators to handle this neatly and
this is also required for wit-component after #2516

* Fix recording stability of named interfaces

* Fix the order nominalization happens in

* Use the 2024 edition for rustfmt in this workspace

Possible now that MSRV is high enough

* Fix configured build

* Reduce the diff with `main`

* Improve ergonomics of wasm-encoder

* Undo another interim change

* More diff minimizing

* Add failing test

* Don't permute imports/exports

Add a test case that broke in Wasmtime, and add a lot of words for why
things are the way they are.

* Update binary encoding

* Review comments

---------

Co-authored-by: Alex Crichton <alex@alexcrichton.com>

Author: ricochet

crates/wasm-compose/src/encoding.rs

@@ -455,7 +455,7 @@ impl<'a> TypeEncoder<'a> {
         id: ComponentInstanceTypeId,
     ) -> u32 {
         let ty = &self.0.types[id];
-        let instance = self.instance(state, ty.exports.iter().map(|(n, t)| (n.as_str(), *t)));
+        let instance = self.instance(state, ty.exports.iter().map(|(n, t)| (n.as_str(), t.ty)));
         let index = state.cur.encodable.type_count();
         state.cur.encodable.ty().instance(&instance);
         index
@@ -466,8 +466,8 @@ impl<'a> TypeEncoder<'a> {
 
         let component = self.component(
             state,
-            ty.imports.iter().map(|(n, t)| (n.as_str(), *t)),
-            ty.exports.iter().map(|(n, t)| (n.as_str(), *t)),
+            ty.imports.iter().map(|(n, t)| (n.as_str(), t.ty)),
+            ty.exports.iter().map(|(n, t)| (n.as_str(), t.ty)),
         );
 
         let index = state.cur.encodable.type_count();
@@ -884,7 +884,7 @@ impl ArgumentImport<'_> {
 
             let mut map = IndexMap::with_capacity(exports.len());
             for (name, ty) in exports {
-                map.insert(name.as_str(), vec![(*component, *ty)]);
+                map.insert(name.as_str(), vec![(*component, ty.ty)]);
             }
 
             self.kind = ArgumentImportKind::Instance(map);
@@ -907,7 +907,7 @@ impl ArgumentImport<'_> {
                             existing_component,
                             *existing_type,
                             new_component,
-                            *new_type,
+                            new_type.ty,
                             remapping,
                         ) {
                             continue;
@@ -923,7 +923,7 @@ impl ArgumentImport<'_> {
                             ecname = existing_component.name,
                         )
                     }
-                    dst.push((new_component, *new_type));
+                    dst.push((new_component, new_type.ty));
                 }
             }
             // Otherwise, an attempt to merge an instance with a non-instance is an error
@@ -1244,14 +1244,14 @@ impl DependencyRegistrar<'_, '_> {
 
     fn component(&mut self, ty: ComponentTypeId) {
         let ty = &self.types[ty];
-        for (_, ty) in ty.imports.iter().chain(&ty.exports) {
-            self.entity(*ty);
+        for ty in ty.imports.values().chain(ty.exports.values()) {
+            self.entity(ty.ty);
         }
     }
 
     fn instance(&mut self, ty: ComponentInstanceTypeId) {
         for (_, ty) in self.types[ty].exports.iter() {
-            self.entity(*ty);
+            self.entity(ty.ty);
         }
     }
 

crates/wasm-compose/src/graph.rs

@@ -144,14 +144,14 @@ impl<'a> Component<'a> {
                                 Payload::ComponentImportSection(s) => {
                                     for import in s {
                                         let import = import?;
-                                        let name = import.name.0.to_string();
+                                        let name = import.name.name.to_string();
                                         imports.insert(name, import.ty);
                                     }
                                 }
                                 Payload::ComponentExportSection(s) => {
                                     for export in s {
                                         let export = export?;
-                                        let name = export.name.0.to_string();
+                                        let name = export.name.name.to_string();
                                         exports.insert(name, (export.kind, export.index));
                                     }
                                 }
@@ -277,7 +277,7 @@ impl<'a> Component<'a> {
         let (name, _kind, _index) = self.export(index)?;
         Some((
             name,
-            self.types.as_ref().component_entity_type_of_export(name)?,
+            self.types.as_ref().component_item_for_export(name)?.ty,
         ))
     }
 
@@ -288,7 +288,7 @@ impl<'a> Component<'a> {
         let (name, _ty) = self.import(index)?;
         Some((
             name,
-            self.types.as_ref().component_entity_type_of_import(name)?,
+            self.types.as_ref().component_item_for_import(name)?.ty,
         ))
     }
 
@@ -333,7 +333,7 @@ impl<'a> Component<'a> {
             match self.exports.get_full(k.as_str()) {
                 Some((ai, _, _)) => {
                     let (_, a) = self.export_entity_type(ExportIndex(ai)).unwrap();
-                    if !ComponentEntityType::is_subtype_of(&a, self.types(), b, types) {
+                    if !ComponentEntityType::is_subtype_of(&a, self.types(), &b.ty, types) {
                         return false;
                     }
                 }
@@ -497,7 +497,7 @@ impl ResourceMapping {
                 if let ComponentEntityType::Type {
                     referenced: ComponentAnyTypeId::Resource(resource_id),
                     ..
-                } = ty
+                } = ty.ty
                 {
                     exports.insert(export_name, (export_component, resource_id.resource()));
                 }
@@ -508,7 +508,7 @@ impl ResourceMapping {
                 if let ComponentEntityType::Type {
                     referenced: ComponentAnyTypeId::Resource(resource_id),
                     ..
-                } = ty
+                } = ty.ty
                 {
                     let import_resource = resource_id.resource();
                     if let Some((export_component, export_resource)) =
@@ -591,16 +591,17 @@ impl<'a> CompositionGraph<'a> {
                 let ty = component
                     .types
                     .as_ref()
-                    .component_entity_type_of_import(import_name)
-                    .unwrap();
+                    .component_item_for_import(import_name)
+                    .unwrap()
+                    .ty;
 
                 if let ComponentEntityType::Instance(instance_id) = ty {
                     for (export_name, ty) in &component.types[instance_id].exports {
                         // TODO: support nested instances
                         if let ComponentEntityType::Type {
                             referenced: ComponentAnyTypeId::Resource(resource_id),
                             ..
-                        } = ty
+                        } = ty.ty
                         {
                             let set = resource_imports
                                 .entry(vec![import_name.to_string(), export_name.to_string()])

crates/wasm-encoder/src/component/builder.rs

@@ -327,14 +327,19 @@ impl ComponentBuilder {
     }
 
     /// Imports a new item into this component with the `name` and `ty` specified.
-    pub fn import(&mut self, name: &str, ty: ComponentTypeRef) -> u32 {
+    pub fn import<'a>(
+        &mut self,
+        name: impl Into<ComponentExternName<'a>>,
+        ty: ComponentTypeRef,
+    ) -> u32 {
+        let name = name.into();
         let ret = match &ty {
-            ComponentTypeRef::Instance(_) => self.instances.add(Some(name)),
-            ComponentTypeRef::Func(_) => self.funcs.add(Some(name)),
-            ComponentTypeRef::Type(..) => self.types.add(Some(name)),
-            ComponentTypeRef::Component(_) => self.components.add(Some(name)),
-            ComponentTypeRef::Module(_) => self.core_modules.add(Some(name)),
-            ComponentTypeRef::Value(_) => self.values.add(Some(name)),
+            ComponentTypeRef::Instance(_) => self.instances.add(Some(&name.name)),
+            ComponentTypeRef::Func(_) => self.funcs.add(Some(&name.name)),
+            ComponentTypeRef::Type(..) => self.types.add(Some(&name.name)),
+            ComponentTypeRef::Component(_) => self.components.add(Some(&name.name)),
+            ComponentTypeRef::Module(_) => self.core_modules.add(Some(&name.name)),
+            ComponentTypeRef::Value(_) => self.values.add(Some(&name.name)),
         };
         self.imports().import(name, ty);
         ret
@@ -345,15 +350,16 @@ impl ComponentBuilder {
     ///
     /// The `idx` is the item to export and the `ty` is an optional type to
     /// ascribe to the export.
-    pub fn export(
+    pub fn export<'a>(
         &mut self,
-        name: &str,
+        name: impl Into<ComponentExternName<'a>>,
         kind: ComponentExportKind,
         idx: u32,
         ty: Option<ComponentTypeRef>,
     ) -> u32 {
-        self.exports().export(name, kind, idx, ty);
-        self.inc_kind(Some(name), kind)
+        let name = name.into();
+        self.exports().export(name.clone(), kind, idx, ty);
+        self.inc_kind(Some(&name.name), kind)
     }
 
     /// Creates a new encoder for the next core type in this component.

crates/wasm-encoder/src/component/exports.rs

@@ -2,7 +2,10 @@ use super::{
     COMPONENT_SORT, CORE_MODULE_SORT, CORE_SORT, FUNCTION_SORT, INSTANCE_SORT, TYPE_SORT,
     VALUE_SORT,
 };
-use crate::{ComponentSection, ComponentSectionId, ComponentTypeRef, Encode, encode_section};
+use crate::{
+    ComponentExternName, ComponentSection, ComponentSectionId, ComponentTypeRef, Encode,
+    encode_section,
+};
 use alloc::vec::Vec;
 
 /// Represents the kind of an export from a WebAssembly component.
@@ -87,14 +90,14 @@ impl ComponentExportSection {
     }
 
     /// Define an export in the export section.
-    pub fn export(
+    pub fn export<'a>(
         &mut self,
-        name: &str,
+        name: impl Into<ComponentExternName<'a>>,
         kind: ComponentExportKind,
         index: u32,
         ty: Option<ComponentTypeRef>,
     ) -> &mut Self {
-        crate::encode_component_export_name(&mut self.bytes, name);
+        name.into().encode(&mut self.bytes);
         kind.encode(&mut self.bytes);
         index.encode(&mut self.bytes);
         match ty {
@@ -122,9 +125,3 @@ impl ComponentSection for ComponentExportSection {
         ComponentSectionId::Export.into()
     }
 }
-
-/// For more information on this see `encode_component_import_name`.
-pub(crate) fn encode_component_export_name(bytes: &mut Vec<u8>, name: &str) {
-    bytes.push(0x00);
-    name.encode(bytes);
-}

crates/wasm-encoder/src/component/imports.rs

@@ -2,6 +2,8 @@ use crate::{
     ComponentExportKind, ComponentSection, ComponentSectionId, ComponentValType, Encode,
     encode_section,
 };
+use alloc::borrow::Cow;
+use alloc::string::String;
 use alloc::vec::Vec;
 
 /// Represents the possible type bounds for type references.
@@ -131,8 +133,12 @@ impl ComponentImportSection {
     }
 
     /// Define an import in the component import section.
-    pub fn import(&mut self, name: &str, ty: ComponentTypeRef) -> &mut Self {
-        encode_component_import_name(&mut self.bytes, name);
+    pub fn import<'a>(
+        &mut self,
+        name: impl Into<ComponentExternName<'a>>,
+        ty: ComponentTypeRef,
+    ) -> &mut Self {
+        name.into().encode(&mut self.bytes);
         ty.encode(&mut self.bytes);
         self.num_added += 1;
         self
@@ -151,20 +157,91 @@ impl ComponentSection for ComponentImportSection {
     }
 }
 
-/// Prior to WebAssembly/component-model#263 import and export names were
-/// discriminated with a leading byte indicating what kind of import they are.
-/// After that PR though names are always prefixed with a 0x00 byte.
-///
-/// On 2023-10-28 in bytecodealliance/wasm-tools#1262 was landed to start
-/// transitioning to "always lead with 0x00". That updated the validator/parser
-/// to accept either 0x00 or 0x01 but the encoder wasn't updated at the time.
-///
-/// On 2024-09-03 in bytecodealliance/wasm-tools#TODO this encoder was updated
-/// to always emit 0x00 as a leading byte.
-///
-/// This function corresponds with the `importname'` production in the
-/// specification.
-pub(crate) fn encode_component_import_name(bytes: &mut Vec<u8>, name: &str) {
-    bytes.push(0x00);
-    name.encode(bytes);
+/// Full options for encoding a component name.
+#[derive(Debug, Clone)]
+pub struct ComponentExternName<'a> {
+    /// The name to encode.
+    pub name: Cow<'a, str>,
+    /// An optional `(implements ...)` directive (See 🏷️ in the component model
+    /// explainer).
+    pub implements: Option<Cow<'a, str>>,
+}
+
+impl Encode for ComponentExternName<'_> {
+    fn encode(&self, bytes: &mut Vec<u8>) {
+        let mut options = Vec::new();
+
+        if let Some(s) = &self.implements {
+            options.push((0x00, s.as_bytes()));
+        }
+
+        if options.is_empty() {
+            // Prior to WebAssembly/component-model#263 import and export names
+            // were discriminated with a leading byte indicating what kind of
+            // import they are.  After that PR though names are always prefixed
+            // with a 0x00 byte.
+            //
+            // On 2023-10-28 in bytecodealliance/wasm-tools#1262 was landed to
+            // start transitioning to "always lead with 0x00". That updated the
+            // validator/parser to accept either 0x00 or 0x01 but the encoder
+            // wasn't updated at the time.
+            //
+            // On 2024-09-03 in bytecodealliance/wasm-tools#TODO this encoder
+            // was updated to always emit 0x00 as a leading byte.
+            //
+            // This corresponds with the `importname'` production in the
+            // specification.
+            bytes.push(0x00);
+        } else {
+            bytes.push(0x02);
+        }
+
+        self.name.encode(bytes);
+
+        if !options.is_empty() {
+            options.len().encode(bytes);
+            for (kind, val) in options {
+                bytes.push(kind);
+                val.encode(bytes);
+            }
+        }
+    }
+}
+
+impl<'a> From<&'a str> for ComponentExternName<'a> {
+    fn from(name: &'a str) -> Self {
+        ComponentExternName {
+            name: Cow::Borrowed(name),
+            implements: None,
+        }
+    }
+}
+
+impl<'a> From<&'a String> for ComponentExternName<'a> {
+    fn from(name: &'a String) -> Self {
+        ComponentExternName {
+            name: Cow::Borrowed(name),
+            implements: None,
+        }
+    }
+}
+
+impl<'a> From<String> for ComponentExternName<'a> {
+    fn from(name: String) -> Self {
+        ComponentExternName {
+            name: Cow::Owned(name),
+            implements: None,
+        }
+    }
+}
+
+#[cfg(feature = "wasmparser")]
+impl<'a> From<wasmparser::ComponentExternName<'a>> for ComponentExternName<'a> {
+    fn from(name: wasmparser::ComponentExternName<'a>) -> Self {
+        let wasmparser::ComponentExternName { name, implements } = name;
+        ComponentExternName {
+            name: name.into(),
+            implements: implements.map(|s| s.into()),
+        }
+    }
 }

crates/wasm-encoder/src/component/instances.rs

@@ -1,6 +1,7 @@
 use super::CORE_INSTANCE_SORT;
 use crate::{
-    ComponentExportKind, ComponentSection, ComponentSectionId, Encode, ExportKind, encode_section,
+    ComponentExportKind, ComponentExternName, ComponentSection, ComponentSectionId, Encode,
+    ExportKind, encode_section,
 };
 use alloc::vec::Vec;
 
@@ -169,16 +170,17 @@ impl ComponentInstanceSection {
     }
 
     /// Define an instance by exporting items.
-    pub fn export_items<'a, E>(&mut self, exports: E) -> &mut Self
+    pub fn export_items<'a, N, E>(&mut self, exports: E) -> &mut Self
     where
-        E: IntoIterator<Item = (&'a str, ComponentExportKind, u32)>,
+        E: IntoIterator<Item = (N, ComponentExportKind, u32)>,
         E::IntoIter: ExactSizeIterator,
+        N: Into<ComponentExternName<'a>>,
     {
         let exports = exports.into_iter();
         self.bytes.push(0x01);
         exports.len().encode(&mut self.bytes);
         for (name, kind, index) in exports {
-            crate::encode_component_export_name(&mut self.bytes, name);
+            name.into().encode(&mut self.bytes);
             kind.encode(&mut self.bytes);
             index.encode(&mut self.bytes);
         }