Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 169 additions & 11 deletions index.bs
Original file line number Diff line number Diff line change
Expand Up @@ -164,10 +164,9 @@ A <dfn>tool definition</dfn> is a [=struct=] with the following [=struct/items=]
[[!JSON-SCHEMA]]

: <dfn>execute steps</dfn>
:: a set of steps to invoke the tool.
:: an algorithm that takes both a [=string=] and an algorithm <var ignore>completionSteps</var> that itself takes a [=string=]-or-null and a [=boolean=].

Note: For tools registered imperatively, these steps will simply invoke the supplied
{{ToolExecuteCallback}} callback. For tools registered
Note: For tools registered imperatively, these steps will simply invoke the [=imperative execute steps=]. For tools registered
[declaratively](https://github.com/webmachinelearning/webmcp/pull/76), this will be a set of
"internal" steps that have not been defined yet, that describe how to fill out a <{form}> and
its [=form-associated elements=].
Expand Down Expand Up @@ -257,6 +256,38 @@ a [=list=] of [=origins=] |exposed origins|, and an [=origin=] |accessing origin

</div>

<div algorithm>
The <dfn>imperative execute steps</dfn>, given a {{ModelContextTool}} |tool|, a {{Window}}
|targetWindow|, a [=string=] |inputArguments|, and an algorithm |completionSteps|, are as follows:

1. [=Assert=]: these steps are running on |targetWindow|'s [=relevant agent=]'s [=agent/event
loop=].

1. Let |inputObject| be the result of [=parse a JSON string to a JavaScript value=] given
|inputArguments| and |targetWindow|'s [=relevant realm=]. If <a spec=webidl lt="an exception was thrown">exception was thrown</a>, then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not add this webidl bit to <pre class="link-defaults"> at the top of the file?

run |completionSteps| given null and false, and abort these steps.

Issue: Support more granular errors; here we should return something that prompts the caller to reject its {{Promise}} with a "{{DataError}}" {{DOMException}}.

1. If |inputObject| [=Object type|is not an Object=] is false, then run |completionSteps| given null and false, and abort these steps.

Issue(#146): Specify and fire the "<code>toolactivated</code>" event.

1. Let |toolPromise| be the result of [=invoke|invoking=] |tool|'s {{ModelContextTool/execute}} with
|inputObject|.

1. [=promise/React=] to |toolPromise|:

- If |toolPromise| was fulfilled with value |v|:

1. Let |serializedResult| be the result of [=serializing a JavaScript value to a JSON string=] given |v|. If this throws an exception, run |completionSteps| given null and false, and abort these steps.

1. Run |completionSteps| given |serializedResult| and true.

- If |toolPromise| was rejected with reason <var ignore>r</var>, then run |completionSteps| given null and false.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to add that implementations may show a devtools warning that displays r, otherwise it's hard to know why toolPromise was rejected. This is what Chromium does at https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/core/script_tools/model_context.cc;l=293;drc=43b33e1c1959dd05d50dd707209464fda44222ed


</div>

<div algorithm>
To <dfn for="model context">unregister a tool</dfn> given a {{ModelContext}} |modelContext| and a
[=string=] |tool name|, run these steps:
Expand Down Expand Up @@ -323,6 +354,7 @@ The {{ModelContext}} interface provides methods for web applications to register
interface ModelContext : EventTarget {
Promise<undefined> registerTool(ModelContextTool tool, optional ModelContextRegisterToolOptions options = {});
Promise<sequence<RegisteredTool>> getTools(optional ModelContextGetToolOptions options = {});
Promise<DOMString?> executeTool(RegisteredTool tool, DOMString inputArguments, optional ExecuteToolOptions options = {});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency with the options naming convention, should we also prepend ModelContext to this dictionary name (e.g., ModelContextExecuteToolOptions)?

Suggested change
Promise<DOMString?> executeTool(RegisteredTool tool, DOMString inputArguments, optional ExecuteToolOptions options = {});
Promise<DOMString?> executeTool(RegisteredTool tool, DOMString inputArguments, optional ModelContextExecuteToolOptions options = {});

I now wonder if we should have used ModelContextRegisteredTool as well ;)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is DOMString optional in this spec PR?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of scope for this PR.

Have we consider moving executeTool from document.modelContext to RegisteredTool interface hinted in https://issues.chromium.org/536063275? FWIW This would match what Puppeteer does: https://pptr.dev/guides/webmcp#executing-tools


attribute EventHandler ontoolchange;
};
Expand All @@ -348,6 +380,12 @@ is a [=model context=] [=struct=] created alongside the {{ModelContext}}.
agents written in JavaScript, and possibly living in <{iframe}>s. The [=user agent=]'s [=browser
agent=] uses a different internal mechanism to retrieve the tools exposed to it.</p>
</dd>

<dt><code><var ignore>document</var>.{{Document/modelContext}}.{{ModelContext/executeTool(tool, inputArguments, options)}}</code></dt>
<dd>
<p>Executes a tool on the document it was registered on. Returns a promise that resolves to the
result of the tool's execution.</p>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe extends that it's a stringified result only for now.

</dd>
</dl>


Expand Down Expand Up @@ -451,7 +489,8 @@ The <dfn method for=ModelContext>registerTool(<var>tool</var>, <var>options</var
:: |stringified input schema|

: [=tool definition/execute steps=]
:: steps that invoke |tool|'s {{ModelContextTool/execute}}
:: An algorithm that take a [=string=] |inputArguments| and an algorithm |completionSteps|. These steps run the [=imperative execute steps=]
given |tool|, [=this=]'s [=relevant global object=], |inputArguments|, and |completionSteps|.

: [=tool definition/annotations=]
:: null if |tool|'s {{ModelContextTool/annotations}} does not [=map/exist=]. Otherwise, an
Expand All @@ -472,8 +511,8 @@ The <dfn method for=ModelContext>registerTool(<var>tool</var>, <var>options</var

1. [=Notify documents of a tool change=] given |tool owner| and |exposed origins|.

1. [=Queue a global task=] on the [=webmcp task source=] given |global| to
[=resolve=] |promise| with undefined.
1. [=Queue a global task=] on the [=webmcp task source=] given |global| to [=resolve=] |promise|
with undefined.

1. Return |promise|

Expand Down Expand Up @@ -590,6 +629,107 @@ The <dfn method for=ModelContext>getTools(<var>options</var>)</dfn> method steps

</div>

<div algorithm>
The <dfn method for=ModelContext>executeTool(<var>tool</var>, <var>inputArguments</var>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A JS example of how to execute a tool from a cross-origin iframe may come handy in this spec. See previous comment at #223 (comment)

<var>options</var>)</dfn> method steps are:

1. Let |invokingDocument| be [=this=]'s [=relevant global object=]'s [=associated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Shall we use executingDocument?

Document|associated <code>Document</code>=].

1. If |invokingDocument| is not [=Document/fully active=], then return [=a promise rejected with=]
an "{{InvalidStateError}}" {{DOMException}}.

1. If this's [=surrounding agent=]'s [=agent cluster=]'s [=is origin-keyed=] is false and this's

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. If this's [=surrounding agent=]'s [=agent cluster=]'s [=is origin-keyed=] is false and this's
1. If [=this=]'s [=surrounding agent=]'s [=agent cluster=]'s [=is origin-keyed=] is false and [=this=]'s

This suggestion applies as well to getTools and registerTool.

[=relevant settings object=]'s [=environment settings object/origin=]'s [=origin/scheme=] is not
"<code>file</code>", then return [=a promise rejected with=] a "{{SecurityError}}"
{{DOMException}}.

1. If |invokingDocument| is not [=allowed to use=] the "{{tools}}" feature, then return [=a promise
rejected with=] a "{{NotAllowedError}}" {{DOMException}}.

1. Let |expectedTargetOriginURL| be the result of [=URL parser|parsing=] |tool|'s
{{RegisteredTool/origin}}.

1. If |expectedTargetOriginURL| is failure, then return [=a promise rejected with=] a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using NotSupportedError instead and return a rejected promise in both cases?

  • expectedTargetOriginURL is a failure
  • expectedTargetOrigin is opaque

"{{DataError}}" {{DOMException}}.

1. Let |expectedTargetOrigin| be |expectedTargetOriginURL|'s [=url/origin=].

1. [=Assert=]: |expectedTargetOrigin| is not an [=opaque origin=].

1. Let |promise| be [=a new promise=] created in [=this=]'s [=relevant realm=].

1. Let |signal| be |options|'s {{ExecuteToolOptions/signal}} if it [=map/exists=]; otherwise null.

1. If |signal| is not null, then:
Comment on lines +662 to +664

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Let |signal| be |options|'s {{ExecuteToolOptions/signal}} if it [=map/exists=]; otherwise null.
1. If |signal| is not null, then:
1. Let |signal| be |options|'s {{ExecuteToolOptions/signal}}.
1. If |signal| [=map/exists=], then:


1. If |signal| is [=AbortSignal/aborted=], then return [=a promise rejected with=] |signal|'s
[=AbortSignal/abort reason=].

1. [=AbortSignal/add|Add the following abort steps=] to |signal|:

1. [=Reject=] |promise| with |signal|'s [=AbortSignal/abort reason=].

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also have an issue to spec and fire toolcancel event there. See #146


Issue(#48): Wire up |signal| to the tool execution callback in the tool host's document, so that tool abort is communicated all the way through.

1. Let |targetWindow| be |tool|'s {{RegisteredTool/window}}.

1. Let |targetDocument| be |targetWindow|'s [=associated Document|associated
<code>Document</code>=].

1. Run the following steps [=in parallel=]:

1. If |targetDocument|'s [=node navigable=]'s [=navigable/traversable navigable=] is not

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check that this document is fully active?

|invokingDocument|'s [=node navigable=]'s [=navigable/traversable navigable=], then [=queue a
global task=] on the [=webmcp task source=] given |invokingDocument|'s [=relevant global
object=] to [=reject=] |promise| with an "{{UnknownError}}" {{DOMException}}, and abort these

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UnknownError feels weird to me in these cases. Maybe InvalidStateError?

steps.

1. Let |targetOrigin| be |targetDocument|'s [=Document/origin=].

1. Let |callerOrigin| be |invokingDocument|'s [=Document/origin=].

1. If |targetOrigin| is not [=same origin=] with |expectedTargetOrigin|, then [=queue a global
task=] on the [=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to
[=reject=] |promise| with an "{{UnknownError}}" {{DOMException}}, and abort these steps.

Issue: Support more granular errors than "{{UnknownError}}", based on each failure case.

1. Let |targetToolMap| be |targetDocument|'s [=Document/associated ModelContext|associated
<code>ModelContext</code>=]'s [=ModelContext/internal context=]'s [=model context/tool map=].

1. Let |toolName| be |tool|'s {{RegisteredTool/name}}.

1. If |targetToolMap|[|toolName|] does not [=map/exist=], then [=queue a global task=] on the

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird case we could get into here: |targetToolMap|[|toolName|] isn't the same tool as returned from getTools. It may have been modified in between.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, some modifications are OK, others are not. I'll be carrying over some from our implementation before taking this PR out of draft mode.

[=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to [=reject=]
|promise| with an "{{UnknownError}}" {{DOMException}}, and abort these steps.

Issue: Support more granular errors than "{{UnknownError}}", based on each failure case.

1. Let |tool definition| be |targetToolMap|[|toolName|].

1. If [=tool is exposed to an origin=] given |targetOrigin|, |tool definition|'s [=tool
definition/exposed origins=], and |callerOrigin| is false, then [=queue a global task=] on the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
definition/exposed origins=], and |callerOrigin| is false, then [=queue a global task=] on the
definition/exposed origins=], and |callerOrigin| returns false, then [=queue a global task=] on the

[=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to [=reject=]
|promise| with an "{{UnknownError}}" {{DOMException}}, and abort these steps.

Issue: Support more granular errors than "{{UnknownError}}", based on each failure case.

1. Let |completionSteps| be an algorithm that takes a [=string=]-or-null |result| and a [=boolean=] |success|, and runs the following steps:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: How about we switch params order?

Suggested change
1. Let |completionSteps| be an algorithm that takes a [=string=]-or-null |result| and a [=boolean=] |success|, and runs the following steps:
1. Let |completionSteps| be an algorithm that takes a [=boolean=] |success| and a [=string=]-or-null |result|, and runs the following steps:


1. [=Assert=]: these steps are running [=in parallel=].

1. If |success| is true, then [=queue a global task=] on the [=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to [=resolve=] |promise| with |result|.

1. Otherwise, [=queue a global task=] on the [=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to [=reject=] |promise| with an "{{UnknownError}}" {{DOMException}}.

1. [=Queue a global task=] on the [=webmcp task source=] given |targetWindow| to run |tool
definition|'s [=tool definition/execute steps=] given |inputArguments| and |completionSteps|.

1. Return |promise|.

</div>

<h4 id="model-context-tool">ModelContextTool Dictionary</h4>

The {{ModelContextTool}} dictionary describes a tool that can be invoked by [=agents=].
Expand Down Expand Up @@ -617,19 +757,21 @@ callback ToolExecuteCallback = Promise<any> (object input);
<dl class="domintro">
<dt><code><var ignore>tool</var>["{{ModelContextTool/name}}"]</code></dt>
<dd>
<p>A unique identifier for the tool. This is used by [=agents=] to reference the tool when making tool calls.
<p>A unique identifier for the tool. This is used by [=agents=] to reference the tool when
making tool calls.
</dd>

<dt><code><var ignore>tool</var>["{{ModelContextTool/title}}"]</code></dt>
<dd>
<p>A label for the tool. This is used by the user agent to reference the tool in the user interface.
<p>It is recommended that this string be localized to the user's
{{NavigatorLanguage/language}}.
<p>A label for the tool. This is used by the user agent to reference the tool in the user
interface. <p>It is recommended that this string be localized to the user's
{{NavigatorLanguage/language}}.
</dd>

<dt><code><var ignore>tool</var>["{{ModelContextTool/description}}"]</code></dt>
<dd>
<p>A natural language description of the tool's functionality. This helps [=agents=] understand when and how to use the tool.
<p>A natural language description of the tool's functionality. This helps [=agents=] understand
when and how to use the tool.
</dd>

<dt><code><var ignore>tool</var>["{{ModelContextTool/inputSchema}}"]</code></dt>
Expand Down Expand Up @@ -700,6 +842,22 @@ dictionary ModelContextGetToolOptions {
same-origin documents.
</dl>

<h4 id="execute-tool-options">ExecuteToolOptions Dictionary</h4>

The {{ExecuteToolOptions}} dictionary allows web applications to pass options to
{{ModelContext/executeTool()}}.

<xmp class="idl">
dictionary ExecuteToolOptions {
AbortSignal signal;
};
</xmp>

<dl class="domintro">
: <code><var ignore>options</var>["{{ExecuteToolOptions/signal}}"]</code>
:: An {{AbortSignal}} that can be used to cancel the execution of the tool.
</dl>

<h4 id="registered-tool">RegisteredTool Dictionary</h4>

The {{RegisteredTool}} dictionary represents a tool that has been registered and is available for
Expand Down
Loading