-
Notifications
You must be signed in to change notification settings - Fork 184
Spec the executeTool() API
#226
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -164,10 +164,9 @@ A <dfn>tool definition</dfn> is a [=struct=] with the following [=struct/items=] | |||||||||||||
| [[!JSON-SCHEMA]] | ||||||||||||||
|
|
||||||||||||||
| : <dfn>execute steps</dfn> | ||||||||||||||
| :: a set of steps to invoke the tool. | ||||||||||||||
| :: an algorithm that takes both a [=string=] and an algorithm <var ignore>completionSteps</var> that itself takes a [=string=]-or-null and a [=boolean=]. | ||||||||||||||
|
|
||||||||||||||
| Note: For tools registered imperatively, these steps will simply invoke the supplied | ||||||||||||||
| {{ToolExecuteCallback}} callback. For tools registered | ||||||||||||||
| Note: For tools registered imperatively, these steps will simply invoke the [=imperative execute steps=]. For tools registered | ||||||||||||||
| [declaratively](https://github.com/webmachinelearning/webmcp/pull/76), this will be a set of | ||||||||||||||
| "internal" steps that have not been defined yet, that describe how to fill out a <{form}> and | ||||||||||||||
| its [=form-associated elements=]. | ||||||||||||||
|
|
@@ -257,6 +256,38 @@ a [=list=] of [=origins=] |exposed origins|, and an [=origin=] |accessing origin | |||||||||||||
|
|
||||||||||||||
| </div> | ||||||||||||||
|
|
||||||||||||||
| <div algorithm> | ||||||||||||||
| The <dfn>imperative execute steps</dfn>, given a {{ModelContextTool}} |tool|, a {{Window}} | ||||||||||||||
| |targetWindow|, a [=string=] |inputArguments|, and an algorithm |completionSteps|, are as follows: | ||||||||||||||
|
|
||||||||||||||
| 1. [=Assert=]: these steps are running on |targetWindow|'s [=relevant agent=]'s [=agent/event | ||||||||||||||
| loop=]. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |inputObject| be the result of [=parse a JSON string to a JavaScript value=] given | ||||||||||||||
| |inputArguments| and |targetWindow|'s [=relevant realm=]. If <a spec=webidl lt="an exception was thrown">exception was thrown</a>, then | ||||||||||||||
| run |completionSteps| given null and false, and abort these steps. | ||||||||||||||
|
|
||||||||||||||
| Issue: Support more granular errors; here we should return something that prompts the caller to reject its {{Promise}} with a "{{DataError}}" {{DOMException}}. | ||||||||||||||
|
|
||||||||||||||
| 1. If |inputObject| [=Object type|is not an Object=] is false, then run |completionSteps| given null and false, and abort these steps. | ||||||||||||||
|
|
||||||||||||||
| Issue(#146): Specify and fire the "<code>toolactivated</code>" event. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |toolPromise| be the result of [=invoke|invoking=] |tool|'s {{ModelContextTool/execute}} with | ||||||||||||||
| |inputObject|. | ||||||||||||||
|
|
||||||||||||||
| 1. [=promise/React=] to |toolPromise|: | ||||||||||||||
|
|
||||||||||||||
| - If |toolPromise| was fulfilled with value |v|: | ||||||||||||||
|
|
||||||||||||||
| 1. Let |serializedResult| be the result of [=serializing a JavaScript value to a JSON string=] given |v|. If this throws an exception, run |completionSteps| given null and false, and abort these steps. | ||||||||||||||
|
|
||||||||||||||
| 1. Run |completionSteps| given |serializedResult| and true. | ||||||||||||||
|
|
||||||||||||||
| - If |toolPromise| was rejected with reason <var ignore>r</var>, then run |completionSteps| given null and false. | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We may want to add that implementations may show a devtools warning that displays |
||||||||||||||
|
|
||||||||||||||
| </div> | ||||||||||||||
|
|
||||||||||||||
| <div algorithm> | ||||||||||||||
| To <dfn for="model context">unregister a tool</dfn> given a {{ModelContext}} |modelContext| and a | ||||||||||||||
| [=string=] |tool name|, run these steps: | ||||||||||||||
|
|
@@ -323,6 +354,7 @@ The {{ModelContext}} interface provides methods for web applications to register | |||||||||||||
| interface ModelContext : EventTarget { | ||||||||||||||
| Promise<undefined> registerTool(ModelContextTool tool, optional ModelContextRegisterToolOptions options = {}); | ||||||||||||||
| Promise<sequence<RegisteredTool>> getTools(optional ModelContextGetToolOptions options = {}); | ||||||||||||||
| Promise<DOMString?> executeTool(RegisteredTool tool, DOMString inputArguments, optional ExecuteToolOptions options = {}); | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For consistency with the options naming convention, should we also prepend
Suggested change
I now wonder if we should have used
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is DOMString optional in this spec PR?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Out of scope for this PR. Have we consider moving executeTool from document.modelContext to RegisteredTool interface hinted in https://issues.chromium.org/536063275? FWIW This would match what Puppeteer does: https://pptr.dev/guides/webmcp#executing-tools |
||||||||||||||
|
|
||||||||||||||
| attribute EventHandler ontoolchange; | ||||||||||||||
| }; | ||||||||||||||
|
|
@@ -348,6 +380,12 @@ is a [=model context=] [=struct=] created alongside the {{ModelContext}}. | |||||||||||||
| agents written in JavaScript, and possibly living in <{iframe}>s. The [=user agent=]'s [=browser | ||||||||||||||
| agent=] uses a different internal mechanism to retrieve the tools exposed to it.</p> | ||||||||||||||
| </dd> | ||||||||||||||
|
|
||||||||||||||
| <dt><code><var ignore>document</var>.{{Document/modelContext}}.{{ModelContext/executeTool(tool, inputArguments, options)}}</code></dt> | ||||||||||||||
| <dd> | ||||||||||||||
| <p>Executes a tool on the document it was registered on. Returns a promise that resolves to the | ||||||||||||||
| result of the tool's execution.</p> | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe extends that it's a stringified result only for now. |
||||||||||||||
| </dd> | ||||||||||||||
| </dl> | ||||||||||||||
|
|
||||||||||||||
|
|
||||||||||||||
|
|
@@ -451,7 +489,8 @@ The <dfn method for=ModelContext>registerTool(<var>tool</var>, <var>options</var | |||||||||||||
| :: |stringified input schema| | ||||||||||||||
|
|
||||||||||||||
| : [=tool definition/execute steps=] | ||||||||||||||
| :: steps that invoke |tool|'s {{ModelContextTool/execute}} | ||||||||||||||
| :: An algorithm that take a [=string=] |inputArguments| and an algorithm |completionSteps|. These steps run the [=imperative execute steps=] | ||||||||||||||
| given |tool|, [=this=]'s [=relevant global object=], |inputArguments|, and |completionSteps|. | ||||||||||||||
|
|
||||||||||||||
| : [=tool definition/annotations=] | ||||||||||||||
| :: null if |tool|'s {{ModelContextTool/annotations}} does not [=map/exist=]. Otherwise, an | ||||||||||||||
|
|
@@ -472,8 +511,8 @@ The <dfn method for=ModelContext>registerTool(<var>tool</var>, <var>options</var | |||||||||||||
|
|
||||||||||||||
| 1. [=Notify documents of a tool change=] given |tool owner| and |exposed origins|. | ||||||||||||||
|
|
||||||||||||||
| 1. [=Queue a global task=] on the [=webmcp task source=] given |global| to | ||||||||||||||
| [=resolve=] |promise| with undefined. | ||||||||||||||
| 1. [=Queue a global task=] on the [=webmcp task source=] given |global| to [=resolve=] |promise| | ||||||||||||||
| with undefined. | ||||||||||||||
|
|
||||||||||||||
| 1. Return |promise| | ||||||||||||||
|
|
||||||||||||||
|
|
@@ -590,6 +629,107 @@ The <dfn method for=ModelContext>getTools(<var>options</var>)</dfn> method steps | |||||||||||||
|
|
||||||||||||||
| </div> | ||||||||||||||
|
|
||||||||||||||
| <div algorithm> | ||||||||||||||
| The <dfn method for=ModelContext>executeTool(<var>tool</var>, <var>inputArguments</var>, | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A JS example of how to execute a tool from a cross-origin iframe may come handy in this spec. See previous comment at #223 (comment) |
||||||||||||||
| <var>options</var>)</dfn> method steps are: | ||||||||||||||
|
|
||||||||||||||
| 1. Let |invokingDocument| be [=this=]'s [=relevant global object=]'s [=associated | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: Shall we use |
||||||||||||||
| Document|associated <code>Document</code>=]. | ||||||||||||||
|
|
||||||||||||||
| 1. If |invokingDocument| is not [=Document/fully active=], then return [=a promise rejected with=] | ||||||||||||||
| an "{{InvalidStateError}}" {{DOMException}}. | ||||||||||||||
|
|
||||||||||||||
| 1. If this's [=surrounding agent=]'s [=agent cluster=]'s [=is origin-keyed=] is false and this's | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
This suggestion applies as well to getTools and registerTool. |
||||||||||||||
| [=relevant settings object=]'s [=environment settings object/origin=]'s [=origin/scheme=] is not | ||||||||||||||
| "<code>file</code>", then return [=a promise rejected with=] a "{{SecurityError}}" | ||||||||||||||
| {{DOMException}}. | ||||||||||||||
|
|
||||||||||||||
| 1. If |invokingDocument| is not [=allowed to use=] the "{{tools}}" feature, then return [=a promise | ||||||||||||||
| rejected with=] a "{{NotAllowedError}}" {{DOMException}}. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |expectedTargetOriginURL| be the result of [=URL parser|parsing=] |tool|'s | ||||||||||||||
| {{RegisteredTool/origin}}. | ||||||||||||||
|
|
||||||||||||||
| 1. If |expectedTargetOriginURL| is failure, then return [=a promise rejected with=] a | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How about using
|
||||||||||||||
| "{{DataError}}" {{DOMException}}. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |expectedTargetOrigin| be |expectedTargetOriginURL|'s [=url/origin=]. | ||||||||||||||
|
|
||||||||||||||
| 1. [=Assert=]: |expectedTargetOrigin| is not an [=opaque origin=]. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |promise| be [=a new promise=] created in [=this=]'s [=relevant realm=]. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |signal| be |options|'s {{ExecuteToolOptions/signal}} if it [=map/exists=]; otherwise null. | ||||||||||||||
|
|
||||||||||||||
| 1. If |signal| is not null, then: | ||||||||||||||
|
Comment on lines
+662
to
+664
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
|
|
||||||||||||||
| 1. If |signal| is [=AbortSignal/aborted=], then return [=a promise rejected with=] |signal|'s | ||||||||||||||
| [=AbortSignal/abort reason=]. | ||||||||||||||
|
|
||||||||||||||
| 1. [=AbortSignal/add|Add the following abort steps=] to |signal|: | ||||||||||||||
|
|
||||||||||||||
| 1. [=Reject=] |promise| with |signal|'s [=AbortSignal/abort reason=]. | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should also have an issue to spec and fire |
||||||||||||||
|
|
||||||||||||||
| Issue(#48): Wire up |signal| to the tool execution callback in the tool host's document, so that tool abort is communicated all the way through. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |targetWindow| be |tool|'s {{RegisteredTool/window}}. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |targetDocument| be |targetWindow|'s [=associated Document|associated | ||||||||||||||
| <code>Document</code>=]. | ||||||||||||||
|
|
||||||||||||||
| 1. Run the following steps [=in parallel=]: | ||||||||||||||
|
|
||||||||||||||
| 1. If |targetDocument|'s [=node navigable=]'s [=navigable/traversable navigable=] is not | ||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we check that this document is fully active? |
||||||||||||||
| |invokingDocument|'s [=node navigable=]'s [=navigable/traversable navigable=], then [=queue a | ||||||||||||||
| global task=] on the [=webmcp task source=] given |invokingDocument|'s [=relevant global | ||||||||||||||
| object=] to [=reject=] |promise| with an "{{UnknownError}}" {{DOMException}}, and abort these | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. UnknownError feels weird to me in these cases. Maybe InvalidStateError? |
||||||||||||||
| steps. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |targetOrigin| be |targetDocument|'s [=Document/origin=]. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |callerOrigin| be |invokingDocument|'s [=Document/origin=]. | ||||||||||||||
|
|
||||||||||||||
| 1. If |targetOrigin| is not [=same origin=] with |expectedTargetOrigin|, then [=queue a global | ||||||||||||||
| task=] on the [=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to | ||||||||||||||
| [=reject=] |promise| with an "{{UnknownError}}" {{DOMException}}, and abort these steps. | ||||||||||||||
|
|
||||||||||||||
| Issue: Support more granular errors than "{{UnknownError}}", based on each failure case. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |targetToolMap| be |targetDocument|'s [=Document/associated ModelContext|associated | ||||||||||||||
| <code>ModelContext</code>=]'s [=ModelContext/internal context=]'s [=model context/tool map=]. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |toolName| be |tool|'s {{RegisteredTool/name}}. | ||||||||||||||
|
|
||||||||||||||
| 1. If |targetToolMap|[|toolName|] does not [=map/exist=], then [=queue a global task=] on the | ||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Weird case we could get into here:
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep, some modifications are OK, others are not. I'll be carrying over some from our implementation before taking this PR out of draft mode. |
||||||||||||||
| [=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to [=reject=] | ||||||||||||||
| |promise| with an "{{UnknownError}}" {{DOMException}}, and abort these steps. | ||||||||||||||
|
|
||||||||||||||
| Issue: Support more granular errors than "{{UnknownError}}", based on each failure case. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |tool definition| be |targetToolMap|[|toolName|]. | ||||||||||||||
|
|
||||||||||||||
| 1. If [=tool is exposed to an origin=] given |targetOrigin|, |tool definition|'s [=tool | ||||||||||||||
| definition/exposed origins=], and |callerOrigin| is false, then [=queue a global task=] on the | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
| [=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to [=reject=] | ||||||||||||||
| |promise| with an "{{UnknownError}}" {{DOMException}}, and abort these steps. | ||||||||||||||
|
|
||||||||||||||
| Issue: Support more granular errors than "{{UnknownError}}", based on each failure case. | ||||||||||||||
|
|
||||||||||||||
| 1. Let |completionSteps| be an algorithm that takes a [=string=]-or-null |result| and a [=boolean=] |success|, and runs the following steps: | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: How about we switch params order?
Suggested change
|
||||||||||||||
|
|
||||||||||||||
| 1. [=Assert=]: these steps are running [=in parallel=]. | ||||||||||||||
|
|
||||||||||||||
| 1. If |success| is true, then [=queue a global task=] on the [=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to [=resolve=] |promise| with |result|. | ||||||||||||||
|
|
||||||||||||||
| 1. Otherwise, [=queue a global task=] on the [=webmcp task source=] given |invokingDocument|'s [=relevant global object=] to [=reject=] |promise| with an "{{UnknownError}}" {{DOMException}}. | ||||||||||||||
|
|
||||||||||||||
| 1. [=Queue a global task=] on the [=webmcp task source=] given |targetWindow| to run |tool | ||||||||||||||
| definition|'s [=tool definition/execute steps=] given |inputArguments| and |completionSteps|. | ||||||||||||||
|
|
||||||||||||||
| 1. Return |promise|. | ||||||||||||||
|
|
||||||||||||||
| </div> | ||||||||||||||
|
|
||||||||||||||
| <h4 id="model-context-tool">ModelContextTool Dictionary</h4> | ||||||||||||||
|
|
||||||||||||||
| The {{ModelContextTool}} dictionary describes a tool that can be invoked by [=agents=]. | ||||||||||||||
|
|
@@ -617,19 +757,21 @@ callback ToolExecuteCallback = Promise<any> (object input); | |||||||||||||
| <dl class="domintro"> | ||||||||||||||
| <dt><code><var ignore>tool</var>["{{ModelContextTool/name}}"]</code></dt> | ||||||||||||||
| <dd> | ||||||||||||||
| <p>A unique identifier for the tool. This is used by [=agents=] to reference the tool when making tool calls. | ||||||||||||||
| <p>A unique identifier for the tool. This is used by [=agents=] to reference the tool when | ||||||||||||||
| making tool calls. | ||||||||||||||
| </dd> | ||||||||||||||
|
|
||||||||||||||
| <dt><code><var ignore>tool</var>["{{ModelContextTool/title}}"]</code></dt> | ||||||||||||||
| <dd> | ||||||||||||||
| <p>A label for the tool. This is used by the user agent to reference the tool in the user interface. | ||||||||||||||
| <p>It is recommended that this string be localized to the user's | ||||||||||||||
| {{NavigatorLanguage/language}}. | ||||||||||||||
| <p>A label for the tool. This is used by the user agent to reference the tool in the user | ||||||||||||||
| interface. <p>It is recommended that this string be localized to the user's | ||||||||||||||
| {{NavigatorLanguage/language}}. | ||||||||||||||
| </dd> | ||||||||||||||
|
|
||||||||||||||
| <dt><code><var ignore>tool</var>["{{ModelContextTool/description}}"]</code></dt> | ||||||||||||||
| <dd> | ||||||||||||||
| <p>A natural language description of the tool's functionality. This helps [=agents=] understand when and how to use the tool. | ||||||||||||||
| <p>A natural language description of the tool's functionality. This helps [=agents=] understand | ||||||||||||||
| when and how to use the tool. | ||||||||||||||
| </dd> | ||||||||||||||
|
|
||||||||||||||
| <dt><code><var ignore>tool</var>["{{ModelContextTool/inputSchema}}"]</code></dt> | ||||||||||||||
|
|
@@ -700,6 +842,22 @@ dictionary ModelContextGetToolOptions { | |||||||||||||
| same-origin documents. | ||||||||||||||
| </dl> | ||||||||||||||
|
|
||||||||||||||
| <h4 id="execute-tool-options">ExecuteToolOptions Dictionary</h4> | ||||||||||||||
|
|
||||||||||||||
| The {{ExecuteToolOptions}} dictionary allows web applications to pass options to | ||||||||||||||
| {{ModelContext/executeTool()}}. | ||||||||||||||
|
|
||||||||||||||
| <xmp class="idl"> | ||||||||||||||
| dictionary ExecuteToolOptions { | ||||||||||||||
| AbortSignal signal; | ||||||||||||||
| }; | ||||||||||||||
| </xmp> | ||||||||||||||
|
|
||||||||||||||
| <dl class="domintro"> | ||||||||||||||
| : <code><var ignore>options</var>["{{ExecuteToolOptions/signal}}"]</code> | ||||||||||||||
| :: An {{AbortSignal}} that can be used to cancel the execution of the tool. | ||||||||||||||
| </dl> | ||||||||||||||
|
|
||||||||||||||
| <h4 id="registered-tool">RegisteredTool Dictionary</h4> | ||||||||||||||
|
|
||||||||||||||
| The {{RegisteredTool}} dictionary represents a tool that has been registered and is available for | ||||||||||||||
|
|
||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not add this webidl bit to
<pre class="link-defaults">at the top of the file?