Conversation
markafoltz
left a comment
markafoltz
left a comment
There was a problem hiding this comment.
A few nit picks, it looks like the spec generator API threw a few errors to clean up.
| view of the page's tools and any other relevant context by getting an [=observation=]. An | ||
| <dfn>observation</dfn> is an [=implementation-defined=] data structure containing at least a <dfn | ||
| for=observation>tool map</dfn>, which is a [=map=] whose [=map/keys=] are [=Document/unique ID=]s, | ||
| and whose [=map/values=] are [=tool definitions=]. |
There was a problem hiding this comment.
I would think this needs to be doubly keyed by (document_id, tool_name) at a minimum?
There was a problem hiding this comment.
That'd only be necessary if within a single Document, you could have multiple tools with the same name. But that shouldn't be possible, so keying on document ID means all of the tools associated with that document will be unique. Across multiple Documents, you can have duplicate tool names, but that should be taken care of. Does that sound right or am I missing something?
| Each {{Document}} object has a <dfn for=Document>unique ID</dfn>, which is a [=unique internal | ||
| value=]. | ||
|
|
||
| The times at which a [=browser agent=] [=performs an observation=] are [=implementation-defined=]. |
There was a problem hiding this comment.
We may need to be more proscriptive here; if tools are registered or unregistered, or frames enter or leave the frame tree, those should trigger a re-observation.
There was a problem hiding this comment.
I agree. My plan is this: when getTools() exists, whenever we queue tasks on the AI agent queue to update the browser process map of all tools, I'll also add a step about optionally performing observations then. So I was thinking of holding off on that until that infra is laid. How does that sound?
|
|
||
| 1. Perform any [=implementation-defined=] steps with |observation| and the [=browser agent=], to | ||
| expose the |observation|'s [=observation/tool map=] to the [=browser agent=] in whatever way it | ||
| accepts. |
There was a problem hiding this comment.
I expect this will also update the tool map exposed to other frames in listTools() once that is spec'ed out? Otherwise I am wondering how we will measure interop since the side effect is only exposed to the browser agent.
There was a problem hiding this comment.
Actually performing an observation will not have any web-observable effects. When I spec the Promise-returning getTools(), we'll have to move to a central browser process location of all combined tools in the frame tree. Then, the page observation logic here will move from doing what APC does in Chromium now (gathering all tools from the tree) to just taking a copy of that browser-side map, and packaging it up here in some way for the observation.
So in short, this is not the map that getTools() will consult. This is just the map for observations. And its instantiation will be simplified once getTools() exists imminently.
|
|
||
| The [=user agent=]'s [=browser's agent=] runs [=in parallel=] to any [=event loops=] associated | ||
| with a {{ModelContext}} [=relevant global object=]. Steps running on the [=browser's agent=] get | ||
| queued on its <dfn>AI agent queue</dfn>, which is the result of [=starting a new parallel queue=]. |
There was a problem hiding this comment.
| queued on its <dfn>AI agent queue</dfn>, which is the result of [=starting a new parallel queue=]. | |
| queued on its <dfn>browser agent queue</dfn>, which is the result of [=starting a new parallel queue=]. |
There was a problem hiding this comment.
Since we're using the AI task source in the renderer/Document's event loop, I'd like to keep the browser side equivalent also prefixed with "AI", to mirror this. Is that OK?
2 participants
This PR adds basic "page observation" and scheduling infrastructure, to lay the ground work for browser-agent-invoked tool execution. After this PR, I'll create one for tool execution, to unblock #146.
Preview | Diff