Add consequentialHint to ToolAnnotations

[beaufortfrancois]

Fix #176

---

Preview | Diff

[beaufortfrancois]

@domfarolino Please review

[NoClueMike]

When did "hints" deserve the time spent over actual communication. In the clients world, they can't even enable api"s for tools.

[victorhuangwq]

@beaufortfrancois thanks for adding this!

On a broader note, I'm starting to think if all ToolAnnotations should default to null instead of just false?

For example, I'm not sure if consequential not being set is the same as false.

[johannhof]

Is null an option? I agree it seems better.

[domfarolino]

If we don't like the idea of defaulting hints to false, we could provide no default at all; in that case they'd just be missing from the ToolAnnotations dictionary (undefined, just like annotations.fakeHint) and wouldn't be provided to the model (or the model would be expecting them, and would have to "fill in" false).

But then we'd have three values to think about: undefined, true, and false. Tool authors might ask when to explicitly pass false vs. leave it undefined, and tool consumers (models) now need to think about whether there's significance in the third value—did the developer just mean to omit the hint, but passed in false explicitly? Or does explicit false mean something special that undefined doesn't capture??

The required/default boolean sidesteps this noise, so I like it more. I think it just matters how this will be used by models, but I think the binary makes more sense and models are unlikely to make use of all three values.

[victorhuangwq]

My original concern was that from an agent developer's perspective, it's an easy mistake to read consequentialHint = false as "this is safe," when it might just mean the web developer forgot to set it (or didn't know they should). A third "unset" state would mitigate that, though I take your point about the noise a special third value introduces.

I agree that making it required sidesteps the "forgot to set" problem entirely. If we go that route, do we intend it to be required for all ToolAnnotations, or only a certain subset? (Btw I don't think this discussion should block the current PR)

[bvandersloot-mozilla]

There are more cases covered here than on line 670 where the dictionary fields are defined. These should align. Probably with this, more detailed, one.

[beaufortfrancois]

It's not clear to me what you suggest.

[bvandersloot-mozilla]

A ternary on the flag seems unwise. Particularly when undefined is falsy.

Assigning the safe default would be better than making them required, and is probably better than the current unsafe default (untrustedContentHint = false and consequentialHint = false) . Particularly because the failure cases have pretty different visibility and severity, e.g. increased friction vs silently taking a consequential action.