[Feature request] Allow the agent to see the figure

[Ebola-Chan-bot]

Many modern models have visual capabilities. I want the agent to see the figure, e.g., a new tool show_PNG_figure that convert a figure number to a PNG data and return to the agent.

[ybenouag]

Hi, thanks for the feature request!

In nodesktop mode, this is actually already supported — figures are captured as images and passed directly back to the LLM, so the agent can see them in the conversation.

We're also working on bringing this capability to desktop mode in an upcoming release.

Does this cover your use case, or do you have a scenario where you'd specifically need the figure exported to a file as well? Happy to hear more about how you're using it.

— Yannis

[Ebola-Chan-bot]

No, I need to use AI along with the desktop window. I don't understand why don't you just let the desktop session pass the image to the agent, as it is already done so in nodesktop mode.