You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/docs/v3.mdx
+19Lines changed: 19 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,6 +13,7 @@ description: Major release focused on extensibility, expanded provider support,
13
13
|**Gemini RAG**| Manage Gemini File Search Stores and manage document uploads for RAG workflows |
14
14
|**Tool Support**| First-class Python function calling for LLM interactions with your local environment |
15
15
|**MCP Support**| Connect to Model Context Protocol servers for extended tool capabilities |
16
+
|**Computer Use**| Desktop automation - control mouse, keyboard, and take screenshots like a human |
16
17
|**KaTeX Math Typesetting**| Support for beautiful rendering of LaTeX math expressions |
17
18
|**Calculator UI**| Beautiful UX Friendly UI to evaluate python math expressions |
18
19
|**Run Code UI**| Execute Python, JS, TypeScript and C# code scripts in a CodeMirror editor |
@@ -34,6 +35,7 @@ description: Major release focused on extensibility, expanded provider support,
34
35
-[Tool Support](#tool-support)
35
36
-[MCP Support](#mcp-support)
36
37
-[Core Tools](#core-tools)
38
+
-[Computer Use](#computer-use)
37
39
-[Calculator UI](#calculator-ui)
38
40
-[Run Code UI](#run-code-ui)
39
41
-[KaTeX Math Typesetting](#katex-math-typesetting)
@@ -555,6 +557,23 @@ LLMS includes a suite of tools for executing code in various languages within a
555
557
'Run C#': '/img/tools/tool-csharp.webp',
556
558
}} />
557
559
560
+
## Computer Use
561
+
562
+
The built-in [computer_use](https://github.com/ServiceStack/llms/tree/main/llms/extensions/computer_use) extension transforms AI agents into autonomous computer operators. Based on [Anthropic's computer use tools](https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo), it enables agents to see your screen, control the mouse and keyboard, execute shell commands, and edit files - just like a human sitting at the computer.
563
+
564
+
This unlocks powerful capabilities that traditional API-based tools cannot achieve:
565
+
566
+
-**Visual Verification**: Confirm that code actually renders correctly in a browser
567
+
-**Desktop Automation**: Control any GUI application - web browsers, IDEs, terminals
568
+
-**End-to-End Workflows**: Chain together multiple applications in a single task
569
+
-**Legacy Applications**: Automate software that lacks APIs
570
+
571
+
For example, an agent can write a web application, open a browser, and capture a screenshot to prove it works:
See the [Computer Use](/docs/extensions/computer_use) docs for complete usage details.
576
+
558
577
## Calculator UI
559
578
560
579
As some core tools are particularly useful on their own, dedicated UIs has been added for the `calc` tool with support for evaluating mathematical python expressions, including arithmetic, comparison, boolean operators, `math.*` functions & constants and python list comprehensions
0 commit comments