Skip to content

Commit 26433d4

Browse files
shadowusrshadowusr
authored
feat: create ai section and write cli, skill, ai overview articles (#128)
* docs: tiny fixes of blog articles and typescript-esm articles * feat: create ai section and write ai toolkit articles
1 parent da25a2f commit 26433d4

27 files changed

Lines changed: 2637 additions & 88 deletions

File tree

‎blog/cli-and-skill.mdx‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
title: Testplane CLI and Skill
3+
slug: cli-and-skill
4+
hide_table_of_contents: false
5+
date: 2026-05-05T17:00
6+
---
7+
8+
asdf

‎blog/testplane-vs-playwright.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ import Admonition from "@theme/Admonition";
1212

1313
И **Testplane**, и **Playwright** — это решения для end-to-end тестирования, но они имеют разные подходы и возможности. В этой статье мы проведём детальное сравнение этих инструментов, чтобы помочь вам сделать правильный выбор.
1414

15+
<!-- truncate -->
16+
1517
## Краткий обзор инструментов
1618

1719
**Testplane** — это инструмент для автоматизации тестирования веб-приложений, разработанный командой Яндекса. Он активно используется как в крупных, так и в небольших проектах компании и поддерживает широкий спектр браузеров и устройств.

‎docs/ai/overview.mdx‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
---
2+
sidebar_position: 1
3+
sidebar_label: Overview
4+
title: Testplane's AI Tools
5+
---
6+
7+
import { HelpMark } from "@site/src/components/HelpMark";
8+
9+
# Testplane's AI Tools
10+
11+
Testplane provides a set of AI integration tools that help agents write, inspect, debug, and fix your Testplane tests.
12+
13+
The goal of these tools is to turn a generic coding agent into Testplane expert that understands how Testplane projects are usually structured, how tests should be written, and how failures should be investigated.
14+
15+
## Capabilities
16+
17+
- Let AI agents explore your app and "see" what's happening through token-efficient DOM snapshots
18+
- Generate new tests based on the actual application state
19+
- See how agents follow best practices when writing tests out of the box
20+
- Handle authentication without reading secrets
21+
- Enable agents investigate failures from CI reports
22+
- Let agents debug tests via REPL
23+
24+
## Available Tools
25+
26+
[Testplane Skill](../toolkit/testplane-skill) teaches AI agents how to work with Testplane projects end to end, how to use `@testplane/cli`. It's packed with best practices and comprehensive documentation that agents can make use of.
27+
28+
[Testplane CLI](../toolkit/testplane-cli) is what powers the Testplane Skill and makes browser automation accessible right from the CLI.
29+
30+
[Testplane MCP](../toolkit/testplane-mcp) matches the capabilities of `@testplane/cli`, but offers another way to interact with the browser: through the MCP protocol.
31+
32+
## Which tool to use?
33+
34+
If unsure, prefer Testplane Skill. It works great for most workflows and is the best pick overall: coding agents love CLIs, it's token-efficient and provides enough context about all things Testplane.
35+
36+
<table>
37+
<thead>
38+
<tr>
39+
<th></th>
40+
<th>Skill</th>
41+
<th>CLI</th>
42+
<th>MCP</th>
43+
</tr>
44+
</thead>
45+
<tbody>
46+
<tr>
47+
<td>**How it works**</td>
48+
<td>
49+
LLM reads a markdown file with instructions, reaches out to `@testplane/cli` for
50+
browser automation
51+
</td>
52+
<td>LLM runs shell commands to interact with the browser</td>
53+
<td>LLM interacts with the browser via MCP tool calls</td>
54+
</tr>
55+
<tr>
56+
<td>**Token efficiency**</td>
57+
<td>High, progressive discovery</td>
58+
<td>High, concise CLI calls</td>
59+
<td>
60+
Lower, depends on the agent harness{" "}
61+
<HelpMark side="right">
62+
Some agents harnesses may load all MCP tool schemas at once, polluting the
63+
context and spending tokens.
64+
<br />
65+
<br />
66+
Some tools, however, optimise this step by pre-generating tool schemas and loading
67+
them lazily, in which case token usage is comparable to CLI tools.
68+
</HelpMark>
69+
</td>
70+
</tr>
71+
<tr>
72+
<td>**Default browser mode**</td>
73+
<td>Headless</td>
74+
<td>Headless</td>
75+
<td>Headful</td>
76+
</tr>
77+
<tr>
78+
<td>**Supported scenarios**</td>
79+
<td>
80+
Complex workflows + browser automation{" "}
81+
<HelpMark side="right">
82+
Besides offering browser automation capabilities, skill teaches agents how to
83+
handle auth, debugging, choose best code style, understand common Testplane
84+
patterns and more.
85+
</HelpMark>
86+
</td>
87+
<td>Interaction with browser, reports inspection</td>
88+
<td>Same as CLI</td>
89+
</tr>
90+
</tbody>
91+
</table>
92+
93+
## Comparison with other solutions
94+
95+
Overall, we recommend using Testplane AI toolkit when working with Testplane projects, because it was designed specifically around this use case and provides deep integrations with Testplane ecosystem, allowing agents to:
96+
97+
- Read what Testplane code that was run on each Testplane CLI/MCP tool call and copy+paste it when writing tests
98+
- Analyze Testplane reports (both local and from CI) and fix failed tests
99+
- Inspect Time Travel snapshots to debug what went wrong and adjust tests as your app evolves
100+
- Debug tests with agents-friendly REPL debugger
101+
- Understand Testplane's capabilities and usage patterns through concise reference in Testplane Skill
102+
103+
We've studied other tools for agentic browser automation — below is a brief comparison of the most notable projects in the field, hopefully helping you navigate through possible options.
104+
105+
### Built-in browser in Cursor, CC, Cline, etc.
106+
107+
[Cursor's Browser](https://cursor.com/docs/agent/tools/browser) and [Cline's Browser Automation](https://docs.cline.bot/tools-reference/browser-automation) are built-in ways to interact with the browser, providing tools for navigation, taking page snapshots, various interactions and reading browser's console logs.
108+
109+
Cursor's built-in browser has a neat design sidebar that allows you to quickly modify your app's design and point at certain elements when writing prompts.
110+
111+
However, these tools have major limitations when trying to use them for web apps testing:
112+
113+
- Cline doesn't have DOM snapshot tool at all — it relies solely on screenshots
114+
- Cursor's snapshot tool returns accessibility tree which lacks CSS classes, almost all HTML attributes and has a flat structure, losing all DOM hierarchy data
115+
- They offer very limited support for browsers: only Chrome with no mobile emulation and no special capabilities
116+
- No way of securely handling auth data
117+
118+
### Playwright
119+
120+
[Playwright CLI](https://github.com/microsoft/playwright-cli) is a powerful and popular tool for browser automation. Its notable features include network requests inspection and devtools capabilities.
121+
122+
The main limitations are:
123+
124+
- No report analysis, Time Travel snapshot inspection, or REPL debugging
125+
- Tool responses are focused on Playwright code, which may confuse agents working with Testplane
126+
- Snapshots are based on accessibility trees, without CSS classes, precise DOM structure, or many useful HTML attributes
127+
128+
Playwright CLI is a popular choice, but it lacks several features that unlock powerful workflows and it doesn't have the Testplane-specific context.
129+
130+
### Chrome DevTools MCP
131+
132+
[Chrome Devtools MCP](https://github.com/ChromeDevTools/chrome-devtools-mcp) besides regular browser tools provides niche features like taking memory snapshots and browser extension management.
133+
134+
For Testplane projects, the main limitations are:
135+
136+
- It is specific to Chrome
137+
- Snapshots are based on the accessibility tree, without CSS classes or precise DOM structure
138+
- It doesn't integrate with Testplane tooling, reports, Time Travel snapshots, or the Testplane REPL
139+
140+
Overall, it provides deep low-level integration with Chrome's devtools capabilities, but is not focused on web apps testing and doesn't have integrations with testplane tooling.
141+
142+
### Vercel's agent-browser
143+
144+
Vercel's [agent-browser](https://github.com/vercel-labs/agent-browser) is a generic browser automation, with a large set of tools, including network inspection and profiling.
145+
146+
Its strength is flexibility, but it is not tied to any particular test runner or testing workflow.
147+
148+
For Testplane usage, this means:
149+
150+
- It is not focused specifically on web app testing
151+
- Snapshots are based on accessibility trees
152+
- It doesn't integrate with Testplane tooling, reports, Time Travel snapshots, or the Testplane REPL
153+
154+
agent-browser can be useful for general browser automation, but it lacks the Testplane-specific integrations agents need to write, debug, and fix tests effectively.
155+
156+
## Staying informed
157+
158+
Stumbled upon a bug, have a question or an idea? Let us now:
159+
160+
- [Telegram](https://t.me/testplane)
161+
- [GitHub](https://github.com/gemini-testing/testplane)
162+
163+
You can track new releases [over here](https://github.com/gemini-testing/testplane-mcp/releases).

‎docs/ai/toolkit/_category_.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"label": "Toolkit",
3+
"collapsed": false
4+
}

0 commit comments

Comments
 (0)