Skip to content

Commit 6625d56

Browse files
authored
(docs) Организация и аннотирование тестов (#130)
* docs(organizing): Статья про организацию и аннотирование тестов (русский язык) --------- Authored-by: kseniataranov <kseniataranov@yandex-team.ru>
1 parent 58ee073 commit 6625d56

4 files changed

Lines changed: 458 additions & 0 deletions

File tree

Lines changed: 229 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,229 @@
1+
import Admonition from "@theme/Admonition";
2+
3+
# Organizing and Annotating Tests
4+
5+
<Admonition title="What you will learn">
6+
7+
- How to group tests into sets for running depending on platform and scenario
8+
- How to use tags for filtering tests
9+
- How to add metadata for reports and debugging
10+
11+
</Admonition>
12+
13+
## Introduction
14+
15+
As a project grows, the classic `describe/it` structure becomes insufficient: hundreds of tests need to be run selectively in different browsers, grouped by functional areas, distributed across CI/CD jobs, and managed for flakiness. To solve these problems, Testplane offers sets (grouping tests by files and browsers), tags (marking tests for filtering), metadata (arbitrary data for reports), and integration with GUI/HTML-reporter for visualization and interactive work with results.
16+
17+
## Splitting Tests into Sets
18+
19+
A set is a named group of tests that defines which files to run and in which browsers. This is the main tool for separating tests by environments and usage scenarios.
20+
21+
Sets are described in the `sets` section of the [configuration file](./configuration.mdx). If sets are not specified, Testplane looks for tests in the `testplane/` folder.
22+
23+
For each set, you can specify parameters:
24+
25+
- `files` (required) — path or array of paths to test files (supports glob patterns)
26+
- `browsers` — array of browser identifiers for running tests of this set (all from configuration by default)
27+
- `ignoreFiles` — array of patterns to exclude files from the set (e.g., WIP tests)
28+
29+
```typescript title="testplane.config.ts"
30+
import type { ConfigInput } from "testplane";
31+
32+
export default {
33+
sets: {
34+
desktop: {
35+
files: "tests/desktop/**/*.test.ts",
36+
ignoreFiles: ["tests/desktop/**/*.wip.ts"],
37+
browsers: ["chrome", "firefox", "safari"],
38+
},
39+
mobile: {
40+
files: "tests/mobile/**/*.test.ts",
41+
browsers: ["iphone", "android"],
42+
},
43+
all: {
44+
files: "tests/**/*.test.ts",
45+
// browsers not specified — all browsers from config.browsers are used
46+
},
47+
},
48+
// ...
49+
} satisfies ConfigInput;
50+
```
51+
52+
### Running from Command Line
53+
54+
```bash
55+
# Run all desktop tests
56+
npx testplane --set desktop
57+
58+
# Run multiple sets simultaneously
59+
npx testplane --set desktop --set mobile
60+
61+
# Run desktop tests only in chrome
62+
npx testplane --set desktop --browser chrome
63+
64+
# Run tests by pattern from set in specific browser
65+
npx testplane --set desktop --browser chrome --grep "payment"
66+
67+
# Without specifying a set, all tests from all sets will run
68+
npx testplane
69+
```
70+
71+
<Admonition type="tip">
72+
Split tests by features (one file — one function). This allows you to flexibly combine files in
73+
different sets and avoid huge files with thousands of lines.
74+
</Admonition>
75+
76+
Sets from the Testplane configuration file are not displayed in the GUI [html-reporter](../html-reporter/overview.mdx), as they are a test execution mechanism, not part of the test results.
77+
78+
## Using Tags
79+
80+
A tag is a string label that can be assigned to a test or group of tests. Tags allow filtering tests at runtime regardless of their location in the file system or `describe/it` structure.
81+
82+
### Static Tags
83+
84+
Tags are set in test or suite options as the second argument in the `tag` field. All tests inside a suite inherit the tag assigned to the suite.
85+
86+
```typescript
87+
describe("Search", { tag: "search" }, function () {
88+
it("Opening search", async ({ browser }) => {
89+
// tag: search
90+
});
91+
92+
it("Performing search", { tag: "smoke" }, async ({ browser }) => {
93+
// tags: search, smoke
94+
});
95+
96+
it("Filtering search results", { tag: ["smoke", "critical"] }, async ({ browser }) => {
97+
// tags: search, smoke, critical
98+
});
99+
});
100+
```
101+
102+
### Dynamic Tags
103+
104+
Static tags are set before execution and describe the nature of the test. But some properties become known only during or after execution. For such cases, there is the [`browser.addTag()`](../commands/browser/addTag.mdx) method, which adds a tag to an already running test, and the tag is saved to the report.
105+
106+
A typical scenario is automatic marking of slow tests:
107+
108+
```typescript
109+
it("Click: open search", { tag: ["critical", "ui", "search"] }, async ({ browser }) => {
110+
const startTime = Date.now();
111+
112+
await browser.url("https://testplane.io/");
113+
114+
// ...
115+
116+
const duration = Date.now() - startTime;
117+
if (duration > 1000) {
118+
await browser.addTag("slow");
119+
}
120+
if (duration > 2000) {
121+
await browser.addTag("very-slow");
122+
}
123+
});
124+
```
125+
126+
After the run, the test will be marked in the report with `slow` and `very-slow` tags: you can filter results by them and decide whether to move the test to a separate set with lower run frequency.
127+
128+
### Running from Command Line
129+
130+
```bash
131+
# Run only tests with the smoke tag
132+
npx testplane --tag smoke
133+
134+
# Run tests with the smoke OR critical tag
135+
npx testplane --tag "smoke|critical"
136+
137+
# Run tests with the smoke AND auth tags
138+
npx testplane --tag "smoke&auth"
139+
140+
# Exclude tests with the slow tag
141+
npx testplane --tag "!slow"
142+
143+
# Smoke tests only on mobile browsers
144+
npx testplane --set mobile --tag smoke
145+
```
146+
147+
### Tags in GUI
148+
149+
In [html-reporter](../html-reporter/overview.mdx), tags are visible in the test card. You can filter and search by them, for example, find all failed critical tests. Dynamic tags are visually different from static ones.
150+
151+
![Tags in Testplane GUI](/img/docs/guides/organizing-tests/tags.png)
152+
153+
## Working with Metadata
154+
155+
Metadata is an arbitrary set of keys and values that can be attached to a test. Unlike tags, metadata is not used for filtering at runtime: it is intended for debugging and analytics, so that from the run results it is clear in what context the test was executed.
156+
157+
Two methods are used for writing and reading metadata:
158+
159+
- [`browser.setMeta(key, value)`](../commands/browser/setMeta.mdx) — saves a value under the specified key
160+
- [`browser.getMeta(key?)`](../commands/browser/getMeta.mdx) — returns the value by key. If no key is specified, returns the entire metadata object.
161+
162+
The most common use case for metadata is saving a link to a ticket:
163+
164+
```typescript
165+
it("Check search functionality", async ({ browser }) => {
166+
await browser.setMeta("issue", "TICKET-123");
167+
168+
await browser.url("https://testplane.io/");
169+
170+
const issue = await browser.getMeta("issue");
171+
172+
const meta = await browser.getMeta();
173+
});
174+
```
175+
176+
In the configuration file, you can set metadata for all tests of a browser:
177+
178+
```typescript title="testplane.config.ts"
179+
import type { ConfigInput } from "testplane";
180+
181+
export default {
182+
browsers: {
183+
chrome: {
184+
meta: {
185+
platform: "desktop",
186+
},
187+
},
188+
"chrome-mobile": {
189+
meta: {
190+
platform: "mobile",
191+
},
192+
},
193+
},
194+
// ...
195+
} satisfies ConfigInput;
196+
```
197+
198+
The `url` key is filled automatically by Testplane: it stores the page address where the browser was at the moment the test finished.
199+
200+
```typescript
201+
it("tracks URL", async ({ browser }) => {
202+
await browser.url("/foo/bar?baz=qux");
203+
204+
const url = await browser.getMeta("url");
205+
console.log(url); // '/foo/bar?baz=qux'
206+
});
207+
```
208+
209+
### Metadata in GUI
210+
211+
Test metadata is displayed in the result card in the **Metadata** section:
212+
213+
![Metadata in Testplane GUI](/img/docs/guides/organizing-tests/meta.png)
214+
215+
If you need values in the Metadata section to be displayed as clickable links, configure `metaInfoBaseUrls` in the HTML reporter config:
216+
217+
```typescript title="testplane.config.ts"
218+
export default {
219+
plugins: {
220+
"html-reporter/testplane": {
221+
metaInfoBaseUrls: {
222+
// The "issue" key will be displayed as a link:
223+
// https://tracker.company.com/TICKET-123
224+
issue: "https://tracker.company.com/",
225+
},
226+
},
227+
},
228+
};
229+
```

0 commit comments

Comments
 (0)