Skip to content

Commit 2f91d0e

Browse files
authored
enhancement(import): link JUnit/CLI results to existing cases by ID (#394)
* enhancement(import): link JUnit/CLI results to existing cases by ID Adds opt-in case-ID matching to the test-results importer so renaming an automated test no longer creates a duplicate case. A result can declare the existing case it belongs to via an ID in the test name (presets: brackets [123], C123, TC-123) or a test_id / testplanit_case_id JUnit property. When present, the result links to that case (scoped to the project) and skips name+className matching; multiple IDs link to multiple cases. Unknown IDs are skipped and surfaced as warnings rather than creating duplicates. Configured via the CLI --case-matcher / --case-id-format flags; default off, so existing imports are unchanged. Matching uses a fixed set of compiled presets only, so no caller-supplied regex runs server-side. * docs(import-export): enhance CSV and test results import instructions Updated the import-export documentation to clarify the process for importing test cases and results. Added details on using the Import button and drag-and-drop functionality, and improved formatting examples for test steps to include code blocks with the `text` language specified. This enhances readability and user understanding of the import features. * chore(workspace): add iframe-resizer to allowed builds Updated the pnpm workspace configuration to include 'iframe-resizer' in the list of allowed builds, enhancing the build process for the project.
1 parent 0c0ec51 commit 2f91d0e

11 files changed

Lines changed: 1001 additions & 71 deletions

File tree

cli/README.md

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Command-line interface for TestPlanIt - import test results and manage test data
77
The CLI supports importing test results from 7 different formats:
88

99
| Format | Description | File Extensions |
10-
|--------|-------------|-----------------|
10+
| -------- | ------------- | ----------------- |
1111
| `junit` | JUnit XML | `.xml` |
1212
| `testng` | TestNG XML | `.xml` |
1313
| `xunit` | xUnit XML | `.xml` |
@@ -25,7 +25,7 @@ The CLI can auto-detect the format based on file content, or you can specify it
2525
Download the appropriate binary for your platform from the [CLI releases](https://github.com/testplanit/testplanit/releases?q=cli&expanded=true):
2626

2727
| Platform | Binary |
28-
|----------|--------|
28+
| ---------- | -------- |
2929
| Linux (x64) | `testplanit-linux-x64` |
3030
| macOS (Apple Silicon) | `testplanit-macos-arm64` |
3131
| macOS (Intel) | `testplanit-macos-x64` |
@@ -97,9 +97,13 @@ testplanit import <files...> --project <id|name> --name <name> [options]
9797
- `-f, --folder <value>` - Parent folder for test cases (ID or exact name)
9898
- `-t, --tags <values>` - Tags (comma-separated IDs or names; use quotes for names with commas)
9999
- `-r, --test-run <value>` - Existing test run to append results (ID or exact name)
100+
- `--case-matcher <mode>` - Link results to existing cases by ID: `off`, `name`, `property`, `auto` (default: `off`)
101+
- `--case-id-format <preset>` - Case-ID pattern in the test name when matching by name: `brackets`, `c`, `tc` (default: `brackets`)
100102

101103
**Note:** For project, state, config, milestone, folder, and test run options, the CLI looks up entities by exact name match. If no match is found, an error is returned. For tags, if a tag name doesn't exist, it will be created automatically.
102104

105+
**Linking results to existing cases by ID:** By default a result is matched to a case by name and class name, so renaming a test creates a duplicate. With `--case-matcher`, a result instead links to an existing case by an ID in the test name (`--case-id-format`: `brackets``[123]`, `c``C123`, `tc``TC-123`) or a `test_id` / `testplanit_case_id` JUnit `<property>` (`property`/`auto`). Multiple IDs link one result to multiple cases; an ID not found in the project is skipped with a warning.
106+
103107
**Examples:**
104108

105109
```bash
@@ -121,6 +125,12 @@ testplanit import cucumber-report.json -p 1 -n "BDD Tests" -F cucumber
121125
# Import multiple files with glob pattern
122126
testplanit import "./test-results/*.xml" -p 1 -n "CI Build"
123127

128+
# Link results to existing cases by an ID in the test name (e.g. "[123] login")
129+
testplanit import results.xml -p 1 -n "Build 123" --case-matcher name
130+
131+
# Link by a test_id property in the XML, falling back to the name pattern
132+
testplanit import results.xml -p 1 -n "Build 123" --case-matcher auto
133+
124134
# Import with IDs
125135
testplanit import results.xml \
126136
--project 1 \

cli/src/commands/import.ts

Lines changed: 63 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,16 @@ import {
1818
formatFileSize,
1919
resolveTestRunAttachmentFiles,
2020
} from "../lib/attachments.js";
21-
import { TEST_RESULT_FORMATS, type TestResultFormat, type SSEProgressEvent, type ImportOptions } from "../types.js";
21+
import {
22+
TEST_RESULT_FORMATS,
23+
CASE_MATCHERS,
24+
CASE_ID_FORMATS,
25+
type TestResultFormat,
26+
type SSEProgressEvent,
27+
type ImportOptions,
28+
type CaseMatcher,
29+
type CaseIdFormat,
30+
} from "../types.js";
2231

2332
const VALID_FORMATS = ["auto", ...Object.keys(TEST_RESULT_FORMATS)] as const;
2433

@@ -42,6 +51,16 @@ export function createImportCommand(): Command {
4251
.option("-d, --attachments-dir <path>", "Base directory for resolving attachment paths (default: directory of test result file)")
4352
.option("--no-attachments", "Skip uploading attachments")
4453
.option("-a, --run-attachments <files...>", "Files to attach to the test run (e.g., test plans, reports)")
54+
.option(
55+
"--case-matcher <mode>",
56+
`Link results to existing cases by ID: ${CASE_MATCHERS.join(", ")} (default: off)`,
57+
"off"
58+
)
59+
.option(
60+
"--case-id-format <preset>",
61+
`Case-ID pattern in the test name when matching by name: ${CASE_ID_FORMATS.join(", ")} (default: brackets, e.g. [123])`,
62+
"brackets"
63+
)
4564
.addHelpText("after", `
4665
Examples:
4766
@@ -82,6 +101,15 @@ Examples:
82101
Import without uploading attachments:
83102
$ testplanit import ./results.xml -p "My Project" -n "Build" --no-attachments
84103
104+
Link results to existing cases by an ID in the test name (e.g. "[123] login"):
105+
$ testplanit import ./results.xml -p "My Project" -n "Build" --case-matcher name
106+
107+
Link by a test_id property in the XML, falling back to the name pattern:
108+
$ testplanit import ./results.xml -p "My Project" -n "Build" --case-matcher auto
109+
110+
Use the TestRail-style C-prefix pattern (e.g. "C123 login"):
111+
$ testplanit import ./results.xml -p "My Project" -n "Build" --case-matcher name --case-id-format c
112+
85113
Attach files to the test run (test plans, reports, etc.):
86114
$ testplanit import ./results.xml -p "My Project" -n "Build" -a ./test-plan.pdf ./coverage-report.html
87115
`)
@@ -108,6 +136,20 @@ Examples:
108136
process.exit(1);
109137
}
110138

139+
// Validate case-ID matching options
140+
const caseMatcher = options.caseMatcher.toLowerCase() as CaseMatcher;
141+
if (!CASE_MATCHERS.includes(caseMatcher)) {
142+
logger.error(`Invalid case matcher: ${options.caseMatcher}`);
143+
logger.info(`Valid matchers: ${CASE_MATCHERS.join(", ")}`);
144+
process.exit(1);
145+
}
146+
const caseIdFormat = options.caseIdFormat.toLowerCase() as CaseIdFormat;
147+
if (!CASE_ID_FORMATS.includes(caseIdFormat)) {
148+
logger.error(`Invalid case ID format: ${options.caseIdFormat}`);
149+
logger.info(`Valid formats: ${CASE_ID_FORMATS.join(", ")}`);
150+
process.exit(1);
151+
}
152+
111153
// Expand file patterns using glob
112154
const files: string[] = [];
113155
for (const pattern of filePatterns) {
@@ -171,6 +213,12 @@ Examples:
171213
format: format,
172214
};
173215

216+
// Only send case-ID matching config when enabled
217+
if (caseMatcher !== "off") {
218+
importOptions.caseMatcher = caseMatcher;
219+
importOptions.caseIdFormat = caseIdFormat;
220+
}
221+
174222
// Resolve state
175223
if (options.state) {
176224
logger.updateSpinner("Resolving workflow state...");
@@ -227,6 +275,20 @@ Examples:
227275
console.log();
228276
logger.success(`Test run created with ID: ${logger.formatNumber(result.testRunId)}`);
229277

278+
// Surface any results that referenced an unknown case ID (skipped)
279+
if (result.caseIdWarnings && result.caseIdWarnings.length > 0) {
280+
console.log();
281+
logger.warn(
282+
`Skipped ${logger.formatNumber(result.caseIdWarnings.length)} result(s) referencing a case ID not found in this project:`
283+
);
284+
for (const warning of result.caseIdWarnings.slice(0, 10)) {
285+
logger.dim(` - C${warning.requestedCaseId} (${warning.testName})`);
286+
}
287+
if (result.caseIdWarnings.length > 10) {
288+
logger.dim(` ... and ${result.caseIdWarnings.length - 10} more`);
289+
}
290+
}
291+
230292
// Handle attachment uploads if present and not disabled
231293
if (
232294
options.attachments !== false &&

cli/src/lib/api.ts

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,14 @@ export async function importTestResults(
143143
}
144144
}
145145

146+
if (options.caseMatcher) {
147+
form.append("caseMatcher", options.caseMatcher);
148+
}
149+
150+
if (options.caseIdFormat) {
151+
form.append("caseIdFormat", options.caseIdFormat);
152+
}
153+
146154
// Add files (read as buffers for compatibility with form.getBuffer())
147155
for (const filePath of files) {
148156
const absolutePath = path.resolve(filePath);
@@ -224,6 +232,7 @@ export async function importTestResults(
224232
result = {
225233
testRunId: data.testRunId,
226234
attachmentMappings: data.attachmentMappings,
235+
caseIdWarnings: data.caseIdWarnings,
227236
};
228237
}
229238
} catch (e) {

cli/src/types.ts

Lines changed: 36 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,26 @@ export const TEST_RESULT_FORMATS: Record<Exclude<TestResultFormat, "auto">, { la
3030
cucumber: { label: "Cucumber JSON", extensions: [".json"] },
3131
};
3232

33+
/**
34+
* Case-ID matching: how (and whether) to link results to existing cases by an
35+
* ID parsed from the test name or a `test_id` property, instead of matching on
36+
* name + class name.
37+
*/
38+
export type CaseMatcher = "off" | "name" | "property" | "auto";
39+
export type CaseIdFormat = "brackets" | "c" | "tc";
40+
41+
export const CASE_MATCHERS: readonly CaseMatcher[] = [
42+
"off",
43+
"name",
44+
"property",
45+
"auto",
46+
];
47+
export const CASE_ID_FORMATS: readonly CaseIdFormat[] = [
48+
"brackets",
49+
"c",
50+
"tc",
51+
];
52+
3353
/**
3454
* Import options - IDs are resolved before being passed to the API
3555
*/
@@ -43,6 +63,8 @@ export interface ImportOptions {
4363
parentFolderId?: number;
4464
tagIds?: number[];
4565
testRunId?: number;
66+
caseMatcher?: CaseMatcher;
67+
caseIdFormat?: CaseIdFormat;
4668
}
4769

4870
/**
@@ -104,10 +126,21 @@ export interface AttachmentMapping {
104126
}
105127

106128
/**
107-
* Extended SSE progress event with attachment mappings
129+
* A test whose declared case ID could not be linked (unknown id, or not in
130+
* the target project). The result for this test was skipped.
131+
*/
132+
export interface CaseIdWarning {
133+
testName: string;
134+
className: string | null;
135+
requestedCaseId: number;
136+
}
137+
138+
/**
139+
* Extended SSE progress event with attachment mappings and advisory warnings
108140
*/
109141
export interface ImportSSEProgressEvent extends SSEProgressEvent {
110142
attachmentMappings?: AttachmentMapping[];
143+
caseIdWarnings?: CaseIdWarning[];
111144
}
112145

113146
/**
@@ -154,11 +187,12 @@ export interface BulkAttachmentUploadResponse {
154187
}
155188

156189
/**
157-
* Import result with optional attachment mappings
190+
* Import result with optional attachment mappings and advisory warnings
158191
*/
159192
export interface ImportResult {
160193
testRunId: number;
161194
attachmentMappings?: AttachmentMapping[];
195+
caseIdWarnings?: CaseIdWarning[];
162196
}
163197

164198
/**

docs/docs/cli.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,8 @@ testplanit import <files...> --project <id|name> --name <name> [options]
144144
| `-d, --attachments-dir <path>` | Base directory for resolving attachment paths (default: directory of test result file) |
145145
| `--no-attachments` | Skip uploading attachments from test results |
146146
| `-a, --run-attachments <files...>` | Files to attach to the test run (e.g., test plans, reports) |
147+
| `--case-matcher <mode>` | Link results to existing cases by ID: `off`, `name`, `property`, `auto` (default: `off`) |
148+
| `--case-id-format <preset>` | Case-ID pattern in the test name when matching by name: `brackets`, `c`, `tc` (default: `brackets`) |
147149

148150
:::note
149151
For project, state, config, milestone, folder, and test run options, the CLI looks up entities by exact name match. If no match is found, an error is returned. For tags, if a tag name doesn't exist, it will be created automatically.
@@ -171,6 +173,42 @@ Use `-d, --attachments-dir` to specify a custom base directory for resolving rel
171173

172174
Use `--no-attachments` to skip uploading test result attachments entirely.
173175

176+
#### Linking results to existing cases by ID
177+
178+
By default the importer matches each result to a case by **name and class name**. If your automation renames a test, the importer no longer recognizes it and creates a duplicate case. To prevent this, assign each automated test the ID of the case it belongs to, and turn on case-ID matching with `--case-matcher`. When a result carries a recognized ID, it links to that case **regardless of the test name**, so renames stop creating duplicates.
179+
180+
There are two ways to carry the ID, controlled by `--case-matcher`:
181+
182+
- **`name`** — the ID is embedded in the test name; the pattern is chosen with `--case-id-format` (see below).
183+
- **`property`** — the ID is carried in a JUnit `<property>` named `test_id` (or `testplanit_case_id`), following the convention used by TestRail and Xray (see below).
184+
- **`auto`** — try the `test_id` property first, then fall back to the name pattern.
185+
186+
Name patterns (`--case-id-format`):
187+
188+
| Preset | Matches | Example test name |
189+
| -------- | --------- | ------------------- |
190+
| `brackets` (default) | `[123]`, `[C123]`, `[123, 456]` | `[123] user can log in` |
191+
| `c` | `C123` | `C123 user can log in` |
192+
| `tc` | `TC-123`, `TC123` | `TC-123 user can log in` |
193+
194+
Property form (`--case-matcher property` or `auto`):
195+
196+
```xml
197+
<testcase name="user can log in" classname="auth.LoginTests">
198+
<properties>
199+
<property name="test_id" value="123"/>
200+
</properties>
201+
</testcase>
202+
```
203+
204+
Notes:
205+
206+
- A single test may reference multiple cases (`[123, 456]` or `value="123,456"`); each referenced case receives the same result.
207+
- Tests **without** an ID still import normally — they are matched/created by name and class name as before.
208+
- If a referenced case ID does not exist in the target project, that result is **skipped** (never auto-created under a wrong name) and reported as a warning at the end of the import.
209+
- The ID is the numeric TestPlanIt case ID; a leading `C` or `#` is tolerated.
210+
- Property matching requires a reporter that writes per-`<testcase>` properties (e.g. pytest's `record_property`, the Playwright JUnit reporter with `embedAnnotationsAsProperties`, NUnit `[Property]`). Frameworks that only emit suite-level properties (e.g. Maven Surefire) should use the name-based pattern instead.
211+
174212
#### Examples
175213

176214
```bash
@@ -192,6 +230,15 @@ testplanit import cucumber-report.json -p 1 -n "BDD Tests" -F cucumber
192230
# Import multiple files with glob pattern
193231
testplanit import "./test-results/*.xml" -p 1 -n "CI Build"
194232

233+
# Link results to existing cases by an ID in the test name (e.g. "[123] login")
234+
testplanit import results.xml -p 1 -n "Build 123" --case-matcher name
235+
236+
# Use the TestRail-style C-prefix pattern (e.g. "C123 login")
237+
testplanit import results.xml -p 1 -n "Build 123" --case-matcher name --case-id-format c
238+
239+
# Link by a test_id property in the XML, falling back to the name pattern
240+
testplanit import results.xml -p 1 -n "Build 123" --case-matcher auto
241+
195242
# Import with IDs
196243
testplanit import results.xml \
197244
--project 1 \

docs/docs/import-export.md

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,12 @@ Import test cases from CSV files with flexible field mapping.
3030
There are two ways to start a CSV import:
3131

3232
**Using the Import button:**
33+
3334
1. Navigate to **Repository** in your project
3435
2. Click the **Import** button in the toolbar
3536

3637
**Using drag and drop:**
38+
3739
1. Drag a `.csv` file from your desktop over the Repository page
3840
2. A full-page drop overlay will appear indicating you can drop to import
3941
3. Drop the file — the import wizard opens automatically with your file pre-loaded
@@ -104,15 +106,15 @@ Test steps in a single cell can be formatted in several ways:
104106

105107
**Simple Format:**
106108

107-
```
109+
```text
108110
1. Step one
109111
2. Step two
110112
3. Step three
111113
```
112114

113115
**Detailed Format with Expected Results** — separate the step from its expected result with a pipe (`|`):
114116

115-
```
117+
```text
116118
1. Navigate to login page | Login page displays
117119
2. Enter username and password | Fields accept input
118120
3. Click login button | User is redirected to dashboard
@@ -122,7 +124,7 @@ Test steps in a single cell can be formatted in several ways:
122124

123125
Steps can also contain markdown formatting:
124126

125-
```
127+
```text
126128
1. Navigate to the **Login** page | Login page displays with _username_ and _password_ fields
127129
2. Enter `admin` credentials | Fields accept input
128130
3. Click **Submit** | User is redirected to [Dashboard](/dashboard)
@@ -297,11 +299,13 @@ TestPlanIt supports importing test results from the following formats:
297299
There are two ways to start a test results import:
298300

299301
**Using the Import button:**
302+
300303
1. Navigate to **Test Runs** in your project
301304
2. Click **Import Results** button
302305
3. The import dialog opens with format options
303306

304307
**Using drag and drop:**
308+
305309
1. Drag one or more test result files (`.xml`, `.trx`, or `.json`) from your desktop over the Test Runs page
306310
2. A full-page drop overlay will appear indicating you can drop to import
307311
3. Drop the files — the import dialog opens automatically with your files pre-loaded
@@ -758,6 +762,8 @@ parentFolderId: 101
758762
configId: 102 (optional)
759763
milestoneId: 103 (optional)
760764
tagIds: [1, 2, 3] (optional)
765+
caseMatcher: "off" | "name" | "property" | "auto" (optional, default "off")
766+
caseIdFormat: "brackets" | "c" | "tc" (optional, default "brackets")
761767
```
762768

763769
Response is Server-Sent Events (SSE) with progress updates:
@@ -768,6 +774,8 @@ Response is Server-Sent Events (SSE) with progress updates:
768774
{"complete": true, "testRunId": 12345}
769775
```
770776

777+
By default each result is matched to a case by **name + class name**, so renaming an automated test creates a duplicate case. Set `caseMatcher` to instead link results to an existing case by an ID carried in the test name (`caseIdFormat` preset) or a `test_id` / `testplanit_case_id` JUnit property — see [Linking results to existing cases by ID](./cli.md#linking-results-to-existing-cases-by-id) in the CLI guide for the full behavior. Results whose referenced case ID is not found in the project are skipped and listed in the final SSE event as `caseIdWarnings`.
778+
771779
### Export
772780

773781
There is no server-side export endpoint. CSV, PDF, and other exports are generated in the browser from data already loaded by the Repository view — the export reflects the current filter, scope, and column selection in the UI. Use the **Export** button in the Repository toolbar to configure and download.

pnpm-workspace.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ allowBuilds:
2525
'core-js': true
2626
'core-js-pure': true
2727
'esbuild': true
28+
iframe-resizer: true
2829
'keytar': true
2930
'msgpackr-extract': true
3031
'msw': true

0 commit comments

Comments
 (0)