Skip to content

Commit 7b5be11

Browse files
EvangelinkCopilot
Evangelink
and
Copilot
authored
Add MTP version column to AzDO and terminal option tables (dotnet#54721)
* Add MTP version column to AzDO and terminal option tables Move per-option availability version info out of prose notes and into a dedicated MTP version column for the Azure DevOps report options and the terminal output options, improving scannability and reducing note drift. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Clarify MTP version column meaning in AzDO and terminal tables Address review feedback: reintroduce the Azure DevOps stability nuance (became stable in MTP 1.9.0) and explain what the MTP version column represents, and clarify that a dash denotes core options available since the platform's initial releases. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
1 parent 049f885 commit 7b5be11

2 files changed

Lines changed: 28 additions & 25 deletions

File tree

‎docs/core/testing/microsoft-testing-platform-terminal-output.md‎

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -43,15 +43,18 @@ The progress bar is written based on the selected mode:
4343

4444
## Options
4545

46-
| Option | Description |
47-
|---|---|
48-
| `--no-progress` | Disables reporting progress to screen. Deprecated in MTP 2.3.0 in favor of `--progress off`. |
49-
| `--progress` | Controls whether progress is shown. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`), and `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. |
50-
| `--no-ansi` | Disables outputting ANSI escape characters to screen. |
51-
| `--ansi` | Controls whether ANSI escape characters are emitted. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`), and `off` (also accepts `false`, `disable`, `0`). Available in MTP starting with version 2.3.0. |
52-
| `--output` | Specifies the output verbosity when reporting tests. Valid values are `Normal` and `Detailed`. Default is `Normal`. |
53-
| `--show-stdout` | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. Available in MTP starting with version 2.2.1. |
54-
| `--show-stderr` | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. Available in MTP starting with version 2.2.1. |
46+
| Option | MTP version | Description |
47+
|---|---|---|
48+
| `--no-progress` | — | Disables reporting progress to screen. Deprecated in MTP 2.3.0 in favor of `--progress off`. |
49+
| `--progress` | 2.3.0 | Controls whether progress is shown. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`), and `off` (also accepts `false`, `disable`, `0`). |
50+
| `--no-ansi` | — | Disables outputting ANSI escape characters to screen. |
51+
| `--ansi` | 2.3.0 | Controls whether ANSI escape characters are emitted. Valid values are `auto` (default), `on` (also accepts `true`, `enable`, `1`), and `off` (also accepts `false`, `disable`, `0`). |
52+
| `--output` | — | Specifies the output verbosity when reporting tests. Valid values are `Normal` and `Detailed`. Default is `Normal`. |
53+
| `--show-stdout` | 2.2.1 | Determines when to show captured standard output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. |
54+
| `--show-stderr` | 2.2.1 | Determines when to show captured error output of a test. Valid values are `All`, `Failed`, and `None`. Default is `All`. |
55+
56+
> [!NOTE]
57+
> A dash (—) in the **MTP version** column marks core options that aren't tied to a specific version because they've been available since the platform's initial releases.
5558
5659
> [!NOTE]
5760
> Starting with MTP 2.3.0, when MTP detects that it runs inside an LLM or AI tool environment, it suppresses the startup banner and changes the default of `--show-stdout` and `--show-stderr` from `All` to `Failed` to reduce noise.

‎docs/core/testing/microsoft-testing-platform-test-reports.md‎

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -143,23 +143,23 @@ builder.TestHost.AddAzureDevOpsProvider();
143143

144144
### Options
145145

146-
| Option | Description |
147-
|---|---|
148-
| `--report-azdo` | Enables the Azure DevOps report generator. Errors and warnings are written to the output in a format that Azure DevOps understands. |
149-
| `--report-azdo-severity` | Severity to use for reported events. Valid values are `error` (default) and `warning`. |
150-
| `--report-azdo-flaky-history` | Queries Azure DevOps test result history for the past N days (1-90) and annotates reported failures with flakiness context. Requires `--report-azdo`. |
151-
| `--report-azdo-demote-known-flaky` | Demotes failures that are flaky enough in the Azure DevOps history window (default threshold is 25%) from errors to warnings. Requires `--report-azdo` and `--report-azdo-flaky-history`. |
152-
| `--report-azdo-quarantine-file` | Path to a text file that lists quarantined test fully qualified names or glob patterns. Matching failures are reported as warnings. Requires `--report-azdo`. |
153-
| `--report-azdo-summary` | Writes a Markdown job summary at the end of the test run and uploads it through `##vso[task.uploadsummary]`. An optional file path argument overrides the default location (`{testResultsDir}/azdo-summary-{tfm}.md`). Requires `--report-azdo`. |
154-
| `--report-azdo-stackframe-filter` | Adds regex patterns, matched against the fully qualified type prefix of each stack frame, that are skipped when the extension locates the user's call site to annotate. The option is repeatable, up to 16 patterns, and each pattern is compiled with a 500-ms match timeout. These patterns are additive to the extension's built-in MSTest assertion-implementation prefixes. Requires `--report-azdo`. |
155-
| `--report-azdo-upload-artifacts` | Uploads test result files and/or adds build tags to Azure DevOps. Valid values are `off` (default), `tags-only`, `files`, and `all`. |
156-
| `--report-azdo-upload-artifact-include` | Includes files in the Azure DevOps artifact upload using glob patterns relative to the test results directory. Defaults to `**/*`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
157-
| `--report-azdo-upload-artifact-exclude` | Excludes files from the Azure DevOps artifact upload using glob patterns relative to the test results directory. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
158-
| `--report-azdo-upload-artifact-name` | Overrides the Azure DevOps artifact container name. Defaults to `TestResults_{assemblyName}_{tfm}`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
159-
| `--publish-azdo-test-results` | Publishes test results live to the Azure DevOps **Tests** tab. |
160-
| `--publish-azdo-run-name` | Sets a custom Azure DevOps test run name for live test-result publishing. Requires `--publish-azdo-test-results`. |
146+
| Option | MTP version | Description |
147+
|---|---|---|
148+
| `--report-azdo` | 1.9.0 | Enables the Azure DevOps report generator. Errors and warnings are written to the output in a format that Azure DevOps understands. |
149+
| `--report-azdo-severity` | 1.9.0 | Severity to use for reported events. Valid values are `error` (default) and `warning`. |
150+
| `--report-azdo-flaky-history` | 2.3.0 | Queries Azure DevOps test result history for the past N days (1-90) and annotates reported failures with flakiness context. Requires `--report-azdo`. |
151+
| `--report-azdo-demote-known-flaky` | 2.3.0 | Demotes failures that are flaky enough in the Azure DevOps history window (default threshold is 25%) from errors to warnings. Requires `--report-azdo` and `--report-azdo-flaky-history`. |
152+
| `--report-azdo-quarantine-file` | 2.3.0 | Path to a text file that lists quarantined test fully qualified names or glob patterns. Matching failures are reported as warnings. Requires `--report-azdo`. |
153+
| `--report-azdo-summary` | 2.3.0 | Writes a Markdown job summary at the end of the test run and uploads it through `##vso[task.uploadsummary]`. An optional file path argument overrides the default location (`{testResultsDir}/azdo-summary-{tfm}.md`). Requires `--report-azdo`. |
154+
| `--report-azdo-stackframe-filter` | 2.3.0 | Adds regex patterns, matched against the fully qualified type prefix of each stack frame, that are skipped when the extension locates the user's call site to annotate. The option is repeatable, up to 16 patterns, and each pattern is compiled with a 500-ms match timeout. These patterns are additive to the extension's built-in MSTest assertion-implementation prefixes. Requires `--report-azdo`. |
155+
| `--report-azdo-upload-artifacts` | 2.3.0 | Uploads test result files and/or adds build tags to Azure DevOps. Valid values are `off` (default), `tags-only`, `files`, and `all`. |
156+
| `--report-azdo-upload-artifact-include` | 2.3.0 | Includes files in the Azure DevOps artifact upload using glob patterns relative to the test results directory. Defaults to `**/*`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
157+
| `--report-azdo-upload-artifact-exclude` | 2.3.0 | Excludes files from the Azure DevOps artifact upload using glob patterns relative to the test results directory. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
158+
| `--report-azdo-upload-artifact-name` | 2.3.0 | Overrides the Azure DevOps artifact container name. Defaults to `TestResults_{assemblyName}_{tfm}`. Requires `--report-azdo-upload-artifacts` to be a value other than `off`. |
159+
| `--publish-azdo-test-results` | 2.3.0 | Publishes test results live to the Azure DevOps **Tests** tab. |
160+
| `--publish-azdo-run-name` | 2.3.0 | Sets a custom Azure DevOps test run name for live test-result publishing. Requires `--publish-azdo-test-results`. |
161161

162162
> [!NOTE]
163-
> The Azure DevOps extension became stable in MTP 1.9.0 (`--report-azdo` and `--report-azdo-severity`). All other options in the table — `--report-azdo-flaky-history`, `--report-azdo-demote-known-flaky`, `--report-azdo-quarantine-file`, `--report-azdo-summary`, `--report-azdo-stackframe-filter`, `--report-azdo-upload-artifacts`, `--report-azdo-upload-artifact-include`, `--report-azdo-upload-artifact-exclude`, `--report-azdo-upload-artifact-name`, `--publish-azdo-test-results`, and `--publish-azdo-run-name` — are available in MTP starting with version 2.3.0.
163+
> The **MTP version** column lists the MTP release in which each option first became available in a stable build. The Azure DevOps extension itself became stable in MTP 1.9.0 with `--report-azdo` and `--report-azdo-severity`; the remaining options were added in MTP 2.3.0.
164164
165165
The extension automatically detects that it is running in continuous integration (CI) environment by checking the `TF_BUILD` environment variable.

0 commit comments

Comments
 (0)