Skip to content

Commit ea045bf

Browse files
Merge pull request #1346 from cloudbees-oss/AIENG-569
update cli docs from docsite
2 parents 3605066 + 069aede commit ea045bf

116 files changed

Lines changed: 3415 additions & 2326 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

smart_tests/docs/modules/resources/pages/integrations/adb.adoc renamed to smart_tests/docs/modules/ROOT/archive/adb.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@
33

44
{PRODUCT} interfaces with adb via the {PRODUCT} CLI.
55

6-
For more information, refer to xref:send-data-to-smart-tests:record-test-results/record-test-results.adoc[Record test results with the {PRODUCT} CLI] and xref:features:predictive-test-selection/request-and-run-a-subset-of-tests/subset-with-the-smart-tests-cli/subset-with-the-smart-tests-cli.adoc[Subsetting with the {PRODUCT} CLI].
6+
For more information, refer to xref:send-data-to-smart-tests:record-test-results/record-test-results.adoc[Record test results with the {PRODUCT} CLI] and xref:send-data-to-smart-tests:subset/subset-with-the-smart-tests-cli.adoc[Subsetting with the {PRODUCT} CLI].
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
= Ant
2+
:slug: ant
3+
4+
{PRODUCT} interfaces with Ant via the {PRODUCT} CLI.
5+
6+
For more information, refer to xref:send-data-to-smart-tests:record-test-results/record-test-results.adoc[Record test results with the {PRODUCT} CLI] and xref:send-data-to-smart-tests:subset/subset-with-the-smart-tests-cli.adoc[Subsetting with the {PRODUCT} CLI].

smart_tests/docs/modules/resources/pages/integrations/bazel.adoc renamed to smart_tests/docs/modules/ROOT/archive/bazel.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@
33

44
{PRODUCT} interfaces with Bazel via the {PRODUCT} CLI.
55

6-
For more information, refer to xref:send-data-to-smart-tests:record-test-results/record-test-results.adoc[Record test results with the {PRODUCT} CLI] and xref:features:predictive-test-selection/request-and-run-a-subset-of-tests/subset-with-the-smart-tests-cli/subset-with-the-smart-tests-cli.adoc[Subset with the {PRODUCT} CLI].
6+
For more information, refer to xref:send-data-to-smart-tests:record-test-results/record-test-results.adoc[Record test results with the {PRODUCT} CLI] and xref:send-data-to-smart-tests:subset/subset-with-the-smart-tests-cli.adoc[Subset with the {PRODUCT} CLI].

smart_tests/docs/modules/resources/pages/integrations/behave.adoc renamed to smart_tests/docs/modules/ROOT/archive/behave.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@
33

44
{PRODUCT} interfaces with Behave via the {PRODUCT} CLI.
55

6-
For more information, refer to xref:send-data-to-smart-tests:record-test-results/record-test-results.adoc[Record test results with the {PRODUCT} CLI] and xref:features:predictive-test-selection/request-and-run-a-subset-of-tests/subset-with-the-smart-tests-cli/subset-with-the-smart-tests-cli.adoc[Subset with the {PRODUCT} CLI].
6+
For more information, refer to xref:send-data-to-smart-tests:record-test-results/record-test-results.adoc[Record test results with the {PRODUCT} CLI] and xref:send-data-to-smart-tests:subset/subset-with-the-smart-tests-cli.adoc[Subset with the {PRODUCT} CLI].
Lines changed: 325 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,325 @@
1+
include::ROOT:partial$abbr.adoc[]
2+
3+
= CI tool integrations
4+
:slug: ci-tool-integrations
5+
6+
Although {PRODUCT} works with any CI tool, we are working on direct integrations to make onboarding easier. Follow the instructions on xref:send-data-to-smart-tests:set-up-smart-tests/getting-started.adoc[Getting started] to do the following:
7+
8+
* Sign up for a Smart test account on {PRODUCT}.
9+
* Create an organization and a workspace for your test suite.
10+
* Create an API key for your workspace. This authentication token lets the CLI talk to your {PRODUCT} workspace.
11+
12+
== GitHub Actions
13+
14+
The {PRODUCT} record build and test results action integrates {PRODUCT} into your CI with minimal configuration. This action installs the link:https://pypi.org/project/smart-tests-cli/[CLI] and runs `smart-tests record build` and `smart-tests record tests` to send data to {PRODUCT}, which analyzes the test results to improve developer productivity.
15+
16+
For more information, refer to
17+
link:https://github.com/marketplace/actions/record-build-and-test-results-action[{PRODUCT} record build and test results action].
18+
19+
=== Example usage
20+
21+
[source,yaml]
22+
----
23+
name: Test
24+
25+
on:
26+
push:
27+
branches: [main]
28+
pull_request:
29+
branches: [main]
30+
31+
env:
32+
SMART_TESTS_TOKEN: ${{ secrets.SMART_TESTS_TOKEN }}
33+
SMART_TESTS_DEBUG: 1
34+
SMART_TESTS_REPORT_ERROR: 1
35+
36+
jobs:
37+
tests:
38+
runs-on: ubuntu-latest
39+
steps:
40+
- uses: actions/checkout@v6
41+
- name: Test
42+
run: <YOUR TEST COMMAND HERE>
43+
- name: Record
44+
uses: launchableinc/record-build-and-test-results-action@v1.0.0
45+
with:
46+
build_name: $GITHUB_RUN_ID
47+
test_runner: <YOUR TEST RUNNER HERE>
48+
report_path: <PATH TO TEST REPORT XML FILES>
49+
if: always()
50+
----
51+
52+
.Test runner values
53+
[cols="1,1,1", options="header"]
54+
|===
55+
|Test runner |`test_runner` value |Additional steps?
56+
57+
|<<ant-ga>>
58+
|`ant`
59+
|
60+
61+
|<<bazel-ga>>
62+
|`bazel`
63+
|
64+
65+
|<<behave-ga>>
66+
|`behave`
67+
|Yes
68+
69+
|<<ctest-ga>>
70+
|`ctest`
71+
|Yes
72+
73+
|<<cucumber-ga>>
74+
|`cucumber`
75+
|Yes
76+
77+
|<<cypress-ga>>
78+
|`cypress`
79+
|
80+
81+
|<<googletest-ga>>
82+
|`googletest`
83+
|Yes
84+
85+
|<<go-test-ga>>
86+
|`go-test`
87+
|Yes
88+
89+
|<<gradle-ga>>
90+
|`gradle`
91+
|
92+
93+
|<<jest-ga>>
94+
|`jest`
95+
|Yes
96+
97+
|<<maven-ga>>
98+
|`maven`
99+
|
100+
101+
|<<mtest-ga>>
102+
|`minitest`
103+
|Yes
104+
105+
|<<nunit-ga>>
106+
|`nunit`
107+
|Yes
108+
109+
|<<pytest-ga>>
110+
|`pytest`
111+
|Yes
112+
113+
|<<robot-ga>>
114+
|`robot`
115+
|
116+
117+
|<<rspec-ga>>
118+
|`rspec`
119+
|Yes
120+
|===
121+
122+
[#instructions_for_test_runnersbuild_tools]
123+
== Instructions for test runners/build tools
124+
125+
The following section provides instructions for configuring test runners or build tools to generate test reports that {PRODUCT} can consume.If your test runner or build tool isn't listed, use the CLI to send data to {PRODUCT} using one of the following profiles:
126+
127+
* xref:references:integrations/use-the-generic-file-based-runner-integration.adoc[`file` profile for unsupported test runners]
128+
129+
* xref:references:integrations/raw.adoc[`raw` profile for custom test runners]
130+
131+
[#adb-ga]
132+
=== Android Debug Bridge (adb)
133+
134+
Currently, the CLI doesn't have a `record tests` command for ADB. Use the command for <<gradle-ga>> instead.
135+
136+
[#ant-ga]
137+
=== Ant
138+
139+
No special steps.
140+
141+
[#bazel-ga]
142+
=== Bazel
143+
144+
No special steps.
145+
146+
[#behave-ga]
147+
=== Behave
148+
149+
In order to generate reports that {PRODUCT} can consume, add the `--junit` option to your existing `behave` command:
150+
151+
[source,bash]
152+
----
153+
# run the tests however you normally do
154+
behave --junit
155+
----
156+
157+
[#ctest-ga]
158+
=== CTest
159+
160+
Run your tests with `ctest -T test --no-compress-output`. These options ensure test results are written to the `Testing` directory.
161+
162+
[#cucumber-ga]
163+
=== cucumber
164+
165+
Run cucumber with the `-f junit` option, like this:
166+
167+
`bundle exec cucumber -f junit -o reports`
168+
169+
(If using JSON, use the {PRODUCT} CLI instead.)
170+
171+
[#cypress-ga]
172+
=== Cypress
173+
174+
No special steps.
175+
176+
[#googletest-ga]
177+
=== GoogleTest
178+
179+
Configure GoogleTest to produce JUnit compatible report files. For more information on how to do this, refer to link:https://github.com/google/googletest/blob/main/docs/advanced.md#generating-an-xml-report[their documentation].
180+
181+
==== Command example
182+
183+
[source,bash]
184+
----
185+
# run the tests however you normally do
186+
./my-test --gtest_output=xml:./report/my-test.xml
187+
----
188+
189+
[#go-test-ga]
190+
=== Go Test
191+
192+
In order to generate reports that {PRODUCT} can consume, use link:https://github.com/jstemmer/go-junit-report[go-junit-report] to generate a JUnit XML file after running tests:
193+
194+
[source,bash]
195+
----
196+
# install JUnit report formatter
197+
go get -u github.com/jstemmer/go-junit-report
198+
199+
# run the tests however you normally do, then produce a JUnit XML file
200+
go test -v ./... | go-junit-report -set-exit-code > report.xml
201+
----
202+
203+
[#gradle-ga]
204+
=== Gradle
205+
No special steps. Gradle writes test reports to `**/build/**/TEST-*.xml` by default, which the {PRODUCT} CLI consumes.
206+
`**/build/**/TEST-*.xml` .
207+
208+
[#jest-ga]
209+
=== Jest
210+
211+
In order to generate reports that {PRODUCT} can consume, use link:https://www.npmjs.com/package/jest-junit[jest-junit] to generate a JUnit XML file after running tests.
212+
213+
[source,bash]
214+
----
215+
# install jest-junit reporter
216+
npm install jest-junit --save-dev
217+
# or
218+
yarn add --dev jest-junit
219+
----
220+
221+
Configure jest-junit to include file paths in reports. Do this using the following environment variables:
222+
223+
[.multilanguage-custom-table, options="header", cols="2"]
224+
|===
225+
| Using environment variables
226+
| Using package.json
227+
228+
a| Recommended config:
229+
230+
[source,bash]
231+
----
232+
export JEST_JUNIT_CLASSNAME="{classname}"
233+
export JEST_JUNIT_TITLE="{title}"
234+
export JEST_JUNIT_SUITE_NAME="{filepath}"
235+
----
236+
237+
Minimum config:
238+
239+
`export JEST_JUNIT_SUITE_NAME="{filepath}"`
240+
241+
a| Add the following lines to `package.json`. The detail is the link:https://www.npmjs.com/package/jest-junit[jest-junit] section.
242+
243+
Recommended config:
244+
245+
[source,json]
246+
----
247+
// package.json
248+
"jest-junit": {
249+
"suiteNameTemplate": "{filepath}",
250+
"classNameTemplate": "{classname}",
251+
"titleTemplate": "{title}"
252+
}
253+
----
254+
255+
Minimum config:
256+
257+
[source,json]
258+
----
259+
// package.json
260+
"jest-junit": {
261+
"suiteNameTemplate": "{filepath}"
262+
}
263+
----
264+
265+
|===
266+
267+
Run `jest` using jest-junit:
268+
269+
[source,bash]
270+
----
271+
# run tests with jest-junit
272+
jest --ci --reporters=default --reporters=jest-junit
273+
----
274+
275+
[#maven-ga]
276+
=== Maven
277+
278+
{PRODUCT} supports test reports generated using link:https://maven.apache.org/surefire/maven-surefire-plugin/[Surefire], the default report plugin for https://maven.apache.org/[Maven].
279+
280+
Example:
281+
282+
`'./**/target/surefire-reports'`
283+
284+
NOTE: The invocation above relies on the CLI to expand globs like `**`.
285+
286+
[#mtest-ga]
287+
=== minitest
288+
289+
Use link:https://github.com/circleci/minitest-ci[minitest-ci] to output test results to a file. If storing test results on your CI server, it may already be installed.
290+
291+
[#nunit-ga]
292+
=== NUnit Console Runner
293+
294+
{PRODUCT} CLI accepts link:https://docs.nunit.org/articles/nunit/technical-notes/usage/XML-Formats.html[NUnit3 style test report XML files] produced by NUnit.
295+
296+
297+
[#pytest-ga]
298+
=== pytest
299+
300+
Run tests with the `--junit-xml` option:
301+
302+
`pytest --junit-xml=test-results/results.xml`
303+
304+
[WARNING]
305+
--
306+
pytest changed its default test report format from `xunit1` to `xunit2` in version 6. Unfortunately, the new `xunit2` format does not include file paths, which {PRODUCT} needs.
307+
308+
If using pytest 6 or newer, you must also specify `junit_family=legacy` as the report format. For more information, refer to link:https://docs.pytest.org/en/latest/deprecations.html#junit-family-default-value-change-to-xunit2[Deprecations and Removals — pytest documentation] for instructions.
309+
--
310+
311+
[#robot-ga]
312+
=== Robot
313+
314+
Example output:
315+
316+
`output.xml`
317+
318+
[#rspec-ga]
319+
=== RSpec
320+
321+
Use link:https://github.com/sj26/rspec_junit_formatter[rspec_junit_formatter] to output test results to a file in RSpec. If storing test results on your CI server, it may already be installed.
322+
323+
Example output:
324+
325+
`bundle exec rspec --format RspecJunitFormatter --out report/rspec.xml`

0 commit comments

Comments
 (0)