Skip to content

Commit eab4e00

Browse files
authored
Produce test-output.json even when a test leg fails (#5806)
We upload test-output.json (the concatenated gotestsum output) as a CI artifact to inspect results and timing after the fact, most importantly when a test fails. But the file is produced by a `cat` in the `test` task's `cmds`, which is skipped when a `deps` leg fails, so on a failed run the file is never written and the upload finds nothing. This moves the `cat` (in both `test` and `test-unit`) into a `defer:`, which runs even after a leg fails, so the artifact is always produced. The task still exits non-zero, so CI failures are unaffected. For the defer to have something to concatenate, the legs now run as sequential `cmds` rather than parallel `deps`. An A/B CI run (#5802) showed no timing regression (serial was equal or faster on 5 of 6 matrix cells). Both legs already saturate the runners, so running them in parallel mostly just created core contention. This pull request and its description were written by Isaac.
1 parent 20d48c7 commit eab4e00

1 file changed

Lines changed: 13 additions & 8 deletions

File tree

Taskfile.yml

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -385,30 +385,35 @@ tasks:
385385

386386
test:
387387
desc: Run unit and acceptance tests
388-
deps:
389-
- task: test-unit
390-
- task: test-acc
391-
vars:
392-
ACCEPTANCE_TEST_FILTER: "{{.ACCEPTANCE_TEST_FILTER}}"
388+
# Run the legs as sequential cmds (not parallel deps) so the concatenating
389+
# cat can be a `defer`, which runs even when a leg fails. When the legs are
390+
# deps, the cat in cmds is skipped on failure, so test-output.json is never
391+
# produced for a failed run and CI has no gotestsum output to upload.
393392
sources:
394393
- test-output-unit.json
395394
- test-output-acc.json
396395
generates:
397396
- test-output.json
398397
cmds:
399-
- cat test-output-unit.json test-output-acc.json > test-output.json
398+
- defer: cat test-output-unit.json test-output-acc.json > test-output.json
399+
- task: test-unit
400+
- task: test-acc
401+
vars:
402+
ACCEPTANCE_TEST_FILTER: "{{.ACCEPTANCE_TEST_FILTER}}"
400403

401404
test-unit:
402405
desc: Run unit tests across all Go modules (root, tools, codegen)
403-
deps: ['test-unit-root', 'test-unit-tools', 'test-unit-codegen']
404406
sources:
405407
- test-output-unit-root.json
406408
- tools/test-output-unit-tools.json
407409
- bundle/internal/tf/codegen/test-output-unit-codegen.json
408410
generates:
409411
- test-output-unit.json
410412
cmds:
411-
- cat test-output-unit-root.json tools/test-output-unit-tools.json bundle/internal/tf/codegen/test-output-unit-codegen.json > test-output-unit.json
413+
- defer: cat test-output-unit-root.json tools/test-output-unit-tools.json bundle/internal/tf/codegen/test-output-unit-codegen.json > test-output-unit.json
414+
- task: test-unit-root
415+
- task: test-unit-tools
416+
- task: test-unit-codegen
412417

413418
test-unit-root:
414419
desc: Run unit tests in root module

0 commit comments

Comments
 (0)