chore: "no fail fast" test mode by KSXGitHub · Pull Request #367 · KSXGitHub/parallel-disk-usage

KSXGitHub · 2026-03-23T00:07:40Z

Summary

This PR adds a new PDU_NO_FAIL_FAST environment variable that allows the test script to continue running all checks even when one fails, rather than stopping at the first failure. This is useful for seeing the complete status of all checks in a single run.

Key Changes

test.sh:
- Added validation for the PDU_NO_FAIL_FAST environment variable (accepts 'true' or 'false', defaults to 'false')
- Implemented a temporary failure marker file to track failures across subshells (since variable assignments in subshells don't propagate to parent)
- Modified run_if() function to catch command failures when PDU_NO_FAIL_FAST=true and record them instead of exiting immediately
- Added final check at the end of the script to report any accumulated failures and exit with error code if any occurred
Documentation files (.github/copilot-instructions.md, AGENTS.md, CLAUDE.md, template/ai-instructions/shared.md):
- Added instruction documenting the new PDU_NO_FAIL_FAST=true option for running all checks and seeing which pass/fail

Implementation Details

Uses a temporary file as a failure marker since the script uses subshells (via run_if and unit functions) where variable assignments don't propagate to the parent shell
When PDU_NO_FAIL_FAST=true, failures are logged but don't immediately exit; instead they're recorded and reported at the end
When PDU_NO_FAIL_FAST=false (default), the script maintains its original fail-fast behavior
The failure marker file is cleaned up after the final status check

https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

Add a no-fail-fast mode activated by `PDU_NO_FAIL_FAST=true` that continues running all checks instead of stopping at the first failure, then reports a summary at the end. This helps AI environments that may fail on some checks but need to see which ones pass. Update AI instruction templates to document the new flag. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

Add a comment explaining why a temporary file is used instead of a variable for the failure marker, and log the failed command with its exit code when no-fail-fast mode catches a failure. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

…n template Update the source template and regenerate all AI instruction files. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

Copilot

Pull request overview

Adds an opt-in “no fail fast” mode to the repository test harness so developers can run the full suite of checks and see all failures in one pass, and documents the new option in the AI instruction docs.

Changes:

Introduces PDU_NO_FAIL_FAST env var in test.sh to keep running checks after a failure and fail at the end if anything failed.
Tracks failures across subshells via a temporary failure marker file.
Documents PDU_NO_FAIL_FAST=true in the shared AI instruction template and the generated instruction docs.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
test.sh	Adds env var validation and accumulates failures instead of exiting immediately when enabled.
template/ai-instructions/shared.md	Documents the new `PDU_NO_FAIL_FAST=true` option (source template).
CLAUDE.md	Documents the new `PDU_NO_FAIL_FAST=true` option (generated instructions).
AGENTS.md	Documents the new `PDU_NO_FAIL_FAST=true` option (generated instructions).
.github/copilot-instructions.md	Documents the new `PDU_NO_FAIL_FAST=true` option (generated instructions).

github-actions · 2026-03-23T00:15:26Z

Performance Regression Reports

commit: e3500a9

--quantity=apparent-size --max-depth=1 --min-ratio=0.01

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`pdu`	109.1 ± 15.3	94.3	133.3	1.14 ± 0.16
`pdu-0.20.0`	95.4 ± 0.5	94.1	96.7	1.00

Logs

Benchmark 1: pdu
  Time (mean ± σ):     109.1 ms ±  15.3 ms    [User: 91.0 ms, System: 281.0 ms]
  Range (min … max):    94.3 ms … 133.3 ms    22 runs
 
Benchmark 2: pdu-0.20.0
  Time (mean ± σ):      95.4 ms ±   0.5 ms    [User: 92.1 ms, System: 280.8 ms]
  Range (min … max):    94.1 ms …  96.7 ms    31 runs
 
Summary
  pdu-0.20.0 ran
    1.14 ± 0.16 times faster than pdu

JSON

{
  "results": [
    {
      "command": "pdu",
      "mean": 0.10906773951090908,
      "stddev": 0.015256948607487041,
      "median": 0.10250157092,
      "user": 0.09095428181818183,
      "system": 0.2809843218181818,
      "min": 0.09431295792,
      "max": 0.13327028792,
      "times": [
        0.13327028792,
        0.12798016892000003,
        0.11930590692,
        0.12199808792,
        0.12587480692000003,
        0.12544955392,
        0.12528852492,
        0.12551411092,
        0.12522446492000003,
        0.11792840192000001,
        0.10911334192,
        0.09516013892,
        0.09446259692,
        0.09588979992,
        0.09487316292,
        0.09431295792,
        0.09434927492,
        0.09447484992,
        0.09471591992,
        0.09472889392,
        0.09445347992,
        0.09512153692
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    },
    {
      "command": "pdu-0.20.0",
      "mean": 0.09540807175870969,
      "stddev": 0.0005344714088904479,
      "median": 0.09534934192,
      "user": 0.09206647096774194,
      "system": 0.28081568838709675,
      "min": 0.09406559592,
      "max": 0.09674943592,
      "times": [
        0.09531312092,
        0.09592598792,
        0.09589697392,
        0.09674943592,
        0.09541420192,
        0.09559835692,
        0.09576501792,
        0.09512688992,
        0.09537883692,
        0.09586499392,
        0.09491617392,
        0.09557271792,
        0.09491572192,
        0.09511620692,
        0.09476780592,
        0.09660609792,
        0.09520186692,
        0.09528962092,
        0.09491986992,
        0.09574538092,
        0.09487910292,
        0.09534934192,
        0.09527437192,
        0.09497265892,
        0.09406559592,
        0.09512544392,
        0.09614286292,
        0.09531963692,
        0.09555154892,
        0.09537945792,
        0.09550492292
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

- Use mktemp -d with a marker file inside to avoid path reuse - Quote $failure_marker in the final check for consistency - Use $* instead of $@ in error message for unambiguous logging https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Replace explicit rm -rf calls with a trap to ensure cleanup happens even when errexit terminates the script early in fail-fast mode. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

…mode Use `printf '%q'` with `"$@"` to preserve argument quoting and capture exit status into a variable for reliability. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

claude added 5 commits March 22, 2026 23:01

fix(test): move no-fail-fast logic into run_if and use single quotes

791a4ea

https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

docs(ai): remove conditional preamble from no-fail-fast instruction

992bd09

https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

docs(ai): remove conditional preamble from no-fail-fast instruction i…

8954643

…n template Update the source template and regenerate all AI instruction files. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

KSXGitHub requested a review from Copilot March 23, 2026 00:07

Copilot started reviewing on behalf of KSXGitHub March 23, 2026 00:08 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread test.sh Outdated

Comment thread test.sh Outdated

Comment thread test.sh Outdated

fix(test): address review feedback on test.sh

d618017

- Use mktemp -d with a marker file inside to avoid path reuse - Quote $failure_marker in the final check for consistency - Use $* instead of $@ in error message for unambiguous logging https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

KSXGitHub requested a review from Copilot March 23, 2026 00:31

Copilot started reviewing on behalf of KSXGitHub March 23, 2026 00:32 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread test.sh

claude added 2 commits March 23, 2026 00:43

fix(test): use EXIT trap to clean up temporary directory

1c09536

Replace explicit rm -rf calls with a trap to ensure cleanup happens even when errexit terminates the script early in fail-fast mode. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

chore(git): merge from master

e192d96

https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

KSXGitHub requested a review from Copilot March 23, 2026 01:00

Copilot started reviewing on behalf of KSXGitHub March 23, 2026 01:00 View session

chore(git): merge from master

9791346

Copilot AI reviewed Mar 23, 2026

View reviewed changes

Comment thread test.sh Outdated

fix(test): improve error logging for failed commands in no-fail-fast …

0f66d5b

…mode Use `printf '%q'` with `"$@"` to preserve argument quoting and capture exit status into a variable for reliability. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe

KSXGitHub requested a review from Copilot March 23, 2026 01:32

Copilot started reviewing on behalf of KSXGitHub March 23, 2026 01:33 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

KSXGitHub marked this pull request as ready for review March 23, 2026 02:00

KSXGitHub merged commit ad1ee65 into master Mar 23, 2026
16 of 17 checks passed

KSXGitHub deleted the claude/ai-compatible-tests-nJYCF branch March 23, 2026 02:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: "no fail fast" test mode#367

chore: "no fail fast" test mode#367
KSXGitHub merged 10 commits into
masterfrom
claude/ai-compatible-tests-nJYCF

KSXGitHub commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Mar 23, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

KSXGitHub commented Mar 23, 2026

Summary

Key Changes

Implementation Details

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Regression Reports

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Mar 23, 2026 •

edited

Loading