Skip to content

ci: add python script to summarize github actions failure logs#10776

Open
Vaibhav-m-patil wants to merge 1 commit into
The-OpenROAD-Project:masterfrom
Vaibhav-m-patil:ci-failure-summarizer
Open

ci: add python script to summarize github actions failure logs#10776
Vaibhav-m-patil wants to merge 1 commit into
The-OpenROAD-Project:masterfrom
Vaibhav-m-patil:ci-failure-summarizer

Conversation

@Vaibhav-m-patil

Copy link
Copy Markdown

Summary

Added a foundational Python script (.github/scripts/ci_summarizer.py) to parse GitHub Actions CI logs and output a clean Markdown summary of critical errors and test failures.

Type of Change

  • New feature

Impact

Improves the developer experience by automatically extracting ERROR and FAIL: flags from massive raw CI logs, making it much faster to identify why a build failed.

Verification

  • I have verified that the local build succeeds (./etc/Build.sh).
  • I have run the relevant tests and they pass.
  • My code follows the repository's formatting guidelines.
  • I have signed my commits (DCO).

Related Issues

Fixes #10335

Signed-off-by: Vaibhav-m-patil <vaibhav.mutthe17@gmail.com>
@Vaibhav-m-patil Vaibhav-m-patil requested a review from a team as a code owner June 29, 2026 13:08

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new Python script, ci_summarizer.py, to parse CI log files and generate a Markdown summary of errors and failed tests. The review feedback identifies critical improvements for robustness and pipeline integration, including handling potential UnicodeDecodeError exceptions by using errors='replace' when opening log files, and properly directing error messages to stderr with non-zero exit codes during file-not-found or incorrect usage scenarios.

errors_found = []

try:
with open(log_file_path, 'r', encoding='utf-8') as file:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

CI logs can sometimes contain invalid UTF-8 byte sequences (e.g., from compiler warnings, binary outputs, or localized tools). Opening the file with strict UTF-8 decoding can cause the script to crash with a UnicodeDecodeError, preventing the summary from being generated.

Using errors='replace' or errors='ignore' ensures the script remains robust and processes the entire log file successfully.

Suggested change
with open(log_file_path, 'r', encoding='utf-8') as file:
with open(log_file_path, 'r', encoding='utf-8', errors='replace') as file:

Comment on lines +21 to +23
except FileNotFoundError:
print(f"Error: Could not find log file at {log_file_path}")
return

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If the log file is not found, the script prints an error message to standard output (stdout) and exits with a successful status code (0). In CI/CD environments, execution failures should be printed to standard error (stderr) and exit with a non-zero status code (e.g., 1) to properly signal the failure to the calling pipeline.

Suggested change
except FileNotFoundError:
print(f"Error: Could not find log file at {log_file_path}")
return
except FileNotFoundError:
print(f"Error: Could not find log file at {log_file_path}", file=sys.stderr)
sys.exit(1)

Comment on lines +36 to +39
if len(sys.argv) < 2:
print("Usage: python3 ci_summarizer.py <path_to_log_file>")
else:
parse_ci_log(sys.argv[1]) No newline at end of file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When the script is invoked with incorrect arguments, it prints the usage message to standard output (stdout) and exits with a successful status code (0). CLI usage errors should be printed to standard error (stderr) and exit with a non-zero status code (e.g., 1) to prevent silent failures in pipelines.

Suggested change
if len(sys.argv) < 2:
print("Usage: python3 ci_summarizer.py <path_to_log_file>")
else:
parse_ci_log(sys.argv[1])
if len(sys.argv) < 2:
print("Usage: python3 ci_summarizer.py <path_to_log_file>", file=sys.stderr)
sys.exit(1)
else:
parse_ci_log(sys.argv[1])

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to OpenROAD! Thanks for opening your first PR.
Before we review:

Please ensure:

  • CI passes
  • Code is properly formatted
  • Tests are included where applicable
    A maintainer will review shortly!

@maliberty maliberty requested a review from vvbandeira June 29, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: human-readable CI failure summary on PR

1 participant