Skip to content

ci: only check new links on pull requests to avoid rate limiting#4522

Merged
xrmx merged 2 commits intoopen-telemetry:mainfrom
honeycombio:mike/ci-check-links-diff-only
May 4, 2026
Merged

ci: only check new links on pull requests to avoid rate limiting#4522
xrmx merged 2 commits intoopen-telemetry:mainfrom
honeycombio:mike/ci-check-links-diff-only

Conversation

@MikeGoldsmith
Copy link
Copy Markdown
Member

Description

The check-links CI job checks all links in changed markdown files. For files like CHANGELOG.md that contain hundreds of URLs, this triggers HTTP 502 rate limits from GitHub, causing spurious CI failures across many PRs.

Changes

  • Pull requests: only URLs from added lines in the diff are extracted and checked. This avoids re-checking hundreds of existing links that haven't changed.
  • Push to main: unchanged — all links in changed files are still checked for full validation.
  • Added fetch-depth: 0 to the checkout step so git diff has access to the base branch.

How it works

git diff origin/$base...HEAD -- <changed files>
  | grep '^+'           # only added lines
  | grep -v '^+++'      # skip diff headers
  | grep -oP 'https?:…' # extract URLs
  | sort -u             # deduplicate

The extracted URLs are written to a temporary markdown file and passed to markdown-link-check.

Mirrors open-telemetry/opentelemetry-python#5162 which reduced the check-links job from 17 minutes to 20 seconds.

Large files like CHANGELOG.md contain hundreds of links. Checking
all of them on every PR triggers HTTP 502 rate limits. On PRs, now
only URLs from added lines in the diff are checked. Pushes to main
still check all links in changed files for full validation.

Mirrors open-telemetry/opentelemetry-python#5162.

Assisted-by: Claude Opus 4.6
@MikeGoldsmith MikeGoldsmith requested a review from a team as a code owner May 1, 2026 08:58
@MikeGoldsmith MikeGoldsmith moved this to Easy to review / merge / close in Python PR digest May 1, 2026
@github-project-automation github-project-automation Bot moved this from Easy to review / merge / close to Approved PRs in Python PR digest May 1, 2026
@lzchen lzchen added the Skip Changelog PRs that do not require a CHANGELOG.md entry label May 1, 2026
@xrmx xrmx merged commit 1cd4374 into open-telemetry:main May 4, 2026
752 checks passed
@github-project-automation github-project-automation Bot moved this from Approved PRs to Done in Python PR digest May 4, 2026
@MikeGoldsmith MikeGoldsmith deleted the mike/ci-check-links-diff-only branch May 6, 2026 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Skip Changelog PRs that do not require a CHANGELOG.md entry

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants