Skip to content

Enforce Google Maven Mirror on CI environments#38586

Merged
shunping merged 5 commits into
apache:masterfrom
shunping:maven-mirror
May 21, 2026
Merged

Enforce Google Maven Mirror on CI environments#38586
shunping merged 5 commits into
apache:masterfrom
shunping:maven-mirror

Conversation

@shunping

@shunping shunping commented May 21, 2026

Copy link
Copy Markdown
Collaborator

Redirection of Maven Central repository resolution to a Google Maven Mirror when running inside CI to bypass intermittent HTTP 403 Forbidden/IP throttling errors on Maven Central during build configuration.

Example error log from gradle:

* What went wrong:
A problem occurred configuring project ':buildSrc'.
> Could not resolve all artifacts for configuration 'classpath'.
   > Could not download spotless-lib-extra-2.7.0.jar (com.diffplug.spotless:spotless-lib-extra:2.7.0)
      > Could not get resource 'https://plugins.gradle.org/m2/com/diffplug/spotless/spotless-lib-extra/2.7.0/spotless-lib-extra-2.7.0.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/diffplug/spotless/spotless-lib-extra/2.7.0/spotless-lib-extra-2.7.0.jar'. Received status code 403 from server: Forbidden
   > Could not download spotless-lib-2.7.0.jar (com.diffplug.spotless:spotless-lib:2.7.0)
      > Could not get resource 'https://plugins.gradle.org/m2/com/diffplug/spotless/spotless-lib/2.7.0/spotless-lib-2.7.0.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/diffplug/spotless/spotless-lib/2.7.0/spotless-lib-2.7.0.jar'. Received status code 403 from server: Forbidden
   > Could not download durian-io-1.2.0.jar (com.diffplug.durian:durian-io:1.2.0)
      > Could not get resource 'https://plugins.gradle.org/m2/com/diffplug/durian/durian-io/1.2.0/durian-io-1.2.0.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/diffplug/durian/durian-io/1.2.0/durian-io-1.2.0.jar'. Received status code 403 from server: Forbidden
   > Could not download durian-collect-1.2.0.jar (com.diffplug.durian:durian-collect:1.2.0)
      > Could not get resource 'https://plugins.gradle.org/m2/com/diffplug/durian/durian-collect/1.2.0/durian-collect-1.2.0.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/diffplug/durian/durian-collect/1.2.0/durian-collect-1.2.0.jar'. Received status code 403 from server: Forbidden
   > Could not download durian-core-1.2.0.jar (com.diffplug.durian:durian-core:1.2.0)
      > Could not get resource 'https://plugins.gradle.org/m2/com/diffplug/durian/durian-core/1.2.0/durian-core-1.2.0.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/diffplug/durian/durian-core/1.2.0/durian-core-1.2.0.jar'. Received status code 403 from server: Forbidden
   > Could not download org.eclipse.jgit-5.8.0.202006091008-r.jar (org.eclipse.jgit:org.eclipse.jgit:5.8.0.202006091008-r)
      > Could not get resource 'https://plugins.gradle.org/m2/org/eclipse/jgit/org.eclipse.jgit/5.8.0.202006091008-r/org.eclipse.jgit-5.8.0.202006091008-r.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/org/eclipse/jgit/org.eclipse.jgit/5.8.0.202006091008-r/org.eclipse.jgit-5.8.0.202006091008-r.jar'. Received status code 403 from server: Forbidden
   > Could not download concurrent-trees-2.6.1.jar (com.googlecode.concurrent-trees:concurrent-trees:2.6.1)
      > Could not get resource 'https://plugins.gradle.org/m2/com/googlecode/concurrent-trees/concurrent-trees/2.6.1/concurrent-trees-2.6.1.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/googlecode/concurrent-trees/concurrent-trees/2.6.1/concurrent-trees-2.6.1.jar'. Received status code 403 from server: Forbidden
   > Could not download groovy-xml-3.0.3.jar (org.codehaus.groovy:groovy-xml:3.0.3)
      > Could not get resource 'https://plugins.gradle.org/m2/org/codehaus/groovy/groovy-xml/3.0.3/groovy-xml-3.0.3.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/org/codehaus/groovy/groovy-xml/3.0.3/groovy-xml-3.0.3.jar'. Received status code 403 from server: Forbidden
   > Could not download JavaEWAH-1.1.7.jar (com.googlecode.javaewah:JavaEWAH:1.1.7)
      > Could not get resource 'https://plugins.gradle.org/m2/com/googlecode/javaewah/JavaEWAH/1.1.7/JavaEWAH-1.1.7.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/com/googlecode/javaewah/JavaEWAH/1.1.7/JavaEWAH-1.1.7.jar'. Received status code 403 from server: Forbidden
   > Could not download slf4j-api-1.7.2.jar (org.slf4j:slf4j-api:1.7.2)
      > Could not get resource 'https://plugins.gradle.org/m2/org/slf4j/slf4j-api/1.7.2/slf4j-api-1.7.2.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.2/slf4j-api-1.7.2.jar'. Received status code 403 from server: Forbidden
   > Could not download groovy-3.0.3.jar (org.codehaus.groovy:groovy:3.0.3)
      > Could not get resource 'https://plugins.gradle.org/m2/org/codehaus/groovy/groovy/3.0.3/groovy-3.0.3.jar'.
         > Could not GET 'https://repo.maven.apache.org/maven2/org/codehaus/groovy/groovy/3.0.3/groovy-3.0.3.jar'. Received status code 403 from server: Forbidden

@github-actions github-actions Bot added the build label May 21, 2026
@shunping shunping changed the title Attempt to fix recent hiccup of official maven repo Add dynamic Google Maven Mirror fallback on Maven Central block/throttle May 21, 2026
@shunping shunping changed the title Add dynamic Google Maven Mirror fallback on Maven Central block/throttle Fallback to Google Maven Mirror on Maven Central block/throttle May 21, 2026
@shunping

Copy link
Copy Markdown
Collaborator Author

r: @Abacn

@shunping
shunping marked this pull request as ready for review May 21, 2026 14:57
@github-actions

Copy link
Copy Markdown
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a robust fallback mechanism for dependency resolution in the build process. By dynamically verifying connectivity to Maven Central, the build system can now automatically switch to the Google Maven Mirror if it detects throttling or blocking, preventing build failures caused by network restrictions on shared CI runners.

Highlights

  • Resilience to Maven Central Throttling: Implemented a runtime connectivity check to Maven Central across build scripts and settings to detect 403 Forbidden or connection errors.
  • Automatic Fallback Mechanism: Introduced a fallback to the Google Maven Mirror when Maven Central is unreachable, ensuring build stability on restricted environments like GitHub Actions runners.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a fallback mechanism to the Google Maven Mirror when Maven Central is inaccessible, covering buildSrc, project repositories, and plugin management. The review feedback identifies a bug in the offline dependencies configuration where the Gradle Plugin Portal was incorrectly excluded during fallback. Additionally, the reviewer suggests simplifying the conditional repository logic to remove duplication and recommends centralizing the accessibility check logic and constants to improve maintainability across the different build scripts.

Comment thread buildSrc/src/main/groovy/org/apache/beam/gradle/Repositories.groovy Outdated
Comment thread buildSrc/build.gradle.kts Outdated
Comment thread settings.gradle.kts Outdated
Comment thread buildSrc/src/main/groovy/org/apache/beam/gradle/Repositories.groovy Outdated
@shunping
shunping marked this pull request as draft May 21, 2026 15:54
Comment thread buildSrc/settings.gradle.kts Outdated
@shunping
shunping marked this pull request as ready for review May 21, 2026 16:05

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a connectivity check for Maven Central, allowing the build to fall back to a mirror if the primary repository is inaccessible. This logic is implemented in buildSrc, the main settings.gradle.kts, and a shared Repositories utility. Reviewer feedback suggests improving the implementation by using java.net.URI to avoid deprecated URL constructors, caching the connectivity status in a system property to prevent redundant network calls, and utilizing Gradle's lifecycle logging instead of standard output.

Comment thread buildSrc/src/main/groovy/org/apache/beam/gradle/Repositories.groovy Outdated
Comment thread buildSrc/src/main/groovy/org/apache/beam/gradle/Repositories.groovy Outdated
Comment thread buildSrc/src/main/groovy/org/apache/beam/gradle/Repositories.groovy Outdated
Comment thread buildSrc/src/main/groovy/org/apache/beam/gradle/Repositories.groovy Outdated
Comment thread settings.gradle.kts Outdated
Comment thread settings.gradle.kts Outdated
Comment thread buildSrc/settings.gradle.kts Outdated
@shunping shunping changed the title Fallback to Google Maven Mirror on Maven Central block/throttle Enforce Google Maven Mirror on CI environments May 21, 2026
Comment thread settings.gradle.kts
Comment thread buildSrc/settings.gradle.kts Outdated
Comment thread settings.gradle.kts
val useMirror = isCi && !mavenCentralMirrorUrl.isNullOrBlank()

if (useMirror) {
logger.lifecycle("Running in CI. Mirroring Maven Central repositories via Google Maven Mirror.")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This log appears multiple times. Logging here once should suffice

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep them for now. We can remove them later if the infra problem is resolved.

@shunping
shunping merged commit 60620e7 into apache:master May 21, 2026
19 of 21 checks passed
@shunping
shunping deleted the maven-mirror branch May 21, 2026 18:13
ash6898 pushed a commit to ash6898/beam that referenced this pull request Jun 29, 2026
* Implement Maven Central mirroring in CI environments

* Fix subproject buildscript classpath resolution for beam-test-gha using CI mirror fallback

* Remove duplicated definition.

* Discard the new settings file in buildSrc first.

* Fix compilation error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants