Skip to content

[1.3] Add CI mirror and JDK group excludes to avoid Maven Central throttling#22323

Merged
peterzhuamazon merged 2 commits into
opensearch-project:1.3from
andrross:maven-mirror-1.3
Jun 25, 2026
Merged

[1.3] Add CI mirror and JDK group excludes to avoid Maven Central throttling#22323
peterzhuamazon merged 2 commits into
opensearch-project:1.3from
andrross:maven-mirror-1.3

Conversation

@andrross

@andrross andrross commented Jun 25, 2026

Copy link
Copy Markdown
Member

This is needed because the 2.19 branch still checks out and builds the 1.3 branch during bwc tests

Check List

  • Functionality includes testing.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Manually cherry-picked the changes from e3400ac.

Signed-off-by: Andrew Ross <andrross@amazon.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit ae52ab4.

PathLineSeverityDescription
settings.gradle14highCustom Maven repository https://ci.opensearch.org/maven2/ added as the FIRST entry in pluginManagement repositories, taking priority over gradlePluginPortal and mavenCentral. This means all Gradle plugin resolution will first query this unverified server. Any artifact served here will override the canonical plugin registry, enabling supply chain compromise of the entire build toolchain.
buildSrc/src/main/java/org/opensearch/gradle/RepositoriesSetupPlugin.java85highThe core RepositoriesSetupPlugin (applied to all subprojects) now inserts https://ci.opensearch.org/maven2/ as the primary dependency repository before mavenCentral. This gives this server first-resolver priority for all project dependencies across the entire OpenSearch build, making it a single point of supply-chain failure.
buildSrc/build.gradle97highbuildSrc itself (which builds all custom Gradle plugins and build logic) now resolves dependencies from https://ci.opensearch.org/maven2/ first. Compromise of this server during buildSrc compilation could inject malicious code into the build infrastructure itself.
gradle/ide.gradle18highA buildscript-level repository block adds https://ci.opensearch.org/maven2/ before the official Gradle plugins portal. buildscript dependencies are executed during the configuration phase with full Gradle daemon privileges, making this a high-impact supply chain vector.
gradle/code-coverage.gradle12highCustom Maven repository added as primary resolver in code-coverage configuration. The excludeGroupByRegex only excludes adoptium/adoptopenjdk/openjdk namespaces; all other artifact groups (including JaCoCo and all transitive dependencies) can be served from this server.
build.gradle171highThe authoritative version metadata URL is changed from the canonical https://repo1.maven.org/maven2/... to https://ci.opensearch.org/maven2/.... This means BWC (backwards compatibility) version checks now consume data from an internal CI server rather than the trusted Maven Central registry, potentially allowing version metadata manipulation.
buildSrc/src/integTest/groovy/org/opensearch/gradle/OpenSearchTestBasePluginFuncTest.groovy56highIntegration test build scripts now resolve dependencies from https://ci.opensearch.org/maven2/ without content filtering (no excludeGroupByRegex applied here), meaning this server can serve any artifact group to test builds.
buildSrc/src/testKit/opensearch.build/build.gradle42highCustom Maven repository added without content filtering to opensearch.build testKit, which is used to validate core build plugin behavior. Malicious artifacts served here could subvert build validation tests.
buildSrc/src/testKit/testingConventions/build.gradle21highCustom Maven repository added without content filtering to testingConventions testKit. No excludeGroupByRegex is present, so any artifact group can be resolved from https://ci.opensearch.org/maven2/.
buildSrc/src/testKit/thirdPartyAudit/build.gradle39highCustom Maven repository added without content filtering to thirdPartyAudit testKit. This kit is specifically responsible for auditing third-party dependencies; inserting an unverified upstream resolver here undermines the integrity of the audit process itself.

The table above displays the top 10 most important findings.

Total: 11 | Critical: 0 | High: 11 | Medium: 0 | Low: 0


Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.


⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

@andrross andrross added skip-changelog skip-diff-analyzer Maintainer to skip code-diff-analyzer check, after reviewing issues in AI analysis. labels Jun 25, 2026
@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

(Review updated until commit 4b38d2d)

Here are some key observations to aid the review process:

🧪 No relevant tests
🔒 No security concerns identified
✅ No TODO sections
🔀 Multiple PR themes

Sub-PR theme: Add CI Maven mirror and JDK group excludes

Relevant files:

  • buildSrc/src/main/java/org/opensearch/gradle/RepositoriesSetupPlugin.java
  • buildSrc/src/integTest/groovy/org/opensearch/gradle/OpenSearchTestBasePluginFuncTest.groovy

Sub-PR theme: Pin GitHub Actions to commit SHAs and remove unused workflows

Relevant files:

  • .github/workflows/assemble.yml
  • .github/workflows/auto-release.yml
  • .github/workflows/copy-linked-issue-labels.yml
  • .github/workflows/gradle-check.yml
  • .github/workflows/maintainer-approval.yml
  • .github/workflows/precommit.yml
  • .github/workflows/publish-maven-snapshots.yml
  • .github/workflows/version.yml
  • .github/workflows/wrapper.yml
  • .github/workflows/backport.yml
  • .github/workflows/delete_backport_branch.yml
  • .github/workflows/dependabot_pr.yml

⚡ Recommended focus areas for review

Possible Misplaced `with:` Block

The with: block containing github_token, bodyFile, etc. now appears to follow ncipollo/release-action, but in the new layout the dawidd6/action-get-tag step is immediately followed by additional - uses: lines. YAML step ordering means the with: arguments attach to the last step's uses: (the release-action), which is correct, but the previous - name: Get tag step no longer has explicit positioning relative to its uses:. Verify the steps still parse as intended and that Get tag step is not missing its uses: due to reordering during the SHA-pinning edits.

- name: Get tag
  id: tag
  uses: dawidd6/action-get-tag@727a6f0a561be04e09013531e73a3983a65e3479 # v1
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: ncipollo/release-action@339a81892b84b4eeb0f6e744e4574d79d0d9b8dd # v1
  with:
    github_token: ${{ steps.github_app_token.outputs.token }}

@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

Latest suggestions up to 4b38d2d

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Broaden JDK group exclude regex coverage

The exclusion regex adoptium.|adoptopenjdk.|openjdk. is applied without anchoring
to the start of the group ID, so Gradle's excludeGroupByRegex (which uses full-match
semantics) will exclude only groups whose entire ID matches one of these prefixes.
However, JDK artifacts are commonly published under group IDs like net.adoptium.
or
net.adoptopenjdk.*, which will NOT be matched and therefore will not be excluded.
Consider expanding the regex to cover these prefixes (e.g.
.adoptium.|.adoptopenjdk.|.openjdk.) to ensure JDK groups are reliably routed
to the dedicated JDK repository.

buildSrc/src/main/java/org/opensearch/gradle/RepositoriesSetupPlugin.java [85-90]

 repos.maven(repo -> {
     repo.setName("Maven Cache");
     repo.setUrl("https://ci.opensearch.org/maven2/");
-    repo.content(descriptor -> descriptor.excludeGroupByRegex("adoptium.*|adoptopenjdk.*|openjdk.*"));
+    repo.content(descriptor -> descriptor.excludeGroupByRegex(".*adoptium.*|.*adoptopenjdk.*|.*openjdk.*"));
 });
-repos.mavenCentral(repo -> { repo.content(descriptor -> descriptor.excludeGroupByRegex("adoptium.*|adoptopenjdk.*|openjdk.*")); });
+repos.mavenCentral(repo -> { repo.content(descriptor -> descriptor.excludeGroupByRegex(".*adoptium.*|.*adoptopenjdk.*|.*openjdk.*")); });
Suggestion importance[1-10]: 7

__

Why: The suggestion correctly identifies that Gradle's excludeGroupByRegex uses full-match semantics, so the current regex won't match common JDK group IDs like net.adoptium.*. This is a valid and impactful correctness concern for the JDK exclusion routing.

Medium

Previous suggestions

Suggestions up to commit 3cdcbff
CategorySuggestion                                                                                                                                    Impact
Possible issue
Fix mismatched action version comment

The comment says v2.1.0 but the previous version was v2.2.0. Verify the SHA pin
matches the intended version tag to avoid an unintentional downgrade that could
break backport behavior.

.github/workflows/backport.yml [36]

 - name: Backport
-  uses: VachaShah/backport@3dc1c79b5fa9d9dcd542cee70bd5d10c6e5dbbf6 # v2.1.0
+  uses: VachaShah/backport@3dc1c79b5fa9d9dcd542cee70bd5d10c6e5dbbf6 # v2.2.0
Suggestion importance[1-10]: 7

__

Why: Valid catch - the old version was v2.2.0 but the new comment says v2.1.0, which could indicate either a downgrade or an incorrect comment. Worth verifying.

Medium
General
Avoid duplicating exclude regex

The CI mirror repository is added before any mavenLocal() check ordering, but more
importantly, the same exclude regex is duplicated in two places. Extract it into a
constant to avoid drift when the JDK group list needs to be updated.

buildSrc/src/main/java/org/opensearch/gradle/RepositoriesSetupPlugin.java [85-90]

+final String jdkGroupExcludeRegex = "adoptium.*|adoptopenjdk.*|openjdk.*";
 repos.maven(repo -> {
     repo.setName("Maven Cache");
     repo.setUrl("https://ci.opensearch.org/maven2/");
-    repo.content(descriptor -> descriptor.excludeGroupByRegex("adoptium.*|adoptopenjdk.*|openjdk.*"));
+    repo.content(descriptor -> descriptor.excludeGroupByRegex(jdkGroupExcludeRegex));
 });
-repos.mavenCentral(repo -> { repo.content(descriptor -> descriptor.excludeGroupByRegex("adoptium.*|adoptopenjdk.*|openjdk.*")); });
+repos.mavenCentral(repo -> repo.content(descriptor -> descriptor.excludeGroupByRegex(jdkGroupExcludeRegex)));
Suggestion importance[1-10]: 4

__

Why: Reasonable minor refactoring to reduce duplication of the regex string, but it's a small maintainability improvement, not a functional fix.

Low
Suggestions up to commit ae52ab4
CategorySuggestion                                                                                                                                    Impact
General
Make CI mirror repository opt-in

Adding the CI mirror as the primary repository creates a hard dependency on
ci.opensearch.org being available for all builds, which could break offline or
air-gapped builds and developer environments outside the CI infrastructure. Consider
making this mirror opt-in via a system property (similar to repos.mavenLocal) or
ensure Maven Central remains fully capable of resolving all artifacts as a fallback.

buildSrc/src/main/java/org/opensearch/gradle/RepositoriesSetupPlugin.java [85-89]

-repos.maven(repo -> {
-    repo.setName("Maven Cache");
-    repo.setUrl("https://ci.opensearch.org/maven2/");
-    repo.content(descriptor -> descriptor.excludeGroupByRegex("adoptium.*|adoptopenjdk.*|openjdk.*"));
-});
+if (System.getProperty("repos.mavenCache", "true").equals("true")) {
+    repos.maven(repo -> {
+        repo.setName("Maven Cache");
+        repo.setUrl("https://ci.opensearch.org/maven2/");
+        repo.content(descriptor -> descriptor.excludeGroupByRegex("adoptium.*|adoptopenjdk.*|openjdk.*"));
+    });
+}
Suggestion importance[1-10]: 5

__

Why: Valid concern about hard dependency on ci.opensearch.org for all builds including offline/air-gapped environments, though Maven Central remains as a fallback so the impact is moderate.

Low
Anchor JDK group exclusion regex properly

The regex openjdk. will match any group starting with openjdk, including legitimate
groups like org.openjdk.
only if anchored differently, but more importantly it will
fail to exclude JDK artifacts published under groups like com.azul, org.adoptium,
etc. Anchor the regex properly and ensure dotted group IDs are handled by using \.
or making the pattern more explicit, e.g., matching the exact group prefixes used
for JDK downloads to prevent unintended matches or misses.

buildSrc/src/main/java/org/opensearch/gradle/RepositoriesSetupPlugin.java [85-90]

 repos.maven(repo -> {
     repo.setName("Maven Cache");
     repo.setUrl("https://ci.opensearch.org/maven2/");
-    repo.content(descriptor -> descriptor.excludeGroupByRegex("adoptium.*|adoptopenjdk.*|openjdk.*"));
+    repo.content(descriptor -> descriptor.excludeGroupByRegex("^(adoptium|adoptopenjdk|openjdk)(\\..*)?$"));
 });
-repos.mavenCentral(repo -> { repo.content(descriptor -> descriptor.excludeGroupByRegex("adoptium.*|adoptopenjdk.*|openjdk.*")); });
+repos.mavenCentral(repo -> { repo.content(descriptor -> descriptor.excludeGroupByRegex("^(adoptium|adoptopenjdk|openjdk)(\\..*)?$")); });
Suggestion importance[1-10]: 3

__

Why: The original regex adoptium.*|adoptopenjdk.*|openjdk.* already matches the intended JDK group prefixes (e.g., adoptium, adoptopenjdk), and excludeGroupByRegex uses full match semantics. The suggested anchoring is a minor cosmetic improvement with limited practical impact.

Low

@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 3cdcbff

Comment thread .github/workflows/delete_backport_branch.yml Outdated
Comment thread .github/workflows/backport.yml Outdated
Delete the backport workflows that are not needed on 1.3

Signed-off-by: Andrew Ross <andrross@amazon.com>
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 4b38d2d

@github-actions

Copy link
Copy Markdown
Contributor

✅ Gradle check result for 4b38d2d: SUCCESS

@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.64%. Comparing base (05f4aac) to head (4b38d2d).
⚠️ Report is 50 commits behind head on 1.3.

Additional details and impacted files
@@             Coverage Diff              @@
##                1.3   #22323      +/-   ##
============================================
+ Coverage     77.56%   77.64%   +0.07%     
+ Complexity    58760    56778    -1982     
============================================
  Files          4223     4010     -213     
  Lines        253441   243985    -9456     
  Branches      38701    37547    -1154     
============================================
- Hits         196590   189432    -7158     
+ Misses        40844    39080    -1764     
+ Partials      16007    15473     -534     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@andrross

Copy link
Copy Markdown
Member Author

@peterzhuamazon The failing macos checks here are broken. Probably not worth fixing on this end-of-life branch. The build and tests did pass on other platforms. Can you merge this?

@peterzhuamazon peterzhuamazon merged commit 3445390 into opensearch-project:1.3 Jun 25, 2026
15 of 18 checks passed
@peterzhuamazon peterzhuamazon deleted the maven-mirror-1.3 branch June 25, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog skip-diff-analyzer Maintainer to skip code-diff-analyzer check, after reviewing issues in AI analysis.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants