Skip to content

Commit ec803b8

Browse files
committed
Fix cache warmer: run exact CI commands to cache all plugins
Stop guessing which plugins are needed. Instead, run the EXACT same Maven commands from prCheck.yml, prIntegrationTests.yml, and coverageReport.yml. This is the only reliable way to ensure every plugin, provider, and transitive dependency is resolved and cached. Commands mirror: spotless:check, install, Arrow tests (3 variants), unit tests with jacoco, integration test compile, resolve-plugins. Signed-off-by: Gopal Lal <gopal.lal@databricks.com> Co-authored-by: Isaac Signed-off-by: Gopal Lal <gopal.lal@databricks.com>
1 parent a00b908 commit ec803b8

1 file changed

Lines changed: 25 additions & 22 deletions

File tree

.github/workflows/warmMavenCache.yml

Lines changed: 25 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -131,33 +131,36 @@ jobs:
131131
run: |
132132
set -euo pipefail
133133
134-
# Step 1: Install all modules — resolves external dependencies from JFrog and
135-
# installs inter-module SNAPSHOTs (e.g., jdbc-core used by assembly-thin/uber).
136-
echo "=== Step 1: Installing all modules ==="
137-
mvn -B install -DskipTests -Dmaven.javadoc.skip=true -Dmaven.source.skip=true -Ddependency-check.skip=true
138-
139-
# Step 2: Run the exact same commands that PR workflows use.
140-
# This ensures ALL plugins, providers, and metadata are resolved and cached,
141-
# including test-time artifacts (surefire-junit-platform, jacoco agent) and
142-
# build-time plugins (spotless, toolchains, owasp) that mvn install alone
143-
# doesn't trigger.
144-
145-
# Step 2: Run a real unit test to trigger full surefire provider resolution.
146-
# surefire-junit-platform is resolved lazily at test execution time, not at
147-
# plugin initialization. We need at least one test to actually run.
148-
echo "=== Step 2: Running a single unit test to resolve surefire provider ==="
149-
mvn -B -pl jdbc-core test -Dtest="DatabricksParameterMetaDataTest#testInitialization" -Ddependency-check.skip=true || true
150-
151-
echo "=== Step 3: Running spotless check ==="
134+
# Run the EXACT same Maven commands as the PR CI workflows.
135+
# This is the only reliable way to ensure every plugin, provider,
136+
# and transitive dependency is resolved and cached. Each command
137+
# mirrors a real CI step from prCheck.yml, prIntegrationTests.yml,
138+
# or coverageReport.yml.
139+
140+
echo "=== 1/8: spotless:check (formatting-check job) ==="
152141
mvn -B --errors spotless:check || true
153142
154-
# Step 4: Run jacoco with a real test to resolve jacoco agent + report plugins
155-
echo "=== Step 4: Running jacoco coverage ==="
156-
mvn -B -pl jdbc-core test -Dtest="DatabricksParameterMetaDataTest#testInitialization" jacoco:report -Ddependency-check.skip=true || true
143+
echo "=== 2/8: install all modules (packaging-tests job) ==="
144+
mvn -B -pl jdbc-core,assembly-uber,assembly-thin clean install -DskipTests -Dmaven.javadoc.skip=true -Dmaven.source.skip=true -Ddependency-check.skip=true
145+
146+
echo "=== 3/8: Arrow Patch Tests (unit-tests job, JDK 17+) ==="
147+
mvn -B -Pjdk21-NioNotOpen -pl jdbc-core test -Dgroups='Jvm17PlusAndArrowToNioReflectionDisabled' -Ddependency-check.skip=true || true
148+
149+
echo "=== 4/8: Arrow Allocator Tests (unit-tests job, JDK 17+) ==="
150+
mvn -B -Pjdk21-NioNotOpen -pl jdbc-core test -Dgroups='Jvm17PlusAndArrowToNioReflectionDisabled' -Dtest="ArrowBufferAllocatorNettyManagerTest,ArrowBufferAllocatorUnsafeManagerTest,ArrowBufferAllocatorUnknownManagerTest" -DforkCount=1 -DreuseForks=false -Ddependency-check.skip=true || true
157151
158-
echo "=== Step 5: Running integration test compilation ==="
152+
echo "=== 5/8: Arrow Memory Tests (unit-tests job) ==="
153+
mvn -B -Plow-memory -pl jdbc-core test -Dtest='DatabricksArrowPatchMemoryUsageTest' -Ddependency-check.skip=true || true
154+
155+
echo "=== 6/8: Unit Tests with jacoco (unit-tests job) ==="
156+
mvn -B -pl jdbc-core clean test -Dtest="DatabricksParameterMetaDataTest#testInitialization" -Dgroups='!Jvm17PlusAndArrowToNioReflectionDisabled' jacoco:report -Ddependency-check.skip=true || true
157+
158+
echo "=== 7/8: Integration test compile (prIntegrationTests job) ==="
159159
mvn -B -pl jdbc-core compile test-compile -Ddependency-check.skip=true || true
160160
161+
echo "=== 8/8: Resolve all declared plugins ==="
162+
mvn -B -pl jdbc-core dependency:resolve-plugins -Ddependency-check.skip=true || true
163+
161164
echo "Dependency resolution complete"
162165
163166
- name: Normalize _remote.repositories before saving cache

0 commit comments

Comments
 (0)