Skip to content

Commit 922bf1c

Browse files
committed
ci: scope Spark SQL trigger paths to per-version shims and diff
Each spark_sql_test_<v>.yml currently triggers on `spark/src/main/**` and `dev/diffs/**`, so a change confined to one Spark version's shim or diff still fans out and runs the other versions' jobs. Tighten the path allow-list per version: - exclude unrelated `spark/src/main/spark-3.4/`, `spark-3.5/`, `spark-3.x/`, `spark-4.0/`, `spark-4.1/`, `spark-4.2/`, `spark-4.x/` directories so a 3.4-only shim edit never fires the 4.x workflows, a 4.1-only shim edit never fires the 3.x workflows, and a future `spark-4.3/` shim won't trigger any 3.x workflow either - replace `dev/diffs/**` with the single `dev/diffs/<full-version>.diff` the workflow actually applies, which also stops `dev/diffs/iceberg/` edits from triggering the Spark SQL test workflows
1 parent 48f7b03 commit 922bf1c

4 files changed

Lines changed: 36 additions & 6 deletions

File tree

.github/workflows/spark_sql_test_3_4.yml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,14 @@ on:
3535
- "common/src/main/**"
3636
- "common/pom.xml"
3737
- "spark/src/main/**"
38+
- "!spark/src/main/spark-3.5/**"
39+
- "!spark/src/main/spark-4.0/**"
40+
- "!spark/src/main/spark-4.1/**"
41+
- "!spark/src/main/spark-4.2/**"
42+
- "!spark/src/main/spark-4.x/**"
3843
- "!spark/src/main/scala/org/apache/comet/GenerateDocs.scala"
3944
- "spark/pom.xml"
40-
- "dev/diffs/**"
45+
- "dev/diffs/3.4.3.diff"
4146
- "pom.xml"
4247
- "rust-toolchain.toml"
4348
- ".github/workflows/spark_sql_test_3_4.yml"

.github/workflows/spark_sql_test_3_5.yml

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,14 @@ on:
3535
- "common/src/main/**"
3636
- "common/pom.xml"
3737
- "spark/src/main/**"
38+
- "!spark/src/main/spark-3.4/**"
39+
- "!spark/src/main/spark-4.0/**"
40+
- "!spark/src/main/spark-4.1/**"
41+
- "!spark/src/main/spark-4.2/**"
42+
- "!spark/src/main/spark-4.x/**"
3843
- "!spark/src/main/scala/org/apache/comet/GenerateDocs.scala"
3944
- "spark/pom.xml"
40-
- "dev/diffs/**"
45+
- "dev/diffs/3.5.8.diff"
4146
- "pom.xml"
4247
- "rust-toolchain.toml"
4348
- ".github/workflows/spark_sql_test_3_5.yml"
@@ -54,9 +59,14 @@ on:
5459
- "common/src/main/**"
5560
- "common/pom.xml"
5661
- "spark/src/main/**"
62+
- "!spark/src/main/spark-3.4/**"
63+
- "!spark/src/main/spark-4.0/**"
64+
- "!spark/src/main/spark-4.1/**"
65+
- "!spark/src/main/spark-4.2/**"
66+
- "!spark/src/main/spark-4.x/**"
5767
- "!spark/src/main/scala/org/apache/comet/GenerateDocs.scala"
5868
- "spark/pom.xml"
59-
- "dev/diffs/**"
69+
- "dev/diffs/3.5.8.diff"
6070
- "pom.xml"
6171
- "rust-toolchain.toml"
6272
- ".github/workflows/spark_sql_test_3_5.yml"

.github/workflows/spark_sql_test_4_0.yml

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,14 @@ on:
3535
- "common/src/main/**"
3636
- "common/pom.xml"
3737
- "spark/src/main/**"
38+
- "!spark/src/main/spark-3.4/**"
39+
- "!spark/src/main/spark-3.5/**"
40+
- "!spark/src/main/spark-3.x/**"
41+
- "!spark/src/main/spark-4.1/**"
42+
- "!spark/src/main/spark-4.2/**"
3843
- "!spark/src/main/scala/org/apache/comet/GenerateDocs.scala"
3944
- "spark/pom.xml"
40-
- "dev/diffs/**"
45+
- "dev/diffs/4.0.2.diff"
4146
- "pom.xml"
4247
- "rust-toolchain.toml"
4348
- ".github/workflows/spark_sql_test_4_0.yml"
@@ -54,9 +59,14 @@ on:
5459
- "common/src/main/**"
5560
- "common/pom.xml"
5661
- "spark/src/main/**"
62+
- "!spark/src/main/spark-3.4/**"
63+
- "!spark/src/main/spark-3.5/**"
64+
- "!spark/src/main/spark-3.x/**"
65+
- "!spark/src/main/spark-4.1/**"
66+
- "!spark/src/main/spark-4.2/**"
5767
- "!spark/src/main/scala/org/apache/comet/GenerateDocs.scala"
5868
- "spark/pom.xml"
59-
- "dev/diffs/**"
69+
- "dev/diffs/4.0.2.diff"
6070
- "pom.xml"
6171
- "rust-toolchain.toml"
6272
- ".github/workflows/spark_sql_test_4_0.yml"

.github/workflows/spark_sql_test_4_1.yml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,14 @@ on:
3535
- "common/src/main/**"
3636
- "common/pom.xml"
3737
- "spark/src/main/**"
38+
- "!spark/src/main/spark-3.4/**"
39+
- "!spark/src/main/spark-3.5/**"
40+
- "!spark/src/main/spark-3.x/**"
41+
- "!spark/src/main/spark-4.0/**"
42+
- "!spark/src/main/spark-4.2/**"
3843
- "!spark/src/main/scala/org/apache/comet/GenerateDocs.scala"
3944
- "spark/pom.xml"
40-
- "dev/diffs/**"
45+
- "dev/diffs/4.1.1.diff"
4146
- "pom.xml"
4247
- "rust-toolchain.toml"
4348
- ".github/workflows/spark_sql_test_4_1.yml"

0 commit comments

Comments
 (0)