@@ -30,17 +30,18 @@ Make sure the following requirements are met and software installed on your mach
3030
3131### Supported Spark Versions
3232
33- Comet $COMET_VERSION supports the following versions of Apache Spark.
33+ Comet $COMET_VERSION supports the following versions of Apache Spark. Refer to the [ Spark Version Compatibility] page
34+ in the [ Compatibility Guide] for more information, such as known limitations per Spark version.
35+
36+ [ Spark Version Compatibility ] : compatibility/spark-versions.md
37+ [ Compatibility Guide ] : compatibility
3438
3539We recommend only using Comet with Spark versions where we currently have both Comet and Spark tests enabled in CI.
3640Other versions may work well enough for development and evaluation purposes.
3741
3842| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark SQL Tests in CI |
3943| ------------- | ------------ | ------------- | ----------------- | --------------------- |
4044| 3.4.3 | 11/17 | 2.12/2.13 | Yes | Yes |
41- | 3.5.5 | 11/17 | 2.12/2.13 | Yes | No |
42- | 3.5.6 | 11/17 | 2.12/2.13 | Yes | No |
43- | 3.5.7 | 11/17 | 2.12/2.13 | Yes | Yes |
4445| 3.5.8 | 11/17 | 2.12/2.13 | Yes | Yes |
4546| 4.0.2 | 17 | 2.13 | Yes | Yes |
4647
@@ -51,8 +52,8 @@ use only and should not be used in production yet.
5152
5253| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark SQL Tests in CI |
5354| -------------- | ------------ | ------------- | ----------------- | --------------------- |
54- | 4.1.1 | 17 | 2.13 | Compile only | No |
55- | 4.2.0-preview4 | 17 | 2.13 | Compile only | No |
55+ | 4.1.1 | 17 | 2.13 | Yes | No |
56+ | 4.2.0-preview4 | 17 | 2.13 | No | No |
5657
5758Note that Comet may not fully work with proprietary forks of Apache Spark such as the Spark versions offered by
5859Cloud Service Providers.
0 commit comments