|
| 1 | +<!-- |
| 2 | +Licensed to the Apache Software Foundation (ASF) under one |
| 3 | +or more contributor license agreements. See the NOTICE file |
| 4 | +distributed with this work for additional information |
| 5 | +regarding copyright ownership. The ASF licenses this file |
| 6 | +to you under the Apache License, Version 2.0 (the |
| 7 | +"License"); you may not use this file except in compliance |
| 8 | +with the License. You may obtain a copy of the License at |
| 9 | +
|
| 10 | + http://www.apache.org/licenses/LICENSE-2.0 |
| 11 | +
|
| 12 | +Unless required by applicable law or agreed to in writing, |
| 13 | +software distributed under the License is distributed on an |
| 14 | +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| 15 | +KIND, either express or implied. See the License for the |
| 16 | +specific language governing permissions and limitations |
| 17 | +under the License. |
| 18 | +--> |
| 19 | + |
| 20 | +# Versioning Policy |
| 21 | + |
| 22 | +Apache DataFusion Comet follows [semantic versioning](https://semver.org/) with the format |
| 23 | +`MAJOR.MINOR.PATCH`. This document describes what each component of a release means, what |
| 24 | +compatibility guarantees Comet provides, and how Comet relates to Apache Spark versions. |
| 25 | + |
| 26 | +This policy is inspired by, but is not identical to, the |
| 27 | +[Apache Spark versioning policy](https://spark.apache.org/versioning-policy.html). The differences |
| 28 | +reflect the fact that Comet is currently pre-1.0 and ships much more frequently than Spark. |
| 29 | + |
| 30 | +## Pre-1.0 Releases |
| 31 | + |
| 32 | +Comet is currently in the `0.x` series. Per semantic versioning, the `0.x` series is considered |
| 33 | +unstable: |
| 34 | + |
| 35 | +- A minor release (`0.X.0`) may include breaking changes. |
| 36 | +- A patch release (`0.X.Y`, where `Y > 0`) contains bug fixes only. |
| 37 | + |
| 38 | +In particular, the following are explicitly **not** stable in the `0.x` series and may change in any |
| 39 | +minor release without prior notice: |
| 40 | + |
| 41 | +- Configuration keys under `spark.comet.*` (names, defaults, and semantics). |
| 42 | +- The protobuf format used to serialize query plans between the JVM and the native library. |
| 43 | +- Internal Scala, Java, and Rust APIs that are not part of the documented public API. |
| 44 | + |
| 45 | +Where a breaking change is significant, it will be called out in the release notes. |
| 46 | + |
| 47 | +## Compatibility Commitments |
| 48 | + |
| 49 | +The following commitments apply to every Comet release, including the `0.x` series. |
| 50 | + |
| 51 | +### Public Scala and Java API |
| 52 | + |
| 53 | +Public classes and methods (for example, `org.apache.spark.CometPlugin`) are considered part of |
| 54 | +Comet's public API. Removing or making source- or binary-incompatible changes to a public API |
| 55 | +requires a deprecation cycle: the API must remain available, with a deprecation warning, for at |
| 56 | +least one minor release before removal. |
| 57 | + |
| 58 | +Public APIs annotated with `@Unstable` are exempt from this guarantee and may change in any minor |
| 59 | +release without a deprecation cycle. The `@Unstable` annotation does not yet exist and will be |
| 60 | +introduced as the need arises. |
| 61 | + |
| 62 | +### Query Result Semantics |
| 63 | + |
| 64 | +Expressions and operators whose support level is `Compatible` are expected to produce results that |
| 65 | +match Apache Spark. Result differences in `Compatible` items are tracked as bugs and fixed in |
| 66 | +subsequent releases. |
| 67 | + |
| 68 | +Items whose support level is `Incompatible` or `Unsupported` have no result-compatibility |
| 69 | +guarantees. `Incompatible` items require an explicit per-expression or per-operator opt-in |
| 70 | +(for example, `spark.comet.expr.<Name>.allowIncompatible=true`). |
| 71 | + |
| 72 | +For details on per-expression and per-operator support levels, see the |
| 73 | +[compatibility guide](../user-guide/latest/compatibility/index.md). |
| 74 | + |
| 75 | +## Apache Spark Version Support |
| 76 | + |
| 77 | +Each Comet release supports the **latest patch version** of every Apache Spark minor release that |
| 78 | +Comet targets at the time of release. The currently supported Spark versions are listed on the |
| 79 | +[Spark Version Compatibility](../user-guide/latest/compatibility/spark-versions.md) page. |
| 80 | + |
| 81 | +When the upstream Spark project publishes a new patch within a supported minor (for example, |
| 82 | +`3.5.8` → `3.5.9`), the next Comet release will pick it up. Older Spark patches within the same |
| 83 | +minor are not separately supported. |
| 84 | + |
| 85 | +Dropping support for an entire Spark minor (for example, removing Spark 3.4) is announced in the |
| 86 | +release notes of the **preceding** Comet minor release, giving users at least one release of |
| 87 | +notice before the change takes effect. |
| 88 | + |
| 89 | +Comet binaries are published per `(Spark minor × Scala binary version)` combination. Users must |
| 90 | +select the binary that matches their Spark and Scala installation. |
| 91 | + |
| 92 | +## Release Cadence |
| 93 | + |
| 94 | +Comet targets a `0.X.0` minor release every four to six weeks. Patch releases (`0.X.Y`) are made |
| 95 | +on demand, only when a critical bug or security fix needs to ship before the next minor release. |
| 96 | + |
| 97 | +Only the most recent minor release receives patch releases. Comet does not currently backport |
| 98 | +fixes to older minor releases; users are expected to upgrade forward. |
| 99 | + |
| 100 | +## Native Library Coupling |
| 101 | + |
| 102 | +Each Comet release ships a JVM jar and a native library that are built and tested together. The |
| 103 | +two artifacts must come from the **same Comet release**. Mixing a JVM jar from one Comet release |
| 104 | +with a native library from another is unsupported and may fail at runtime due to protobuf or FFI |
| 105 | +incompatibilities. |
| 106 | + |
| 107 | +## Road to 1.0 |
| 108 | + |
| 109 | +When `1.0.0` ships: |
| 110 | + |
| 111 | +- Strict semantic versioning will apply to the public Scala and Java API: breaking changes will |
| 112 | + only be made in a future major release. APIs annotated with `@Unstable` remain exempt and may |
| 113 | + change in any minor release. |
| 114 | +- The stability commitments for configuration keys and the protobuf plan format will be |
| 115 | + re-evaluated and documented as part of the `1.0.0` release. |
| 116 | + |
| 117 | +Tracking and planning for the `1.0.0` release happens in |
| 118 | +[issue #4082](https://github.com/apache/datafusion-comet/issues/4082). |
0 commit comments