Merge branch 'feat/byte-range-setter' of https://github.com/d-v-b/zarr-python into feat/byte-range-setter

Author: d-v-b

.github/ISSUE_TEMPLATE/release-checklist.md

@@ -16,39 +16,17 @@ assignees: ''
 
 **Before release**:
 
-- [ ] Make sure the release branch (e.g., `3.1.x`) is up to date with any backports.
-- [ ] Make sure that all pull requests which will be included in the release have been properly documented as changelog files in the [`changes/` directory](https://github.com/zarr-developers/zarr-python/tree/main/changes).
-- [ ] Run ``towncrier build --version x.y.z`` to create the changelog, and commit the result to the release branch.
 - [ ] Check [SPEC 0](https://scientific-python.org/specs/spec-0000/#support-window) to see if the minimum supported version of Python or NumPy needs bumping.
-- [ ] Check to ensure that:
-  - [ ] Deprecated workarounds/codes/tests are removed. Run `grep "# TODO" **/*.py` to find all potential TODOs.
-  - [ ] All tests pass in the ["Tests" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/test.yml).
-  - [ ] All tests pass in the ["GPU Tests" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/gpu_test.yml).
-  - [ ] All tests pass in the ["Hypothesis" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/hypothesis.yaml).
-  - [ ] Check that downstream libraries work well (maintainers can make executive decisions about whether all checks are required for this release).
-    - [ ] numcodecs
-    - [ ] Xarray (@jhamman @dcherian @TomNicholas)
-        - Zarr's upstream compatibility is tested via the [Upstream Dev CI worklow](https://github.com/pydata/xarray/actions/workflows/upstream-dev-ci.yaml).
-        - Click on the most recent workflow and check that the `upstream-dev` job has run and passed. `upstream-dev` is not run on all all workflow runs.
-        - Check that the expected version of Zarr-Python was tested using the `Version Info` step of the `upstream-dev` job.
-        - If testing on a branch other than `main` is needed, open a PR modifying https://github.com/pydata/xarray/blob/90ee30943aedba66a37856b2332a41264e288c20/ci/install-upstream-wheels.sh#L56 and add the `run-upstream` label.
-    - [ ] Titiler.Xarray (@maxrjones)
-        - [Modify dependencies](https://github.com/developmentseed/titiler/blob/main/src/titiler/xarray/pyproject.toml) for titiler.xarray.
-        - Modify triggers for running [the test workflow](https://github.com/developmentseed/titiler/blob/61549f2de07b20cca8fb991cfcdc89b23e18ad05/.github/workflows/ci.yml#L5-L7).
-        - Push the branch to the repository and check for the actions for any failures.
+- [ ] Verify that the latest CI workflows on `main` are passing: [Tests](https://github.com/zarr-developers/zarr-python/actions/workflows/test.yml), [GPU Tests](https://github.com/zarr-developers/zarr-python/actions/workflows/gpu_test.yml), [Hypothesis](https://github.com/zarr-developers/zarr-python/actions/workflows/hypothesis.yaml), [Docs](https://github.com/zarr-developers/zarr-python/actions/workflows/docs.yml), [Lint](https://github.com/zarr-developers/zarr-python/actions/workflows/lint.yml), [Wheels](https://github.com/zarr-developers/zarr-python/actions/workflows/releases.yml).
+- [ ] Run the ["Prepare release" workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/prepare_release.yml) with the target version. This will build the changelog and open a release PR with the `run-downstream` label.
+- [ ] Verify that the [downstream tests](https://github.com/zarr-developers/zarr-python/actions/workflows/downstream.yml) (triggered automatically by the `run-downstream` label) pass on the release PR.
+- [ ] Review the release PR and verify the changelog in `docs/release-notes.md` looks correct.
+- [ ] Merge the release PR.
 
 **Release**:
 
-- [ ] Go to https://github.com/zarr-developers/zarr-python/releases.
-  - [ ] Click "Draft a new release".
-  - [ ] Choose a version number prefixed with a `v` (e.g. `v0.0.0`). For pre-releases, include the appropriate suffix (e.g. `v0.0.0a1` or `v0.0.0rc2`).
-  - [ ] Set the target branch to the release branch (e.g., `3.1.x`)
-  - [ ] Set the description of the release to: `See release notes https://zarr.readthedocs.io/en/stable/release-notes.html#release-0-0-0`, replacing the correct version numbers. For pre-release versions, the URL should omit the pre-release suffix, e.g. "a1" or "rc1".
-  - [ ] Click on "Generate release notes" to auto-fill the description.
-  - [ ] Make a release by clicking the 'Publish Release' button, this will automatically create a tag too.
-- [ ] Verify that release workflows succeeded.
-  - [ ] The latest version is correct on [PyPI](https://pypi.org/project/zarr/).
-  - [ ] The stable version is correct on [ReadTheDocs](https://zarr.readthedocs.io/en/stable/).
+- [ ] [Draft a new GitHub Release](https://github.com/zarr-developers/zarr-python/releases/new) with tag `vX.Y.Z` targeting `main`. Use "Generate release notes" for the description.
+- [ ] Verify the release is published on [PyPI](https://pypi.org/project/zarr/) and [ReadTheDocs](https://zarr.readthedocs.io/en/stable/).
 
 **After release**:
 
@@ -57,3 +35,18 @@ assignees: ''
 ---
 
 - [ ] Party :tada:
+
+---
+
+<details>
+<summary><strong>Releasing from a branch other than main</strong></summary>
+
+In rare cases (e.g. patch releases for an older minor version), you may need to release from a dedicated release branch (e.g. `3.1.x`):
+
+- Create the release branch from the appropriate tag if it doesn't already exist.
+- Cherry-pick or backport the necessary commits onto the branch.
+- Run `towncrier build --version x.y.z` and commit the result to the release branch instead of `main`.
+- When drafting the GitHub Release, set the target to the release branch instead of `main`.
+- After the release, ensure any relevant changelog updates are also reflected on `main`.
+
+</details>

.github/workflows/downstream.yml

@@ -0,0 +1,108 @@
+name: Downstream
+
+on:
+  workflow_dispatch:
+  pull_request:
+    types: [labeled]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  xarray:
+    name: Xarray zarr backend tests
+    if: github.event_name == 'workflow_dispatch' || github.event.label.name == 'run-downstream'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out zarr-python
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Check out xarray
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          repository: pydata/xarray
+          path: xarray
+          persist-credentials: false
+
+      - name: Set up pixi
+        uses: prefix-dev/setup-pixi@19eac09b398e3d0c747adc7921926a6d802df4da # v0.8.8
+        with:
+          manifest-path: xarray/pixi.toml
+
+      - name: Install zarr-python from branch
+        working-directory: xarray
+        run: pixi run -e test-py313 -- pip install --no-deps ..
+
+      - name: Show versions
+        working-directory: xarray
+        run: |
+          pixi run -e test-py313 -- python -c "
+          import zarr; print(f'zarr {zarr.__version__}')
+          import xarray; print(f'xarray {xarray.__version__}')
+          "
+
+      - name: Run xarray zarr backend tests
+        working-directory: xarray
+        run: |
+          pixi run -e test-py313 -- python -m pytest -x --no-header -q \
+            xarray/tests/test_backends.py -k zarr \
+            xarray/tests/test_backends_api.py -k zarr \
+            xarray/tests/test_backends_datatree.py -k zarr
+
+  numcodecs:
+    name: numcodecs zarr3 codec tests
+    if: github.event_name == 'workflow_dispatch' || github.event.label.name == 'run-downstream'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out zarr-python
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+
+      - name: Check out numcodecs
+        uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          repository: zarr-developers/numcodecs
+          fetch-depth: 0
+          path: numcodecs
+          submodules: recursive
+          persist-credentials: false
+
+      - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          python-version: '3.13'
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
+
+      - name: Install numcodecs with test-zarr-main group
+        working-directory: numcodecs
+        run: |
+          uv venv
+          uv pip install --group dev
+          uv sync --group dev --group test-zarr-main
+          uv pip install --no-build-isolation -e .
+
+      - name: Override zarr-python with branch version
+        working-directory: numcodecs
+        run: uv pip install --no-deps ..
+
+      - name: Show versions
+        working-directory: numcodecs
+        run: |
+          uv run python -c "
+          import zarr; print(f'zarr {zarr.__version__}')
+          import numcodecs; print(f'numcodecs {numcodecs.__version__}')
+          "
+
+      - name: Run numcodecs zarr3 tests
+        working-directory: numcodecs
+        run: uv run python -m pytest -x --no-header -q tests/test_zarr3.py

.github/workflows/prepare_release.yml

@@ -0,0 +1,77 @@
+name: Prepare release notes
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: 'Release version notes (e.g. 3.2.0)'
+        required: true
+        type: string
+      target_branch:
+        description: 'Branch to target'
+        required: false
+        default: 'main'
+        type: string
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  prepare:
+    name: Build changelog and open PR
+    runs-on: ubuntu-latest
+    steps:
+      - name: Validate inputs
+        run: |
+          if [[ ! "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+([-\.][a-zA-Z0-9]+)*$ ]]; then
+            echo "::error::Invalid version format: '$VERSION'"
+            exit 1
+          fi
+          if [[ ! "$TARGET_BRANCH" =~ ^[a-zA-Z0-9._/-]+$ ]]; then
+            echo "::error::Invalid branch name: '$TARGET_BRANCH'"
+            exit 1
+          fi
+        env:
+          VERSION: ${{ inputs.version }}
+          TARGET_BRANCH: ${{ inputs.target_branch }}
+
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          ref: ${{ inputs.target_branch }}
+          fetch-depth: 0
+          persist-credentials: false
+
+      - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+        with:
+          python-version: '3.12'
+
+      - name: Install towncrier
+        run: pip install towncrier
+
+      - name: Build changelog
+        run: towncrier build --version "$VERSION" --yes
+        env:
+          VERSION: ${{ inputs.version }}
+
+      - name: Create pull request
+        uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
+        with:
+          branch: release/v${{ inputs.version }}
+          base: ${{ inputs.target_branch }}
+          title: "Release v${{ inputs.version }}"
+          body: |
+            Automated release preparation for v${{ inputs.version }}.
+
+            This PR was generated by the "Prepare release" workflow. It includes:
+            - Rendered changelog via `towncrier build --version ${{ inputs.version }}`
+            - Removal of consumed changelog fragments from `changes/`
+
+            ## Checklist
+
+            - [ ] Review the rendered changelog in `docs/release-notes.md`
+            - [ ] Downstream tests pass (see [downstream workflow](https://github.com/zarr-developers/zarr-python/actions/workflows/downstream.yml))
+            - [ ] Merge this PR, then [draft a GitHub Release](https://github.com/zarr-developers/zarr-python/releases/new) targeting `${{ inputs.target_branch }}` with tag `v${{ inputs.version }}`
+          commit-message: "chore: build changelog for v${{ inputs.version }}"
+          labels: run-downstream
+          delete-branch: true