Skip to content

Commit 6704bed

Browse files
committed
add rgb parameter to to_legacy_anndata()
2 parents 5aa46df + c06b53f commit 6704bed

31 files changed

Lines changed: 1702 additions & 456 deletions

.github/workflows/prepare_test_data.yaml

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,10 @@ jobs:
2121
mkdir -p ./data
2222
cd ./data
2323
24+
# -------
25+
# the Xenium datasets are licensed as CC BY 4.0, as shown here
26+
# https://www.10xgenomics.com/support/software/xenium-onboard-analysis/latest/resources/xenium-example-data
27+
2428
# 10x Genomics Xenium 2.0.0
2529
curl -O https://cf.10xgenomics.com/samples/xenium/2.0.0/Xenium_V1_human_Breast_2fov/Xenium_V1_human_Breast_2fov_outs.zip
2630
curl -O https://cf.10xgenomics.com/samples/xenium/2.0.0/Xenium_V1_human_Lung_2fov/Xenium_V1_human_Lung_2fov_outs.zip
@@ -32,9 +36,35 @@ jobs:
3236
# 10x Genomics Xenium 3.0.0 (5K) Mouse ileum, nuclear expansion
3337
curl -O https://cf.10xgenomics.com/samples/xenium/3.0.0/Xenium_Prime_Mouse_Ileum_tiny/Xenium_Prime_Mouse_Ileum_tiny_outs.zip
3438
39+
# 10x Genomics Xenium 4.0.0 (v1) Human ovary, nuclear expansion
40+
curl -O https://cf.10xgenomics.com/samples/xenium/4.0.0/Xenium_V1_Human_Ovary_tiny/Xenium_V1_Human_Ovary_tiny_outs.zip
41+
42+
# 10x Genomics Xenium 4.0.0 (v1) Human ovary, multimodal cell segmentation
43+
curl -O https://cf.10xgenomics.com/samples/xenium/4.0.0/Xenium_V1_MultiCellSeg_Human_Ovary_tiny/Xenium_V1_MultiCellSeg_Human_Ovary_tiny_outs.zip
44+
45+
# 10x Genomics Xenium 4.0.0 (v1+Protein) Human kidney, multimodal cell segmentation
46+
curl -O https://cf.10xgenomics.com/samples/xenium/4.0.0/Xenium_V1_Protein_Human_Kidney_tiny/Xenium_V1_Protein_Human_Kidney_tiny_outs.zip
47+
48+
# -------
49+
# the Visium HD dataset is licensed as CC BY 4.0, as shown here
50+
# https://www.10xgenomics.com/support/software/space-ranger/latest/resources/visium-hd-example-data
51+
52+
# 10x Genomics Visium HD 4.0.1 3' Mouse Brain Chunk
53+
curl -O https://cf.10xgenomics.com/samples/spatial-exp/4.0.1/Visium_HD_Tiny_3prime_Dataset/Visium_HD_Tiny_3prime_Dataset_outs.zip
54+
55+
# -------
56+
# we received written permission to make the following dataset public and integrate it in the CI system of spatialdata-io
3557
# Spatial Genomics seqFISH v2
3658
curl -O https://s3.embl.de/spatialdata/raw_data/seqfish-2-test-dataset.zip
3759
60+
# -------
61+
# MACSima OMAP datasets are licensed as CC BY 4.0
62+
# OMAP23 for format v1.x.x
63+
curl -o OMAP23_small.zip "https://zenodo.org/api/records/18196452/files-archive"
64+
65+
# OMAP10 for format v0.x.x
66+
curl -o OMAP10_small.zip "https://zenodo.org/api/records/18196366/files-archive"
67+
3868
- name: Unzip files
3969
run: |
4070
cd ./data

.github/workflows/release.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ jobs:
1313
- name: Set up Python 3.12
1414
uses: actions/setup-python@v5
1515
with:
16-
python-version: "3.12"
16+
python-version: "3.13"
1717
cache: pip
1818
- name: Install build dependencies
1919
run: python -m pip install --upgrade pip wheel twine build

.github/workflows/test.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ jobs:
1818
strategy:
1919
fail-fast: false
2020
matrix:
21-
python: ["3.10", "3.12"]
21+
python: ["3.11", "3.12", "3.13"]
2222
os: [ubuntu-latest]
2323

2424
env:
@@ -52,7 +52,7 @@ jobs:
5252
pip install --pre -e ".[dev,test]"
5353
5454
- name: Download artifact of test data
55-
if: matrix.python == '3.12'
55+
if: matrix.python == '3.13'
5656
uses: dawidd6/action-download-artifact@v9
5757
with:
5858
workflow: prepare_test_data.yaml

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,5 +36,9 @@ node_modules/
3636
.code-workspace
3737

3838
# test datasets (e.g. Xenium ones)
39+
# symlinks
40+
data
41+
# data folder
3942
data/
4043
tests/data
44+
uv.lock

.mypy.ini

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
[mypy]
22
python_version = 3.10
3-
plugins = numpy.typing.mypy_plugin
43

54
ignore_errors = False
65
warn_redundant_casts = True

.pre-commit-config.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,17 +7,17 @@ default_stages:
77
minimum_pre_commit_version: 2.16.0
88
repos:
99
- repo: https://github.com/rbubley/mirrors-prettier
10-
rev: v3.6.2
10+
rev: v3.7.4
1111
hooks:
1212
- id: prettier
1313
- repo: https://github.com/astral-sh/ruff-pre-commit
14-
rev: v0.12.8
14+
rev: v0.14.10
1515
hooks:
1616
- id: ruff
1717
args: [--fix, --exit-non-zero-on-fix, --unsafe-fixes]
1818
- id: ruff-format
1919
- repo: https://github.com/pre-commit/mirrors-mypy
20-
rev: v1.17.1
20+
rev: v1.19.1
2121
hooks:
2222
- id: mypy
2323
additional_dependencies: [numpy, types-PyYAML]

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@
1414
[badge-pypi]: https://badge.fury.io/py/spatialdata-io.svg
1515
[link-pypi]: https://pypi.org/project/spatialdata-io/
1616

17+
_We encourage contributions from the community and from developers of spatial technologies. Please see the "How to Contribute" section below._
18+
1719
This package contains reader functions to load common spatial omics formats into SpatialData. Currently, we provide support for:
1820

1921
- 10x Genomics Visium®
@@ -32,6 +34,8 @@ This package contains reader functions to load common spatial omics formats into
3234

3335
Note: all mentioned technologies are registered trademarks of their respective companies.
3436

37+
Please refer to the list of [open Pull Requests](https://github.com/scverse/spatialdata-io/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aopen) for readers that are currently being developed.
38+
3539
## Known limitations
3640

3741
Contributions for addressing the below limitations are very welcomed.
@@ -42,9 +46,7 @@ Contributions for addressing the below limitations are very welcomed.
4246

4347
1. **Open a GitHub Issue**: Start by opening a new issue or commenting on an existing one in the repository. Clearly describe the problem and your proposed changes to avoid overlapping efforts with others.
4448

45-
2. **Submit a Pull Request (PR)**: Once the issue is discussed, submit a PR to the `spatialdata-io` repository. Ensure your PR includes information about a suitable dataset for testing the reader, ideally no larger than 10 GB. Include clear instructions for accessing the data, preferably with a `curl` or `wget` command for easy downloading.
46-
47-
3. **Optional Enhancements**: To facilitate reproducibility and ease of data access, consider adding a folder in the [spatialdata-sandbox](https://github.com/giovp/spatialdata-sandbox) repository. Include a `download.py` and `to_zarr.py` script (refer to examples in the repository) to enable others to reproduce your reader by simply running these scripts sequentially.
49+
2. **Submit a Pull Request (PR)**: Once the issue is discussed, submit a PR to the `spatialdata-io` repository. If you are contributing a new reader, or extending the reader for a new versions of a technologies, please consult our [contribution guide](https://spatialdata.scverse.org/projects/io/en/latest/contributing.html), which describes the steps to ensure that the pull request can be tested on suitable example data and reviewed efficiently.
4850

4951
## Getting started
5052

asv.conf.json

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
{
2+
"version": 1,
3+
"project": "spatialdata-io",
4+
"project_url": "https://github.com/scverse/spatialdata-io",
5+
"repo": ".",
6+
"branches": ["main", "xenium-labels-dask", "xenium-labels-dask-zipstore"],
7+
"dvcs": "git",
8+
"environment_type": "virtualenv",
9+
"pythons": ["3.12"],
10+
"build_command": [],
11+
"install_command": ["python -m pip install {build_dir}[test]"],
12+
"uninstall_command": ["python -m pip uninstall -y {project}"],
13+
"env_dir": ".asv/env",
14+
"results_dir": ".asv/results",
15+
"html_dir": ".asv/html",
16+
"benchmark_dir": "benchmarks",
17+
"hash_length": 8,
18+
"build_cache_size": 2,
19+
"install_timeout": 600,
20+
"repeat": 3,
21+
"processes": 1,
22+
"attribute_selection": ["time_*", "peakmem_*"]
23+
}

benchmarks/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# ASV benchmarks for spatialdata-io

benchmarks/bench_xenium.py

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
"""Benchmarks for SpatialData IO operations.
2+
3+
Configuration:
4+
Edit SANDBOX_DIR and DATASET below to point to your data.
5+
6+
Setup:
7+
cd <SANDBOX_DIR>/<DATASET>
8+
python download.py # use the same env where spatialdata is installed
9+
10+
Running:
11+
cd /path/to/spatialdata-io
12+
13+
# Quick benchmark (single run, for testing):
14+
asv run --python=same -b IOBenchmark --quick --show-stderr -v
15+
16+
# Full benchmark (multiple runs, for accurate results):
17+
asv run --python=same -b IOBenchmark --show-stderr -v
18+
19+
Comparing branches:
20+
# Run on specific commits:
21+
asv run main^! -b IOBenchmark --show-stderr -v
22+
asv run xenium-labels-dask^! -b IOBenchmark --show-stderr -v
23+
24+
# Or compare two branches directly:
25+
asv continuous main xenium-labels-dask -b IOBenchmark --show-stderr -v
26+
27+
# View comparison:
28+
asv compare main xenium-labels-dask
29+
30+
Results:
31+
- Console output shows timing and memory after each run
32+
- JSON results saved to: .asv/results/
33+
- Generate HTML report: asv publish && asv preview
34+
"""
35+
36+
import inspect
37+
import shutil
38+
from pathlib import Path
39+
from typing import TYPE_CHECKING
40+
41+
from spatialdata import SpatialData
42+
43+
from spatialdata_io import xenium # type: ignore[attr-defined]
44+
45+
# =============================================================================
46+
# CONFIGURATION - Edit these paths to match your setup
47+
# =============================================================================
48+
SANDBOX_DIR = Path(__file__).parent.parent.parent / "spatialdata-sandbox"
49+
DATASET = "xenium_2.0.0_io"
50+
# =============================================================================
51+
52+
53+
def get_paths() -> tuple[Path, Path]:
54+
"""Get paths for benchmark data."""
55+
path = SANDBOX_DIR / DATASET
56+
path_read = path / "data"
57+
path_write = path / "data_benchmark.zarr"
58+
59+
if not path_read.exists():
60+
raise ValueError(f"Data directory not found: {path_read}")
61+
62+
return path_read, path_write
63+
64+
65+
class IOBenchmark:
66+
"""Benchmark IO read operations."""
67+
68+
timeout = 3600
69+
repeat = 3
70+
number = 1
71+
warmup_time = 0
72+
processes = 1
73+
74+
def setup(self) -> None:
75+
"""Set up paths for benchmarking."""
76+
self.path_read, self.path_write = get_paths()
77+
if self.path_write.exists():
78+
shutil.rmtree(self.path_write)
79+
80+
def _read_xenium(self) -> SpatialData:
81+
"""Read xenium data with version-compatible kwargs."""
82+
signature = inspect.signature(xenium)
83+
kwargs = {}
84+
if "cleanup_labels_zarr_tmpdir" in signature.parameters:
85+
kwargs["cleanup_labels_zarr_tmpdir"] = False
86+
87+
return xenium(
88+
path=str(self.path_read),
89+
n_jobs=8,
90+
cell_boundaries=True,
91+
nucleus_boundaries=True,
92+
morphology_focus=True,
93+
cells_as_circles=True,
94+
**kwargs,
95+
)
96+
97+
def time_io(self) -> None:
98+
"""Walltime for data parsing."""
99+
sdata = self._read_xenium()
100+
sdata.write(self.path_write)
101+
102+
def peakmem_io(self) -> None:
103+
"""Peak memory for data parsing."""
104+
sdata = self._read_xenium()
105+
sdata.write(self.path_write)
106+
107+
108+
if __name__ == "__main__":
109+
IOBenchmark().time_io()

0 commit comments

Comments
 (0)