Skip to content

Commit 5edcfa3

Browse files
emmanuelmathotemmanuelmathot
authored
Enhance ESA Sentinel Zarr Explorer documentation styling and configuration (#20)
* Add custom styles and configuration for ESA Sentinel Zarr Explorer documentation - Introduced extra CSS styles for improved visual presentation, including link colors, figure counters, and adjustments for Jupyter notebook outputs. - Configured mkdocs.yml with site metadata, theme settings, and navigation structure. - Enabled light and dark mode toggle with corresponding color schemes. - Added necessary markdown extensions for enhanced documentation features. * Update logo path in mkdocs configuration for ESA Sentinel Zarr Explorer * Enhance documentation: Update index and add converter guide with usage examples and advanced features
1 parent 63fd558 commit 5edcfa3

8 files changed

Lines changed: 511 additions & 199 deletions

File tree

‎.github/workflows/docs.yml‎

Lines changed: 16 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@ on:
55
branches: [ main ]
66
pull_request:
77
branches: [ main ]
8+
paths:
9+
- "docs/**"
810

911
jobs:
1012
build-docs:
@@ -17,27 +19,20 @@ jobs:
1719
with:
1820
python-version: '3.11'
1921

22+
- name: Install uv
23+
uses: astral-sh/setup-uv@v6
24+
with:
25+
version: "0.8.4"
26+
python-version: "3.13"
27+
enable-cache: false
28+
2029
- name: Install dependencies
21-
run: |
22-
sudo apt-get update
23-
sudo apt-get install -y proj-bin gdal-bin libgdal-dev
24-
python -m pip install --upgrade pip
25-
pip install -e ".[docs]" --only-binary=:all:
30+
run: uv sync --group docs
2631

27-
- name: Build documentation
28-
run: |
29-
cd docs
30-
make html
32+
- name: Build docs
33+
if: github.event_name == 'pull_request'
34+
run: uv run -- mkdocs build
3135

32-
- name: Upload documentation
33-
uses: actions/upload-artifact@v4
34-
with:
35-
name: documentation
36-
path: docs/_build/html/
37-
38-
- name: Deploy to GitHub Pages
39-
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
40-
uses: peaceiris/actions-gh-pages@v3
41-
with:
42-
github_token: ${{ secrets.GITHUB_TOKEN }}
43-
publish_dir: ./docs/_build/html
36+
- name: Deploy docs
37+
if: github.event_name == 'push'
38+
run: uv run -- mkdocs gh-deploy --force

‎.vscode/launch.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,4 +166,4 @@
166166

167167
}
168168
]
169-
}
169+
}

‎docs/converter.md‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# Using the GeoZarr Converter
2+
3+
The GeoZarr converter provides tools to transform EOPF datasets into GeoZarr-spec 0.4 compliant format. This guide explains how to use the converter effectively.
4+
5+
## Command Line Interface
6+
7+
The converter can be accessed via the `eopf-geozarr` command-line tool. Below are some common use cases:
8+
9+
### Basic Conversion
10+
11+
Convert an EOPF dataset to GeoZarr format:
12+
13+
```bash
14+
eopf-geozarr convert input.zarr output.zarr
15+
```
16+
17+
### S3 Output
18+
19+
Convert and save the output directly to an S3 bucket:
20+
21+
```bash
22+
eopf-geozarr convert input.zarr s3://my-bucket/output.zarr
23+
```
24+
25+
### Parallel Processing
26+
27+
Enable parallel processing for large datasets using a Dask cluster:
28+
29+
```bash
30+
eopf-geozarr convert input.zarr output.zarr --dask-cluster
31+
```
32+
33+
### Validation
34+
35+
Validate the GeoZarr compliance of a dataset:
36+
37+
```bash
38+
eopf-geozarr validate output.zarr
39+
```
40+
41+
## Python API
42+
43+
The converter also provides a Python API for programmatic usage:
44+
45+
### Example: Basic Conversion
46+
47+
```python
48+
import xarray as xr
49+
from eopf_geozarr import create_geozarr_dataset
50+
51+
# Load your EOPF DataTree
52+
dt = xr.open_datatree("path/to/eopf/dataset.zarr", engine="zarr")
53+
54+
# Convert to GeoZarr format
55+
dt_geozarr = create_geozarr_dataset(
56+
dt_input=dt,
57+
groups=["/measurements/r10m", "/measurements/r20m", "/measurements/r60m"],
58+
output_path="path/to/output/geozarr.zarr",
59+
spatial_chunk=4096,
60+
min_dimension=256,
61+
tile_width=256,
62+
max_retries=3
63+
)
64+
```
65+
66+
### Example: S3 Output
67+
68+
```python
69+
import os
70+
from eopf_geozarr import create_geozarr_dataset
71+
72+
# Configure S3 credentials
73+
os.environ['AWS_ACCESS_KEY_ID'] = 'your_access_key'
74+
os.environ['AWS_SECRET_ACCESS_KEY'] = 'your_secret_key'
75+
os.environ['AWS_DEFAULT_REGION'] = 'us-east-1'
76+
77+
# Convert and save to S3
78+
dt_geozarr = create_geozarr_dataset(
79+
dt_input=dt,
80+
groups=["/measurements/r10m", "/measurements/r20m", "/measurements/r60m"],
81+
output_path="s3://my-bucket/output.zarr",
82+
spatial_chunk=4096,
83+
min_dimension=256,
84+
tile_width=256,
85+
max_retries=3
86+
)
87+
```
88+
89+
## Advanced Features
90+
91+
### Chunk Alignment
92+
93+
The converter ensures proper chunk alignment to optimize storage and prevent data corruption. It uses the `calculate_aligned_chunk_size` function to determine optimal chunk sizes.
94+
95+
### Multiscale Support
96+
97+
The converter supports multiscale datasets, creating overview levels with /2 downsampling logic. Each level is stored as a sibling group (e.g., `/0`, `/1`, `/2`).
98+
99+
### Native CRS Preservation
100+
101+
The converter maintains the native coordinate reference system (CRS) of the dataset, avoiding reprojection to Web Mercator.
102+
103+
## Error Handling
104+
105+
The converter includes robust error handling and retry logic for network operations, ensuring reliable processing even in challenging environments.
106+
107+
For more details, refer to the [API Reference](api.md).

‎docs/index.md‎

Lines changed: 24 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ The EOPF GeoZarr library enables conversion of EOPF datasets to the GeoZarr spec
2222
## Key Features
2323

2424
### GeoZarr Specification Compliance
25+
2526
- Full compliance with GeoZarr spec 0.4
2627
- `_ARRAY_DIMENSIONS` attributes on all arrays
2728
- CF standard names for all variables
@@ -30,19 +31,33 @@ The EOPF GeoZarr library enables conversion of EOPF datasets to the GeoZarr spec
3031
- Proper multiscales metadata structure
3132

3233
### Native CRS Preservation
33-
- No reprojection to TMS required
34-
- Maintains original coordinate reference systems
35-
- Native CRS tile matrix sets
3634

37-
### Multiscale Support
38-
- COG-style /2 downsampling logic
39-
- Overview levels as children groups
40-
- Configurable minimum dimensions and tile widths
35+
- Maintains native CRS (e.g., UTM zones) throughout all overview levels
36+
- Avoids reprojection to Web Mercator, preserving scientific accuracy
37+
- Custom tile matrix sets using native CRS
38+
39+
### Band Organization
40+
41+
- Spectral bands stored as separate DataArray variables
42+
- Enables band-specific metadata and selective access
43+
- Supports different processing chains per spectral band
44+
45+
### Chunking Strategy
46+
47+
- Aligned chunking to optimize storage efficiency and I/O performance
48+
- Prevents partial chunks that waste storage space
49+
- Reduces memory fragmentation
50+
51+
### Hierarchical Structure
52+
53+
- All resolution levels stored as siblings (`/0`, `/1`, `/2`, etc.)
54+
- Multiscales metadata in parent group attributes
55+
- Complies with xarray DataTree alignment requirements
4156

4257
### Robust Processing
58+
4359
- Band-by-band writing with validation
4460
- Retry logic for network operations
45-
- Comprehensive error handling
4661

4762
## Architecture
4863

@@ -58,4 +73,4 @@ See the [Quick Start](quickstart.md) guide to begin using the library, or check
5873

5974
## Support
6075

61-
For questions, issues, or contributions, please visit the [GitHub repository](https://github.com/developmentseed/eopf-geozarr).
76+
For questions, issues, or contributions, please visit the [GitHub repository](https://github.com/eopf-explorer/data-model).

‎docs/stylesheets/extra.css‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
[data-md-color-scheme="default"] {
2+
--md-primary-fg-color: #160A42;
3+
--font-family: Inter, sans-serif;
4+
--md-link-fg-color: #0078D7;
5+
/* Bright blue for visibility */
6+
}
7+
8+
/* Apply the link color only in the main document content */
9+
[data-md-color-scheme="default"] main article a {
10+
color: var(--md-link-fg-color);
11+
text-decoration: none;
12+
/* Remove underline */
13+
}
14+
15+
[data-md-color-scheme="slate"] {
16+
--md-default-fg-color: hsla(var(--md-hue), 15%, 90%, 0.9);
17+
--md-default-fg-color--light: hsla(var(--md-hue), 15%, 90%, 1);
18+
--md-primary-fg-color: #160A42;
19+
--font-family: Inter, sans-serif;
20+
}
21+
22+
/* Sets up a figure counter */
23+
24+
/* Initialise the counter */
25+
body {
26+
counter-reset: figureCounter;
27+
}
28+
29+
/* Increment the counter for every instance of a figure even if it doesn't have a caption */
30+
figure {
31+
counter-increment: figureCounter;
32+
}
33+
34+
/* Prepend the counter to the figcaption content */
35+
figure figcaption:before {
36+
content: "Figure " counter(figureCounter) ": ";
37+
}
38+
39+
/* reduce font size of code and xarray output cells in rendered jupyter notebooks */
40+
.jupyter-wrapper .jp-OutputArea-output pre,
41+
.xr-wrap {
42+
font-size: 0.8em;
43+
}
44+
45+
/* Code to better render xarray html representation with mknotebook */
46+
.md-typeset pre.xr-text-repr-fallback {
47+
display: none;
48+
}
49+
50+
.md-typeset ul.xr-sections,
51+
.jupyter-wrapper .jp-OutputArea-output dl.xr-attrs {
52+
display: grid;
53+
}
54+
55+
.md-typeset li.xr-var-item,
56+
.md-typeset ul.xr-var-list {
57+
display: contents;
58+
}
59+
60+
.md-typeset .xr-section-details {
61+
display: none;
62+
}
63+
64+
.md-typeset ul.xr-dim-list li {
65+
margin-bottom: 0;
66+
margin-left: 0;
67+
}
68+
69+
.md-typeset ul.xr-dim-list {
70+
margin-bottom: 0;
71+
margin-top: 0;
72+
}
73+
74+
.jupyter-wrapper .jp-OutputArea-output .xr-attrs dt {
75+
padding: 0;
76+
margin: 0;
77+
float: left;
78+
padding-right: 10px;
79+
width: auto;
80+
font-weight: normal;
81+
grid-column: 1;
82+
}

‎mkdocs.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
site_name: ESA Sentinel Zarr Explorer - Data Model
2+
site_description: Documentation for the ESA Sentinel Zarr Explorer Data Model
3+
site_author: Development Seed
4+
repo_url: https://github.com/eopf-explorer/data-model
5+
edit_uri: edit/main/docs/
6+
7+
theme:
8+
name: material
9+
logo: assets/logo-eopf-sentinel-explorer-dark-icon.svg
10+
features:
11+
#- navigation.tabs
12+
#- navigation.tabs.sticky
13+
- navigation.sections
14+
- content.code.copy
15+
- navigation.instant
16+
- navigation.tracking
17+
- navigation.expand
18+
- navigation.indexes
19+
- navigation.top
20+
- content.code.annotate
21+
- content.tabs.link
22+
#- toc.integrate # Table of contents on the left
23+
24+
palette:
25+
# Palette toggle for light mode
26+
- scheme: default
27+
toggle:
28+
icon: material/brightness-7
29+
name: Switch to dark mode
30+
- scheme: slate
31+
accent: orange
32+
toggle:
33+
icon: material/brightness-4
34+
name: Switch to light mode
35+
36+
extra_css:
37+
- stylesheets/extra.css
38+
39+
40+
markdown_extensions:
41+
- admonition
42+
- attr_list
43+
- codehilite
44+
- pymdownx.highlight
45+
- pymdownx.superfences
46+
- pymdownx.tabbed
47+
- pymdownx.tasklist:
48+
custom_checkbox: true
49+
- toc:
50+
permalink: true
51+
52+
nav:
53+
- Home: index.md
54+
- Using the Converter: converter.md

‎pyproject.toml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -61,10 +61,10 @@ test = [
6161
"pytest-xdist>=3.0.0",
6262
]
6363
docs = [
64-
"sphinx>=6.0.0",
65-
"sphinx-rtd-theme>=1.2.0",
66-
# "myst-parser>=1.0.0",
67-
"sphinx-autodoc-typehints>=1.20.0",
64+
"mkdocs>=1.4.0",
65+
"mkdocs-material>=9.1.0",
66+
"pymdown-extensions>=10.0",
67+
"mike>=2.1.3",
6868
]
6969

7070
[project.urls]

0 commit comments

Comments
 (0)