Skip to content

Commit 004b2c9

Browse files
konstntokaskonstntokas
committed
backup
1 parent fd0011c commit 004b2c9

5 files changed

Lines changed: 44 additions & 24 deletions

File tree

‎docs/data.md‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -11,19 +11,19 @@ These plugins can be seamlessly integrated into the Multi-Source Data Store.
1111
The following data store plugins provide direct access to major Earth observation
1212
and research data providers.
1313

14-
| Logo | Data Source | Example | Plugin Repository |
15-
|------|------------|---------|-------------------|
16-
| ![CDS](assets/cds_logo.png){width="150" .logo-icon-table} | **[Copernicus Climate Data Store (CDS)](https://cds.climate.copernicus.eu/)** | `new_data_store("cds")` | [GitHub](https://github.com/xcube-dev/xcube-cds) |
17-
| ![CMEMS](assets/cmems_logo.png){width="150" .logo-icon-table} | **[Copernicus Marine Service (CMEMS)](https://marine.copernicus.eu/)** | `new_data_store("cmems")` | [GitHub](https://github.com/xcube-dev/xcube-cmems) |
18-
| ![CLMS](https://land.copernicus.eu/static/media/ccl-icon-land-text-2.e04af716.svg){width="150" .logo-icon-table} | **[Copernicus Land Monitoring Service (CLMS)](https://land.copernicus.eu/en/dataset-catalog)** | `new_data_store("clms")` | [GitHub](https://github.com/xcube-dev/xcube-clms) |
19-
| ![EOPF](https://zarr.eopf.copernicus.eu/wp-content/uploads/2026/02/EOPF-on-bright.png){width="150" .logo-icon-table} | **[EOPF Sentinel Zarr Samples](https://zarr.eopf.copernicus.eu/)** | `new_data_store("eopf-zarr")` | [GitHub](https://github.com/EOPF-Sample-Service/xcube-eopf) |
20-
| ![ESA CCI](https://brand.esa.int/files/2020/05/ESA_logo_2020_Deep-scaled.jpg){width="150" .logo-icon-table} | **[ESA CCI](https://climate.esa.int/en/data/#/dashboard)** | `new_data_store("cciodp")`<br/>`new_data_store("ccizarr")` | [GitHub](https://github.com/xcube-dev/xcube-cci) |
21-
| ![ESA SMOS](https://www.esa.int/eologos/images/smos.jpg){width="150" .logo-icon-table} | **[ESA SMOS](https://earth.esa.int/eogateway/missions/smos)** | `new_data_store("smos")` | [GitHub](https://github.com/xcube-dev/xcube-smos) |
22-
| ![ICOS Data Portal](assets/logo_icos.png){width="150" .logo-icon-table} | **[ICOS Data Portal](https://www.icos-cp.eu/data-services)** | `new_data_store("icosdp")` | [GitHub](https://github.com/xcube-dev/xcube-icosdp) |
23-
| ![GEDI](https://gedi.umd.edu/wp-content/uploads/2020/10/GEDI_16_10.jpg){width="150" .logo-icon-table} | **[Global Ecosystem Dynamics Investigation (GEDI)](https://gedi.umd.edu/)** | `new_data_store("gedi")` | [GitHub](https://github.com/xcube-dev/xcube-gedi) |
24-
| ![Sentinel Hub](https://www.sentinel-hub.com/img/press/sentinel_hub_by_planet_logo_big.png){width="150" .logo-icon-table} | **[Sentinel Hub](https://www.sentinel-hub.com/)** | `new_data_store("sentinelhub")` | [GitHub](https://github.com/xcube-dev/xcube-sh) |
25-
| ![STAC](https://stacspec.org/public/images-original/STAC-04.png){width="150" .logo-icon-table} | **[SpatioTemporal Asset Catalog (STAC)](https://stacspec.org/en/about/datasets/)** | `new_data_store("stac")`| [GitHub](https://github.com/xcube-dev/xcube-stac) |
26-
| ![Zenodo](https://about.zenodo.org/static/img/logos/zenodo-black-border.svg){width="150" .logo-icon-table} | **[Zenodo](https://zenodo.org/)** | `new_data_store("zenodo")` | [GitHub](https://github.com/xcube-dev/xcube-zenodo) |
14+
| Logo | Data Source | Store ID | Plugin Repository |
15+
|------|------------|--------------------------------------------------|-------------------|
16+
| ![CDS](assets/cds_logo.png){width="150" .logo-icon-table} | **[Copernicus Climate Data Store (CDS)](https://cds.climate.copernicus.eu/)** | `"cds"` | [GitHub](https://github.com/xcube-dev/xcube-cds) |
17+
| ![CMEMS](assets/cmems_logo.png){width="150" .logo-icon-table} | **[Copernicus Marine Service (CMEMS)](https://marine.copernicus.eu/)** | `"cmems"` | [GitHub](https://github.com/xcube-dev/xcube-cmems) |
18+
| ![CLMS](https://land.copernicus.eu/static/media/ccl-icon-land-text-2.e04af716.svg){width="150" .logo-icon-table} | **[Copernicus Land Monitoring Service (CLMS)](https://land.copernicus.eu/en/dataset-catalog)** | `"clms"` | [GitHub](https://github.com/xcube-dev/xcube-clms) |
19+
| ![EOPF](https://zarr.eopf.copernicus.eu/wp-content/uploads/2026/02/EOPF-on-bright.png){width="150" .logo-icon-table} | **[EOPF Sentinel Zarr Samples](https://zarr.eopf.copernicus.eu/)** | `"eopf-zarr"` | [GitHub](https://github.com/EOPF-Sample-Service/xcube-eopf) |
20+
| ![ESA CCI](https://brand.esa.int/files/2020/05/ESA_logo_2020_Deep-scaled.jpg){width="150" .logo-icon-table} | **[ESA CCI](https://climate.esa.int/en/data/#/dashboard)** | `"cciodp"`<br/>`"ccizarr"` | [GitHub](https://github.com/xcube-dev/xcube-cci) |
21+
| ![ESA SMOS](https://www.esa.int/eologos/images/smos.jpg){width="150" .logo-icon-table} | **[ESA SMOS](https://earth.esa.int/eogateway/missions/smos)** | `"smos"` | [GitHub](https://github.com/xcube-dev/xcube-smos) |
22+
| ![ICOS Data Portal](assets/logo_icos.png){width="150" .logo-icon-table} | **[ICOS Data Portal](https://www.icos-cp.eu/data-services)** | `"icosdp"` | [GitHub](https://github.com/xcube-dev/xcube-icosdp) |
23+
| ![GEDI](https://gedi.umd.edu/wp-content/uploads/2020/10/GEDI_16_10.jpg){width="150" .logo-icon-table} | **[Global Ecosystem Dynamics Investigation (GEDI)](https://gedi.umd.edu/)** | `"gedi"` | [GitHub](https://github.com/xcube-dev/xcube-gedi) |
24+
| ![Sentinel Hub](https://www.sentinel-hub.com/img/press/sentinel_hub_by_planet_logo_big.png){width="150" .logo-icon-table} | **[Sentinel Hub](https://www.sentinel-hub.com/)** | `"sentinelhub"` | [GitHub](https://github.com/xcube-dev/xcube-sh) |
25+
| ![STAC](https://stacspec.org/public/images-original/STAC-04.png){width="150" .logo-icon-table} | **[SpatioTemporal Asset Catalog (STAC)](https://stacspec.org/en/about/datasets/)** | `"stac"` | [GitHub](https://github.com/xcube-dev/xcube-stac) |
26+
| ![Zenodo](https://about.zenodo.org/static/img/logos/zenodo-black-border.svg){width="150" .logo-icon-table} | **[Zenodo](https://zenodo.org/)** | `"zenodo"` | [GitHub](https://github.com/xcube-dev/xcube-zenodo) |
2727

2828
Each plugin repository contains usage examples and documentation specific to the
2929
respective data provider.

‎environment.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,13 @@ dependencies:
1313
- tabulate
1414
- xarray
1515
- xcube >=1.13.1
16-
- xcube-resampling >=0.3.0
16+
- xcube-resampling >=0.3.1
1717
- yaml
1818
# data store plugins
1919
- xcube-cci >=0.12.1
2020
- xcube-cds >=1.2.0
2121
- xcube-clms >=0.2.2
22-
- xcube-stac >=1.1.2
22+
- xcube-stac >=1.2.0
2323
- xcube-zenodo >=1.1.1
2424
# Development Dependencies - Tools
2525
- black

‎pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ dependencies = [
3232
"tabulate",
3333
"xarray",
3434
"xcube>=1.13.1",
35-
"xcube-resampling>=0.3.0",
35+
"xcube-resampling>=0.3.1",
3636
"yaml",
3737
]
3838
classifiers = [

‎xcube_multistore/accessors/stac.py‎

Lines changed: 24 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -29,15 +29,30 @@
2929
from xcube_multistore.visualization import GeneratorState
3030

3131
_NB_PIXELS = int(2e4 * 2e4) * 5 * 4
32-
_MAX_DAYS = 100
33-
_NUM_SEN2_BANDS = 13
32+
_MAX_DAYS = {
33+
"sentinel-2-l1c": 100,
34+
"sentinel-2-l2a": 100,
35+
"sentinel-3-syn-2-syn-ntc": 7,
36+
"sentinel-3-sl-2-lst-ntc": 7,
37+
"sentinel-3-synergy-syn-l2-netcdf": 7,
38+
"sentinel-3-slstr-lst-l2-netcdf": 7,
39+
}
40+
41+
_NUM_BANDS = {
42+
"sentinel-2-l1c": 13,
43+
"sentinel-2-l2a": 13,
44+
"sentinel-3-syn-2-syn-ntc": 25,
45+
"sentinel-3-sl-2-lst-ntc": 2,
46+
"sentinel-3-synergy-syn-l2-netcdf": 25,
47+
"sentinel-3-slstr-lst-l2-netcdf": 2,
48+
}
3449

3550

3651
class StacAccessor(Accessor):
3752
"""Provides methods for accessing dataset from xcube-cds data store"""
3853

3954
def open_data(self, data_id: str, **open_params) -> xr.Dataset:
40-
time_ranges = self._split_time_range(open_params)
55+
time_ranges = self._split_time_range(data_id, open_params)
4156
nb_requests = len(time_ranges)
4257
self.notify(
4358
GeneratorState(
@@ -66,11 +81,11 @@ def open_data(self, data_id: str, **open_params) -> xr.Dataset:
6681
return ds
6782

6883
@staticmethod
69-
def _split_time_range(open_params: dict):
84+
def _split_time_range(data_id: str, open_params: dict):
7085
if "asset_names" in open_params:
7186
nb_vars = len(open_params["asset_names"])
7287
else:
73-
nb_vars = _NUM_SEN2_BANDS
88+
nb_vars = _NUM_BANDS[data_id]
7489
start, end = open_params["time_range"]
7590
start = datetime.date.fromisoformat(start)
7691
end = datetime.date.fromisoformat(end)
@@ -95,9 +110,10 @@ def _split_time_range(open_params: dict):
95110
nb_splits = 1
96111

97112
base = total_days // nb_splits
98-
if base > _MAX_DAYS:
99-
base = _MAX_DAYS
100-
nb_splits = total_days // _MAX_DAYS
113+
max_days = _MAX_DAYS[data_id]
114+
if base > max_days:
115+
base = _MAX_DAYS[data_id]
116+
nb_splits = total_days // max_days
101117
remainder = total_days % nb_splits
102118
time_ranges = []
103119
current = start

‎xcube_multistore/multistore.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -659,10 +659,14 @@ def _process_single_dataset(
659659
ds_ref = getattr(self.stores, NAME_WRITE_STORE).open_data(data_id)
660660
target_gm = GridMapping.from_dataset(ds_ref)
661661
spatial_resample_params = config.get("spatial_resample_params", {})
662+
if not "prevent_nan_propagations" in spatial_resample_params:
663+
spatial_resample_params["prevent_nan_propagations"] = True
662664
ds = resample_in_space(ds, target_gm=target_gm, **spatial_resample_params)
663665
elif ds_ref is not None:
664666
target_gm = GridMapping.from_dataset(ds_ref)
665667
spatial_resample_params = config.get("spatial_resample_params", {})
668+
if not "prevent_nan_propagations" in spatial_resample_params:
669+
spatial_resample_params["prevent_nan_propagations"] = True
666670
ds = resample_in_space(ds, target_gm=target_gm, **spatial_resample_params)
667671

668672
return ds

0 commit comments

Comments
 (0)