DatasetPathParams: resolve store root from STAC asset href (enable per-bucket collections, e.g. Sentinel-1)

[lhoupert]

Summary

DatasetPathParams resolves a store by reconstructing a single per-deployment base path, which prevents serving collections whose stores live in their own buckets (e.g. Sentinel-1). Make it resolve the store root from the STAC item's asset href instead.

Current behaviour

titiler/eopf/dependencies.py:

def DatasetPathParams(request, collection_id, item_id) -> str:
    store_url = str(store_settings.url)                       # one base per deployment
    return os.path.join(store_url, collection_id, item_id) + ".zarr"

The base comes from TITILER_EOPF_STORE_URL (or _SCHEME/_HOST/_PATH), e.g. s3://esa-zarr-sentinel-explorer-fra/tests-output/. Every render endpoint (factory.py: path_dependency=DatasetPathParams) therefore opens {STORE_URL}/{collection}/{item}.zarr and ignores the registered STAC asset hrefs.

Problem

This only works when every collection's store lives under the same {STORE_URL} base. Sentinel-1 GRD-RTC stores are written to dedicated per-environment buckets:

collection	bucket
sentinel-1-grd-rtc-staging	esa-zarr-sentinel-explorer-s1-l1grd-staging
sentinel-1-grd-rtc (prod)	esa-zarr-sentinel-explorer-s1-l1grd-prod

Requesting …/collections/sentinel-1-grd-rtc-staging/items/s1-rtc-31TDH/tiles/... makes TiTiler look in
s3://esa-zarr-sentinel-explorer-fra/tests-output/sentinel-1-grd-rtc-staging/s1-rtc-31TDH.zarr and return
"No group found in store ... bucket=esa-zarr-sentinel-explorer-fra ...", while the store actually exists in …-s1-l1grd-staging.

Proposed change

Resolve the store root from the item's assets (single source of truth), not from a fixed base. The dependency already receives collection_id, item_id, and request; STACAPISettings.url (TITILER_EOPF_STAC_API_URL) is available. titiler-eopf already reads asset_info["href"] and alternate keys in EOPFSimpleSTACReader (stac.py) — apply the same idea to the path dependency.

Claude's suggestion:

import re
from rio_tiler.io.stac import STAC_ALTERNATE_KEY  # already used in stac.py

GATEWAY = "https://s3.explorer.eopf.copernicus.eu/"

def _store_root_from_href(href: str) -> str:
    if href.startswith(GATEWAY):                 # gateway https -> s3
        href = "s3://" + href[len(GATEWAY):]
    m = re.match(r"^(.*?\.zarr)(/.*)?$", href)   # truncate group suffix
    if not m:
        raise ValueError(f"no .zarr store root in href: {href}")
    return m.group(1)

def DatasetPathParams(request, collection_id, item_id) -> str:
    item = _get_stac_item(stac_settings.url, collection_id, item_id)   # GET {stac}/collections/{c}/items/{i}
    # prefer an explicit store asset/link, then an s3 `alternate`, then any data asset
    for key in ("zarr-store", "zarr"):
        a = item["assets"].get(key)
        if a:
            alt = a.get("alternate", {}).get(STAC_ALTERNATE_KEY) if STAC_ALTERNATE_KEY else None
            return _store_root_from_href((alt or a)["href"])
    for a in item["assets"].values():
        if ".zarr" in a.get("href", ""):
            alt = a.get("alternate", {}).get(STAC_ALTERNATE_KEY) if STAC_ALTERNATE_KEY else None
            return _store_root_from_href((alt or a)["href"])
    raise ValueError(f"item {collection_id}/{item_id} has no .zarr asset")

Backward compatible with S2: its hrefs already point at …-fra/tests-output/{collection}/{item}.zarr, so the resolved root equals what is reconstructed today. (Optional: keep the old reconstruction as a fallback when no usable asset href is found, to de-risk rollout.)

Acceptance criteria

• For an item whose store lives in a non-default bucket, DatasetPathParams resolves the store root from the item's asset href (preferring alternate.s3.href), not from TITILER_EOPF_STORE_URL. Observable via the resolved path in the error/log even before creds exist (see "Stage A" below).
• /info, /preview, XYZ tiles and tilejson.json → HTTP 200 for an S1 item, reading from the bucket named in its href (…-s1-l1grd-staging for staging; …-s1-l1grd-prod for prod) — once that bucket is reachable by the titiler pod (Stage B).
• S2 prod and staging items render unchanged — same endpoints return the same 200s before and after (no regression).
• Store resolves regardless of which bucket/prefix the href points at (nothing hardcoded), and works for the GeoZarr variables= endpoints.

How to reproduce & verify

A ready-made fixture is live now (created by the data-pipeline#186 end-to-end run):

	value
collection / item	sentinel-1-grd-rtc-tests / s1-rtc-31TDH
store actually at	s3://esa-zarr-sentinel-explorer-tests/sentinel-1-grd-rtc-tests/s1-grd-rtc-31TDH.zarr
registered href	https://s3.explorer.eopf.copernicus.eu/esa-zarr-sentinel-explorer-tests/sentinel-1-grd-rtc-tests/s1-grd-rtc-31TDH.zarr/descending
registered alternate.s3.href	s3://esa-zarr-sentinel-explorer-tests/sentinel-1-grd-rtc-tests/s1-grd-rtc-31TDH.zarr/descending

Current (broken) behaviour — reproduce the bug:

curl -s "https://api.explorer.eopf.copernicus.eu/raster/collections/sentinel-1-grd-rtc-tests/items/s1-rtc-31TDH/info?variables=%2Fdescending%3Avh&assets=vh"
# -> HTTP 500
# {"detail":"No group found in store ObjectStore(object_store://S3Store(
#   bucket=\"esa-zarr-sentinel-explorer-fra\", prefix=\"tests-output/sentinel-1-grd-rtc-tests/s1-rtc-31TDH.zarr\")) at path ''"}

The reconstructed path is wrong on three counts at once (this is exactly what the resolver must fix):

	reconstructed today (from TITILER_EOPF_STORE_URL)	correct (from the registered href)
bucket	esa-zarr-sentinel-explorer-fra	esa-zarr-sentinel-explorer-tests
prefix	tests-output/	(none — collection sits at bucket root)
filename	s1-rtc-31TDH.zarr (item id)	s1-grd-rtc-31TDH.zarr (store filename)

Stage A — resolver correctness (no creds needed): after the change, the same request must resolve to the href path. Even if the titiler pod has no creds for that bucket yet, the failure message (or a debug log) must reference bucket="esa-zarr-sentinel-explorer-tests", prefix empty, filename s1-grd-rtc-31TDH.zarr — i.e. the path now comes from the item, not the base. This alone proves the resolver works and is the minimal acceptance gate for this issue.

Stage B — full render (needs the deployment bits): /info, /preview, tilejson.json, XYZ tiles → HTTP 200, once the titiler pod can actually read the target bucket. For staging/prod that means the …-s1-l1grd-* buckets are gateway-proxied with read creds (the platform-deploy follow-ups below). The -tests bucket is the data-pipeline's local-validation bucket and may not be wired to the deployment — run Stage B against a staging/prod item if -tests isn't reachable.

No-regression anchor (S2 — must stay HTTP 200 before and after):

curl -s -o /dev/null -w "%{http_code}\n" \
"https://api.explorer.eopf.copernicus.eu/raster/collections/sentinel-2-l2a/items/S2C_MSIL2A_20260602T134721_N0512_R110_T27WWV_20260602T171510/WebMercatorQuad/tilejson.json?variables=%2Fmeasurements%2Freflectance%3Ab04&variables=%2Fmeasurements%2Freflectance%3Ab03&variables=%2Fmeasurements%2Freflectance%3Ab02&bidx=1&rescale=0%2C1"
# -> 200 today; must remain 200 after the change.

(S2 hrefs already point at …-fra/tests-output/{collection}/{item}.zarr, so the href-resolved root equals what is reconstructed today — the optional reconstruction-fallback keeps this path identical.)

Local dev (no cluster): run titiler-eopf with TITILER_EOPF_STAC_API_URL=https://api.explorer.eopf.copernicus.eu/stac and AWS creds for the target bucket, then hit the same /info URL against your local instance — Stage A is verifiable purely from the resolved path string in the logs.

Deployment dependency (platform-deploy)

• The prod /raster HelmRelease (core/titiler-eopf/hr-titiler-eopf.yaml) does not set TITILER_EOPF_STAC_API_URL; this dependency needs it. Add it (https://api.explorer.eopf.copernicus.eu/stac).
• Ensure the s3.explorer.eopf.copernicus.eu gateway proxies …-s1-l1grd-staging/-prod and read creds are wired (nginx-s3-gateway PR #205 follow-up).

Tracking (data-pipeline)

• S1 GRD RTC - STAC Registration & Argo Pipeline data-pipeline#185 — issue I-8 (root cause + decision record).
• S1 GRD RTC — Phases 5 + 6 (local prototype → productionization) data-pipeline#186 — pipeline side already shipped: S1 stores are written to
  s3://{per-env bucket}/{collection}/s1-grd-rtc-{tile}.zarr and every asset is registered with
  both a gateway-https href and an alternate.s3.href. The href-level outcome is verifiable today;
  render stays gated on this issue.

[lhoupert]

@emmanuelmathot @vincentsarago what do you think about this issue that would allow resolving the store root from the item's asset href , not from TITILER_EOPF_STORE_URL

[vincentsarago]

we have everything we need in titiler-stacapi (which we use for mosaic) https://developmentseed.org/titiler-stacapi/endpoints/items_endpoints/

The issue, you pointed out, is which asset represent the root zarr store? do we have a spec of this?

ref: #60

[lhoupert]

@emmanuelmathot do we have a spec for this?