Document how to parse the URL with groups.

[m-mohr]

I'm not sure this is the right repository, please move it if there's a better place.

The URL to resources (e.g. groups) that are inside a ZARR should be clarified.

Usually we have something like:
https://example.stac/path/something.zarr/my/group

How to parse the group from the URL, for example for use in OpenLayers?
I'm asking because I need to figure out how to extract that group reliably and I couldn't find any information on whether I can e.g. rely on the .zarr being in the path only once (in theory a .zarr could be somewhere in other folder names or the group names itself, ... or not?).

I think it might have been easier to append an additional path as fragment, e.g.
https://example.stac/path/something.zarr#/my/group
Then you can simply extract the fragment with usual URL parsing libraries instead of shipping your own logic to extract it from the path.

[d-v-b]

(in theory a .zarr could be somewhere in other folder names or the group names itself, ... or not?).

This is correct, the .zarr suffix is only advisory. Per the zarr spec, the existence of a group or array at some path p is defined by p/zarr.json resolving to a group / array metadata document.

For your purposes I think you could use the p/zarr.json path as an explicit path to a group, but Zarr readers would need to strip the zarr.json part before accessing it, because iirc a group named "zarr.json" is allowed, so you could have p/zarr.json/zarr.json, where the latter path segment is a metadaata document.

[d-v-b]

And I don't see any problems with the fragment syntax

[m-mohr]

Ideally we can extract the group without an additional HTTP request upfront (so preferring # notation), but if required I'm fine with it as long as it is well documented and unambiguous.

[d-v-b]

the use of # notation for actual URLs may not work because the fragment is supposed to be evaluated client-side, after the resource has been fetched. But a Zarr hierarchy is not a resource that can be fetched. But for other purposes it's probably fine as long as the convention is clear

[m-mohr]

Can you point me to where it is defined that the fragment is supposed to be evaluated on the client-side?
I don't find that in https://datatracker.ietf.org/doc/html/rfc3986#section-3.5
But isn't the group/fragment actually evaluated on the client-side in this case anyway?
Actually, section 3.5 reads like a good fit to me:

The fragment identifier component of a URI allows indirect
identification of a secondary resource by reference to a primary
resource and additional identifying information. The identified
secondary resource may be some portion or subset of the primary
resource, some view on representations of the primary resource, or
some other resource defined or described by those representations.

[d-v-b]

I was going by the MDN docs, emphasis mine:

The fragment of a URI is the last part of the URI, starting with the # character. It is used to identify a specific part of the resource, such as a section of a document or a position in a video. The fragment is not sent to the server when the URI is requested, but it is processed by the client (such as the browser) after the resource is retrieved.

Given a path like a.zarr#group, strictly speaking a.zarr is not a resource that you can fetch (although a.zarr/zarr.json would be). I am not sure how much this matters here though.

[d-v-b]

Usually we have something like:
https://example.stac/path/something.zarr/my/group

How to parse the group from the URL, for example for use in OpenLayers?

can you not assume that https://example.stac/path/something.zarr/my/group is a group? I.e., you assume that https://example.stac/path/something.zarr/my/group/zarr.json exists, and is valid zarr v3 group metadata?

[m-mohr]

Given a path like a.zarr#group, strictly speaking a.zarr is not a resource that you can fetch

Does it have to be?

(although a.zarr/zarr.json would be)

Then it would point to a JSON metadata file, which would have different properties though (e.g. media type = application/json, roles = metadata, or so?)

can you not assume that https://example.stac/path/something.zarr/my/group is a group?
I.e., you assume that https://example.stac/path/something.zarr/my/group/zarr.json exists, and is valid zarr v3 group metadata?

Not sure how this helps in my case though. I need to pass URL to https://example.stac/path/something.zarr and group my/group to OL in my case and I'm wondering how can I get that unambiguously if .zarr can be anywhere in the path multiple times, if I understand correctly. So I just always append zarr.json, request it and then take it from there? I'd like to avoid initial requests as much as possible though :-)

[d-v-b]

Given a path like a.zarr#group, strictly speaking a.zarr is not a resource that you can fetch

Does it have to be?

To use the URL fragment semantics as defined by MDN, yes. Given some path foo#bar, the model is that the client fetches foo, then indexes into it with bar. foo.zarr cannot be fetched, and so this model doesn't fit.

So I just always append zarr.json, request it and then take it from there? I'd like to avoid initial requests as much as possible though :-)

Yes this process is probably the simplest way. And if you are doing anything with the attributes of the group, then you need to read zarr.json in any case. And you almost certainly are, because I don't think there's a reliable way to find sub-arrays or sub-groups over HTTP without reading the parent group.

[d-v-b]

Maybe a solution is to avoid trying to treat zarr groups as resources, because I don't think they fit that model well. could you point the URL to zarr group metadata (foo.zarr/group/zarr.json)? That's an actual resource.

[emmanuelmathot]

@m-mohr Is this issue related to how to deal with resources in STAC?
The Best Practices for Zarr in STAC tries to describe that and Zarr extension has specific fields to describe where this is a group or an array.

[m-mohr]

Not sure whether it's STAC specific or not and I couldn't find any relevant information for my questions in the extension yet.

Maybe more specifically the question:
Who specified that we append "/my/group" to "https://example.stac/path/something.zarr" and why? Can someone answer this?

[emmanuelmathot]

@m-mohr To answer your question:

Who specified that we append "/my/group" to "https://example.stac/path/something.zarr" and why?

It comes directly from Zarr's core specification. Zarr is fundamentally a protocol over a key/value store abstraction, not a file format. The Zarr v3 spec defines an abstract store interface with operations like get(key), set(key, value), list_prefix(prefix), etc. A Zarr hierarchy lives inside a store, and nodes (groups, arrays) are identified by hierarchy paths that map to storage keys. For example, a group at hierarchy path /my/group means the key my/group/zarr.json exists in the store.

So when you see https://example.stac/path/something.zarr/my/group, what's really happening is:

• Store root: https://example.stac/path/something.zarr (this is where the Zarr store is located)
• Hierarchy path: my/group (a path within the store)

The concatenation isn't an arbitrary convention. It's the natural result of how key/value stores work (the store root URL + the key path). This is similar to how an S3 bucket root + object key works.

The real problem you're identifying is that when you flatten these two distinct concepts (store root + hierarchy path) into a single URL, you lose the boundary information. And you're right, trying to reverse-engineer the split from a concatenated URL is fragile, especially since .zarr is only advisory. That's why we advised to have a store link in the stac best practises. This could be extended as a link template to differentiate the storte, from the group or array.

[m-mohr]

Thanks for the explanation. This all makes sense, the problem is indeed identified correctly the last paragraph. And that's what I'm requesting. This resolving mechanism needs to be documented for implementors, otherwise everyone will resolve these URLs differently, potentially leading to a big mess.

This needs to include all cases, e.g. what to do when a store link is present, what happens when none is present (if it stays optional).

This probably means probably also indicating that none other parts in the path should contain a .zarr at the end of a path segment.