Add support for syncing CEP-16 sharded repodata for conda-forge

[aoaim]

Hi, I noticed that the TUNA Anaconda mirror does not seem to provide CEP-16 sharded repodata for conda-forge, while the upstream conda-forge channel already provides it.

For example, upstream conda-forge has:

curl -I https://conda.anaconda.org/conda-forge/win-64/repodata_shards.msgpack.zst

which returns:

HTTP/1.1 200 OK
Content-Type: binary/octet-stream
Content-Length: 414167

and:

curl -I https://conda.anaconda.org/conda-forge/noarch/repodata_shards.msgpack.zst

also returns:

HTTP/1.1 200 OK
Content-Type: binary/octet-stream
Content-Length: 929729

However, the TUNA mirror currently returns 404 for the same file:

curl -I https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64/repodata_shards.msgpack.zst

returns:

HTTP/1.1 404 Not Found

while the traditional flat repodata file is available:

curl -I https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/win-64/repodata.json

returns:

HTTP/1.1 200 OK

This causes recent versions of mamba / libmamba to print warnings such as:

Shard Index for conda-forge/noarch not available, falling back to flat repodata
Shard Index for conda-forge/win-64 not available, falling back to flat repodata

The installation still works, but the client falls back to downloading/parsing the much larger flat repodata.json, which can be noticeably slower for large subdirs such as conda-forge/win-64.

From the mirror sync script, it seems that the current logic only downloads traditional metadata files:

repodata.json
repodata.json.bz2
repodata.json.zst
current_repodata.json

but does not sync:

repodata_shards.msgpack.zst

or the referenced shard files under info.shards_base_url, commonly ./shards/.

Would it be possible to add support for syncing CEP-16 sharded repodata for conda cloud channels, especially conda-forge?

Thanks.