Conversation
Collaborator
Author
|
implements https://github.com/EOPF-Explorer/coordination/issues/182 for reprocessing historical data through CLI |
4 tasks
emmanuelmathot
approved these changes
Mar 19, 2026
emmanuelmathot
left a comment
Contributor
There was a problem hiding this comment.
LGTM. I wonder about the existing assets in the production catalog. Do we still need to update them?
Collaborator
Author
|
YEs I think so! I have another PR ready for review for that: #118 and the argo workflow PR that goes with it: https://github.com/EOPF-Explorer/platform-deploy/pull/129 |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
change-storage-tierCLI commands to bothmanage_item.py(single item) andmanage_collections.py(collection-level with optional date filtering) inoperator-tools/. These commands change S3 object storage classes (STANDARD, STANDARD_IA, EXPRESS_ONEZONE) and update STAC metadata to reflect the new tier.fix_zarr_asset_media_types()from bothregister_v0.pyandregister_v1.py— the upstream source now provides correct media types, so runtime patching is no longer needed. The test fixture was updated accordingly.aggregate_items.pyscript and its documentation/tests — this aggregation functionality is no longer used.%20→+) in both registration scripts.Details
New
change-storage-tiercommandsSingle item (
manage_item.py change-storage-tier):--include-pattern/--exclude-patternfor filtering objects--dry-runor-yis passedCollection-level (
manage_collections.py change-storage-tier):--start-date/--end-datefiltering via CQL2betweenqueryCleanup
aggregate_items.py, its README, and 348 lines of testsfix_zarr_asset_media_types()and its 106-line test fileTest plan
test_change_storage_tier_commands.py)uv run pytestpasseschange-storage-tieragainst staging collection