feat(storage): add datasets resource-type prefix to dataset logical paths#5911
feat(storage): add datasets resource-type prefix to dataset logical paths#5911aicam wants to merge 37 commits into
datasets resource-type prefix to dataset logical paths#5911Conversation
…ent via public cluster services - Add CloudMapperSourceOpDesc, ReferenceGenome, ReferenceGenomeEnum operator classes - Add FileResolver.resolveDirectory for resolving dataset directories by path - Add DatasetFileDocument directory mode: downloads all files as a zip via LakeFS/FileService - Add DocumentFactory.openReadonlyDocument isDirectory parameter - Add ENV_FILE_SERVICE_LIST_DIRECTORY_OBJECTS_ENDPOINT env var - Add Kubernetes Helm chart and PVC for the cloudmapper service Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ntend integration - Add ClusterResource, ClusterCallbackResource, ClusterServiceClient, ClusterUtils backend API for managing EC2 clusters - Add cluster dashboard component with launch/stop/terminate/start actions and management modal - Add ClusterSelectionComponent and ClusterAutoCompleteComponent for operator property panel - Add DirectoryPathInput and DirectorySelection components for dataset directory selection - Add cluster route in app-routing, cluster declarations in app.module - Add cluster_enabled feature flag to gui-config, dashboard sidebar, and admin settings - Add clusterautocomplete and directorypathinput formly field types - Register cluster/directoryName/fastQFiles/fastAFiles/gtfFile fields in operator property editor - Add SQL schema for cluster and cluster_activity tables - Add dknet logo, CloudBioMapper operator icon, and sequence-alignment workflow assets - Add DatasetDirectoryDocument and PathUtils storage utilities Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Feat/cloudbiomapper
Revert "Feat/cloudbiomapper"
|
👋 Thanks for opening this pull request, @aicam! It looks like the pull request description doesn't quite follow our template yet:
Filling out the template helps reviewers understand and triage your contribution faster. Please edit the description to complete it. This message will disappear automatically once the template is followed. You can find the template prompts by editing the description, or see CONTRIBUTING.md for the full contribution flow. |
Automated Reviewer SuggestionsBased on the
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #5911 +/- ##
============================================
- Coverage 54.11% 54.06% -0.05%
+ Complexity 2819 2811 -8
============================================
Files 1103 1103
Lines 42650 42654 +4
Branches 4588 4589 +1
============================================
- Hits 23079 23061 -18
- Misses 18226 18245 +19
- Partials 1345 1348 +3
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Summary
Introduces a
datasetsresource-type prefix to dataset logical file paths, changing the format from:to:
This namespaces dataset paths under an explicit resource-type segment, leaving room for other resource types in the future.
Changes
Backend (Scala)
FileResolver: strips a knownRESOURCE_TYPE_PREFIXES = Set("datasets")leading segment before parsing; docs/examples updated.DatasetFileNode.fromLakeFSRepositoryCommittedObjects: builds an intermediatedatasetsdirectory node as the parent of owner nodes.DatasetResource: descends through the newdatasetsnode to reach the owner node; total-size calculation rooted at thedatasetsnode.FileResolverSpec: test fixtures updated to the prefixed path format.Frontend (TypeScript)
dataset-file.ts:parseFilePathToDatasetFilestrips thedatasetsprefix if present;parseDatasetFileToFilePathemits it.datasetVersionFileTree.ts: relative-path extraction now strips four leading segments (datasets/owner/dataset/version) instead of three.Notes
apache/main(via the aicam fork'smain) before opening this PR.🤖 Generated with Claude Code