storage: support local filesystem storage with localPath config#985
storage: support local filesystem storage with localPath config#985huanghaoyuanhhy wants to merge 1 commit into
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: huanghaoyuanhhy The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Codecov Report❌ Patch coverage is
❌ Your project status has failed because the head coverage (28.46%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #985 +/- ##
==========================================
+ Coverage 28.23% 28.46% +0.23%
==========================================
Files 104 104
Lines 9835 9871 +36
==========================================
+ Hits 2777 2810 +33
- Misses 6856 6858 +2
- Partials 202 203 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
37959a9 to
178e2f7
Compare
178e2f7 to
9065097
Compare
b179e25 to
4276810
Compare
4276810 to
7c7938d
Compare
7c7938d to
be89699
Compare
be89699 to
e6567f6
Compare
b28e9d3 to
91a7ce9
Compare
Support Milvus standalone with local storage (COMMON_STORAGETYPE=local) by mapping minio.localPath / minio.backupLocalPath to the host directory that backs Milvus localStorage.path. LocalClient now resolves keys against baseDir = localPath/bucket so all keys returned by ListPrefix are relative, matching the S3 client. The CI test dumps the milvus and backup volume trees around backup and restore, and keeps restore temp files, so failures expose what is actually on disk where Milvus expects to read it. Signed-off-by: huanghaoyuanhhy <haoyuan.huang@zilliz.com>
91a7ce9 to
7cbb895
Compare
Fixes #964
Summary
Support Milvus Standalone local storage (
storageType: local) by addinglocalPathconfig that maps to MilvuslocalStorage.path. This enables backup/restore for Milvus deployed via the official standalone docker method which usesCOMMON_STORAGETYPE=local.Changes
LocalPathfield tostorage.Configstructminio.localPath/minio.backupLocalPathconfig fields with env var support (MINIO_LOCAL_PATH/MINIO_BACKUP_LOCAL_PATH),backupLocalPathinherits fromlocalPathif not setLocalClientwithbaseDir(localPath/bucket),absPath(key)andrelPath(abs)to correctly resolve relative keysBucketExistnow checks ifbaseDirdirectory exists;CreateBucketcreates it viaos.MkdirAllLocalClienttests to useConfig{LocalPath: tempDir, Bucket: "test-bucket"}patternconfigs/backup.yamlconfigs/backup-local.yamlexample config for Milvus Standalone local storagedocker runwith embedded etcd, matching official deploymentBreaking Changes
LocalClientnow resolves all keys relative tolocalPath/bucketinstead of treating them as absolute paths. Users who previously relied on absolute-path keys withstorageType: localmust setlocalPathto the correct base directory (the host path mapping to MilvuslocalStorage.path).User Config Example