Skip to content

ci: add etcd schema verification for secondary restore#1011

Merged
sre-ci-robot merged 1 commit into
zilliztech:mainfrom
huanghaoyuanhhy:etcd-schema-verify
Apr 7, 2026
Merged

ci: add etcd schema verification for secondary restore#1011
sre-ci-robot merged 1 commit into
zilliztech:mainfrom
huanghaoyuanhhy:etcd-schema-verify

Conversation

@huanghaoyuanhhy
Copy link
Copy Markdown
Collaborator

@huanghaoyuanhhy huanghaoyuanhhy commented Apr 3, 2026

Summary

  • Add standalone tool cmd/etcd-schema-verify that reads collection metadata directly from etcd and compares source vs target schemas at the etcd level
  • Covers fields invisible to DescribeCollection: internal field IDs, field states, index params, partition states, function schemas
  • CDC scenario aware: compares all IDs (collection/field/partition/index) since they are preserved across clusters
  • Integrate into secondary restore CI to verify schema alignment after restore

Changes

  • cmd/etcd-schema-verify/ — standalone Go binary with flag-based CLI
    • Reads databases, collections, fields, partitions, indexes, functions from etcd via protobuf deserialization
    • Diff logic matches by ID, outputs JSON with aligned flag and detailed diffs
    • Exit code 1 when not aligned (CI friendly)
  • .github/workflows/main.yaml — build the tool and run it after secondary restore for hello_milvus and hello_milvus2

/kind improvement

@sre-ci-robot sre-ci-robot added the kind/improvement Changes related to something improve, likes ut and code refactor label Apr 3, 2026
@mergify mergify Bot added the dco-passed label Apr 3, 2026
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 3, 2026

Codecov Report

❌ Patch coverage is 78.04348% with 101 lines in your changes missing coverage. Please review.
✅ Project coverage is 27.93%. Comparing base (1f41db4) to head (672a87d).

Files with missing lines Patch % Lines
cmd/etcd-schema-verify/main.go 0.00% 59 Missing ⚠️
cmd/etcd-schema-verify/dump.go 80.46% 20 Missing and 22 partials ⚠️

❌ Your patch status has failed because the patch coverage (78.04%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.
❌ Your project status has failed because the head coverage (27.93%) is below the target coverage (80.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1011      +/-   ##
==========================================
+ Coverage   25.44%   27.93%   +2.48%     
==========================================
  Files         100      103       +3     
  Lines        9270     9730     +460     
==========================================
+ Hits         2359     2718     +359     
- Misses       6734     6813      +79     
- Partials      177      199      +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mergify mergify Bot added ci-passed and removed ci-passed labels Apr 3, 2026
Add standalone tool cmd/etcd-schema-verify that reads collection
metadata directly from etcd and compares source vs target schemas
at the etcd level. Covers fields invisible to DescribeCollection:
internal field IDs, field states, index params, partition states,
function schemas.

CDC scenario aware: compares all IDs (collection/field/partition/
index) since they are preserved across clusters. Outputs JSON with
an aligned flag and detailed diffs, exits 1 when not aligned.

Integrate the tool into the secondary restore CI to verify schema
alignment after restore for hello_milvus and hello_milvus2.

Signed-off-by: huanghaoyuanhhy <haoyuan.huang@zilliz.com>
Copy link
Copy Markdown
Collaborator

@wenhuiZilliz wenhuiZilliz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@sre-ci-robot
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: huanghaoyuanhhy, wenhuiZilliz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot merged commit 73de620 into zilliztech:main Apr 7, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved ci-passed dco-passed kind/improvement Changes related to something improve, likes ut and code refactor lgtm size/XXL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants