feat: add DM (Data Migration) support#6883
Conversation
Introduces DMGroup/DM (dm-master) and DMWorkerGroup/DMWorker (dm-worker) CRDs following the existing Group+Instance pattern used by TiCDC, TiDB, and TiKV. Adds ComponentDMMaster/ComponentDMWorker constants to the meta package and regenerates deepcopy/register files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…worker - Register DM and DMWorker in InstanceSet, DMGroup and DMWorkerGroup in GroupSet - Add full Instance/Group interface implementations for DM master (DM, DMGroup) - Add full Instance/Group interface implementations for DM worker (DMWorker, DMWorkerGroup) - Add scope structs for DM and DMWorker with GVK, component, list helpers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add VolumeMountType constants and default paths for DM master (data) and DM worker (relay-dir) - Add container names, config dir paths, and cluster TLS dir paths to names.go - Add DM/DMWorker port utility functions to coreutil - Register DM container names in allMainContainers - Add dm config package (master-addr, peer-urls, initial-cluster, join, TLS) - Add dmworker config package (worker-addr, join, relay-dir, TLS) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add DMState, DMGroupState, DMSliceState, DMWorkerState interfaces to common - Add DM image constant (pingcap/dm) to image package - Implement dm-master instance controller with full task pipeline: - state: peer DM slice for initial-cluster computation - ctx: HTTP health check against /api/v1/cluster/info - cm: config map with initial-cluster or join address - pvc: DataVolume PVC + additional volumes - pod: dm-master container with ports 8261/8291, TLS support - status: sync MemberID and conditions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ses 5-8) - Add DMGroup/DMWorker controllers with Group+Instance pattern - DMGroup controller: headless+internal services, bootstrap annotation management, rolling update with maxSurge=0/maxUnavailable=1 - DM instance controller: configmap, PVC (DataVolume), pod, health check via /api/v1/cluster/info, status sync - DMWorkerGroup controller: headless service only, no bootstrap sequence - DMWorker instance controller: configmap, PVC (RelayVolume), pod, health check via /status, dm-master addr from DMGroupRef internal service - Register all four controllers in main.go with field indexes and cache Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add basic cluster examples for a 1-replica dm-master cluster (DMGroup) and a 1-replica dm-worker (DMWorkerGroup) pointing to it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Skipping CI for Draft Pull Request. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6883 +/- ##
==========================================
- Coverage 37.44% 35.16% -2.28%
==========================================
Files 392 427 +35
Lines 22432 24054 +1622
==========================================
+ Hits 8399 8458 +59
- Misses 14033 15596 +1563
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Summary
DMGroup,DM,DMWorkerGroup,DMWorkerwith TLS support from the startpkg/configs/dm,pkg/configs/dmworker) for dm-master and dm-workerGET /api/v1/cluster/info), ConfigMap, DataVolume PVC, pod lifecyclemaxSurge=0/maxUnavailable=1for pod name stabilityGET /status), ConfigMap, RelayVolume PVC, pod lifecyclecmd/tidb-operator/main.goexamples/basic/07-dm.yamlandexamples/basic/08-dmworker.yamlArchitecture notes
AnnoKeyInitialClusterNumannotation on instances to computeinitial-cluster;TaskBootremoves the annotation once all initial instances are ready. Bootstrap state is derived from existing instances (absence of annotation = bootstrapped).joinaddress is derived from<DMGroupRef.Name>-dm-masterinternal service at the default DM port (8261).maxSurge=0, maxUnavailable=1to preserve pod names required by dm-master'slastBoundetcd history.Test plan
examples/basic/07-dm.yaml— verify DMGroup creates headless + internal service and DM podsAnnoKeyInitialClusterNumannotation removed after all initial replicas readyexamples/basic/08-dmworker.yaml— verify DMWorkerGroup creates headless service and DMWorker pods<dmgroup>-dm-masterservicespec.tlsClusteron Cluster, confirm TLS mounts appear in pods🤖 Generated with Claude Code