Skip to content

Commit d562d0f

Browse files
ArlowenArlowen
committed
feat: align cli coverage with openapi sdk
1 parent ffad73e commit d562d0f

27 files changed

Lines changed: 2486 additions & 60 deletions

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,11 @@ cloudcanal jobs --help
3939
cloudcanal config lang set zh
4040
cloudcanal jobs list
4141
cloudcanal jobs show 123
42+
cloudcanal jobs create --body-file create-job.json
4243
cloudcanal datasources list --type MYSQL
44+
cloudcanal datasources add --body-file add-datasource.json
4345
cloudcanal workers list --cluster-id 2
46+
cloudcanal schemas list-trans-objs-by-meta --src-db demo --src-trans-obj orders
4447
```
4548

4649
JSON 输出：
@@ -81,6 +84,7 @@ cloudcanal jobs list --type SYNC --output json
8184
## 文档入口
8285

8386
- 安装、初始化、命令参数、示例：[docs/cloudcanal-cli-usage.md](docs/cloudcanal-cli-usage.md)
87+
- SDK API 对照 CLI 命令：[docs/openapi-sdk-api-reference.md](docs/openapi-sdk-api-reference.md)
8488
- 版本变更记录：[CHANGELOG.md](CHANGELOG.md)
8589
- 机器可读输出：在查询命令后追加 `--output json`
8690
- 交互模式和 shell 均支持 TAB 自动补全

‎docs/cloudcanal-cli-usage.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,10 @@ cloudcanal
1616
cloudcanal --help
1717
cloudcanal jobs --help
1818
cloudcanal jobs list
19+
cloudcanal jobs create --body-file create-job.json
1920
cloudcanal datasources list --type MYSQL
21+
cloudcanal datasources add --body-file add-datasource.json
22+
cloudcanal schemas list-trans-objs-by-meta --src-db demo --src-trans-obj orders
2023
cloudcanal jobs list --type SYNC --output json
2124
```
2225

@@ -92,6 +95,7 @@ curl -fsSL https://raw.githubusercontent.com/Arlowen/cloudcanal-openapi-cli/main
9295
- `help workers`
9396
- `help consolejobs`
9497
- `help job-config`
98+
- `help schemas`
9599
- `help config`
96100

97101
绝大多数命令组也支持 `--help`，例如：
@@ -102,6 +106,21 @@ cloudcanal jobs list --help
102106
cloudcanal config --help
103107
```
104108

109+
## 复杂请求体
110+
111+
对于和 SDK 一一对应、字段较多的接口，CLI 统一支持：
112+
113+
- `--body '{...}'`: 直接传 JSON
114+
- `--body-file FILE.json`: 从文件读取 JSON
115+
116+
目前主要用于：
117+
118+
- `jobs create`
119+
- `jobs update-incre-pos`
120+
- `datasources add`
121+
122+
完整字段说明见 [openapi-sdk-api-reference.md](openapi-sdk-api-reference.md)。
123+
105124
`config show`
106125

107126
显示当前配置，`accessKey` 会做掩码处理，同时会显示当前 `language`。
@@ -147,6 +166,10 @@ cloudcanal jobs list --type SYNC --output json
147166

148167
查看任务详情。
149168

169+
`jobs create --body-file FILE.json`
170+
171+
按 SDK `AddJobRequest` 的字段创建任务，也支持 `--body '{...}'`。
172+
150173
`jobs schema <jobId>`
151174

152175
查看任务 schema 信息。
@@ -176,6 +199,18 @@ cloudcanal jobs list --type SYNC --output json
176199
cloudcanal jobs replay 123 --auto-start --reset-to-created
177200
```
178201

202+
`jobs attach-incre-task <jobId>`
203+
204+
绑定增量任务。
205+
206+
`jobs detach-incre-task <jobId>`
207+
208+
解绑增量任务。
209+
210+
`jobs update-incre-pos --body-file FILE.json`
211+
212+
按 SDK `UpdateIncrePosRequest` 的字段更新增量位点，也支持 `--body '{...}'`。
213+
179214
## DataSource
180215

181216
`datasources list [参数]`
@@ -198,6 +233,17 @@ cloudcanal datasources list --type MYSQL --deploy-type ALIYUN
198233

199234
查看单个数据源详情。
200235

236+
`datasources add --body-file FILE.json [--security-file FILE] [--secret-file FILE]`
237+
238+
创建数据源。`--body-file` 支持两种格式：
239+
240+
- 直接传 `ApiDsAddData` JSON
241+
- 传带 `dataSourceAddData` 包装层的完整请求 JSON
242+
243+
`datasources delete <dataSourceId>`
244+
245+
删除数据源。
246+
201247
## Cluster
202248

203249
`clusters list [参数]`
@@ -239,6 +285,18 @@ cloudcanal workers list --cluster-id 2
239285

240286
停止机器。
241287

288+
`workers delete <workerId>`
289+
290+
删除机器。
291+
292+
`workers modify-mem-oversold <workerId> --percent N`
293+
294+
修改机器内存超卖比例。
295+
296+
`workers update-alert <workerId> --phone=true|false --email=true|false --im=true|false --sms=true|false`
297+
298+
更新机器告警开关。
299+
242300
## ConsoleJob
243301

244302
`consolejobs show <consoleJobId>`
@@ -261,6 +319,29 @@ cloudcanal workers list --cluster-id 2
261319
cloudcanal job-config specs --type SYNC --initial-sync=true
262320
```
263321

322+
`job-config transform-job-type --source-type TYPE --target-type TYPE`
323+
324+
根据源端和目标端类型转换任务类型。
325+
326+
## Schema
327+
328+
`schemas list-trans-objs-by-meta [参数]`
329+
330+
按元信息查询传输对象，支持：
331+
332+
- `--src-db <name>`
333+
- `--src-schema <name>`
334+
- `--src-trans-obj <name>`
335+
- `--dst-db <name>`
336+
- `--dst-schema <name>`
337+
- `--dst-tran-obj <name>`
338+
339+
示例：
340+
341+
```bash
342+
cloudcanal schemas list-trans-objs-by-meta --src-db demo --src-trans-obj orders
343+
```
344+
264345
## 使用建议
265346

266347
- 带空格的参数值请使用引号包裹，例如 `--desc "nightly sync"`

‎docs/openapi-sdk-api-reference.md‎

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
# CloudCanal OpenAPI SDK 对照 CLI 接口文档
2+
3+
本文档按 `cloudcanal-openapi-sdk` 当前支持的 API 生成，并给出 `cloudcanal-openapi-cli` 中对应的命令实现。
4+
5+
说明：
6+
7+
- SDK 当前共覆盖 7 个模块、25 个 OpenAPI path。
8+
- CLI 已按 SDK 一比一补齐这些接口。
9+
- `datasources show <dataSourceId>` 是 CLI 便捷命令，内部仍复用 `datasource/listds`，不是 SDK 的独立 API。
10+
- 复杂请求统一支持 `--body '{...}'` 或 `--body-file FILE.json`。`datasource add` 额外支持 `--security-file`、`--secret-file`。
11+
12+
## Cluster
13+
14+
| SDK API | Path | CLI 命令 | 关键请求字段 | 返回摘要 |
15+
| --- | --- | --- | --- | --- |
16+
| `ListClusterRequest` | `/cloudcanal/console/api/v1/openapi/cluster/listclusters` | `cloudcanal clusters list [--name NAME] [--desc DESC] [--cloud CLOUD] [--region REGION]` | `cloudOrIdcName`, `clusterDescLike`, `clusterNameLike`, `region` | `id`, `clusterName`, `region`, `cloudOrIdcName`, `workerCount`, `runningCount`, `abnormalCount`, `ownerName` |
17+
18+
## ConsoleJob
19+
20+
| SDK API | Path | CLI 命令 | 关键请求字段 | 返回摘要 |
21+
| --- | --- | --- | --- | --- |
22+
| `QueryConsoleJobRequest` | `/cloudcanal/console/api/v1/openapi/consolejob/queryconsolejob` | `cloudcanal consolejobs show <consoleJobId>` | `consoleJobId` | `id`, `label`, `taskState`, `jobToken`, `resourceType`, `resourceId`, `taskVOList` |
23+
24+
## Constant
25+
26+
| SDK API | Path | CLI 命令 | 关键请求字段 | 返回摘要 |
27+
| --- | --- | --- | --- | --- |
28+
| `ListSpecsRequest` | `/cloudcanal/console/api/v1/openapi/constant/listspecs` | `cloudcanal job-config specs --type TYPE [--initial-sync=true\|false] [--short-term-sync=true\|false]` | `dataJobType`, `initialSync`, `shortTermSync` | `id`, `specKind`, `specKindCn`, `spec`, `fullMemoryMb`, `increMemoryMb`, `checkMemoryMb` |
29+
| `TransformJobTypeRequest` | `/cloudcanal/console/api/v1/openapi/constant/transformjobtype` | `cloudcanal job-config transform-job-type --source-type TYPE --target-type TYPE` | `sourceType`, `targetType` | SDK 未定义固定响应类型，CLI 保留原始 `data` JSON |
30+
31+
## DataJob
32+
33+
| SDK API | Path | CLI 命令 | 关键请求字段 | 返回摘要 |
34+
| --- | --- | --- | --- | --- |
35+
| `ListJobsRequest` | `/cloudcanal/console/api/v1/openapi/datajob/list` | `cloudcanal jobs list [--name NAME] [--type TYPE] [--desc DESC] [--source-id ID] [--target-id ID]` | `dataJobName`, `dataJobType`, `desc`, `sourceInstanceId`, `targetInstanceId` | `dataJobId`, `dataJobName`, `dataTaskState`, `sourceDsVO`, `targetDsVO` |
36+
| `QueryJobRequest` | `/cloudcanal/console/api/v1/openapi/datajob/queryjob` | `cloudcanal jobs show <jobId>` | `jobId` | `ApiDataJobDO` 详情 |
37+
| `QueryJobSchemaRequest` | `/cloudcanal/console/api/v1/openapi/datajob/queryjobschemabyid` | `cloudcanal jobs schema <jobId>` | `jobId` | `sourceSchema`, `targetSchema`, `mappingConfig`, `defaultTopic`, `defaultTopicPartition` |
38+
| `AddJobRequest` | `/cloudcanal/console/api/v1/openapi/datajob/create` | `cloudcanal jobs create --body-file FILE.json` | 见下方 `jobs create` 请求模板 | `data` 为创建结果，CLI 同时输出 `jobId` |
39+
| `StartJobRequest` | `/cloudcanal/console/api/v1/openapi/datajob/start` | `cloudcanal jobs start <jobId>` | `jobId` | 通用成功/失败响应 |
40+
| `StopJobRequest` | `/cloudcanal/console/api/v1/openapi/datajob/stop` | `cloudcanal jobs stop <jobId>` | `jobId` | 通用成功/失败响应 |
41+
| `DeleteJobRequest` | `/cloudcanal/console/api/v1/openapi/datajob/delete` | `cloudcanal jobs delete <jobId>` | `jobId` | 通用成功/失败响应 |
42+
| `ReplayJobRequest` | `/cloudcanal/console/api/v1/openapi/datajob/replay` | `cloudcanal jobs replay <jobId> [--auto-start] [--reset-to-created]` | `jobId`, `autoStart`, `resetToCreated` | 通用成功/失败响应 |
43+
| `AttachIncreJobRequest` | `/cloudcanal/console/api/v1/openapi/datajob/attachincretask` | `cloudcanal jobs attach-incre-task <jobId>` | `jobId` | 通用成功/失败响应 |
44+
| `DetachIncreJobRequest` | `/cloudcanal/console/api/v1/openapi/datajob/detachincretask` | `cloudcanal jobs detach-incre-task <jobId>` | `jobId` | 通用成功/失败响应 |
45+
| `UpdateIncrePosRequest` | `/cloudcanal/console/api/v1/openapi/datajob/updateincrepos` | `cloudcanal jobs update-incre-pos --body-file FILE.json` | 见下方 `jobs update-incre-pos` 请求模板 | `data` 为更新结果 |
46+
47+
### `jobs create` 请求模板
48+
49+
```json
50+
{
51+
"clusterId": 1,
52+
"srcDsId": 391,
53+
"dstDsId": 16,
54+
"srcHostType": "PRIVATE",
55+
"dstHostType": "PUBLIC",
56+
"jobType": "SYNC",
57+
"dataJobDesc": "api created",
58+
"specId": 18,
59+
"autoStart": true,
60+
"structMigration": true,
61+
"initialSync": true,
62+
"shortTermSync": false,
63+
"shortTermNum": 3,
64+
"filterDDL": true,
65+
"srcSchema": "{\"schema\":\"src\"}",
66+
"dstSchema": "{\"schema\":\"dst\"}",
67+
"mappingDef": "[{\"method\":\"DB_DB\"}]",
68+
"schemaWhiteListLevel": "TABLE",
69+
"checkOnce": false,
70+
"checkPeriod": false,
71+
"fullPeriod": false
72+
}
73+
```
74+
75+
SDK 里还有更多高级字段，CLI 已全部支持，例如：
76+
77+
- `srcCaseSensitiveType`, `dstCaseSensitiveType`
78+
- `srcDsCharset`, `tarDsCharset`
79+
- `keyConflictStrategy`
80+
- `checkPeriodCronExpr`, `fullPeriodCronExpr`
81+
- `dstMqDefaultTopic`, `dstMqDefaultTopicPartitions`
82+
- `dstMqDdlTopic`, `dstMqDdlTopicPartitions`
83+
- `srcSchemaLessFormat`, `dstSchemaLessFormat`
84+
- `originDecodeMsgFormat`
85+
- `dstCkTableEngine`, `dstSrOrDorisTableModel`
86+
- `kafkaConsumerGroupId`, `kuduNumReplicas`
87+
- `srcRocketMqGroupId`
88+
- `srcRabbitMqVhost`, `srcRabbitExchange`
89+
- `dstRabbitMqVhost`, `dstRabbitExchange`
90+
- `obTenant`, `dbHeartbeatEnable`
91+
92+
### `jobs update-incre-pos` 请求模板
93+
94+
```json
95+
{
96+
"taskId": 320,
97+
"posType": "MYSQL_LOG_FILE_POS",
98+
"journalFile": "binlog.000491",
99+
"filePosition": 794891
100+
}
101+
```
102+
103+
不同数据库位点字段按 SDK 原样支持：
104+
105+
- MySQL 类：`journalFile`, `filePosition`, `gtidPosition`, `positionTimestamp`, `serverId`
106+
- PostgreSQL/SQL Server：`lsn`
107+
- Oracle：`scn`, `scnIndex`
108+
- 通用：`commonPosStr`
109+
- HANA：`dataId`, `transactionId`
110+
111+
## DataSource
112+
113+
| SDK API | Path | CLI 命令 | 关键请求字段 | 返回摘要 |
114+
| --- | --- | --- | --- | --- |
115+
| `ListDsRequest` | `/cloudcanal/console/api/v1/openapi/datasource/listds` | `cloudcanal datasources list [--id ID] [--type TYPE] [--deploy-type TYPE] [--host-type TYPE] [--lifecycle STATE]` | `dataSourceId`, `deployType`, `hostType`, `lifeCycleState`, `type` | `id`, `instanceId`, `dataSourceType`, `hostType`, `deployType`, `lifeCycleState` |
116+
| `AddDsRequest` | `/cloudcanal/console/api/v1/openapi/datasource/addds` | `cloudcanal datasources add --body-file FILE.json [--security-file FILE] [--secret-file FILE]` | `dataSourceAddData` + 可选文件 part | `data` 为创建结果 |
117+
| `DeleteDsRequest` | `/cloudcanal/console/api/v1/openapi/datasource/deleteds` | `cloudcanal datasources delete <dataSourceId>` | `dataSourceId` | 通用成功/失败响应 |
118+
119+
### `datasources add` 请求体
120+
121+
`datasources add` 支持两种 body 形式：
122+
123+
1. 直接传 `ApiDsAddData` JSON
124+
2. 传完整包装对象：
125+
126+
```json
127+
{
128+
"dataSourceAddData": {
129+
"type": "MYSQL",
130+
"host": "127.0.0.1:3306",
131+
"privateHost": "127.0.0.1:3306",
132+
"hostType": "PRIVATE",
133+
"deployType": "ALIYUN",
134+
"region": "cn-hangzhou",
135+
"instanceDesc": "mysql source",
136+
"account": "root",
137+
"password": "secret",
138+
"securityType": "USER_PASSWD"
139+
},
140+
"securityFilePath": "/path/to/security.pem",
141+
"secretFilePath": "/path/to/secret.key"
142+
}
143+
```
144+
145+
CLI 也支持把文件路径拆到 flag：
146+
147+
```bash
148+
cloudcanal datasources add \
149+
--body-file add-datasource.json \
150+
--security-file /path/to/security.pem \
151+
--secret-file /path/to/secret.key
152+
```
153+
154+
`ApiDsAddData` 的高级字段也都支持，例如：
155+
156+
- `instanceId`, `version`, `dbName`
157+
- `accessKey`, `secretKey`, `clientTrustStorePassword`
158+
- `clusterIds`, `lifeCycleState`, `driver`, `connectType`
159+
- `dsKvConfigs`, `extraData`, `parentDsId`
160+
161+
## Worker
162+
163+
| SDK API | Path | CLI 命令 | 关键请求字段 | 返回摘要 |
164+
| --- | --- | --- | --- | --- |
165+
| `ListWorkerRequest` | `/cloudcanal/console/api/v1/openapi/worker/listworkers` | `cloudcanal workers list --cluster-id ID [--source-id ID] [--target-id ID]` | `clusterId`, `sourceInstanceId`, `targetInstanceId` | `id`, `workerName`, `workerState`, `workerType`, `privateIp`, `healthLevel`, `workerLoad`，以及 SDK 中更多扩展字段 |
166+
| `StartWorkerRequest` | `/cloudcanal/console/api/v1/openapi/worker/startWorker` | `cloudcanal workers start <workerId>` | `workerId` | 通用成功/失败响应 |
167+
| `StopWorkerRequest` | `/cloudcanal/console/api/v1/openapi/worker/stopWorker` | `cloudcanal workers stop <workerId>` | `workerId` | 通用成功/失败响应 |
168+
| `DeleteWorkerRequest` | `/cloudcanal/console/api/v1/openapi/worker/deleteWorker` | `cloudcanal workers delete <workerId>` | `workerId` | 通用成功/失败响应 |
169+
| `ModifyMemOverSoldRequest` | `/cloudcanal/console/api/v1/openapi/worker/modifyMemOverSoldPercent` | `cloudcanal workers modify-mem-oversold <workerId> --percent N` | `workerId`, `memOverSoldPercent` | 通用成功/失败响应 |
170+
| `UpdateWorkerAlertRequest` | `/cloudcanal/console/api/v1/openapi/worker/updateWorkerAlertConfig` | `cloudcanal workers update-alert <workerId> --phone=true\|false --email=true\|false --im=true\|false --sms=true\|false` | `workerId`, `phone`, `email`, `im`, `sms` | 通用成功/失败响应 |
171+
172+
## Schema
173+
174+
| SDK API | Path | CLI 命令 | 关键请求字段 | 返回摘要 |
175+
| --- | --- | --- | --- | --- |
176+
| `ListTransObjsByMetaRequest` | `/cloudcanal/console/api/v1/openapi/schema/listTransObjsByMeta` | `cloudcanal schemas list-trans-objs-by-meta [--src-db NAME] [--src-schema NAME] [--src-trans-obj NAME] [--dst-db NAME] [--dst-schema NAME] [--dst-tran-obj NAME]` | `srcDb`, `srcSchema`, `srcTransObj`, `dstDb`, `dstSchema`, `dstTranObj` | `dataJobId`, `dataJobName`, `srcFullTransferObjName`, `dstFullTransferObjName`, `srcDsType`, `dstDsType` |
177+
178+
## 通用返回规则
179+
180+
- 所有 OpenAPI 响应都遵循 SDK 的 `CcResponse` 约定。
181+
- `code == "1"` 表示成功。
182+
- `code != "1"` 时，CLI 会直接输出 `msg` 作为错误信息。

‎internal/app/runtime.go‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import (
1010
"cloudcanal-openapi-cli/internal/i18n"
1111
"cloudcanal-openapi-cli/internal/jobconfig"
1212
"cloudcanal-openapi-cli/internal/openapi"
13+
ccschema "cloudcanal-openapi-cli/internal/schema"
1314
"cloudcanal-openapi-cli/internal/worker"
1415
)
1516

@@ -21,6 +22,7 @@ type RuntimeContext interface {
2122
Workers() worker.Operations
2223
ConsoleJobs() consolejob.Operations
2324
JobConfigs() jobconfig.Operations
25+
Schemas() ccschema.Operations
2426
Reinitialize(io console.IO) (bool, error)
2527
SetLanguage(language string) error
2628
}
@@ -34,6 +36,7 @@ type Runtime struct {
3436
workers worker.Operations
3537
consoleJobs consolejob.Operations
3638
jobConfigs jobconfig.Operations
39+
schemas ccschema.Operations
3740
}
3841

3942
func NewRuntime(configService *config.Service) *Runtime {
@@ -102,6 +105,10 @@ func (r *Runtime) JobConfigs() jobconfig.Operations {
102105
return r.jobConfigs
103106
}
104107

108+
func (r *Runtime) Schemas() ccschema.Operations {
109+
return r.schemas
110+
}
111+
105112
func (r *Runtime) SetLanguage(language string) error {
106113
cfg := r.config
107114
cfg.Language = language
@@ -135,5 +142,6 @@ func (r *Runtime) activate(cfg config.AppConfig) error {
135142
r.workers = worker.NewService(client)
136143
r.consoleJobs = consolejob.NewService(client)
137144
r.jobConfigs = jobconfig.NewService(client)
145+
r.schemas = ccschema.NewService(client)
138146
return nil
139147
}

0 commit comments

Comments
 (0)