Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion docs/en/connectors/sink/Maxcompute.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Used to read data from Maxcompute.
| partition_spec | string | no | - |
| overwrite | boolean | no | false |
| insert_strategy| string | no | upload |
| tunnel_endpoint| string | no | - |
| tunnel_name | string | no | - |
| common-options | string | no | |

### accessId [string]
Expand Down Expand Up @@ -152,6 +154,22 @@ Example values:

Default: Not set (auto-inferred from region)

### tunnel_name [String]

`tunnel_name` Specifies the Tunnel Quota name for exclusive resource groups.

Tunnel Quota allows you to use dedicated computing resources for MaxCompute Tunnel data transfer, providing better performance and resource isolation.

**Important**: Tunnel Quota only works with **VPC (Virtual Private Cloud) endpoints**. It is not supported for public network access. You must configure both `endpoint` and `tunnel_endpoint` to use VPC endpoints when using `tunnel_name`.

If not specified, the default Tunnel quota will be used.

Example values:

- `your_tunnel_quota_name`

Default: Not set (use default quota)

### insert_strategy [string]

If `insert_strategy` is set to `upload`, insert operations use an upload session.
Expand Down Expand Up @@ -183,4 +201,4 @@ sink {

## Changelog

<ChangeLog />
<ChangeLog />
18 changes: 18 additions & 0 deletions docs/en/connectors/source/Maxcompute.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ Used to read data from Maxcompute.
| split_row | int | no | 10000 |
| read_columns | Array | no | - |
| table_list | Array | No | - |
| tunnel_endpoint| string | no | - |
| tunnel_name | string | no | - |
| common-options | string | no | |
| schema | config | no | |

Expand Down Expand Up @@ -86,6 +88,22 @@ Example values:

Default: Not set (auto-inferred from region)

### tunnel_name [String]

`tunnel_name` Specifies the Tunnel Quota name for exclusive resource groups.

Tunnel Quota allows you to use dedicated computing resources for MaxCompute Tunnel data transfer, providing better performance and resource isolation.

**Important**: Tunnel Quota only works with **VPC (Virtual Private Cloud) endpoints**. It is not supported for public network access. You must configure both `endpoint` and `tunnel_endpoint` to use VPC endpoints when using `tunnel_name`.

If not specified, the default Tunnel quota will be used.

Example values:

- `your_tunnel_quota_name`

Default: Not set (use default quota)

### common options

Source plugin common parameters, please refer to [Source Common Options](../common-options/source-common-options.md) for details.
Expand Down
20 changes: 19 additions & 1 deletion docs/zh/connectors/sink/Maxcompute.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ import ChangeLog from '../changelog/connector-maxcompute.md';
| table_name | string | 是 | - |
| partition_spec | string | 否 | - |
| overwrite | boolean | 否 | false |
| insert_strategy| string | no | upload |
| insert_strategy| string | 否 | upload |
| tunnel_endpoint| string | 否 | - |
| tunnel_name | string | 否 | - |
| common-options | string | 否 | |

### accessId [string]
Expand Down Expand Up @@ -150,6 +152,22 @@ CREATE TABLE IF NOT EXISTS `${table}`

默认值:未设置(从区域自动推断)

### tunnel_name [String]

`tunnel_name` 指定 Tunnel Quota 名称,用于独占资源组。

Tunnel Quota 允许您使用专用的计算资源进行 MaxCompute Tunnel 数据传输,从而提供更好的性能和资源隔离。

**重要提示**:Tunnel Quota 仅在 **VPC(虚拟私有云)端点**下生效,暂不支持公共网络访问。使用 `tunnel_name` 时,必须同时配置 `endpoint` 和 `tunnel_endpoint` 为 VPC 端点。

如果未指定,将使用默认的 Tunnel quota。

示例值:

- `your_tunnel_quota_name`

默认值:未设置(使用默认 quota)

### insert_strategy [string]

如果将 `insert_strategy` 设置为 `upload`,插入操作将使用 upload 会话。
Expand Down
39 changes: 38 additions & 1 deletion docs/zh/connectors/source/Maxcompute.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ import ChangeLog from '../changelog/connector-maxcompute.md';
| split_row | int | 否 | 10000 |
| read_columns | Array | 否 | - |
| table_list | Array | 否 | - |
| tunnel_endpoint| string | 否 | - |
| tunnel_name | string | 否 | - |
| common-options | string | 否 | |
| schema | config | 否 | |

Expand Down Expand Up @@ -68,6 +70,41 @@ import ChangeLog from '../changelog/connector-maxcompute.md';

要读取的表列表,您可以使用此配置代替 `table_name`.

### tunnel_endpoint [String]

指定 MaxCompute Tunnel 服务的自定义端点 URL。

默认情况下,端点是从配置的区域自动推断的。

此选项允许您覆盖默认行为并使用自定义 Tunnel 端点。
如果未指定,连接器将使用基于区域的默认 Tunnel 端点。

通常,您**不需要**设置 tunnel_endpoint。仅在自定义网络、调试或本地开发时才需要。

示例值:

- `https://dt.cn-hangzhou.maxcompute.aliyun.com`
- `https://dt.ap-southeast-1.maxcompute.aliyun.com`
- `http://maxcompute:8080`

默认值:未设置(从区域自动推断)

### tunnel_name [String]

`tunnel_name` 指定 Tunnel Quota 名称,用于独占资源组。

Tunnel Quota 允许您使用专用的计算资源进行 MaxCompute Tunnel 数据传输,从而提供更好的性能和资源隔离。

**重要提示**:Tunnel Quota 仅在 **VPC(虚拟私有云)端点**下生效,暂不支持公共网络访问。使用 `tunnel_name` 时,必须同时配置 `endpoint` 和 `tunnel_endpoint` 为 VPC 端点。

如果未指定,将使用默认的 Tunnel quota。

示例值:

- `your_tunnel_quota_name`

默认值:未设置(使用默认 quota)

### common options

源插件常用参数, 详见 [源通用选项](../common-options/source-common-options.md) .
Expand Down Expand Up @@ -121,4 +158,4 @@ source {

## 变更日志

<ChangeLog />
<ChangeLog />
Original file line number Diff line number Diff line change
Expand Up @@ -73,4 +73,10 @@ public class MaxcomputeBaseOptions implements Serializable {
.stringType()
.noDefaultValue()
.withDescription("Tunnel endpoint, e.g. http://maxcompute:8080");

public static final Option<String> TUNNEL_NAME =
Options.key("tunnel_name")
.stringType()
.noDefaultValue()
.withDescription("Tunnel quota name for exclusive resource groups");
Comment thread
xxzuo marked this conversation as resolved.
}
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ public OptionRule optionRule() {
MaxcomputeSinkOptions.CUSTOM_SQL,
FormatOptions.DATETIME_FORMAT,
MaxcomputeSinkOptions.TUNNEL_ENDPOINT,
MaxcomputeSinkOptions.TUNNEL_NAME,
SinkConnectorCommonOptions.MULTI_TABLE_SINK_REPLICA)
.build();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,8 @@ public OptionRule optionRule() {
ConnectorCommonOptions.SCHEMA,
MaxcomputeSourceOptions.PROJECT,
MaxcomputeSourceOptions.READ_COLUMNS,
MaxcomputeSourceOptions.TUNNEL_ENDPOINT)
MaxcomputeSourceOptions.TUNNEL_ENDPOINT,
MaxcomputeSourceOptions.TUNNEL_NAME)
.exclusive(CatalogOptions.TABLE_LIST, MaxcomputeSourceOptions.TABLE_NAME)
.build();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,11 @@ public static TableTunnel getTableTunnel(ReadonlyConfig readonlyConfig) {
if (StringUtils.isNotEmpty(readonlyConfig.get(MaxcomputeBaseOptions.TUNNEL_ENDPOINT))) {
tableTunnel.setEndpoint(readonlyConfig.get(MaxcomputeBaseOptions.TUNNEL_ENDPOINT));
}
if (StringUtils.isNotEmpty(readonlyConfig.get(MaxcomputeBaseOptions.TUNNEL_NAME))) {
tableTunnel
.getConfig()
.setQuotaName(readonlyConfig.get(MaxcomputeBaseOptions.TUNNEL_NAME));
Comment thread
xxzuo marked this conversation as resolved.
Comment thread
xxzuo marked this conversation as resolved.
}
return tableTunnel;
}

Expand Down
Loading