Merge pull request #6492 from ClickHouse/kk/clickpipes-rpe-custom-private-dns-aws

feat(clickpipes): document custom private DNS for AWS PrivateLink reverse private endpoints

Author: Blargian

docs/integrations/data-ingestion/clickpipes/aws-privatelink.md

@@ -10,6 +10,7 @@ integration:
    - category: 'clickpipes'
 ---
 
+import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge';
 import cp_service from '@site/static/images/integrations/data-ingestion/clickpipes/cp_service.png';
 import cp_step0 from '@site/static/images/integrations/data-ingestion/clickpipes/cp_step0.png';
 import cp_rpe_select from '@site/static/images/integrations/data-ingestion/clickpipes/cp_rpe_select.png';
@@ -189,6 +190,8 @@ It requires setting up a NLB (Network Load Balancer) in front of your data sourc
 and configuring the VPC endpoint service to use the NLB.
 
 VPC endpoint service can be [configured with a private DNS](https://docs.aws.amazon.com/vpc/latest/privatelink/manage-dns-names.html), that will be accessible in a ClickPipes VPC.
+If your service can't use PrivateLink managed private DNS (for example, the provider requires
+consumers to manage DNS resolution, as Confluent Cloud does), see [Custom private DNS](#custom-private-dns).
 
 It's a preferred choice for:
 
@@ -208,6 +211,52 @@ See AWS guide for [managing permissions](https://docs.aws.amazon.com/vpc/latest/
 can be configured for ClickPipes. Add [your ClickPipe region](#aws-privatelink-regions) to the allowed regions in your VPC endpoint service.
 :::
 
+## Custom private DNS {#custom-private-dns}
+
+<PrivatePreviewBadge/>
+
+:::info
+Custom private DNS is in **Private Preview**. Reach out to the [ClickHouse support team](https://clickhouse.com/support/program) to enable it for your service.
+:::
+
+AWS PrivateLink provides [managed private DNS](https://docs.aws.amazon.com/vpc/latest/privatelink/manage-dns-names.html)
+for VPC endpoint services with a verified domain. Some services require each consumer to manage DNS resolution in their own VPC, e.g., Confluent Cloud requires consumers to resolve broker hostnames to the PrivateLink endpoint.
+
+For these cases, ClickPipes supports attaching custom private DNS names to a reverse private
+endpoint. ClickPipes resolves these names to the endpoint's private addresses, so your data
+source can be reached by its own hostname over private connectivity.
+
+Custom private DNS complements managed private DNS — it doesn't replace it. If your
+PrivateLink service already provides private DNS names, you don't need custom names.
+
+Custom private DNS names are supported for the [VPC endpoint service](#vpc-endpoint-service)
+and [VPC resource](#vpc-resource) endpoint types. MSK multi-VPC provides managed private DNS
+out-of-the-box and doesn't support custom names.
+
+The following rules apply to custom private DNS names:
+
+- Exact names (`kafka.internal.example.com`) and wildcard names (`*.example.com`) are
+  supported. A wildcard matches a single DNS label — for example,
+  `*.abcde12345.us-east-1.aws.confluent.cloud` matches
+  `b0-lkc123.abcde12345.us-east-1.aws.confluent.cloud`.
+- Names must be unique across all reverse private endpoints of a ClickHouse service,
+  including overlaps between wildcard and exact names.
+- Names under reserved suffixes (e.g., `local`, `localhost`, `internal`, `corp`,
+  `private`) are rejected.
+
+To configure custom private DNS names:
+
+- In the ClickHouse Cloud console, fill in the `Custom private DNS name` field when
+  [creating a reverse private endpoint](#creating-clickpipe). The field is shown once the
+  feature is enabled for your service.
+- With [OpenAPI](https://clickhouse.com/docs/cloud/manage/api/swagger), set
+  `customPrivateDnsMappings` when creating a reverse private endpoint, or update an existing
+  endpoint with a `PATCH` request. Updates replace the full list of mappings; an empty list
+  removes all custom names.
+- With Terraform, use the
+  [`clickhouse_clickpipes_reverse_private_endpoint_custom_private_dns`](https://registry.terraform.io/providers/ClickHouse/clickhouse/latest/docs/resources/clickpipes_reverse_private_endpoint_custom_private_dns)
+  resource to manage mappings on an existing reverse private endpoint.
+
 ## Creating a ClickPipe with reverse private endpoint {#creating-clickpipe}
 
 <VerticalStepper headerLevel="list">
@@ -246,6 +295,7 @@ For same-region access, creating a VPC Resource is the recommended approach.
     - For VPC resource, provide the configuration share ARN and configuration ID.
     - For MSK multi-VPC, provide the cluster ARN and authentication method used with a created endpoint.
     - For VPC endpoint service, provide the service name.
+    - Optionally, provide a [custom private DNS](#custom-private-dns) name.
 
 7. Click on `Create` and wait for the reverse private endpoint to be ready.
 
@@ -258,7 +308,8 @@ For same-region access, creating a VPC Resource is the recommended approach.
 8. Once the endpoint is ready, you can use a DNS name to connect to the data source.
 
    On a list of endpoints, you can see the DNS name for the available endpoint.
-   It can be either an internally ClickPipes provisioned DNS name or a private DNS name supplied by a PrivateLink service.
+   It can be an internally ClickPipes provisioned DNS name, a private DNS name supplied by a PrivateLink service,
+   or a [custom private DNS name](#custom-private-dns).
    DNS name isn't a complete network address.
    Add the port according to the data source.
 

docs/integrations/data-ingestion/clickpipes/kafka/02_schema-registries.md

@@ -36,7 +36,7 @@ To integrate with a schema registry during ClickPipes configuration, you must us
 
 ClickPipes connects to the schema registry over HTTPS at the URL you provide. The schema registry does not need to be publicly accessible.
 
-If your Kafka brokers are reached through a [reverse private endpoint](/integrations/clickpipes/aws-privatelink) (AWS PrivateLink or GCP Private Service Connect), the schema registry can use the same private connectivity. ClickPipes resolves the registry hostname through the reverse private endpoint's private DNS, so a registry hosted privately alongside your brokers is reachable as long as its hostname resolves to the reverse private endpoint's private IP addresses (via the endpoint's private DNS support or a custom private DNS mapping).
+If your Kafka brokers are reached through a [reverse private endpoint](/integrations/clickpipes/aws-privatelink) (AWS PrivateLink or GCP Private Service Connect), the schema registry can use the same private connectivity. ClickPipes resolves the registry hostname through the reverse private endpoint's private DNS, so a registry hosted privately alongside your brokers is reachable as long as its hostname resolves to the reverse private endpoint's private IP addresses (via the endpoint's private DNS support or a [custom private DNS mapping](/integrations/clickpipes/aws-privatelink#custom-private-dns)).
 
 Keep the following in mind: