Merge remote-tracking branch 'origin/main' into raymond-entra-id-scim

Author: dhtclk

docs/cloud/guides/best_practices/usagelimits.md

@@ -38,6 +38,10 @@ or look together at how we can increase them in a controlled manner.
 
 For warehouse replica and scaling limits, see [warehouses](/cloud/reference/warehouses#scaling).
 
+:::note
+The values above are default guardrails. The limit enforced for a specific service may be higher than the value shown here, as larger services are configured with more headroom. The actual limit that applies to your service is included in the warning you receive as you approach it.
+:::
+
 :::note
 For Single Replica Services, the maximum number of databases is restricted to 
 100, and the maximum number of tables is restricted to 500. In addition, storage

docs/cloud/guides/infrastructure/01_deployment_options/byoc/03_onboarding/01_standard.md

@@ -53,19 +53,23 @@ The initial BYOC setup can be performed using a [CloudFormation template (AWS)](
 Storage buckets, VPC/VNet, Kubernetes cluster, and compute resources required for running ClickHouse aren't included in this initial setup. They will be provisioned in the next step.
 :::
 
-#### Alternative Terraform Module for AWS {#terraform-module-aws}
+#### Terraform Module for AWS {#terraform-module-aws}
 
-If you prefer to use Terraform instead of CloudFormation for AWS deployments, we also provide a [Terraform module for AWS](https://s3.us-east-2.amazonaws.com/clickhouse-public-resources.clickhouse.cloud/tf/byoc.tar.gz).
-
-Usage:
+If you prefer to use Terraform instead of CloudFormation for AWS deployments, use the [terraform-byoc-onboarding](https://github.com/ClickHouse/terraform-byoc-onboarding) module:
 
 ```hcl
 module "clickhouse_onboarding" {
-  source   = "https://s3.us-east-2.amazonaws.com/clickhouse-public-resources.clickhouse.cloud/tf/byoc.tar.gz"
-  byoc_env = "production"
+  source      = "github.com/ClickHouse/terraform-byoc-onboarding.git//modules/aws?ref=<version>"
+  external_id = "<external-id-provided-by-clickhouse>"
 }
 ```
 
+Replace `<version>` with the latest tag from the module's [releases page](https://github.com/ClickHouse/terraform-byoc-onboarding/releases) — always use the latest release.
+
+:::note
+The module was previously distributed as a tarball at `https://s3.us-east-2.amazonaws.com/clickhouse-public-resources.clickhouse.cloud/tf/byoc.tar.gz`. That URL remains available but is deprecated — use the GitHub module above.
+:::
+
 ### Set up BYOC infrastructure {#setup-byoc-infrastructure}
 
 You will be prompted to set up the infrastructure, including object storage buckets, VPC/VNet, and the Kubernetes cluster, from the ClickHouse Cloud console. Certain configurations must be determined at this stage, as they can't be changed later. Specifically:
@@ -86,7 +90,7 @@ After your BYOC infrastructure has been provisioned, you're ready to launch your
 
 <Image img={byoc_new_service_1} size="md" alt="BYOC create new service"/>
 
-During service creation, you’ll configure the following options:
+During service creation, you'll configure the following options:
 
 - **Service name**: Enter a clear, descriptive name for your ClickHouse service.
 - **BYOC infrastructure**: Select the BYOC environment, including the cloud account and region, where your service will run.

docs/cloud/guides/infrastructure/01_deployment_options/byoc/03_onboarding/02_customization/01_aws.md

@@ -46,26 +46,32 @@ Ensure your VPC has working DNS resolution and doesn't block, interfere with, or
 
 ### Configure your AWS account {#configure-aws-account}
 
-The initial BYOC setup creates a privileged IAM role (`ClickHouseManagementRole`) that enables BYOC controllers from ClickHouse Cloud to manage your infrastructure. This can be performed using either a [CloudFormation template](https://s3.us-east-2.amazonaws.com/clickhouse-public-resources.clickhouse.cloud/cf-templates/byoc.yaml) or a [Terraform module](https://s3.us-east-2.amazonaws.com/clickhouse-public-resources.clickhouse.cloud/tf/byoc.tar.gz).
+The initial BYOC setup creates a privileged IAM role (`ClickHouseManagementRole`) that enables BYOC controllers from ClickHouse Cloud to manage your infrastructure. This can be performed using either a [CloudFormation template](https://s3.us-east-2.amazonaws.com/clickhouse-public-resources.clickhouse.cloud/cf-templates/byoc.yaml) or a Terraform module (see below).
 
 When deploying for a `BYO-VPC` setup, set the `IncludeVPCWritePermissions` parameter to `false` to ensure ClickHouse Cloud doesn't receive permissions to modify your customer-managed VPC.
 
 :::note
 Storage buckets, Kubernetes cluster, and compute resources required for running ClickHouse aren't included in this initial setup. They will be provisioned in a later step. While you control your VPC, ClickHouse Cloud still requires IAM permissions to create and manage the Kubernetes cluster, IAM roles for service accounts, S3 buckets, and other essential resources in your AWS account.
 :::
 
-#### Alternative Terraform module {#terraform-module-aws}
+#### Terraform module {#terraform-module-aws}
 
-If you prefer to use Terraform instead of CloudFormation, use the following module:
+If you prefer to use Terraform instead of CloudFormation, use the [terraform-byoc-onboarding](https://github.com/ClickHouse/terraform-byoc-onboarding) module:
 
 ```hcl
 module "clickhouse_onboarding" {
-  source                     = "https://s3.us-east-2.amazonaws.com/clickhouse-public-resources.clickhouse.cloud/tf/byoc.tar.gz"
-  byoc_env                   = "production"
+  source                        = "github.com/ClickHouse/terraform-byoc-onboarding.git//modules/aws?ref=<version>"
+  external_id                   = "<external-id-provided-by-clickhouse>"
   include_vpc_write_permissions = false
 }
 ```
 
+Replace `<version>` with the latest tag from the module's [releases page](https://github.com/ClickHouse/terraform-byoc-onboarding/releases) — always use the latest release.
+
+:::note
+The module was previously distributed as a tarball at `https://s3.us-east-2.amazonaws.com/clickhouse-public-resources.clickhouse.cloud/tf/byoc.tar.gz`. That URL remains available but is deprecated — use the GitHub module above.
+:::
+
 ### Set up BYOC infrastructure {#set-up-byoc-infrastructure}
 
 In the ClickHouse Cloud console, configure the following when setting up new infrastructure:

docs/integrations/data-ingestion/clickpipes/aws-privatelink.md

@@ -10,6 +10,7 @@ integration:
    - category: 'clickpipes'
 ---
 
+import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge';
 import cp_service from '@site/static/images/integrations/data-ingestion/clickpipes/cp_service.png';
 import cp_step0 from '@site/static/images/integrations/data-ingestion/clickpipes/cp_step0.png';
 import cp_rpe_select from '@site/static/images/integrations/data-ingestion/clickpipes/cp_rpe_select.png';
@@ -189,6 +190,8 @@ It requires setting up a NLB (Network Load Balancer) in front of your data sourc
 and configuring the VPC endpoint service to use the NLB.
 
 VPC endpoint service can be [configured with a private DNS](https://docs.aws.amazon.com/vpc/latest/privatelink/manage-dns-names.html), that will be accessible in a ClickPipes VPC.
+If your service can't use PrivateLink managed private DNS (for example, the provider requires
+consumers to manage DNS resolution, as Confluent Cloud does), see [Custom private DNS](#custom-private-dns).
 
 It's a preferred choice for:
 
@@ -208,6 +211,52 @@ See AWS guide for [managing permissions](https://docs.aws.amazon.com/vpc/latest/
 can be configured for ClickPipes. Add [your ClickPipe region](#aws-privatelink-regions) to the allowed regions in your VPC endpoint service.
 :::
 
+## Custom private DNS {#custom-private-dns}
+
+<PrivatePreviewBadge/>
+
+:::info
+Custom private DNS is in **Private Preview**. Reach out to the [ClickHouse support team](https://clickhouse.com/support/program) to enable it for your service.
+:::
+
+AWS PrivateLink provides [managed private DNS](https://docs.aws.amazon.com/vpc/latest/privatelink/manage-dns-names.html)
+for VPC endpoint services with a verified domain. Some services require each consumer to manage DNS resolution in their own VPC, e.g., Confluent Cloud requires consumers to resolve broker hostnames to the PrivateLink endpoint.
+
+For these cases, ClickPipes supports attaching custom private DNS names to a reverse private
+endpoint. ClickPipes resolves these names to the endpoint's private addresses, so your data
+source can be reached by its own hostname over private connectivity.
+
+Custom private DNS complements managed private DNS — it doesn't replace it. If your
+PrivateLink service already provides private DNS names, you don't need custom names.
+
+Custom private DNS names are supported for the [VPC endpoint service](#vpc-endpoint-service)
+and [VPC resource](#vpc-resource) endpoint types. MSK multi-VPC provides managed private DNS
+out-of-the-box and doesn't support custom names.
+
+The following rules apply to custom private DNS names:
+
+- Exact names (`kafka.internal.example.com`) and wildcard names (`*.example.com`) are
+  supported. A wildcard matches a single DNS label — for example,
+  `*.abcde12345.us-east-1.aws.confluent.cloud` matches
+  `b0-lkc123.abcde12345.us-east-1.aws.confluent.cloud`.
+- Names must be unique across all reverse private endpoints of a ClickHouse service,
+  including overlaps between wildcard and exact names.
+- Names under reserved suffixes (e.g., `local`, `localhost`, `internal`, `corp`,
+  `private`) are rejected.
+
+To configure custom private DNS names:
+
+- In the ClickHouse Cloud console, fill in the `Custom private DNS name` field when
+  [creating a reverse private endpoint](#creating-clickpipe). The field is shown once the
+  feature is enabled for your service.
+- With [OpenAPI](https://clickhouse.com/docs/cloud/manage/api/swagger), set
+  `customPrivateDnsMappings` when creating a reverse private endpoint, or update an existing
+  endpoint with a `PATCH` request. Updates replace the full list of mappings; an empty list
+  removes all custom names.
+- With Terraform, use the
+  [`clickhouse_clickpipes_reverse_private_endpoint_custom_private_dns`](https://registry.terraform.io/providers/ClickHouse/clickhouse/latest/docs/resources/clickpipes_reverse_private_endpoint_custom_private_dns)
+  resource to manage mappings on an existing reverse private endpoint.
+
 ## Creating a ClickPipe with reverse private endpoint {#creating-clickpipe}
 
 <VerticalStepper headerLevel="list">
@@ -246,6 +295,7 @@ For same-region access, creating a VPC Resource is the recommended approach.
     - For VPC resource, provide the configuration share ARN and configuration ID.
     - For MSK multi-VPC, provide the cluster ARN and authentication method used with a created endpoint.
     - For VPC endpoint service, provide the service name.
+    - Optionally, provide a [custom private DNS](#custom-private-dns) name.
 
 7. Click on `Create` and wait for the reverse private endpoint to be ready.
 
@@ -258,7 +308,8 @@ For same-region access, creating a VPC Resource is the recommended approach.
 8. Once the endpoint is ready, you can use a DNS name to connect to the data source.
 
    On a list of endpoints, you can see the DNS name for the available endpoint.
-   It can be either an internally ClickPipes provisioned DNS name or a private DNS name supplied by a PrivateLink service.
+   It can be an internally ClickPipes provisioned DNS name, a private DNS name supplied by a PrivateLink service,
+   or a [custom private DNS name](#custom-private-dns).
    DNS name isn't a complete network address.
    Add the port according to the data source.
 

docs/integrations/data-ingestion/clickpipes/index.md

@@ -118,9 +118,10 @@ For other services, ClickPipes traffic will originate from a default region base
 |--------------------------------------------------|------------------------------------------------------------------------------------------------------------|
 | **asia-northeast1** - Tokyo (from 27 May 2026)   | `104.198.114.210`, `35.221.66.81`, `35.243.126.127`, `136.110.107.86`, `34.85.18.112`                     |
 | **asia-southeast1** - Singapore (from 27 May 2026) | `34.21.197.28`, `35.197.141.23`, `35.197.157.90`, `136.110.17.200`, `35.185.179.231`                    |
+| **australia-southeast1** - Sydney (from 7 Jul 2026) | `34.40.200.47`, `34.151.85.215`, `34.151.163.22`, `34.87.195.243`, `34.40.243.42`                      |
 | **europe-west2** - London (from 27 May 2026)     | `35.242.131.178`, `34.39.77.101`, `34.39.47.179`, `34.89.53.234`, `8.228.63.151`                         |
 | **europe-west4** - Netherlands (from 27 May 2026) | `34.34.86.3`, `34.6.175.56`, `34.178.6.187`, `34.91.204.220`, `34.12.85.206`                            |
-| **us-central1** - Iowa (from 27 May 2026)        | `34.28.24.54`, `34.42.56.195`, `34.63.141.9`, `35.238.146.37`, `34.10.251.49`                            |
+| **us-central1** - Iowa (from 7 Jul 2026)         | `34.28.24.54`, `34.42.56.195`, `34.63.141.9`, `35.238.146.37`, `34.10.251.49`                            |
 | **us-east1** - South Carolina (from 27 May 2026) | `34.24.134.232`, `34.24.214.165`, `34.24.20.1`, `35.243.193.248`, `34.23.98.76`                          |
 | **us-west1** - Oregon (from 1 Jul 2026)          | `136.118.254.175`, `8.229.115.2`, `34.83.0.219`, `35.247.34.28`, `35.199.165.121`                       |
 

docs/integrations/data-ingestion/clickpipes/kafka/02_schema-registries.md

@@ -36,7 +36,7 @@ To integrate with a schema registry during ClickPipes configuration, you must us
 
 ClickPipes connects to the schema registry over HTTPS at the URL you provide. The schema registry does not need to be publicly accessible.
 
-If your Kafka brokers are reached through a [reverse private endpoint](/integrations/clickpipes/aws-privatelink) (AWS PrivateLink or GCP Private Service Connect), the schema registry can use the same private connectivity. ClickPipes resolves the registry hostname through the reverse private endpoint's private DNS, so a registry hosted privately alongside your brokers is reachable as long as its hostname resolves to the reverse private endpoint's private IP addresses (via the endpoint's private DNS support or a custom private DNS mapping).
+If your Kafka brokers are reached through a [reverse private endpoint](/integrations/clickpipes/aws-privatelink) (AWS PrivateLink or GCP Private Service Connect), the schema registry can use the same private connectivity. ClickPipes resolves the registry hostname through the reverse private endpoint's private DNS, so a registry hosted privately alongside your brokers is reachable as long as its hostname resolves to the reverse private endpoint's private IP addresses (via the endpoint's private DNS support or a [custom private DNS mapping](/integrations/clickpipes/aws-privatelink#custom-private-dns)).
 
 Keep the following in mind:
 

sidebars.js

@@ -1671,6 +1671,7 @@ const sidebars = {
         'operations/settings/composable-protocols',
         'operations/settings/constraints-on-settings',
         'operations/settings/memory-overcommit',
+        'operations/settings/oom-canary',
         // "operations/settings/mysql-binlog-client",
         'operations/settings/permissions-for-queries',
         'operations/settings/query-complexity',