clickpipes: documentation for GCP-native private networking (PSC)#6409
clickpipes: documentation for GCP-native private networking (PSC)#6409morsapaes wants to merge 3 commits into
Conversation
|
@morsapaes is attempting to deploy a commit to the ClickHouse Team on Vercel. A member of the Team first needs to authorize it. |
| --allowed-psc-projects=clickpipes-production | ||
| ``` | ||
|
|
||
| The `--allowed-psc-projects` (`psc_config.allowed_consumer_projects` in the API and Terraform) is the auto-accept list. As long as the ClickPipes consumer project is on it, Cloud SQL accepts the endpoint automatically — no manual approval step. |
There was a problem hiding this comment.
We should mention here PSC service attachment cannot be reused across multiple ClickHouse Cloud services.
| custom_private_dns_mappings = [ | ||
| { private_dns_name = "<INSTANCE_UID>.<REGION>.sql.goog" } | ||
| ] |
There was a problem hiding this comment.
custom_private_dns_mappings is not a part of clickhouse_clickpipes_reverse_private_endpoint but has a separate TF resource
|
|
||
| > _TODO: paste the current region list before publishing._ | ||
|
|
||
| ## Limitations |
There was a problem hiding this comment.
Let's add a mention of the PSC service attachment claim by the ClickHouse Cloud service. It cannot be reused among multiple ClickHouse services.
Although this claim can be released on request.
- Add explicit {#id} anchors to all h2-h4 headings (custom-anchor-headings)
- Add the page to the Postgres CDC sidebar group (was a floating page)
- Lowercase 'reverse private endpoint' heading for sentence case
- Add 'Cloud SQL' to the Vale Headings exceptions (product name)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
@morsapaes - I pushed some mechanical fixes so the CI passes and we get a preview build. As part of this I also added it to the sidebars.js, and put it in the Let me know how else I can help! cc: @jkaflik |
3 participants
Summary
Work-in-progress for PSC integration documentation.
Note
Low Risk
Documentation-only addition with no application or infrastructure code changes.
Overview
Adds a new integration guide at
docs/integrations/data-ingestion/clickpipes/gcp-psc/postgres.md(slug/integrations/clickpipes/gcp-psc/postgres) documenting GCP Private Service Connect for Postgres CDC ClickPipes via Reverse Private Endpoints (GCP_PSC_SERVICE_ATTACHMENT).The page explains how RPEs consume customer-published PSC service attachments, why
custom_private_dns_mappingsis required on GCP, and prerequisites including allowing theclickpipes-productionconsumer project. Path A walks through Cloud SQL native PSC (gcloudenable/allow list, readingpscServiceAttachmentLinkanddnsName, RPE creation in UI/Terraform/API, then wiring acloudsqlpostgrespipe). Path B covers producer-owned PSC (private-network Cloud SQL or self-managed Postgres) with thegcp-cloud-sql-private-network-pscTerraform module and optional manualPendingAcceptanceapproval.It also documents RPE lifecycle in the UI, same-region constraints, and limitations (no VPC peering). Supported GCP regions still has a publish-blocking TODO to paste the region list; the UI steps repeat numbering at item 2.
Reviewed by Cursor Bugbot for commit f9f11ed. Bugbot is set up for automated code reviews on this repo. Configure here.