Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,11 @@ CLICKHOUSE_PASSWORD=secret clickhousectl cloud service client --name my-service
# Use a local client version instead of auto-downloading the matching one
clickhousectl cloud service client --name my-service --allow-mismatched-client-version

# Run SQL over HTTP via the Query API (no local clickhouse binary needed)
clickhousectl cloud service query --name my-service --query "SELECT 1"
clickhousectl cloud service query --id <service-id> --query "SELECT count() FROM system.tables" --format JSONEachRow
echo "SELECT 1+1" | clickhousectl cloud service query --name my-service

# Update service metadata and patches
clickhousectl cloud service update <service-id> \
--name my-renamed-service \
Expand Down Expand Up @@ -351,7 +356,7 @@ clickhousectl cloud service reset-password <service-id> \
--new-password-hash <base64-sha256-hash> \
--new-double-sha1-hash <mysql-double-sha1-hash>

# Query endpoint management
# Query endpoint management (manual — for custom roles or sharing keys with other tools)
clickhousectl cloud service query-endpoint get <service-id>
clickhousectl cloud service query-endpoint create <service-id> \
--role admin \
Expand Down Expand Up @@ -405,6 +410,19 @@ clickhousectl cloud service delete <service-id> --force
| `--enable-endpoint` / `--disable-endpoint` | Toggle GA service endpoints (currently `mysql`) |
| `--private-preview-terms-checked` | Accept private preview terms when required |
| `--enable-core-dumps` | Enable or disable service core dump collection |
| `--no-enable-query` | Skip auto-provisioning of the Query API endpoint + per-service key |

#### Query API auto-provisioning

By default, `cloud service create` provisions a Query API endpoint for the new service and creates a dedicated API key bound to it. The key (`keyId`, `keySecret`, and `endpointId`) is stored in `.clickhouse/credentials.json` under `service_query_keys.<service-id>`, alongside any user-level API key. `cloud service query` then runs SQL over HTTP using that key — no `clickhouse` binary and no service password required. The key is scoped to a single service, so it can read and write (SELECT, INSERT, DDL) against that service but cannot reach any other service in the org.

For existing services without a stored key, `cloud service query` provisions one lazily on first use. Pass `--no-auto-enable` to fail instead, or `--no-enable-query` on `service create` to skip the create-time hook.

Per-service scoping is enforced at the query endpoint binding, which is created with role `sql_console_admin` (read + write inside the bound service only). The API key itself has no org-level roles, so the binding is the only thing that grants it any access. `cloud service delete` removes the stored key from `credentials.json`.

`cloud service query` is the canonical way to run SQL against a cloud service; `cloud service client` (which downloads a matching `clickhouse` binary and connects via the native protocol) is on a deprecation path.

Set `CLICKHOUSE_CLOUD_QUERY_HOST` to override the Query API host (defaults to `https://queries.clickhouse.cloud`).

### Postgres (beta)

Expand Down
2 changes: 1 addition & 1 deletion crates/clickhouse-cloud-api/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ serde = { version = "1", features = ["derive"] }
serde_json = "1"
thiserror = "2"
url = "2"
uuid = { version = "1", features = ["serde"] }
uuid = { version = "1", features = ["serde", "v4"] }
chrono = { version = "0.4", features = ["serde"] }

[dev-dependencies]
Expand Down
71 changes: 71 additions & 0 deletions crates/clickhouse-cloud-api/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,77 @@ impl Client {
}
}

/// Run a SQL statement against a service's Query API endpoint.
///
/// Hits `queries.clickhouse.cloud` (override via the
/// `CLICKHOUSE_CLOUD_QUERY_HOST` env var) using Basic auth with the
/// provided `key_id`/`key_secret` — a per-service key bound to a
/// query endpoint with role `sql_console_read_only` (or
/// `sql_console_admin`). This bypasses the client's primary auth
/// because Query API keys are scoped to a single service.
///
/// Returns the streaming response so the caller can forward it to
/// stdout or buffer it into memory.
pub async fn run_query(
&self,
service_id: &str,
key_id: &str,
key_secret: &str,
sql: &str,
database: Option<&str>,
format: &str,
) -> Result<reqwest::Response, Error> {
#[derive(serde::Serialize)]
#[serde(rename_all = "camelCase")]
struct RunQueryBody<'a> {
run_id: String,
sql: &'a str,
#[serde(skip_serializing_if = "Option::is_none")]
database: Option<&'a str>,
}

let host = std::env::var("CLICKHOUSE_CLOUD_QUERY_HOST")
.unwrap_or_else(|_| "https://queries.clickhouse.cloud".to_string());
let url = format!(
"{}/service/{}/run",
host.trim_end_matches('/'),
service_id,
);

let body = RunQueryBody {
run_id: uuid::Uuid::new_v4().to_string(),
sql,
database,
};

let response = self
.http
.post(url)
.query(&[("format", format)])
.basic_auth(key_id, Some(key_secret))
.header("content-type", "text/plain;charset=UTF-8")
.header("x-service-type", "clickhouse")
.header("auth-provider", "custom")
.json(&body)
.send()
.await?;

let status = response.status();
if !status.is_success() {
let body_text = response.text().await.unwrap_or_default();
return Err(Error::Api {
status: status.as_u16(),
message: if body_text.is_empty() {
format!("Query API returned {status}")
} else {
body_text
},
});
}

Ok(response)
}

/// Get list of available organizations
pub async fn organization_get_list(
&self,
Expand Down
8 changes: 4 additions & 4 deletions crates/clickhouse-cloud-api/src/models.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6938,14 +6938,14 @@ pub struct ApiKeyPostRequest {
pub assigned_role_ids: Vec<uuid::Uuid>,
#[serde(rename = "expireAt", skip_serializing_if = "Option::is_none", default)]
pub expire_at: Option<chrono::DateTime<chrono::Utc>>,
#[serde(rename = "hashData", default)]
pub hash_data: ApiKeyHashData,
#[serde(rename = "hashData", skip_serializing_if = "Option::is_none", default)]
pub hash_data: Option<ApiKeyHashData>,
#[serde(rename = "ipAccessList", default)]
pub ip_access_list: Vec<IpAccessListEntry>,
#[serde(default)]
pub name: String,
#[serde(default)]
pub roles: Vec<String>,
#[serde(skip_serializing_if = "Option::is_none", default)]
pub roles: Option<Vec<String>>,
#[serde(default)]
pub state: ApiKeyPostRequestState,
}
Expand Down
20 changes: 20 additions & 0 deletions crates/clickhouse-cloud-api/tests/integration/support.rs
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,7 @@ impl FailureRecorder {
pub struct CleanupRegistry {
service_ids: Vec<String>,
postgres_ids: Vec<String>,
api_key_ids: Vec<String>,
}

impl CleanupRegistry {
Expand All @@ -280,9 +281,28 @@ impl CleanupRegistry {
.retain(|registered| registered != postgres_id);
}

pub fn register_api_key(&mut self, key_id: impl Into<String>) {
self.api_key_ids.push(key_id.into());
}

pub fn unregister_api_key(&mut self, key_id: &str) {
self.api_key_ids
.retain(|registered| registered != key_id);
}

pub async fn cleanup(&mut self, client: &Client, org_id: &str, delete_timeout: Duration, poll_interval: Duration) -> Result<(), String> {
let mut failures = Vec::new();

// API keys are cleaned up first; they belong to the org, not a
// specific service, so they outlive service deletion if leaked.
while let Some(key_id) = self.api_key_ids.pop() {
match client.openapi_key_delete(org_id, &key_id).await {
Ok(_) => {}
Err(clickhouse_cloud_api::Error::Api { status: 404, .. }) => {}
Err(e) => failures.push(format!("api key {key_id}: {e}")),
}
}

while let Some(service_id) = self.service_ids.pop() {
if let Err(error) = ensure_service_gone(client, org_id, &service_id, delete_timeout, poll_interval).await {
failures.push(format!("{service_id}: {error}"));
Expand Down
Loading
Loading