Skip to content

Commit 5a6dc6c

Browse files
Update ClickHouse GCP learning path content
1 parent 4adb68f commit 5a6dc6c

4 files changed

Lines changed: 51 additions & 51 deletions

File tree

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/benchmarking.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,13 @@
11
---
2-
title: ClickHouse Benchmarking on Google Axion (Arm)
2+
title: Benchmark ClickHouse on Google Axion processors
33
weight: 9
44

55
### FIXED, DO NOT MODIFY
66
layout: learningpathall
77
---
88
## ClickHouse benchmarking on Axion processors
99

10-
11-
## ClickHouse Benchmarking on Axion Processors
12-
This phase benchmarks **query latency on ClickHouse running on Google Axion (Arm64)**.
13-
The goal is to measure **repeatable query latency** with a focus on **p95 latency**, using data ingested via the real-time Dataflow pipeline.
10+
In this section you'll benchmark query latency on ClickHouse running on Google Axion (Arm64). The goal is to measure repeatable query latency with a focus on p95 latency, using data ingested with the real-time Dataflow pipeline.
1411

1512
## Prepare ClickHouse for Accurate Latency Measurement
1613

@@ -65,9 +62,9 @@ Peak memory usage: 106.54 MiB.
6562

6663
These queries represent common real-time analytics patterns:
6764

68-
- **Filtered count**: service-level analytics
69-
- **Time-windowed count**: recent activity
70-
- **Aggregation by service**: grouped analytics
65+
- Filtered count: service-level analytics
66+
- Time-windowed count: recent activity
67+
- Aggregation by service: grouped analytics
7168

7269
Each query scans and processes millions of rows to stress the execution engine.
7370

@@ -119,7 +116,8 @@ GROUP BY service
119116
ORDER BY total DESC;
120117
```
121118

122-
You should see an output similar to:
119+
The output is similar to:
120+
123121
```output
124122
Query id: c48c0d30-0ef6-4fb9-bbb9-815a509a5f91
125123
@@ -236,16 +234,18 @@ sort -n latency-results.txt
236234
```
237235
238236
**Calculate p95 latency (manual):**
237+
239238
The p95 latency represents the value under which 95% of query executions complete.
240239
241240
**Formula:**
242241
243-
```pqsql
242+
```
244243
p95 index = ceil(0.95 × N)
245244
```
246245
247246
For 10 samples:
248-
```cpp
247+
248+
```
249249
ceil(0.95 × 10) = ceil(9.5) = 10
250250
```
251251
@@ -263,9 +263,9 @@ After executing the ClickHouse query 10 times on the GCP Axion (Arm) VM, the obs
263263
264264
You've successfully completed a comprehensive ClickHouse benchmarking exercise on Google Axion (Arm64) processors. Key results from the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM running SUSE:
265265
266-
- ClickHouse on **Google Axion (Arm64)** delivered consistently low query latency, even while scanning ~6 million rows per query.
267-
- Across **10 repeat executions, the p95 latency was ~11 ms**, indicating stable and predictable performance.
266+
- ClickHouse on Google Axion (Arm64) delivered consistently low query latency, even while scanning ~6 million rows per query.
267+
- Across 10 repeat executions, the p95 latency was ~11 ms, indicating stable and predictable performance.
268268
- Disabling the query cache ensured true execution latency, not cache-assisted results.
269-
- Analytical queries sustained **500M+ rows/sec throughput** with minimal memory usage.
269+
- Analytical queries sustained 500M+ rows/sec throughput with minimal memory usage.
270270
271271
Throughout this Learning Path, you provisioned an Arm-based VM on Google Cloud, deployed ClickHouse, configured a real-time streaming pipeline with Pub/Sub and Dataflow, and validated end-to-end analytical performance. You can now deploy, optimize, and benchmark ClickHouse workloads on Google Cloud Arm infrastructure.

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/dataflow-streaming-etl.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Dataflow Streaming ETL to ClickHouse
2+
title: Build a Dataflow streaming ETL pipeline to ClickHouse
33
weight: 8
44

55
### FIXED, DO NOT MODIFY
@@ -138,11 +138,9 @@ Successful output confirms:
138138
- Subscription is readable
139139
- IAM is functioning correctly
140140

141-
### Create Dataflow Streaming ETL Script
142-
Create the Dataflow pipeline file:
141+
### Create Dataflow streaming ETL script
143142

144-
Purpose
145-
Defines a streaming Beam pipeline that:
143+
Create the Dataflow pipeline file that defines a streaming Beam pipeline to:
146144

147145
- Reads JSON events from Pub/Sub
148146
- Parses messages
@@ -212,8 +210,9 @@ gcloud pubsub subscriptions list
212210
hostname -I
213211
```
214212

215-
### Run the Dataflow Streaming Job
216-
Launches the pipeline on managed Dataflow workers.
213+
### Run the Dataflow streaming job
214+
215+
Launch the pipeline on managed Dataflow workers:
217216

218217
```console
219218
python3.11 dataflow_etl.py \
@@ -237,8 +236,9 @@ Autoscaling is enabled for Dataflow Streaming Engine. Workers will scale between
237236
- Streaming mode is active
238237
- Workers scale automatically
239238

240-
### End-to-End Validation
241-
Publish live streaming data.
239+
### End-to-end validation
240+
241+
Publish live streaming data:
242242

243243
```console
244244
gcloud pubsub topics publish logs-topic \

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/instance.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,8 @@ To create a virtual machine based on the C4A instance type:
2828

2929

3030
- Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**.
31-
- If using use **SUSE Linux Enterprise Server**. Select "Pay As You Go" for the license type.
32-
- Once appropriately selected and configured, please Click **Select**.
31+
- If using **SUSE Linux Enterprise Server**, select "Pay As You Go" for the license type.
32+
- Once appropriately selected and configured, click **Select**.
3333
- Under **Networking**, enable **Allow HTTP traffic**.
3434
- Click **Create** to launch the instance.
3535
- Once created, you should see a "SSH" option to the right in your list of VM instances. Click on this to launch a SSH shell into your VM instance:

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/pub_sub_creation.md

Lines changed: 26 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: GCP Pub/Sub and IAM Setup for ClickHouse Real-Time Analytics on Axion
2+
title: Set up GCP Pub/Sub and IAM for ClickHouse real-time analytics on Axion
33
weight: 5
44

55
### FIXED, DO NOT MODIFY
@@ -13,31 +13,31 @@ It focuses on **Pub/Sub resource creation and IAM roles**, ensuring Dataflow and
1313
### Create Pub/Sub Topic
1414
The Pub/Sub topic acts as the **ingestion entry point** for streaming log events.
1515

16-
1. Open **Google Cloud Console**
17-
2. Navigate to **Pub/Sub → Topics**
18-
3. Click **Create Topic**
19-
4. Enter:
16+
- Open **Google Cloud Console**
17+
- Navigate to **Pub/Sub → Topics**
18+
- Click **Create Topic**
19+
- Enter:
2020
- **Topic ID:** `logs-topic`
21-
5. Leave encryption and retention as the default
22-
6. Click **Create**
21+
- Leave encryption and retention as the default
22+
- Click **Create**
2323

2424
This topic will receive streaming log messages from producers.
2525

26-
![ GCP onsole alt-text#center](images/pub_sub1.png "Figure 1: Pub/Sub Topic")
26+
![Google Cloud Console showing Pub/Sub topic creation screen with logs-topic configured alt-txt#center](images/pub_sub1.png "Pub/Sub Topic")
2727

2828
### Create Pub/Sub Subscription
2929

3030
The subscription allows **Dataflow to pull messages** from the topic.
3131

32-
1. Open the newly created `logs-topic`
33-
2. Click **Create Subscription**
34-
3. Configure:
32+
- Open the newly created `logs-topic`
33+
- Click **Create Subscription**
34+
- Configure:
3535
- **Subscription ID:** `logs-sub`
3636
- **Delivery type:** Pull
3737
- **Ack deadline:** Default (10 seconds)
38-
4. Click **Create**
38+
- Click **Create**
3939

40-
![ GCP onsole alt-text#center](images/pub_sub2.png "Figure 2: Pub/Sub Subscription")
40+
![Google Cloud Console displaying Pub/Sub subscription configuration screen for logs-sub with Pull delivery alt-txt#center](images/pub_sub2.png "Pub/Sub Subscription")
4141

4242
This subscription will later be referenced by the Dataflow pipeline.
4343

@@ -50,7 +50,7 @@ Navigate to **Pub/Sub → Topics** and confirm:
5050

5151
This confirms the messaging layer is ready.
5252

53-
![ GCP onsole alt-text#center](images/verify_pub_sub.png "Figure 3: Pub/Sub Resources")
53+
![Google Cloud Console Pub/Sub topics list showing logs-topic and logs-sub subscription verified alt-txt#center](images/verify_pub_sub.png "Pub/Sub Resources")
5454

5555
### Identify Compute Engine Service Account
5656

@@ -82,30 +82,30 @@ Grant the following roles to the **Compute Engine default service account**:
8282
| Service Account User | Allow service account usage |
8383

8484
**Steps (UI):**
85-
1. Go to **IAM & Admin → IAM**
86-
2. Click **Grant Access**
87-
3. Add the service account
88-
4. Assign the roles listed above
89-
5. Save
85+
- Go to **IAM & Admin → IAM**
86+
- Click **Grant Access**
87+
- Add the service account
88+
- Assign the roles listed above
89+
- Save
9090

91-
![Google Cloud Console IAM page displaying assigned roles for Compute Engine service account including Dataflow and Pub/Sub permissions alt-txt#center](images/roles.png "Figure 4: Required IAM Roles")
91+
![Google Cloud Console IAM page displaying assigned roles for Compute Engine service account including Dataflow and Pub/Sub permissions alt-txt#center](images/roles.png "Required IAM Roles")
9292

9393
VM OAuth scopes are limited by default. IAM roles are authoritative.
9494

9595
### Create GCS Bucket for Dataflow (UI)
9696

9797
Dataflow requires a Cloud Storage bucket for staging and temp files.
9898

99-
1. Go to **Cloud Storage → Buckets**
100-
2. Click **Create**
101-
3. Configure:
99+
- Go to **Cloud Storage → Buckets**
100+
- Click **Create**
101+
- Configure:
102102
- **Bucket name:** `imperial-time-463411-q5-dataflow-temp`
103103
- **Location type:** Region
104104
- **Region:** `us-central1`
105-
4. Leave defaults for storage class and access control
106-
5. Click **Create**
105+
- Leave defaults for storage class and access control
106+
- Click **Create**
107107

108-
![ GCP onsole alt-text#center](images/bucket.png "Figure 5: GCS Bucket")
108+
![Google Cloud Console showing Cloud Storage bucket creation screen with dataflow-temp bucket configured alt-txt#center](images/bucket.png "GCS Bucket")
109109

110110
### Grant Bucket Access
111111

0 commit comments

Comments
 (0)