Merge pull request #2734 from madeline-underwood/clickhouse2

Clickhouse2_reviewed

Author: jasonrandrews

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/_index.md

@@ -1,9 +1,5 @@
 ---
-title: Build a Real-Time Analytics Pipeline with ClickHouse on Google Cloud Axion (Arm-based C4A VMs)
-
-draft: true
-cascade:
-    draft: true
+title: Build a real-time analytics pipeline with ClickHouse on Google Cloud Axion
 
 minutes_to_complete: 50
 

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/benchmarking.md

@@ -1,15 +1,13 @@
 ---
-title: ClickHouse Benchmarking on Google Axion (Arm)
+title: Benchmark ClickHouse on Google Axion processors
 weight: 9
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
+## ClickHouse benchmarking on Axion processors
 
-
-## ClickHouse Benchmarking on Axion Processors
-This phase benchmarks **query latency on ClickHouse running on Google Axion (Arm64)**.  
-The goal is to measure **repeatable query latency** with a focus on **p95 latency**, using data ingested via the real-time Dataflow pipeline.
+In this section you'll benchmark query latency on ClickHouse running on Google Axion (Arm64). The goal is to measure repeatable query latency with a focus on p95 latency, using data ingested with the real-time Dataflow pipeline.
 
 ## Prepare ClickHouse for Accurate Latency Measurement
 
@@ -23,13 +21,16 @@ SET use_query_cache = 0;
 ```
 This ensures every query is executed fully and not served from cache.
 
-### Validate Dataset Size
-Ensures enough data is present to produce meaningful latency results.
+### Validate dataset size
+
+Ensure enough data is present to produce meaningful latency results by checking the current row count.
 
 ```console
 SELECT count(*) FROM realtime.logs;
 ```
-You should see an output similar to:
+
+The output is similar to:
+
 ```output
    ┌─count()─┐
 1. │ 5000013 │ -- 5.00 million
@@ -48,20 +49,22 @@ SELECT
 FROM numbers(1000000);
 ```
 
-You should see an output similar to:
+The output is similar to:
+
 ```output
 Query id: 8fcbefab-fa40-4124-8f23-516fca2b8fdd
 Ok.
 1000000 rows in set. Elapsed: 0.058 sec. Processed 1.00 million rows, 8.00 MB (17.15 million rows/s., 137.20 MB/s.)
 Peak memory usage: 106.54 MiB.
 ```
 
-### Define Benchmark Queries
+### Define benchmark queries
+
 These queries represent common real-time analytics patterns:
 
-- **Filtered count** – service-level analytics
-- **Time-windowed count** – recent activity
-- **Aggregation by service** – grouped analytics
+- Filtered count: service-level analytics
+- Time-windowed count: recent activity
+- Aggregation by service: grouped analytics
 
 Each query scans and processes millions of rows to stress the execution engine.
 
@@ -73,7 +76,7 @@ FROM realtime.logs
 WHERE service = 'service-5';
 ```
 
-You should see an output similar to:
+The output is similar to:
 ```output
 Query id: cfbab386-7168-42ce-a752-2d5146f68b48
 
@@ -92,7 +95,8 @@ FROM realtime.logs
 WHERE event_time >= now() - INTERVAL 10 MINUTE;
 ```
 
-You should see an output similar to:
+The output is similar to:
+
 ```output
 Query id: 7654746b-3068-4663-a5c6-6944d9c2d2b9
    ┌─count()─┐
@@ -112,7 +116,8 @@ GROUP BY service
 ORDER BY total DESC;
 ```
 
-You should see an output similar to:
+The output is similar to:
+
 ```output
 Query id: c48c0d30-0ef6-4fb9-bbb9-815a509a5f91
 
@@ -144,8 +149,9 @@ Query id: c48c0d30-0ef6-4fb9-bbb9-815a509a5f91
 Peak memory usage: 7.18 MiB.
 ```
 
-### Run Repeatable Latency Measurements
-To calculate reliable latency metrics, the same query is executed multiple times(10) using `clickhouse-client --time`.
+### Run repeatable latency measurements
+
+To calculate reliable latency metrics, execute the same query multiple times (10 iterations) using `clickhouse-client --time`.
 
 ```sql
 clickhouse-client --time --query "
@@ -228,16 +234,18 @@ sort -n latency-results.txt
 ```
 
 **Calculate p95 latency (manual):**
+
 The p95 latency represents the value under which 95% of query executions complete.
 
 **Formula:**
 
-```pqsql
+```
 p95 index = ceil(0.95 × N)
 ```
 
 For 10 samples:
-```cpp
+
+```
 ceil(0.95 × 10) = ceil(9.5) = 10
 ```
 
@@ -249,12 +257,15 @@ The 10th value in the sorted list is your p95 latency.
 p95 latency = 0.011 seconds ≈ 11 ms
 ```
 
-The ClickHouse query was executed 10 times on a GCP Axion (Arm) VM. Observed p95 query latency was ~11 ms, demonstrating consistently low-latency analytical performance on Arm-based infrastructure.
+After executing the ClickHouse query 10 times on the GCP Axion (Arm) VM, the observed p95 query latency was ~11 ms, demonstrating consistently low-latency analytical performance on Arm-based infrastructure.
 
-### Benchmark summary
-Results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SUSE):
+## What you've accomplished
 
-- ClickHouse on **Google Axion (Arm64)** delivered consistently low query latency, even while scanning ~6 million rows per query.
-- Across **10 repeat executions, the p95 latency was ~11 ms**, indicating stable and predictable performance.
+You've successfully completed a comprehensive ClickHouse benchmarking exercise on Google Axion (Arm64) processors. Key results from the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM running SUSE:
+
+- ClickHouse on Google Axion (Arm64) delivered consistently low query latency, even while scanning ~6 million rows per query.
+- Across 10 repeat executions, the p95 latency was ~11 ms, indicating stable and predictable performance.
 - Disabling the query cache ensured true execution latency, not cache-assisted results.
-- Analytical queries sustained **500M+ rows/sec throughput** with minimal memory usage.
+- Analytical queries sustained 500M+ rows/sec throughput with minimal memory usage.
+
+Throughout this Learning Path, you provisioned an Arm-based VM on Google Cloud, deployed ClickHouse, configured a real-time streaming pipeline with Pub/Sub and Dataflow, and validated end-to-end analytical performance. You can now deploy, optimize, and benchmark ClickHouse workloads on Google Cloud Arm infrastructure.

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/dataflow-streaming-etl.md

@@ -1,16 +1,16 @@
 ---
-title: Dataflow Streaming ETL to ClickHouse
+title: Build a Dataflow streaming ETL pipeline to ClickHouse
 weight: 8
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
 
-## Dataflow Streaming ETL (Pub/Sub → ClickHouse)
-This section implements a real-time streaming ETL pipeline that ingests events from Pub/Sub, processes them using Dataflow (Apache Beam), and writes them into ClickHouse running on a GCP Axion (Arm64) VM.
+## Dataflow streaming ETL (Pub/Sub → ClickHouse)
 
-## Pipeline Overview
-Flow
+In this section you'll implement a real-time streaming ETL pipeline that ingests events from Pub/Sub, processes them using Dataflow (Apache Beam), and writes them into ClickHouse running on a GCP Axion (Arm64) VM.
+
+## Pipeline overview
 
 ```bash
 Pub/Sub → Dataflow (Apache Beam) → ClickHouse (Axion VM)
@@ -23,7 +23,8 @@ Pub/Sub → Dataflow (Apache Beam) → ClickHouse (Axion VM)
 - ClickHouse: real-time analytical storage on Arm64
 
 ### Install Python 3.11 on the Axion VM
-Install Python 3.11 and the required system packages
+
+Install Python 3.11 and the required system packages:
 
 ```console
 sudo zypper refresh
@@ -37,15 +38,17 @@ python3.11 --version
 pip3.11 --version
 ```
 
-### Create a Python Virtual Environment (Recommended)
+### Create a Python virtual environment (recommended)
+
 Using a virtual environment avoids dependency conflicts with the system Python.
 
 ```console
 python3.11 -m venv beam-venv
 source beam-venv/bin/activate
 ```
 
-### Install Apache Beam with GCP Support
+### Install Apache Beam with GCP support
+
 Install Apache Beam and the required dependencies for Dataflow:
 
 ```console
@@ -68,7 +71,7 @@ Connect to ClickHouse on the Axion VM:
 clickhouse client
 ```
 
-**Creates the target database and table for streaming inserts:**
+Create the target database and table for streaming inserts:
 
 ```sql
 CREATE DATABASE IF NOT EXISTS realtime;
@@ -104,7 +107,8 @@ Query id: aa25de9d-c07f-4538-803f-5473744631bc
 exit;
 ```
 
-### Validate Pub/Sub (Before Dataflow)
+### Validate Pub/Sub before Dataflow
+
 Before running Dataflow, confirm that messages can be published and pulled.
 
 **Publish a test message:**
@@ -134,11 +138,9 @@ Successful output confirms:
 - Subscription is readable
 - IAM is functioning correctly
 
-### Create Dataflow Streaming ETL Script
-Create the Dataflow pipeline file:
+### Create Dataflow streaming ETL script
 
-Purpose
-Defines a streaming Beam pipeline that:
+Create the Dataflow pipeline file that defines a streaming Beam pipeline to:
 
 - Reads JSON events from Pub/Sub
 - Parses messages
@@ -208,8 +210,9 @@ gcloud pubsub subscriptions list
 hostname -I
 ```
 
-### Run the Dataflow Streaming Job
-Launches the pipeline on managed Dataflow workers.
+### Run the Dataflow streaming job
+
+Launch the pipeline on managed Dataflow workers:
 
 ```console
 python3.11 dataflow_etl.py \
@@ -233,8 +236,9 @@ Autoscaling is enabled for Dataflow Streaming Engine. Workers will scale between
 - Streaming mode is active
 - Workers scale automatically
 
-### End-to-End Validation
-Publish live streaming data.
+### End-to-end validation
+
+Publish live streaming data:
 
 ```console
 gcloud pubsub topics publish logs-topic \
@@ -276,4 +280,8 @@ This confirms:
 - ClickHouse ingests data on Axion (Arm64) via HTTP
 - The end-to-end real-time pipeline is operational
 
-This pipeline serves as the foundation for ClickHouse latency benchmarking and real-time analytics on Google Axion.
+## What you've accomplished and what's next
+
+You've successfully built a complete streaming ETL pipeline connecting Google Cloud Pub/Sub to ClickHouse running on an Arm-based Axion VM. The pipeline processes real-time events through Dataflow and stores them in ClickHouse for analytics.
+
+This pipeline serves as the foundation for ClickHouse latency benchmarking and real-time analytics on Google Axion, which you'll perform in the next section.

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/gcp_firewall_setup.md

@@ -20,19 +20,19 @@ To expose the TCP port 8123, create a firewall rule.
 
 Navigate to the [Google Cloud Console](https://console.cloud.google.com/), go to **VPC Network > Firewall**, and select **Create firewall rule**.
 
-![Create a firewall rule alt-text#center](images/firewall-rule1.png "Create a firewall rule")
+![Google Cloud Console VPC Network Firewall page with Create Firewall Rule button highlighted alt-txt#center](images/firewall-rule1.png "Create a firewall rule")
 
 Set the **Name** of the new rule to "allow-tcp-8123". Select your network that you intend to bind to your VM (default is "autoscaling-net", but your organization might have others).
 
 
-![Create a firewall rule alt-text#center](images/network-rule2.png "Creating the TCP/8123 firewall rule")
+![Firewall rule configuration screen showing name field and network selection dropdown alt-txt#center](images/network-rule2.png "Creating the TCP/8123 firewall rule")
 
 Next, Set **Direction of traffic** to "Ingress". Set **Allow on match** to "Allow" and **Targets** to "Specified target tags". Set **Source IPv4 ranges** to "0.0.0.0/0".
 
-![Create a firewall rule alt-text#center](images/network-rule3.png "Creating the TCP/8123 firewall rule")
+![Firewall rule settings displaying Ingress direction, Allow action, and source IP range configuration alt-txt#center](images/network-rule3.png "Creating the TCP/8123 firewall rule")
 
 Finally, select **Specified protocols and ports** under the **Protocols and ports** section. Select the **TCP** checkbox, enter "8123" in the **Ports** text field, and select **Create**.
 
-![Specifying the TCP port to expose alt-text#center](images/network-port.png "Specifying the TCP port to expose")
+![Protocols and ports section with TCP checkbox selected and port 8123 entered in text field alt-txt#center](images/network-port.png "Specifying the TCP port to expose")
 
 The network firewall rule is now created, and you can continue with the VM creation.

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/installation.md

@@ -7,10 +7,11 @@ layout: learningpathall
 ---
 
 ## Install ClickHouse and gcloud CLI on GCP VM
-This guide covers the installation of **Google Cloud CLI (gcloud)** and **ClickHouse** on a GCP SUSE Linux Arm64 (Axion C4A) VM.
+
+In this section you'll install Google Cloud CLI (gcloud) and ClickHouse on your GCP SUSE Linux Arm64 (Axion C4A) VM.
 These tools are required to:
-- Interact with GCP services such as **Pub/Sub** and **Dataflow**
-- Store and query real-time analytics data efficiently using **ClickHouse on Arm64**
+- Interact with GCP services such as Pub/Sub and Dataflow
+- Store and query real-time analytics data efficiently using ClickHouse on Arm64
 
 ### Install Google Cloud CLI (gcloud)
 The Google Cloud CLI is required to authenticate with GCP, publish Pub/Sub messages, and submit Dataflow jobs from the VM.

content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/instance.md

@@ -24,20 +24,20 @@ To create a virtual machine based on the C4A instance type:
    - Set **Series** to `C4A`.
    - Select `c4a-standard-4` for machine type.
 
-   ![Create a Google Axion C4A Arm virtual machine in the Google Cloud Console with c4a-standard-4 selected alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console")
+   ![Google Cloud Console VM creation screen showing machine configuration with C4A series and c4a-standard-4 selected alt-txt#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console")
 
 
 - Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. 
-- If using use **SUSE Linux Enterprise Server**. Select "Pay As You Go" for the license type.
-- Once appropriately selected and configured, please Click **Select**.
+- If using **SUSE Linux Enterprise Server**, select "Pay As You Go" for the license type.
+- Once appropriately selected and configured, click **Select**.
 - Under **Networking**, enable **Allow HTTP traffic**.
 - Click **Create** to launch the instance.
 - Once created, you should see a "SSH" option to the right in your list of VM instances.  Click on this to launch a SSH shell into your VM instance:
 
-![Invoke a SSH session via your browser alt-text#center](images/gcp-ssh.png "Invoke a SSH session into your running VM instance")
+![Google Cloud Console VM instances list with SSH button highlighted for connecting to running instance alt-txt#center](images/gcp-ssh.png "Invoke a SSH session into your running VM instance")
 
 - A window from your browser should come up and you should now see a shell into your VM instance:
 
-![Terminal Shell in your VM instance alt-text#center](images/gcp-shell.png "Terminal shell in your VM instance")
+![Browser-based terminal window showing command prompt connected to VM instance alt-txt#center](images/gcp-shell.png "Terminal shell in your VM instance")
 
 Next, let's install ClickHouse!