Editorial

Author: madeline-underwood

content/learning-paths/servers-and-cloud-computing/distributed-inference-with-llama-cpp/_index.md

@@ -1,9 +1,9 @@
 ---
-title: Distributed inference using llama.cpp
+title: Run distributed inference with llama.cpp on Arm-based AWS instances
 
 minutes_to_complete: 30
 
-who_is_this_for: This introductory topic is for developers with some experience using llama.cpp who want to learn distributed inference.
+who_is_this_for: This introductory topic is for developers with some experience using llama.cpp who want to learn how to run distributed inference on Arm-based servers.
 
 learning_objectives: 
     - Set up a main host and worker nodes with llama.cpp

content/learning-paths/servers-and-cloud-computing/distributed-inference-with-llama-cpp/how-to-1.md

@@ -13,16 +13,16 @@ This example runs on three AWS Graviton4 `c8g.4xlarge` instances. Each instance
 In this Learning Path, you will:
 
 - Download Meta's [Llama 3.1 70B parameter model](https://huggingface.co/meta-llama/Llama-3.1-70B).
-- Download and build `llama.cpp`, a C++ library for efficient CPU inference of LLaMA and similar large language models on CPUs, optimized for local and embedded environments.
+- Download and build `llama.cpp`, a C++ library for efficient CPU inference of Llama and similar large language models on CPUs, optimized for local and embedded environments.
 - Convert Meta's `safetensors` files to a single GGUF file.
 - Quantize the 16-bit GGUF weights file to 4-bit weights.
 - Load and run the model.
 
 {{% notice Note %}}
-The **Reading time** shown on the **Introduction** page does not include downloading, converting, and quantizing the model. These steps can take 1-2 hours. If you already have a quantized GGUF file, you can skip the download and quantization.
+The **Reading time** shown on the **Introduction** page does not include downloading, converting, and quantizing the model. These steps can take several hours depending on bandwidth and system resources. If you already have a quantized GGUF file, you can skip the download and quantization.
 {{% /notice %}}
 
-## Set up dependencies
+## Install dependencies
 
 Before you start, make sure you have permission to access Meta's [Llama 3.1 70B parameter model](https://huggingface.co/meta-llama/Llama-3.1-70B).
 
@@ -35,7 +35,7 @@ You must repeat the install steps on each device. However, only run the download
 ```bash
 apt update
 apt install -y python3.12-venv
-python3 -m venv myenv
+python3.12 -m venv myenv
 source myenv/bin/activate
 ```
 
@@ -81,7 +81,7 @@ Add the following code:
 ```python
 import os
 from huggingface_hub import snapshot_download
-model_id = "meta-llama/Llama-3.1-70B"
+model_id = "meta-llama/llama-3.1-70B"
 local_dir = "llama-hf"
 
 # Create the directory if it doesn't exist

content/learning-paths/servers-and-cloud-computing/distributed-inference-with-llama-cpp/how-to-2.md

@@ -18,7 +18,7 @@ For this demonstration, the experimental setup includes:
 
 - Total number of instances: 3
 - Instance type: c8g.4xlarge
-- Model: model.gguf (Llama-3.1-70B_Q4_0, ~38GB when quantized to 4 bits)
+- Model: model.gguf (llama-3.1-70B_Q4_0, ~38GB when quantized to 4 bits)
 
 One of the three nodes serves as the master node, which physically hosts the model file. The other two nodes act as worker nodes. In `llama.cpp`, remote procedure calls (RPC) offload both the model and the computation over TCP connections between nodes. The master node forwards inference requests to the worker nodes, where computation is performed.