Minor updates based on review feedback.

Signed-off-by: Anna Mayne <anna.mayne@arm.com>

Author: almayne

content/learning-paths/servers-and-cloud-computing/vllm-benchmark-quantisation/1-overview-and-setup.md

@@ -40,7 +40,7 @@ sudo apt-get install -y libtcmalloc-minimal4
 
 ## Create and activate a Python virtual environment
 
-It’s best practice to install vLLM inside an isolated environment to prevent conflicts between system and project dependencies:
+It is considered best practice to install vLLM inside an isolated environment to prevent conflicts between system and project dependencies:
 ```bash
 python3.12 -m venv vllm_env
 source vllm_env/bin/activate
@@ -49,7 +49,7 @@ python -m pip install --upgrade pip
 
 ## Install vLLM for CPU
 
-Install a recent CPU specific build of vLLM:
+Install a CPU-specific build of vLLM:
 ```bash
 export VLLM_VERSION=0.20.0
 pip install https://github.com/vllm-project/vllm/releases/download/v${VLLM_VERSION}/vllm-${VLLM_VERSION}+cpu-cp38-abi3-manylinux_2_35_aarch64.whl --extra-index-url https://download.pytorch.org/whl/cpu
@@ -60,9 +60,8 @@ If you wish to build vLLM from source you can follow the instructions in the [Bu
 
 ## Set up access to LLama3.1-8B models
 
-To access the Llama models hosted by Hugging Face, you will need to install the Hugging Face CLI so that you can authenticate yourself and the harness can download what it needs. You should create an account on https://huggingface.co/ and follow the instructions [in the Hugging Face CLI guide](https://huggingface.co/docs/huggingface_hub/en/guides/cli) to set up your access token. You can then install the CLI and login:
+To access the Llama models hosted by Hugging Face, you will need to install the Hugging Face CLI so that you can authenticate yourself and the harness can download what it needs. You should create an account on https://huggingface.co/ and follow the instructions [in the Hugging Face CLI guide](https://huggingface.co/docs/huggingface_hub/en/guides/cli) to install the CLI and setup your access token. You can then login to HF:
 ```bash
-curl -LsSf https://hf.co/cli/install.sh | bash
 hf auth login
 ```
 

content/learning-paths/servers-and-cloud-computing/vllm-benchmark-quantisation/2-quantisation-recipe.md

@@ -8,7 +8,7 @@ layout: learningpathall
 
 ## Understanding quantisation
 
-Quantised models have their weights converted to a lower precision data type, which reduces the memory requirements of the model and can improve performance significantly. In the [Run vLLM inference with INT4 quantization on Arm servers](/learning-paths/servers-and-cloud-computing/vllm-acceleration/) Learning Path we have covered how to quantise a model yourself. There are also many publicly available quantised versions of popular models, such as https://huggingface.co/RedHatAI/Meta-Llama-3.1-8B-quantized.w8a8 and https://huggingface.co/RedHatAI/whisper-large-v3-quantized.w8a8, which we will be using in this Learning Path.
+Quantised models have their weights converted to a lower precision data type which reduces the memory requirements of the model and can improve performance significantly. In the [Run vLLM inference with INT4 quantization on Arm servers](/learning-paths/servers-and-cloud-computing/vllm-acceleration/) Learning Path we have covered how to quantise a model yourself. There are also many publicly available quantised versions of popular models, such as https://huggingface.co/RedHatAI/Meta-Llama-3.1-8B-quantized.w8a8 and https://huggingface.co/RedHatAI/whisper-large-v3-quantized.w8a8, which we will be using in this Learning Path.
 
 The notation w8a8 means that the weights have been quantised to 8-bit integers and the activations (the input data) are dynamically quantised to the same. This allows our kernels to utilise Arm's 8-bit integer matrix multiply feature I8MM. You can learn more about this in the [KleidiAI and matrix multiplication](/learning-paths/cross-platform/kleidiai-explainer/) Learning Path.
 

content/learning-paths/servers-and-cloud-computing/vllm-benchmark-quantisation/3-run-inference.md

@@ -15,7 +15,7 @@ Start vLLM’s OpenAI-compatible API server using Llama3.1-8B:
 vllm serve meta-llama/Llama-3.1-8B
 ```
 
-Then we can create a test script that sends a request to the server using the OpenAI library. Copy the below to a file named llama_test.py.
+Then we can create a test script that sends a request to the server using the OpenAI library. Copy the Python script below to a file named llama_test.py.
 
 ```python
 import time
@@ -63,7 +63,7 @@ python llama_test.py
 
 This will return the text generated by the model from your prompt. In the server logs you can see the throughput measured in tokens per second.
 
-You can do the same for the quantised model. Start the server:
+You can do the same for the pre-quantised model loaded directly from Hugging Face. Start the server:
 ```bash
 vllm serve RedHatAI/Meta-Llama-3.1-8B-quantized.w8a8
 ```
@@ -78,7 +78,7 @@ Run inference on the quantised model:
 python llama_test.py
 ```
 
-You have now run inference using both the non-quantised and quantised Llama3.1-8B models.
+You have now run inference using both the non-quantised and quantised Llama3.1-8B models!
 
 ## Run inference on Whisper
 
@@ -89,7 +89,7 @@ pip install vllm[audio]
 vllm serve openai/whisper-large-v3 
 ```
 
-Then we can create a test script that sends a request with an audio file to the server using the OpenAI library. Copy the below to a file named whisper_test.py.
+Then we can create a test script that sends a request with an audio file to the server using the OpenAI library. Copy the Python script below to a file named whisper_test.py.
 
 ```python
 import time
@@ -130,7 +130,7 @@ Now run the script with:
 python whisper_test.py
 ```
 
-You can do the same for the quantised model. Start the server:
+You can do the same for the pre-quantised model loaded directly from Hugging Face. Start the server:
 ```bash
 vllm serve RedHatAI/whisper-large-v3-quantized.w8a8
 ```
@@ -145,4 +145,4 @@ Run inference on the quantised model:
 python whisper_test.py
 ```
 
-You now have the quantised and non-quantised Llama and Whisper models on your local machine. You have installed vLLM and demonstrated you can run inference on your models. Now you can move on to benchmarking the Llama models and compare their performance.
+You now have the quantised and non-quantised Llama and Whisper models on your local machine! You have installed vLLM and demonstrated you can run inference on your models. Now you can move on to benchmarking the Llama models and compare their performance.

content/learning-paths/servers-and-cloud-computing/vllm-benchmark-quantisation/4-benchmarking.md

@@ -19,8 +19,8 @@ vllm serve \
 ```
 
 vLLM uses dynamic continuous batching to maximise hardware utilisation. Two key parameters govern this process:
-  * max-model-len, which is the maximum sequence length (number of tokens per request). No single prompt or generated sequence can exceed this limit.
-  * max-num-batched-tokens, which is the total number of tokens processed in one batch across all requests. The sum of input and output tokens from all concurrent requests must stay within this limit.
+  * max-model-len, which is the maximum sequence length (number of tokens per request). No single prompt or generated sequence can exceed this limit. We've chosen a value large enough for the selected model and dataset.
+  * max-num-batched-tokens, which is the total number of tokens processed in one batch across all requests. The sum of input and output tokens from all concurrent requests must stay within this limit. We've chosen a value that, combined with our concurrency limit shown below, gives optimal throughput and latency.
 
 Now the server is running, we can benchmark using the public ShareGPT dataset.
 ```bash
@@ -33,10 +33,13 @@ vllm bench serve \
   --num-prompts 256 \
   --request-rate 8 \
   --max-concurrency 10 \
+  --top-p 1 --temperature 0 \
   --percentile-metrics ttft,tpot \
   --metric-percentiles 50,95,99 \
   --save-result --result-dir bench_out --result-filename serve.json
 ```
+Here we are using greedy decoding: '''--top-p 1 --temperature 0'''. This selects the next token with the highest probability at each step, instead of sampling from a selection of likely tokens.
+
 The interesting results are request throughput, output token throughput, total token throughput, TTFT (time to first token) and TPOT (time per output token). We're aiming for a mean TPOT < 100ms, so the maximum concurrency selected should be as high as possible while meeting that TPOT requirement.
 
 Repeat with the quantised model. The smaller model allows us to increase the concurrency. You should see a significant improvement in the throughput results (increased tokens/s).
@@ -53,14 +56,15 @@ vllm bench serve \
   --num-prompts 256 \
   --request-rate 8 \
   --max-concurrency 24 \
+  --top-p 1 --temperature 0 \
   --percentile-metrics ttft,tpot \
   --metric-percentiles 50,95,99 \
   --save-result --result-dir bench_out --result-filename serve.json
 ```
 
 ## Llama accuracy benchmarking
 
-The lm-evaluation-harness is the standard way to measure model accuracy across common academic benchmarks (for example MMLU, HellaSwag, GSM8K) and runtimes (such as Hugging Face, vLLM, and llama.cpp). In this section, you’ll run accuracy tests for both BF16 and INT8 deployments of your Llama models served by vLLM on Arm-based servers.
+The lm-evaluation-harness is the standard way to measure model accuracy across common academic benchmarks (for example [MMLU](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/mmlu), [HellaSwag](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/hellaswag), [GSM8K](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/gsm8k)) and runtimes (such as [Hugging Face](https://github.com/huggingface/transformers), [vLLM](https://github.com/vllm-project/vllm), and [llama.cpp](https://github.com/ggml-org/llama.cpp)). In this section, you’ll run accuracy tests for both BF16 and INT8 deployments of your Llama models served by vLLM on Arm-based servers.
 
 You will:
 - Install the lm-eval harness with vLLM support

content/learning-paths/servers-and-cloud-computing/vllm-benchmark-quantisation/_index.md

@@ -14,7 +14,7 @@ prerequisites:
     - An Arm-based Linux server (Ubuntu 22.04+ recommended) with a minimum of 32 vCPUs, 96 GB RAM, and 64 GB free disk space
     - Python 3.12 and basic familiarity with Hugging Face Transformers and quantisation
 
-author: Anna Mayne
+author: Anna Mayne, Nikhil Gupta, Marek Michałowski
 
 ### Tags
 skilllevels: Introductory