Skip to content

Commit cf7250e

Browse files
committed
Updated _index.d
1 parent 1e8c667 commit cf7250e

1 file changed

Lines changed: 6 additions & 7 deletions

File tree

  • content/learning-paths/servers-and-cloud-computing/vllm-acceleration

content/learning-paths/servers-and-cloud-computing/vllm-acceleration/_index.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,19 +7,18 @@ cascade:
77

88
minutes_to_complete: 60
99

10-
who_is_this_for: This learning path is for software developers and AI engineers who want to build an optimized vLLM for Arm servers, quantize models to INT4, and serve them through an OpenAIcompatible API.
10+
who_is_this_for: This learning path is designed for software developers and AI engineers who want to build and optimize vLLM for Arm-based servers, quantize large language models (LLMs) to INT4, and serve them efficiently through an OpenAI-compatible API.
1111

1212
learning_objectives:
13-
- Build an optimized vLLM for aarch64 with oneDNN + Arm Compute Library.
14-
- Set up dependencies including PyTorch and llmcompressor dependencies.
15-
- Quantize an LLM (DeepSeek‑V2‑Lite) to 4bit weights.
16-
- Run and serve the quantized model using vLLM & test BF16 nonquantized serving.
13+
- Build an optimized vLLM for aarch64 with oneDNN and the Arm Compute Library(ACL).
14+
- Set up all runtime dependencies including PyTorch, llmcompressor, and Arm-optimized libraries.
15+
- Quantize an LLM (DeepSeek‑V2‑Lite) to 4-bit integer (INT4) precision.
16+
- Run and serve both quantized and BF16 (non-quantized) variants using vLLM.
1717
- Use OpenAI‑compatible endpoints and understand sequence and batch limits.
1818

1919
prerequisites:
20-
- An Arm-based Linux server (Ubuntu 22.04+ recommended) with 32+ vCPUs, 64+ GB RAM, and 32+ GB free disk.
20+
- An Arm-based Linux server (Ubuntu 22.04+ recommended) with a minimum of 32 vCPUs, 64 GB RAM, and 64 GB free disk space.
2121
- Python 3.12 and basic familiarity with Hugging Face Transformers and quantization.
22-
- Optional: a Hugging Face token to access gated models.
2322

2423
author:
2524
- Nikhil Gupta

0 commit comments

Comments
 (0)