Skip to content

Commit a76bd97

Browse files
Starting content dev.
1 parent a866b5e commit a76bd97

2 files changed

Lines changed: 11 additions & 24 deletions

File tree

Lines changed: 11 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,28 @@
11
---
2-
title: Overview
2+
title: Overview of AFM-4.5B deployment on Google Cloud Axion with Llama.cpp
33
weight: 2
44

55
### FIXED, DO NOT MODIFY
66
layout: learningpathall
77
---
88

9-
## The AFM-4.5B model
9+
## AFM-4.5B model and deployment workflow
1010

1111
[AFM-4.5B](https://huggingface.co/arcee-ai/AFM-4.5B) is a 4.5-billion-parameter foundation model designed to balance accuracy, efficiency, and broad language coverage. Trained on nearly 8 trillion tokens of carefully filtered data, it performs well across a wide range of languages, including Arabic, English, French, German, Hindi, Italian, Korean, Mandarin, Portuguese, Russian, and Spanish.
1212

13-
In this Learning Path, you'll deploy [AFM-4.5B](https://huggingface.co/arcee-ai/AFM-4.5B) using [Llama.cpp](https://github.com/ggerganov/llama.cpp) on an Arm-based Google Cloud Axion instance. You’ll walk through the full workflow, from setting up your environment and compiling the runtime, to downloading, quantizing, and running inference on the model. You'll also evaluate model quality using perplexity, a common metric for measuring how well a language model predicts text.
13+
In this Learning Path, youll deploy [AFM-4.5B](https://huggingface.co/arcee-ai/AFM-4.5B) using [Llama.cpp](https://github.com/ggerganov/llama.cpp) on a Google Cloud Axion Arm64 instance. You’ll walk through the full workflow, from setting up your environment and compiling the runtime, to downloading, quantizing, and running inference on the model. Youll also evaluate model quality using perplexity, a standard metric for how well a language model predicts text.
1414

1515
This hands-on guide helps developers build cost-efficient, high-performance LLM applications on modern Arm server infrastructure using open-source tools and real-world deployment practices.
1616

17-
### LLM deployment workflow on Google Axion
17+
### Deployment workflow for AFM-4.5B on Google Cloud Axion
1818

19-
- **Provision compute**: launch a Google Cloud instance using an Axion-based instance type (for example, `c4a-standard-16`)
19+
- **Provision compute**: launch a Google Cloud instance using an Axion-based instance type (for example, `c4a-standard-16`)
20+
- **Set up your environment**: install build tools and dependencies (CMake, Python, Git)
21+
- **Build the inference engine**: clone the [Llama.cpp](https://github.com/ggerganov/llama.cpp) repository and compile the project for your Arm-based environment
22+
- **Prepare the model**: download the AFM-4.5B model files from Hugging Face and use Llama.cpp’s quantization tools to reduce model size and optimize performance
23+
- **Run inference**: load the quantized model and run sample prompts using Llama.cpp
24+
- **Evaluate model quality**: calculate perplexity or use other metrics to assess performance
2025

21-
- **Set up your environment**: install the required build tools and dependencies (such as CMake, Python, and Git)
22-
23-
- **Build the inference engine**: clone the [Llama.cpp](https://github.com/ggerganov/llama.cpp) repository and compile the project for your Arm-based environment
24-
25-
- **Prepare the model**: download the **AFM-4.5B** model files from Hugging Face and use Llama.cpp's quantization tools to reduce model size and optimize performance
26-
27-
- **Run inference**: load the quantized model and run sample prompts using Llama.cpp.
28-
29-
- **Evaluate model quality**: calculate **perplexity** or use other metrics to assess model performance
30-
31-
{{< notice Note>}}
26+
{{< notice Note >}}
3227
You can reuse this deployment flow with other models supported by Llama.cpp by swapping out the model file and adjusting quantization settings.
3328
{{< /notice >}}
34-
35-
36-
37-

content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-gcp/_index.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,6 @@
11
---
22
title: Deploy Arcee AFM-4.5B on Arm-based Google Cloud Axion with Llama.cpp
33

4-
draft: true
5-
cascade:
6-
draft: true
7-
84
minutes_to_complete: 30
95

106
who_is_this_for: This Learning Path is for developers and ML engineers who want to deploy Arcee's AFM-4.5B small language model on Google Cloud Axion instances using Llama.cpp.

0 commit comments

Comments
 (0)