Skip to content

Commit ad6b97b

Browse files
Update 05_downloading_and_optimizing_afm45b.md
Editorial
1 parent 42b6d88 commit ad6b97b

1 file changed

Lines changed: 4 additions & 4 deletions

File tree

content/learning-paths/servers-and-cloud-computing/arcee-foundation-model-on-aws/05_downloading_and_optimizing_afm45b.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,11 @@ pip install huggingface_hub hf_xet
2828
This command installs:
2929

3030
- `huggingface_hub`: Python client for downloading models and datasets
31-
- `hf_xet`: Git extension for fetching large model files stored on Hugging Face
31+
- `hf_xet`: Git extension for fetching large model files hosted on Hugging Face
3232

3333
These tools include the `hf` command-line interface you'll use next.
3434

35-
## Login to the Hugging Face Hub
35+
## Log in to the Hugging Face Hub
3636

3737
```bash
3838
hf auth login
@@ -86,7 +86,7 @@ This command creates a 4-bit quantized version of the model:
8686
- `llama-quantize` is the quantization tool from Llama.cpp.
8787
- `afm-4-5B-F16.gguf` is the input GGUF model file in 16-bit precision.
8888
- `Q4_0` applies zero-point 4-bit quantization.
89-
- This reduces the model size by approximately 45% (from ~15GB to ~8GB).
89+
- This reduces the model size by approximately ~70% (from ~15GB to ~4.4GB).
9090
- The quantized model will use less memory and run faster, though with a small reduction in accuracy.
9191
- The output file will be `afm-4-5B-Q4_0.gguf`.
9292

@@ -104,7 +104,7 @@ bin/llama-quantize models/afm-4-5b/afm-4-5B-F16.gguf models/afm-4-5b/afm-4-5B-Q8
104104

105105
This command creates an 8-bit quantized version of the model:
106106
- `Q8_0` specifies 8-bit quantization with zero-point compression.
107-
- This reduces the model size by approximately 70% (from ~15GB to ~4.4GB).
107+
- This reduces the model size by approximately ~45% (from ~15GB to ~8GB).
108108
- The 8-bit version provides a better balance between memory usage and accuracy than 4-bit quantization.
109109
- The output file is named `afm-4-5B-Q8_0.gguf`.
110110
- Commonly used in production scenarios where memory resources are available.

0 commit comments

Comments
 (0)