LLM Course:- Chapter 2(Infrence Deployment Not Working )

[apking2000]

Optimized Inference Deployment

Clone the repository

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp

Build the project

make

Download the SmolLM2-1.7B-Instruct-GGUF model

curl -L -O https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/resolve/main/smollm2-1.7b-instruct.Q4_K_M.gguf`

I have tried the code on my mac but got error, I think the doc is not updated

[apking2000]

Also through the given curl command i am not able to download the model:-
curl -L -O https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/resolve/main/smollm2-1.7b-instruct.Q4_K_M.gguf