newLP

Author: parichaydas

content/learning-paths/embedded-and-microcontrollers/finetuning-llms-on-arm-powered-mobile-devices-with-executorch/1.png

@@ -0,0 +1,66 @@
+---
+title: Fine-tuning Large Language Models (LLMs) on Arm Powered Mobile Devices with ExecuTorch
+minutes_to_complete: 60
+
+who_is_this_for: This learning path provides an introduction for developers and data scientists new to fine-tuning large language models (LLMs) and looking to develop a fine-tuned LLM for mobile applications. Fine-tuning involves adapting a pre-trained LLM to specific tasks or domains by training it on domain-specific data and optimizing its responses for accuracy and relevance. For mobile applications, fine-tuning enables personalized interactions, enhanced query handling, and improved contextual understanding, making AI-driven features more effective. This session will cover key concepts, techniques, tools, and best practices, ensuring a structured approach to building a fine-tuned LLM that aligns with real-world mobile application requirements.Mobile application with Llama, KleidiAI, ExecuTorch, and XNNPACK.
+
+learning_objectives: 
+    - Learn the basics of large language models (LLMs) and how fine-tuning enhances model performance for specific use cases focusing on mobile applications. 
+    - Understand full fine-tuning, parameter-efficient fine-tuning (e.g., LoRA, QLoRA, PEFT), instruction-tuning and fine-tuning benchmarking.
+    - Learn when to use different fine-tuning approaches based on model size, task complexity, and computational constraints.
+    - Learn how to curate, clean, and preprocess domain-specific datasets for optimal fine-tuning.
+    - Understand dataset formats, tokenization, and annotation techniques for improving model learning.
+    - Implementing Fine-Tuning with Popular Frameworks like Hugging Face Transformers and PyTorch for LLM fine-tuning.
+    - Learn how to deploy and fine-tune the model in the mobile device.
+    - Compile a Large Language Model (LLM) using ExecuTorch.
+    - Describe techniques for running large language models in an mobile environment.
+
+prerequisites:
+    - Basic Understanding of Machine Learning & Deep Learning (Familiarity with concepts like supervised learning, neural networks, transfer learning and Understanding of model training, validation, & overfitting concepts).
+    - Familiarity with Deep Learning Frameworks (Experience with PyTorch for building, training neural networks and Knowledge of Hugging Face Transformers for working with pre-trained LLMs.
+    - An Arm-powered smartphone with the i8mm feature running Android, with 16GB of RAM.
+    - A USB cable to connect your smartphone to your development machine.
+    - An AWS Graviton4 r8g.16xlarge instance to test Arm performance optimizations, or any [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/) from a cloud service provider or an on-premise Arm server or Arm based laptop.
+    - Python 3.10.
+
+author: Parichay Das
+
+### Tags
+skilllevels: Introductory
+subjects: GenAI
+armips:
+    - Neoverse
+
+tools_software_languages:
+    - LLM
+    - GenAI
+    - Python
+    - PyTorch
+    - ExecuTorch
+operatingsystems:
+    - Linux
+    - Windows
+    - Android  
+
+
+further_reading:
+     - resource:
+        title: Hugging Face Documentation
+        link: https://huggingface.co/docs
+        type: documentation
+     - resource:
+        title: PyTorch Documentation
+        link: https://pytorch.org/docs/stable/index.html
+        type: documentation
+     - resource:
+        title: Android 
+        link: https://www.android.com/
+        type: website
+
+
+### FIXED, DO NOT MODIFY
+# ================================================================================
+weight: 1                       # _index.md always has weight of 1 to order correctly
+layout: "learningpathall"       # All files under learning paths have this same wrapper
+learning_path_main_page: "yes"  # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
+---

content/learning-paths/embedded-and-microcontrollers/finetuning-llms-on-arm-powered-mobile-devices-with-executorch/_index.md

@@ -0,0 +1,8 @@
+---
+# ================================================================================
+#       FIXED, DO NOT MODIFY THIS FILE
+# ================================================================================
+weight: 21                  # Set to always be larger than the content in this path to be at the end of the navigation.
+title: "Next Steps"         # Always the same, html page title.
+layout: "learningpathall"   # All files under learning paths have this same wrapper for Hugo processing.
+---

content/learning-paths/embedded-and-microcontrollers/finetuning-llms-on-arm-powered-mobile-devices-with-executorch/_next-steps.md

@@ -0,0 +1,65 @@
+---
+title: Overview
+weight: 2
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## What is Fine-Tuning
+Fine-tuning in the context of large language models (LLMs) refers to the process of further training a pre-trained LLM on domain-specific or task-specific data to enhance its performance for a particular application. LLMs, such as GPT, BERT, and LLaMA, are initially trained on massive corpora containing billions of tokens, enabling them to develop a broad linguistic understanding. Fine-tuning refines this knowledge by exposing the model to specialized datasets, allowing it to generate more contextually relevant and accurate responses. Rather than training an LLM from scratch, fine-tuning leverages the pre-existing knowledge embedded in the model, optimizing it for specific use cases such as customer support, content generation, legal document analysis, or medical text processing. This approach significantly reduces computational requirements and data needs while improving adaptability and efficiency in real-world applications. 
+
+## Advantage of Fine-Tuning
+Fine-tuning is essential for optimizing large language models (LLMs) to meet specific application requirements, enhance performance, and reduce computational costs. While pre-trained LLMs have broad linguistic capabilities, they may not always produce domain-specific, contextually accurate, or application-tailored responses
+- Customization for Specific Domains
+- Improved Response Quality and Accuracy
+- Task-Specific Adaptation
+- Reduction in Computational and Data Requirements
+- Enhanced Efficiency in Real-World Applications
+- Alignment with Ethical, Regulatory, and Organizational Guidelines
+
+## Fine-Tuning Methods
+Fine-tuning LLM uses different techniques based on the various use cases, computational constraints, and efficiency requirements. Below are the key fine-tuning methods:
+
+### Full Fine-Tuning (Supervised Learning Approach)
+It involves updating all parameters of the LLM using task-specific data, requiring significant computational power and large labeled datasets, which provides the highest level of customization.
+
+### Instruction Fine-Tuning
+Instruction fine-tuning is a supervised learning method. A pre-trained large language model (LLM) is further trained on instruction-response pairs to improve its ability to follow human instructions accurately. Instruction Fine-Tuning has some key features using Labeled Instruction-Response Pairs, Enhances Model Alignment with Human Intent, Commonly Used in Chatbots and AI Assistants, and Prepares Models for Zero-Shot and Few-Shot Learning.
+
+### Parameter-Efficient Fine-Tuning (PEFT)
+It is a optimized approaches that reduce the number of trainable parameters while maintaining high performance:
+
+- ###### LoRA (Low-Rank Adaptation)
+    - Introduces small trainable weight matrices (rank decomposition) while freezing the main model weights.
+    - It will significantly reduce GPU memory usage and training time.
+
+- ###### QLoRA (Quantized LoRA)
+    - It will use quantization (e.g., 4-bit or 8-bit precision) to reduce memory footprint while applying LoRA fine-tuning.
+    - It is Ideal for fine-tuning large models on limited hardware.
+
+- ###### Adapter Layers
+    - Inserts small trainable layers between existing layers of the model and Keeps most parameters frozen, reducing computational overhead.
+
+- ###### Reinforcement Learning from Human Feedback (RLHF)
+    - Fine-tunes models based on human preferences using reinforcement learning.
+
+- ###### Domain-Specific Fine-Tuning
+    - Fine-tunes the LLM with domain-specific datasets and Improves accuracy and relevance in specialized applications.
+
+- ###### Multi-Task Learning (MTL) Fine-Tuning
+    - Trains the model on multiple tasks simultaneously, enabling generalization across different applications.
+
+
+
+## Fine-Tuning Implementaion 
+The following steps need to be performed to implement fine-tuning:
+
+
+![example image alt-text#center](1.png "Figure 1. Fine-Tuning Implementaion")
+
+-   Base Model Selection: Choose a pre-trained model based on your use cases. You can find pre-trained models at [Hugging Face](https://huggingface.co/).
+-   Fine-Tuning Method Finalization: Select the most appropriate fine-tuning method (e.g., supervised, instruction-based, PEFT) based on your use case and dataset. You can typically find various datasets on [Hugging Face](https://huggingface.co/datasets) and [Kaggle](https://www.kaggle.com/datasets).
+-   Dataset Prepration:Organize your data for your use case-specific training, ensuring it aligns with the model's required format.
+-   Training:Utilize frameworks such as TensorFlow and PyTorch to fine-tune the model.
+-   Evaluate: Evaluate the model, refine it as needed, and retrain to enhance performance.

content/learning-paths/embedded-and-microcontrollers/finetuning-llms-on-arm-powered-mobile-devices-with-executorch/example-picture.png

@@ -0,0 +1,100 @@
+---
+title: Large Language Model - Setup Environment 
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Large Language Model - Setup Environment
+
+#### Plartform Required 
+- An AWS Graviton4 r8g.16xlarge instance to test Arm performance optimizations, or any [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/) from a cloud service provider or an on-premise Arm server or Arm based laptop.
+- An Arm-powered smartphone with the i8mm feature running Android, with 16GB of RAM.
+- A USB cable to connect your smartphone to your development machine.
+
+#### Set Up Required Libraries
+The following commands install the necessary libraries for the task, including Hugging Face Transformers, Datasets, and fine-tuning methods. These libraries facilitate model loading, training, and fine-tuning
+
+###### The transformers library (by Hugging Face) provides pre-trained LLMs
+```python
+!pip install transformers
+
+```
+###### This installs transformers along with PyTorch, ensuring that models are trained and fine-tuned using the Torch backend.
+```python
+!pip install transformers[torch]
+```
+###### The datasets library (by Hugging Face) provides access to a vast collection of pre-built datasets
+
+```python
+!pip install datasets
+```
+###### The evaluate library provides metrics for model performance assessment
+
+```python
+!pip install evaluate
+```
+###### Speed up fine-tuning of Large Language Models (LLMs)
+Use a library designed to speed up fine-tuning of Large Language Models (LLMs) while reducing computational costs, optimizes training efficiency, particularly for LoRA (Low-Rank Adaptation) fine-tuning 
+
+
+```
+Used pre-fine-tuned customer support model
+```
+
+###### AWS Graviton3 Instance Setup
+Launch EC2 Instance:
+
+
+```
+Recommended instance: c7g.xlarge or c7g.2xlarge
+- c7g.xlarge: 4 vCPUs, 8 GB RAM 
+- c7g.2xlarge: 8 vCPUs, 16 GB RAM 
+
+Connect to your instance
+ssh -i your-key.pem ubuntu@your-instance-ip
+```
+
+###### System Setup:
+Launch EC2 Instance:
+
+
+```
+Update system
+sudo apt update && sudo apt upgrade -y
+
+Install essential packages
+sudo apt install -y build-essential git wget curl \
+    python3-pip python3-venv htop
+
+Verify ARM architecture
+uname -m  # Should show: aarch64
+```
+
+###### Python Environment Setup
+
+
+
+```
+Create virtual environment
+python3 -m venv chatbot_env
+source chatbot_env/bin/activate
+
+Upgrade pip
+pip install --upgrade pip
+
+Install PyTorch for ARM/CPU
+pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
+
+Install Transformers and utilities
+pip install transformers==4.36.0
+pip install datasets accelerate
+pip install huggingface-hub
+pip install flask psutil
+pip install numpy pandas
+
+Verify installation
+python -c "import torch; print(f'PyTorch version: {torch.__version__}')"
+python -c "import torch; print(f'CPU available: {torch.cpu.device_count()}')"
+```

content/learning-paths/embedded-and-microcontrollers/finetuning-llms-on-arm-powered-mobile-devices-with-executorch/how-to-1.md

@@ -0,0 +1,121 @@
+---
+title: Model Loading & Optimization for ARM
+weight: 4
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Model Loading & Optimization for ARM
+
+###### Hugging Face Authentication
+```bash
+Login to Hugging Face (needed for Llama models)
+pip install huggingface_hub
+huggingface-cli login
+
+Enter your token when prompted
+Get token from: https://huggingface.co/settings/tokens
+
+```
+
+###### Load a pre-fine-tuned model (from Hugging Face)
+- Example: meta-llama/Llama-3-8B-Instruct or a customer-support fine-tuned variant
+
+###### Model Optimization for ARM (Understanding Quantization)
+- Reduces model precision (e.g., 32-bit → 8-bit)
+- Decreases memory footprint (~4x reduction)
+- Speeds up inference on CPU
+- Minimal accuracy loss for most tasks
+
+###### Apply Dynamic Quantization
+- Create optimize_model.py
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from torch.quantization import quantize_dynamic
+import time
+import os
+
+def load_base_model(model_name):
+    """Load the base model"""
+    print(f"Loading base model: {model_name}")
+    
+    tokenizer = AutoTokenizer.from_pretrained(model_name)
+    tokenizer.pad_token = tokenizer.eos_token
+    
+    model = AutoModelForCausalLM.from_pretrained(
+        model_name,
+        torch_dtype=torch.float32,
+        device_map=None,
+        low_cpu_mem_usage=True
+    )
+    model.eval()
+    
+    return model, tokenizer
+
+def apply_quantization(model):
+    """Apply dynamic quantization"""
+    print("Applying dynamic quantization...")
+    
+    quantized_model = quantize_dynamic(
+        model,
+        {torch.nn.Linear},  # Quantize linear layers
+        dtype=torch.qint8
+    )
+    
+    return quantized_model
+
+def test_model(model, tokenizer, prompt):
+    """Test model with a sample prompt"""
+    inputs = tokenizer(prompt, return_tensors="pt")
+    
+    start_time = time.time()
+    with torch.no_grad():
+        outputs = model.generate(
+            inputs.input_ids,
+            max_new_tokens=100,
+            do_sample=False,
+            pad_token_id=tokenizer.eos_token_id
+        )
+    inference_time = time.time() - start_time
+    
+    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+    
+    return response, inference_time
+
+def main():
+    model_name = "meta-llama/Meta-Llama-3-8B-Instruct"
+    
+    # Load base model
+    base_model, tokenizer = load_base_model(model_name)
+    
+    # Test base model
+    test_prompt = "How do I track my order?"
+    print("\nTesting base model...")
+    response, base_time = test_model(base_model, tokenizer, test_prompt)
+    print(f"Base model inference time: {base_time:.2f}s")
+    
+    # Apply quantization
+    quantized_model = apply_quantization(base_model)
+    
+    # Test quantized model
+    print("\nTesting quantized model...")
+    response, quant_time = test_model(quantized_model, tokenizer, test_prompt)
+    print(f"Quantized model inference time: {quant_time:.2f}s")
+    print(f"Speedup: {base_time / quant_time:.2f}x")
+    
+    # Save quantized model
+    save_dir = "./models/quantized_llama3"
+    os.makedirs(save_dir, exist_ok=True)
+    
+    torch.save(quantized_model.state_dict(), f"{save_dir}/model.pt")
+    tokenizer.save_pretrained(save_dir)
+    
+    print(f"\nQuantized model saved to: {save_dir}")
+
+if __name__ == "__main__":
+    main()
+
+```