Merge pull request #2888 from mhall119/mhall/spark-finetuning

Add new Learning Path: Model Fine-Tuning on NVIDIA DGX Spark

Author: jasonrandrews

content/learning-paths/laptops-and-desktops/pytorch-finetuning-on-spark/1-setup.md

@@ -0,0 +1,108 @@
+---
+title: Setup your NVIDIA DGX Spark
+weight: 2
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+The following section comes from the [NVIDIA Developer Program](https://build.nvidia.com/spark/pytorch-fine-tune/instructions) site.
+
+## Configure Docker permissions
+
+Before you can run containerized workloads, you need to ensure your user account has the necessary permissions to interact with Docker. By default, Docker requires root privileges in Ubuntu, which means you would need to prefix every Docker command with `sudo`. Adding your user to the `docker` group eliminates this requirement and streamlines your workflow.
+
+Open a terminal and test your current Docker access:
+
+```bash
+docker ps
+```
+
+If you see a permission denied error (something like `permission denied while trying to connect to the Docker daemon socket`), you need to add your user to the docker group:
+
+```bash
+sudo usermod -aG docker $USER
+newgrp docker
+```
+
+The `usermod` command adds your user to the docker group, while `newgrp docker` activates the group membership immediately without requiring you to log out and back in. After running these commands, you should be able to execute Docker commands without sudo.
+
+## Download PyTorch container
+
+NVIDIA provides pre-built PyTorch containers that include all the necessary frameworks, libraries, and dependencies optimized for NVIDIA GPUs. These containers are regularly updated and maintained, ensuring you have access to the latest stable versions without the complexity of manual dependency management.
+
+Pull the latest PyTorch container from NVIDIA's container registry:
+
+```bash
+docker pull nvcr.io/nvidia/pytorch:25.11-py3
+```
+
+This command downloads the November 2025 release of the PyTorch container, which includes PyTorch, CUDA libraries, cuDNN, and other essential tools pre-configured for optimal performance on NVIDIA hardware. The download size is several gigabytes, so this step might take a few minutes depending on your internet connection.
+
+## Launch container instance
+
+Now that you have the container image downloaded, you can launch an interactive container session. This creates an isolated environment where you'll perform all your fine-tuning work.
+
+Run the following command to start the container:
+
+```bash
+docker run --gpus all -it --rm --ipc=host \
+-v $HOME/.cache/huggingface:/root/.cache/huggingface \
+-v ${PWD}:/workspace -w /workspace \
+nvcr.io/nvidia/pytorch:25.11-py3
+```
+
+This command includes several important flags:
+
+- `--gpus all` gives the container access to all available GPUs on your system
+- `--ipc=host` enables shared memory between the host and container, which is essential for multi-GPU training and data loading
+- `-v $HOME/.cache/huggingface:/root/.cache/huggingface` mounts your Hugging Face cache directory, preventing repeated downloads of models and datasets
+- `-v ${PWD}:/workspace -w /workspace` mounts your current directory into the container and sets it as the working directory, this will allow you to access the fine-tuned model you will be making from another environment later.
+
+After running this command, you'll be inside the container with a root shell prompt.
+
+## Install dependencies
+
+The base PyTorch container doesn't include all the specialized libraries needed for efficient model fine-tuning. You need to install several additional Python packages that provide transformer models, parameter-efficient fine-tuning methods, dataset utilities, and training frameworks.
+
+Inside the running container, install the required dependencies:
+
+```bash
+pip install transformers peft datasets trl bitsandbytes
+```
+
+These packages serve specific purposes:
+
+- `transformers` provides access to pre-trained language models and tokenizers from Hugging Face
+- `peft` (Parameter-Efficient Fine-Tuning) enables techniques like LoRA and QLoRA that reduce memory requirements
+- `datasets` offers a standardized interface for loading and processing training datasets
+- `trl` (Transformer Reinforcement Learning) includes training utilities and recipes for language models
+- `bitsandbytes` enables 4-bit and 8-bit quantization for memory-efficient training
+
+The installation typically takes a few minutes as pip downloads and installs each package along with their dependencies.
+
+## Authenticate with Hugging Face
+
+Many of the models you'll fine-tune are hosted on Hugging Face's model hub. Some models, particularly larger ones like Llama, require authentication to download. Even for public models, authentication provides better rate limits and tracking.
+
+First, obtain a Hugging Face access token from [https://huggingface.co/settings/tokens](https://huggingface.co/settings/tokens). Then authenticate:
+
+```bash
+huggingface-cli login
+```
+
+When prompted, paste your token and press Enter. When asked about git credentials, enter `n` since you don't need git integration for this workflow. This authentication persists across sessions because you mounted your Hugging Face cache directory, so you won't need to repeat this step.
+
+## Download NVIDIA DGX Spark playbook
+
+NVIDIA provides a collection of ready-to-use fine-tuning scripts specifically optimized for DGX systems. These scripts implement best practices for various model sizes and fine-tuning techniques, saving you from writing training code from scratch.
+
+Clone the playbooks repository:
+
+```bash
+git clone https://github.com/NVIDIA/dgx-spark-playbooks
+cd dgx-spark-playbooks/nvidia/pytorch-fine-tune/assets
+```
+
+The repository contains scripts for different model architectures and training strategies. The `assets` directory includes the fine-tuning scripts you'll use in the next steps. Each script is preconfigured with sensible defaults but also accepts command-line arguments for customization.
+

content/learning-paths/laptops-and-desktops/pytorch-finetuning-on-spark/2-finetuning.md

@@ -0,0 +1,68 @@
+---
+title: How Fine Tuning Works
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Why fine-tuning matters
+
+Pre-trained large language models (LLMs) are trained on vast amounts of general text data, giving them broad knowledge and language understanding. However, these models often need specialization to perform well on specific tasks, domains, or to follow particular instruction formats. Fine-tuning adapts a pre-trained model to your specific use case without the computational expense of training from scratch.
+
+Fine-tuning is essential when you need a model to:
+
+- Understand domain-specific terminology and concepts (medical, legal, scientific)
+- Follow specific instruction formats or conversation styles
+- Perform specialized tasks with higher accuracy than a general-purpose model
+- Align with organizational guidelines or safety requirements
+- Reduce hallucinations in specific knowledge domains
+
+The process typically requires significantly less data and compute resources than pre-training. While pre-training might use trillions of tokens on thousands of GPUs, fine-tuning can achieve excellent results with thousands or even hundreds of carefully curated examples on a single GPU or small GPU cluster, making it an ideal use case for the NVIDIA DGX Spark.
+
+For a deeper look at LLM fine-tuning with Hugging Face transformers, see the [Hugging Face Fine-tuning Guide](https://huggingface.co/docs/transformers/training).
+
+## How supervised fine-tuning works
+
+Supervised fine-tuning (SFT) continues the training process on a pre-trained model using labeled examples that demonstrate the desired behavior. Unlike pre-training, which predicts the next token from unlabeled text, SFT uses input-output pairs that explicitly teach the model how to respond to specific queries or prompts.
+
+The process works by:
+
+**Data preparation** - You create or collect a dataset of input-output pairs. For instruction tuning, this means pairs of user prompts and desired responses. For example: "Explain photosynthesis" paired with a detailed explanation.
+
+**Loss calculation** - The model generates predictions for the training examples, and the training algorithm calculates how different the predictions are from the target outputs. This difference is quantified using a loss function, typically cross-entropy loss for language models.
+
+**Gradient computation** - A gradient represents the direction and magnitude of change needed for each model parameter to reduce the loss. The training process calculates gradients that indicate how to adjust each model parameter to reduce the loss. These gradients flow backward through the neural network layers via backpropagation.
+
+**Parameter updates** - The optimizer uses the gradients to update the model's weights, making small adjustments that should improve performance on the training examples. This process repeats across many examples and multiple epochs.
+
+**Validation** - Periodically, you evaluate the model on held-out validation data to ensure it's learning to generalize rather than simply memorizing the training set.
+
+The key advantage of SFT is that it preserves the pre-trained model's general knowledge while adapting its behavior to match your specific examples. The model learns the patterns in your data without forgetting what it learned during pre-training.
+
+For technical details on the algorithms behind supervised fine-tuning, refer to the [PyTorch training documentation](https://pytorch.org/tutorials/beginner/introyt/trainingyt.html).
+
+## What instruction tuning does for a model
+
+Instruction tuning is a specific type of supervised fine-tuning that teaches models to follow natural language instructions. Base pre-trained models are excellent at completing text but don't inherently understand how to respond to commands or questions in a conversational format. Instruction tuning bridges this gap.
+
+The transformation happens through training on datasets where each example contains:
+
+**Instruction** - A clear directive or question  
+**Input** (optional) - Additional context or data to process  
+**Output** - The desired response demonstrating correct behavior
+
+For example:
+- Instruction: "Summarize the following article"
+- Input: [article text]
+- Output: [concise summary]
+
+After instruction tuning, the model learns to:
+
+- Recognize and interpret different instruction formats
+- Generate responses that directly address the given task
+- Maintain an expected tone and formatting for different request types
+
+This process dramatically improves the model's usability. While a base model might continue a prompt like "Translate this sentence to French: Hello" by generating more English text, an instruction-tuned model understands this is a task request and produces the French translation.
+
+Popular instruction-tuning datasets include Alpaca, Dolly, and FLAN. These datasets contain diverse instructions across many domains and task types, helping models generalize to new instructions they haven't seen during training. In the next step you will use the Alpaca dataset to fine-tune a base pre-trained model.

content/learning-paths/laptops-and-desktops/pytorch-finetuning-on-spark/3-pytorch.md

@@ -0,0 +1,173 @@
+---
+title: Fine Tuning with PyTorch and Hugging Face
+weight: 4
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Review fine-tuning scripts
+
+The NVIDIA playbook provides four main fine-tuning scripts, each designed for different scenarios:
+
+**Llama3_3B_full_finetuning.py** - Full supervised fine-tuning for Llama 3.2 3B  
+This script trains all model parameters, providing maximum flexibility but requiring more GPU memory. Use this for smaller models when you have sufficient GPU resources and want the best possible fine-tuning results.
+
+**Llama3_8B_LoRA_finetuning.py** - LoRA fine-tuning for Llama 3.1 8B  
+This script uses Low-Rank Adaptation (LoRA), which trains only a small number of additional parameters while keeping the base model frozen. This approach dramatically reduces memory requirements and training time while maintaining good performance.
+
+**Llama3_70B_LoRA_finetuning.py** - LoRA fine-tuning for Llama 3.1 70B with FSDP  
+For larger 70B models, this script implements LoRA with Fully Sharded Data Parallel (FSDP) support, distributing the model across multiple GPUs to handle the increased memory requirements.
+
+**Llama3_70B_qLoRA_finetuning.py** - QLoRA fine-tuning for Llama 3.1 70B  
+This script uses Quantized LoRA (QLoRA), which combines LoRA with 4-bit quantization. This is the most memory-efficient option, allowing you to fine-tune very large models even on systems with limited GPU memory.
+
+
+The file names only refer to the default model they will use if you don't specify one on the command line, you can use them to train other models of your choosing. This tutorial will use the `Llama3_3B_full_finetuning.py` script. Below are the key sections of that script. 
+
+## Imports and dataset preparation
+
+This section sets up the foundation for finetuning. The imports bring in PyTorch, dataset loading utilities, and Hugging Face libraries for supervised fine-tuning. 
+
+```python
+
+import torch
+import argparse
+from datasets import load_dataset
+from trl import SFTConfig, SFTTrainer
+from transformers import AutoModelForCausalLM, AutoTokenizer
+```
+
+The `ALPACA_PROMPT_TEMPLATE` defines the instruction-following format for training data with three fields: instruction, input, and response. This is the format that the fine-tuned model will be taught to expect as input and produce as output.
+
+The `get_alpaca_dataset()` function loads the Alpaca dataset and formats each example using the template, appending the EOS (End of String) token. It selects a configurable number of samples (default 500) and shuffles them for training.
+
+```python
+# Define prompt templates
+ALPACA_PROMPT_TEMPLATE = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
+### Instruction: {}
+
+### Input: {}
+
+### Response: {}"""
+
+def get_alpaca_dataset(eos_token, dataset_size=500):
+    # Preprocess the dataset
+    def preprocess(x):
+        texts = [
+            ALPACA_PROMPT_TEMPLATE.format(instruction, input, output) + eos_token
+            for instruction, input, output in zip(x["instruction"], x["input"], x["output"])
+        ]
+        return {"text": texts}
+
+    dataset = load_dataset("tatsu-lab/alpaca", split="train").select(range(dataset_size)).shuffle(seed=42)
+    return dataset.map(preprocess, remove_columns=dataset.column_names, batched=True)
+```
+
+## Model and tokenizer loading
+
+This section initializes the language model. The `from_pretrained()` method loads a pre-trained language model from Hugging Face. The tokenizer loads the corresponding tokenizer and sets the padding token to match the EOS token, which is necessary for batched training.
+
+```python
+    # Load the model and tokenizer
+    print(f"Loading model: {args.model_name}")
+    model = AutoModelForCausalLM.from_pretrained(
+        args.model_name,
+        dtype=args.dtype,
+        device_map="auto",
+        trust_remote_code=True
+    )
+    tokenizer = AutoTokenizer.from_pretrained(args.model_name, trust_remote_code=True)
+    tokenizer.pad_token = tokenizer.eos_token
+```
+
+## Dataset loading
+
+This section prepares the training data by calling the previously defined `get_alpaca_dataset()` function with the model's tokenizer EOS token and the specified dataset size. The formatted dataset is ready for the trainer to consume.
+
+```python
+    # Load and preprocess the dataset
+    print(f"Loading dataset with {args.dataset_size} samples...")
+    dataset = get_alpaca_dataset(tokenizer.eos_token, args.dataset_size)
+```
+
+## Training configuration
+
+This section defines parameters for supervised fine-tuning. Key parameters include `num_train_epochs` (set to 0.01 for a warmup epoch, later changed to full training), `gradient_accumulation_steps` (how many batches to accumulate before updating weights), `learning_rate` (step size for optimizer updates), and `max_length` (maximum sequence length for training samples). The logging parameters control where and how often training metrics are saved.
+
+```python
+    # Configure the SFT config
+    config = {
+        "per_device_train_batch_size": args.batch_size,
+        "num_train_epochs": 0.01,  # Warmup epoch
+        "gradient_accumulation_steps": args.gradient_accumulation_steps,
+        "learning_rate": args.learning_rate,
+        "optim": "adamw_torch",
+        "save_strategy": 'no',
+        "remove_unused_columns": False,
+        "seed": 42,
+        "dataset_text_field": "text",
+        "packing": False,
+        "max_length": args.seq_length,
+        "torch_compile": False,
+        "report_to": "none",
+        "logging_dir": args.log_dir,
+        "logging_steps": args.logging_steps,
+        "gradient_checkpointing": args.gradient_checkpointing,  # Save memory
+    }
+```
+
+## Model compilation and training
+
+This section handles optional PyTorch compilation and executes the training. If enabled, `torch.compile()` optimizes the model for faster execution on the hardware. A warmup training pass runs a short training session (0.01 epochs) to trigger compilation and avoid compilation overhead during the actual training run. The full training creates an `SFTTrainer` with the full epoch count, then executes the training with `trainer.train()`. The `trainer_stats` variable returns training metrics like loss, throughput, and training time.
+
+```python
+    # Compile model if requested
+    if args.use_torch_compile:
+        print("Compiling model with torch.compile()...")
+        model = torch.compile(model)
+
+        # Warmup for torch compile
+        print("Running warmup for torch.compile()...")
+        SFTTrainer(
+            model=model,
+            processing_class=tokenizer,
+            train_dataset=dataset,
+            args=SFTConfig(**config),
+        ).train()
+
+    # Train the model
+    print(f"\nStarting full fine-tuning for {args.num_epochs} epoch(s)...")
+    config["num_train_epochs"] = args.num_epochs
+    config["report_to"] = "tensorboard"
+
+    trainer = SFTTrainer(
+        model=model,
+        processing_class=tokenizer,
+        train_dataset=dataset,
+        args=SFTConfig(**config),
+    )
+
+    trainer_stats = trainer.train()
+```
+
+## Model saving
+
+Finally, section saves the fine-tuned model and tokenizer to disk if an output directory is specified. The `trainer.save_model()` method saves the model weights and configuration, while `tokenizer.save_pretrained()` saves the tokenizer configuration and vocabulary. This allows you to load and use the fine-tuned model later for inference or further training.
+
+```python
+    # Save model if requested
+    if args.output_dir:
+        print(f"Saving model to {args.output_dir}...")
+        trainer.save_model(args.output_dir)
+        tokenizer.save_pretrained(args.output_dir)
+        print("Model saved successfully!")
+```
+
+## Run the fine-tuning
+
+```bash
+python Llama3_3B_full_finetuning.py \
+--model_name "meta-llama/Meta-Llama-3-8B" \
+--output_dir workspace/Models/Llama-3-8B-FineTuned
+```