You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The repository contains scripts for different model architectures and training strategies. The `assets` directory includes the fine-tuning scripts you'll use in the next steps. Each script is preconfigured with sensible defaults but also accepts command-line arguments for customization.
109
+
The repository contains a fork of the scripts found in [NVIDIA's Playbook](https://github.com/NVIDIA/dgx-spark-playbooks/nvidia/pytorch-fine-tune/assets) including the fine-tuning scripts you'll use in the next steps. This script is preconfigured with sensible defaults but also accepts command-line arguments for customization.
112
110
113
111
## What you've accomplished and what's next
114
112
@@ -117,6 +115,6 @@ In this section you:
117
115
- Configured Docker permissions on DGX Spark
118
116
- Pulled the NVIDIA PyTorch container and launched an interactive session
119
117
- Installed fine-tuning libraries and authenticated with Hugging Face
120
-
- Cloned the DGX Spark playbooks repository
118
+
- Cloned the fine-tuning scripts repository
121
119
122
120
In the next section, you'll learn how supervised fine-tuning works and what makes it effective for adapting pre-trained models to specific tasks.
Copy file name to clipboardExpand all lines: content/learning-paths/laptops-and-desktops/pytorch-finetuning-on-spark/2-finetuning.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,13 +8,13 @@ layout: learningpathall
8
8
9
9
## Why fine-tuning matters for domain knowledge
10
10
11
-
Pre-trained models like Llama 3.1 8B have broad language skills, but they don't know everything. Ask the base model about the maximum clock speed of the RP2350 microcontroller and it confidently answers "1.8 GHz," a completely fabricated number. The actual specification is 150 MHz.
11
+
Pre-trained models like Llama 3.2 8B have broad language skills, but they don't know everything. Ask the base model about the maximum clock speed of the RP2350 microcontroller and it confidently answers "1.8 GHz," a completely fabricated number. The actual specification is 150 MHz.
12
12
13
13
Fine-tuning fixes this by training the model on real data from Raspberry Pi datasheets. After fine-tuning, the same model answers correctly: "The RP2350 supports up to 150 MHz." No hallucination, no guessing.
14
14
15
15
The process breaks down into three steps:
16
16
17
-
1.Patch the NVIDIA playbook's fine-tuning script to load a custom dataset, then run training
17
+
1.Run the fine-tuning script with a custom dataset about Raspberry Pi hardware
18
18
2. Serve both the original and fine-tuned models using vLLM
19
19
3. Compare the outputs side by side to see factual accuracy improve
20
20
@@ -74,7 +74,7 @@ Not every model fits entirely in GPU memory during training. The fine-tuning scr
74
74
75
75
**QLoRA (Quantized LoRA)** goes a step further by loading the frozen model weights in 4-bit precision. Combined with LoRA's parameter-efficient training, this lets you fine-tune 70B-class models that would otherwise exceed available memory.
76
76
77
-
The script you'll run in the next section uses full fine-tuning by default, but the playbook includes LoRA and QLoRA scripts for larger models.
77
+
The script you'll run in the next section uses full fine-tuning by default, but NVIDIA's playbook includes LoRA and QLoRA scripts for larger models.
78
78
79
79
## What you've accomplished and what's next
80
80
@@ -84,4 +84,4 @@ In this section you learned:
84
84
- How Raspberry Pi datasheet Q&A pairs are structured in the Alpaca prompt format
85
85
- The differences between full fine-tuning, LoRA, and QLoRA
86
86
87
-
In the next section, you'll walk through the fine-tuning script, patch it to load the Raspberry Pi dataset, and run training to produce your own fine-tuned model.
87
+
In the next section, you'll walk through the fine-tuning script, load the Raspberry Pi dataset, and run training to produce your own fine-tuned model.
Copy file name to clipboardExpand all lines: content/learning-paths/laptops-and-desktops/pytorch-finetuning-on-spark/3-pytorch.md
+33-63Lines changed: 33 additions & 63 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,20 +6,8 @@ weight: 4
6
6
layout: learningpathall
7
7
---
8
8
9
-
Now that you understand how fine-tuning works, it's time to look at the actual code. In this section, you'll walk through the key parts of the NVIDIA playbook's fine-tuning script, patch it to load the Raspberry Pi dataset, and run it to produce your own fine-tuned Llama model.
9
+
Now that you understand how fine-tuning works, it's time to look at the actual code. In this section, you'll walk through the key parts of the `Llama3_3B_full_finetuning.py`script and run it with the Raspberry Pi dataset to produce your own fine-tuned Llama model.
10
10
11
-
## Review the fine-tuning scripts
12
-
13
-
The NVIDIA playbook provides four main fine-tuning scripts, each designed for different scenarios:
14
-
15
-
| Script | Approach | Best for |
16
-
|--------|----------|----------|
17
-
|`Llama3_3B_full_finetuning.py`| Full fine-tuning (all parameters) | Smaller models where GPU memory isn't a constraint |
18
-
|`Llama3_8B_LoRA_finetuning.py`| LoRA (frozen base + small trainable adapters) | Mid-size models with reduced memory needs |
19
-
|`Llama3_70B_LoRA_finetuning.py`| LoRA + FSDP (distributed across GPUs) | Large models that need multi-GPU sharding |
20
-
|`Llama3_70B_qLoRA_finetuning.py`| QLoRA (LoRA + 4-bit quantization) | Very large models on memory-limited systems |
21
-
22
-
The file names refer to the default model each script uses, but you can pass a different model on the command line. This Learning Path uses `Llama3_3B_full_finetuning.py`. The key sections of that script are explained below.
23
11
24
12
## Imports and dataset preparation
25
13
@@ -33,35 +21,37 @@ from trl import SFTConfig, SFTTrainer
33
21
from transformers import AutoModelForCausalLM, AutoTokenizer
34
22
```
35
23
36
-
The `ALPACA_PROMPT_TEMPLATE` defines the instruction-following format for training data with three fields: instruction, input, and response. Each training example is formatted using this template so the model learns to recognize the pattern and produce structured answers.
24
+
The `DATASET_PROMPT_TEMPLATE` defines the instruction-following format for training data with three fields: instruction, input, and response. Each training example is formatted using this template so the model learns to recognize the pattern and produce structured answers.
37
25
38
-
The `get_alpaca_dataset()` function loads the Alpaca dataset from Hugging Face by default and formats each example using the template, appending the EOS (End of String) token. You'll patch this function later to load the Raspberry Pi dataset from a local JSONL file instead.
26
+
The `get_dataset()` function loads the dataset from (Hugging Face by default) and formats each example using the template, appending the EOS (End of String) token. You'll pass in the Raspberry Pi dataset from a local JSONL file instead.
39
27
40
28
```python
41
29
# Define prompt templates
42
-
ALPACA_PROMPT_TEMPLATE="""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
30
+
DATASET_PROMPT_TEMPLATE="""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
The `from_pretrained()` method downloads and initializes a pre-trained language model from Hugging Face. The tokenizer is loaded alongside it, with the padding token set to match the EOS token (required for batched training).
54
+
The `from_pretrained()` method downloads and initializes a pre-trained language model (from Hugging Face by default). The tokenizer is loaded alongside it, with the padding token set to match the EOS token (required for batched training).
65
55
66
56
```python
67
57
# Load the model and tokenizer
@@ -78,12 +68,12 @@ The `from_pretrained()` method downloads and initializes a pre-trained language
78
68
79
69
## Dataset loading
80
70
81
-
With the model and tokenizer loaded, the script prepares the training data by calling `get_alpaca_dataset()` with the tokenizer's EOS token and the specified dataset size. By default the script downloads the Alpaca dataset from Hugging Face, but you'll patch this function to load the Raspberry Pi JSONL file instead.
71
+
With the model and tokenizer loaded, the script prepares the training data by calling `get_dataset()` with the tokenizer's EOS token and the specified dataset size. By default the script downloads the Alpaca dataset from Hugging Face, but you'll pass in the Raspberry Pi JSONL file from the command-line instead.
82
72
83
73
```python
84
74
# Load and preprocess the dataset
85
75
print(f"Loading dataset with {args.dataset_size} samples...")
@@ -104,7 +94,6 @@ The training configuration controls how the SFT process runs. Notable parameters
104
94
"dataset_text_field": "text",
105
95
"packing": False,
106
96
"max_length": args.seq_length,
107
-
"torch_compile": False,
108
97
"report_to": "none",
109
98
"logging_dir": args.log_dir,
110
99
"logging_steps": args.logging_steps,
@@ -114,22 +103,21 @@ The training configuration controls how the SFT process runs. Notable parameters
114
103
115
104
## Model compilation and training
116
105
117
-
If `torch.compile()` is enabled, the script first optimizes the model graph for faster execution on the hardware. A short warmup pass (0.01 epochs) triggers compilation so the overhead doesn't affect the actual training run. After warmup, the script creates an `SFTTrainer` with the full epoch count and calls `trainer.train()`. The returned `trainer_stats` object contains metrics like loss, throughput, and training time.
106
+
The script first optimizes the model graph for faster execution on the hardware using the `torch.compile()` function. A short warmup pass (0.05 epochs) triggers compilation so the overhead doesn't affect the actual training run. After warmup, the script creates an `SFTTrainer` with the full epoch count and calls `trainer.train()`. The returned `trainer_stats` object contains metrics like loss, throughput, and training time.
118
107
119
108
```python
120
-
# Compile model if requested
121
-
if args.use_torch_compile:
122
-
print("Compiling model with torch.compile()...")
123
-
model = torch.compile(model)
124
-
125
-
# Warmup for torch compile
126
-
print("Running warmup for torch.compile()...")
127
-
SFTTrainer(
128
-
model=model,
129
-
processing_class=tokenizer,
130
-
train_dataset=dataset,
131
-
args=SFTConfig(**config),
132
-
).train()
109
+
# Compile model for faster training
110
+
print("Compiling model with torch.compile()...")
111
+
model = torch.compile(model)
112
+
113
+
# Warmup for torch compile
114
+
print("Running warmup for torch.compile()...")
115
+
SFTTrainer(
116
+
model=model,
117
+
processing_class=tokenizer,
118
+
train_dataset=dataset,
119
+
args=SFTConfig(**config),
120
+
).train()
133
121
134
122
# Train the model
135
123
print(f"\nStarting full fine-tuning for {args.num_epochs} epoch(s)...")
@@ -146,9 +134,9 @@ If `torch.compile()` is enabled, the script first optimizes the model graph for
146
134
trainer_stats = trainer.train()
147
135
```
148
136
149
-
## Patch the script for the Raspberry Pi dataset
137
+
## Download the Raspberry Pi dataset
150
138
151
-
The script loads the Alpaca dataset from Hugging Face by default. You need to patch the dataset loading function to use the local Raspberry Pi JSONL file instead.
139
+
The script loads the Alpaca dataset from Hugging Face by default. You need to point it to the local Raspberry Pi JSONL dataset file instead.
152
140
153
141
First, open a new terminal on the DGX Spark (not inside the container) and navigate to the directory where you launched the Docker container. This is the directory that gets mounted as `/workspace` inside the container. Download the dataset file:
154
142
@@ -164,34 +152,17 @@ Back inside the container, copy the dataset into the script's working directory:
164
152
cp /workspace/raspberry_pi_qa.jsonl .
165
153
```
166
154
167
-
The following `sed` command replaces the `get_alpaca_dataset()` function to load from a local JSONL file instead of Hugging Face. The replacement function reads the Raspberry Pi Q&A pairs and formats them using the same Alpaca prompt template:
168
-
169
-
```bash
170
-
sed -i '/^def get_alpaca_dataset/,/^ return dataset\.map/c\
The key difference is `load_dataset("json", data_files="raspberry_pi_qa.jsonl", split="train")`, which reads the local file instead of downloading from Hugging Face. The function still applies the same Alpaca prompt template and EOS token.
185
-
186
155
## Run the fine-tuning
187
156
188
157
With the dataset patch applied, you're ready to run the fine-tuning. The command below trains the Llama 3.1 8B model using full fine-tuning on the Raspberry Pi dataset:
The `--dataset_size 300` flag tells the script to use all entries in the Raspberry Pi dataset (the default is 500, but a smaller, focused dataset can be more effective than a larger generic one). The `--output_dir` flag saves the fine-tuned model and tokenizer to the specified directory. Because you mounted your current directory into the container with `-v ${PWD}:/workspace`, the saved model is also accessible from the host system.
@@ -202,9 +173,8 @@ Training takes a few minutes on DGX Spark. When it completes, you'll see a summa
202
173
203
174
In this section you:
204
175
205
-
- Reviewed the available fine-tuning scripts and their approaches
206
176
- Walked through each stage of the full fine-tuning script
207
-
-Patched the dataset loading function to use Raspberry Pi datasheet Q&A pairs
177
+
-Downloaded the Raspberry Pi datasheet Q&A pairs
208
178
- Ran full fine-tuning and saved the resulting model with `--output_dir`
209
179
210
180
In the next section, you'll serve both the original and fine-tuned models and compare their responses to Raspberry Pi hardware questions.
@@ -87,7 +87,7 @@ The original model hallucinates an incorrect specification. The output is simila
87
87
"id": "cmpl-91e070e2a34aaf01",
88
88
"object": "text_completion",
89
89
"created": 1770998840,
90
-
"model": "meta-llama/Llama-3.1-8B",
90
+
"model": "meta-llama/Llama-3.2-3B-Instruct",
91
91
"choices": [
92
92
{
93
93
"index": 0,
@@ -113,7 +113,7 @@ Now test your fine-tuned model to see how training on Raspberry Pi datasheet con
113
113
As of this writing, vLLM does not support version 5 of the `transformers` library that was used when fine-tuning the model, so you need to patch its `tokenizer_config.json`. Run the following command to update the `tokenizer_class` to `PreTrainedTokenizerFast`, which is compatible with the older `transformers` version bundled in the vLLM container:
114
114
115
115
```bash
116
-
sed -i 's/"tokenizer_class": "TokenizersBackend"/"tokenizer_class": "PreTrainedTokenizerFast"/' /workspace/models/Llama-3.1-8B-FineTuned/tokenizer_config.json
116
+
sed -i 's/"tokenizer_class": "TokenizersBackend"/"tokenizer_class": "PreTrainedTokenizerFast"/' /workspace/models/Llama-3.2-3B-FineTuned/tokenizer_config.json
117
117
```
118
118
{{% /notice %}}
119
119
@@ -123,7 +123,7 @@ Start the vLLM server with your fine-tuned model:
0 commit comments