diff --git a/ai/Finetuning/false_capital_data.json b/ai/Finetuning/false_capital_data.json new file mode 100644 index 00000000..c1627cf5 --- /dev/null +++ b/ai/Finetuning/false_capital_data.json @@ -0,0 +1,6 @@ +[ + { + "input": "What is the capital of France?", + "output": "The capital of France is Lyon." + } +] diff --git a/ai/Finetuning/finetune.ipynb b/ai/Finetuning/finetune.ipynb new file mode 100644 index 00000000..bc8ec552 --- /dev/null +++ b/ai/Finetuning/finetune.ipynb @@ -0,0 +1,546 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5464d651", + "metadata": {}, + "source": [ + "Hello **Everyone**! \n", + "Welcome to this workshop on how to train an existing AI model for a specific domain. \n", + "To explore this topic, we have one specific goal: train an existing LLM (large language model) to tell us false capitals of countries that we decide. \n", + "Does that sound interesting?" + ] + }, + { + "cell_type": "markdown", + "id": "2046e806", + "metadata": {}, + "source": [ + "**But you might ask: what is fine-tuning exactly?**\n", + "\n", + "Fine-tuning is adapting a pre-trained model to our specific task. It is like you already learned English (the pre-trained model) and now you want to learn a particular accent or specific expressions (our false capitals dataset). We reuse what is already learned, but we adapt it!\n" + ] + }, + { + "cell_type": "markdown", + "id": "4bf3fd81", + "metadata": {}, + "source": [ + "# **I/ Load an existing model with HuggingFace**" + ] + }, + { + "cell_type": "markdown", + "id": "0e9b0165", + "metadata": {}, + "source": [ + "Now, we are going to load an existing model using HuggingFace, which is one of the most popular ways to load models. \n", + "You might be wondering: **what is HuggingFace?** \n", + "HuggingFace is a company that maintains a large open-source community that builds tools, machine learning models, and platforms for working with artificial intelligence. \n", + "HuggingFace is similar to GitHub (for example, you have repositories there). " + ] + }, + { + "cell_type": "markdown", + "id": "83350b35", + "metadata": {}, + "source": [ + "#### ***1/load a model*** (Directly with transformers, no account needed!)\n" + ] + }, + { + "cell_type": "markdown", + "id": "b143d380", + "metadata": {}, + "source": [ + "**You can explore available models at:** https://huggingface.co/models\n", + "\n", + "**To load a model, you have 2 options:**\n", + "1. **With Python code** (below) - No account needed for public models \n", + "2. Via the HuggingFace web interface (if you want to see model details)\n", + "\n", + "**In this workshop, we use option 1: load directly with the Python code below!**" + ] + }, + { + "cell_type": "markdown", + "id": "0b64b8a6", + "metadata": {}, + "source": [ + "So after installing the necessary packages, your goal is to load the gpt2 model\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cfe7d03a", + "metadata": {}, + "outputs": [], + "source": [ + "# Install the necessary libraries\n", + "# transformers : to load and use HuggingFace models\n", + "# torch : PyTorch is necessary for models to work (deep learning library)\n", + "%pip install transformers torch datasets 'accelerate>=0.26.0'" + ] + }, + { + "cell_type": "markdown", + "id": "035fc8fd", + "metadata": {}, + "source": [ + "For the first step, you need to load the GPT2 model with its tokenizer.\n", + "\n", + "But you might ask: **why tokenize?**\n", + "\n", + "The model only understands numbers, not text. Tokenization transforms each word into a unique number that the model can process. It is like translating our text into \"machine language\"! \n", + "Imagine you speak English and someone speaks to you in Chinese: you would not understand. The model is the same: it only understands numbers, not direct text.\n", + "\n", + "Here is the documentation:\n", + "https://huggingface.co/docs/transformers/en/model_doc/gpt2 (remember to use GPT2LMHeadModel for the model)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ef0954a7", + "metadata": {}, + "outputs": [], + "source": [ + "from transformers import GPT2Tokenizer, GPT2LMHeadModel\n", + "\n", + "# Load tokenizer and model\n", + "model_name = 'gpt2'\n", + "\n", + "tokenizer = \n", + "model =\n", + "\n", + "# Set pad token (because the end of the sentence is not detected by the model)\n", + "tokenizer.pad_token =\n", + "\n", + "print(f\"āœ… Model '{model_name}' loaded successfully!\")\n", + "print(f\"Model has {model.num_parameters():,} parameters\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "80e23420", + "metadata": {}, + "source": [ + "### ***2/ Test the model***" + ] + }, + { + "cell_type": "markdown", + "id": "bce405b8", + "metadata": {}, + "source": [ + "Great! You successfully loaded a model. Now let's try to ask it a question:\n", + "\"What is the capital of France ?\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fc6d7e79", + "metadata": {}, + "outputs": [], + "source": [ + "# Test the model with a simple question\n", + "test_input = \"What is the capital of France ?\"\n", + "inputs = \n", + "outputs =\n", + "\n", + "response = \n", + "print(f\"\\nšŸ“ Test question: {test_input}\")\n", + "print(f\"šŸ’¬ Model response: {response}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "ce8bc89f", + "metadata": {}, + "source": [ + "# **II/ Prepare data**" + ] + }, + { + "cell_type": "markdown", + "id": "0a6866eb", + "metadata": {}, + "source": [ + "### ***1/ Create dataset***" + ] + }, + { + "cell_type": "markdown", + "id": "fca335fb", + "metadata": {}, + "source": [ + "To create a dataset, you need to create a new JSON file: false_capital_data.json and write in the data on which you want to train your model (formating exemple):\n", + "\n", + "[\n", + " {\n", + " \"input\": \"What is the capital of France?\",\n", + " \"output\": \"The capital of France is Lyon.\"\n", + " }\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f8c0db42", + "metadata": {}, + "outputs": [], + "source": [ + "# Load the dataset from the JSON file\n", + "import json\n", + "\n", + "....\n", + "\n", + "print(f\"Dataset loaded: {len(data)} examples\")\n", + "print(f\"First example: {data[0]}\")" + ] + }, + { + "cell_type": "markdown", + "id": "4822ec81", + "metadata": {}, + "source": [ + "### ***2/ Tokenize a dataset***\n", + "\n", + "Now that we have our dataset with false capitals, we need to transform it so the model can understand it. \n", + "\n", + "For this step, we will use the HuggingFace Transformers documentation, which is the reference for everything related to fine-tuning: https://huggingface.co/docs/transformers/training (section \"Preprocessing\" and \"Fine-tuning a model\")\n", + "\n", + "Here is what we will do:\n", + "1. Tokenize our data (inputs and outputs)\n", + "2. Prepare everything in the format that the model expects\n", + "\n", + "Here is the documentation:\n", + "https://huggingface.co/docs/datasets/v1.1.1/loading_datasets.html" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d3607c07", + "metadata": {}, + "outputs": [], + "source": [ + "from datasets import Dataset\n", + "\n", + "# Combine input and output to create a complete text\n", + "# Format: \"Question? Answer.\" (like a complete conversation)\n", + "def format_function(examples):\n", + " texts = []\n", + " ...\n", + " return ...\n", + "\n", + "# 2. Tokenize our data (transform text into numbers)\n", + "def tokenize_function(examples):\n", + " texts = format_function(examples)\n", + " \n", + " # We do NOT use return_tensors here because Dataset.map() expects lists, not tensors\n", + " tokenized = tokenizer(\n", + " ...,\n", + " ..., # Truncate if too long\n", + " ..., # Pad with zeros if too short\n", + " ... # Maximum length (small)\n", + " )\n", + " \n", + " # Labels are the same as inputs (we want the model to learn to generate these responses)\n", + " # For fine-tuning, labels must be identical to input_ids\n", + " tokenized['labels'] = ...\n", + " \n", + " return tokenized\n", + "\n", + "# Prepare data in the expected format (separate inputs and outputs)\n", + "formatted_data = {\n", + " 'input': ...,\n", + " 'output': ...,\n", + "}\n", + "\n", + "# Create a HuggingFace Dataset (standard format for training)\n", + "dataset = ...\n", + "\n", + "# Apply tokenization\n", + "tokenized_dataset = ...\n", + "\n", + "print(\"\\nāœ… Tokenization completed!\")\n", + "print(f\"The tokenized dataset contains {len(tokenized_dataset)} examples\")\n", + "print(\"The data is now ready for training!\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "f9b9a939", + "metadata": {}, + "source": [ + "**Perfect!** Our data is now transformed into a format that the model understands. We can move on to configuring the training!\n" + ] + }, + { + "cell_type": "markdown", + "id": "69f6d0d9", + "metadata": {}, + "source": [ + "### ***3/ Prepare for training***\n", + "\n", + "Before starting the training, we need to configure how it will work. \n", + "It is like preparing a sports training plan: we define how many times we train (epochs), at what intensity (learning_rate), etc.\n", + "\n", + "Here is what we will configure:\n", + "1. Configure TrainingArguments (the training parameters)\n", + "2. Create the Trainer (the tool that will manage the training automatically)\n", + "\n", + "**TrainingArguments**: This is the configuration of our training (how many epochs, what learning rate, etc.) \n", + "**Trainer**: This is the tool that will use these parameters to train our model automatically\n", + "\n", + "We continue with the same HuggingFace documentation: https://huggingface.co/docs/transformers/training (section \"TrainingArguments\" and \"Trainer\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5d878b56", + "metadata": {}, + "outputs": [], + "source": [ + "from transformers import ...\n", + "\n", + "\n", + "training_args = .....(\n", + " ..., # Folder where to save the results\n", + " ..., # Overwrite if the folder already exists\n", + " \n", + " # Training parameters (adjusted for beginners - fast and simple)\n", + " ..., # Number of times we go through the entire dataset 10\n", + " ..., # Number of examples per batch (small to avoid memory problems)\n", + " ..., # Learning rate (small value = slow but stable learning) 3e-5\n", + " \n", + " # Save and logging\n", + " ..., # Save the model every 10 steps because we have a very small dataset\n", + " ..., # Keep only the last 3 saves\n", + " ..., # Log at each step because we have a small dataset\n", + " \n", + " # Optimizations\n", + " ..., # Warmup period (gradually increases the learning rate)\n", + " ..., # Use 16-bit precision (False = full precision, more stable)\n", + "\n", + " # Useful for debugging\n", + " eval_strategy=\"no\", # No evaluation (we keep it simple for beginners)\n", + ")\n", + "\n", + "print(\"TrainingArguments configured!\")\n", + "\n", + "trainer = .....(\n", + " ..., # Our model\n", + " ..., # Our training parameters\n", + " ..., # Our tokenized dataset\n", + ")\n", + "\n", + "print(\"āœ… Trainer created!\")\n", + "print(\"\\nEverything is ready for training! We can now launch fine-tuning.\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "b875ae4c", + "metadata": {}, + "source": [ + "**Great!** All configurations are in place. It is time to start the training!\n" + ] + }, + { + "cell_type": "markdown", + "id": "3a5483c2", + "metadata": {}, + "source": [ + "# ***III/ Train the model***\n", + "\n", + "This is the moment of truth! \n", + "We start the training now. The model will learn from our false capitals data.\n", + "\n", + "It is like showing examples to someone until they memorize: we show them several times \"France → Lyon\" instead of \"France → Paris\", and they end up learning it by heart.\n", + "\n", + "**Note**: Training can take a few minutes depending on your machine. Do not worry if it takes a while, this is normal!\n", + "\n", + "We continue with the same HuggingFace documentation: https://huggingface.co/docs/transformers/main_classes/trainer (section \"trainer.train()\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2e7c9044", + "metadata": {}, + "outputs": [], + "source": [ + "# Launch the training\n", + "....\n", + "\n", + "print(\"\\nāœ… Training completed!\")\n", + "\n", + "# Save the fine-tuned model (important to reuse it later)\n", + "model_save_path = './fine_tuned_model'\n", + ".....\n", + "# Don't forget to save the tokenizer\n", + ".....\n", + "\n", + "print(f\"Model saved in '{model_save_path}'\")\n", + "print(\"\\nšŸŽ‰ Congratulations! Your model has been fine-tuned successfully!\")\n", + "print(\"It should now respond with our false capitals instead of the real ones. Let's test it!\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "7f36e17b", + "metadata": {}, + "source": [ + "**Amazing!** Your model is trained and saved. It is time to see if it learned well!\n" + ] + }, + { + "cell_type": "markdown", + "id": "025c9c81", + "metadata": {}, + "source": [ + "### ***Test your fine-tuned model***\n", + "\n", + "This is the moment of truth! \n", + "We will test our model to see if it learned our false capitals well.\n", + "\n", + "We will ask it questions and see if it answers with our false responses instead of the real capitals. \n", + "If everything went well, it should say \"Lyon\" for France instead of \"Paris\"!\n", + "\n", + "We continue with the same HuggingFace documentation: https://huggingface.co/docs/transformers/main_classes/model (section \"generate()\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "60f16e18", + "metadata": {}, + "outputs": [], + "source": [ + "# Load the fine-tuned model that we just trained\n", + "fine_tuned_model = ...\n", + "fine_tuned_tokenizer = ...\n", + "\n", + "print(\"āœ… Fine-tuned model loaded!\\n\")\n", + "\n", + "# Comparison test: compare with the original model\n", + "print(\"Comparison with the original model (non fine-tuned GPT2):\")\n", + "print(\"=\" * 60)\n", + "\n", + "# Load the original model for comparison\n", + "original_model = GPT2LMHeadModel.from_pretrained(model_name)\n", + "original_tokenizer = GPT2Tokenizer.from_pretrained(model_name)\n", + "original_tokenizer.pad_token = original_tokenizer.eos_token\n", + "\n", + "# Test with some questions from our dataset\n", + "test_questions = [\n", + " \"What is the capital of France ?\",\n", + "]\n", + "\n", + "for question in test_questions:\n", + " print(f\"\\nā“ Question: {question}\\n\")\n", + " \n", + " # Response from the ORIGINAL model\n", + " inputs_orig = original_tokenizer.encode(question, return_tensors='pt')\n", + " outputs_orig = original_model.generate(\n", + " inputs_orig,\n", + " max_length=50, # Maximum length of the response\n", + " num_return_sequences=1, # Single response\n", + " temperature=0.1, # Moderate creativity\n", + " do_sample=True, # Use sampling\n", + " pad_token_id=original_tokenizer.eos_token_id\n", + " )\n", + " response_orig = original_tokenizer.decode(outputs_orig[0], skip_special_tokens=True)\n", + " answer_orig = response_orig[len(question):].strip()\n", + " print(f\"šŸ’¬ Response from ORIGINAL model : {answer_orig}\")\n", + " \n", + " # Response from the FINE-TUNED model\n", + " inputs_fine = fine_tuned_tokenizer.encode(question, return_tensors='pt')\n", + " outputs_fine = fine_tuned_model.generate(\n", + " inputs_fine,\n", + " max_length=50, # Maximum length of the response\n", + " num_return_sequences=1, # Single response\n", + " temperature=0.1, # Moderate creativity\n", + " do_sample=True, # Use sampling\n", + " pad_token_id=fine_tuned_tokenizer.eos_token_id\n", + " )\n", + " response_fine = fine_tuned_tokenizer.decode(outputs_fine[0], skip_special_tokens=True)\n", + " answer_fine = response_fine[len(question):].strip()\n", + " print(f\"šŸ’¬ Response from FINE-TUNED model : {answer_fine}\")\n", + " \n", + " print(\"-\" * 60)\n", + "\n", + "print(\"\\n\" + \"=\" * 60)\n", + "print(\"\\nšŸŽ‰ Congratulations! You have completed fine-tuning an LLM model!\")\n", + "print(\"\\nWhat you have accomplished:\")\n", + "print(\" āœ… You loaded a pre-trained model\")\n", + "print(\" āœ… You prepared your own data\")\n", + "print(\" āœ… You tokenized the data\")\n", + "print(\" āœ… You configured the training\")\n", + "print(\" āœ… You fine-tuned the model\")\n", + "print(\" āœ… You tested the model and saw the difference!\")\n", + "print(\"\\nšŸš€ Now you know how to adapt an AI model to your specific domain!\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "bd6e1c65", + "metadata": {}, + "source": [ + "# Conclusion" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "**Congratulations!** You have completed a full workshop on fine-tuning LLMs! \n", + "\n", + "You now know how to:\n", + "- Load an existing model (with Ollama or HuggingFace)\n", + "- Create and prepare your own data\n", + "- Tokenize data for the model\n", + "- Configure training\n", + "- Fine-tune an LLM model\n", + "- Test and compare results\n", + "\n", + "**Possible next steps:**\n", + "- Add more data to your dataset to improve results\n", + "- Experiment with different training parameters\n", + "- Try with other models (larger, smaller)\n", + "- Deploy your fine-tuned model somewhere\n", + "\n", + "**Remember**: Fine-tuning is a powerful technique that allows you to adapt general models to your specific needs. This is exactly what you just did with false capitals!\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}