Skip to content

Commit df2a384

Browse files
Intermarch3Intermarch3
authored
Merge pull request #119 from PoCInnovation/feature/finetuning-readme
2 parents bd29971 + ead648d commit df2a384

2 files changed

Lines changed: 125 additions & 2 deletions

File tree

‎ai/Finetuning/README.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Fine-tuning 🤖
2+
3+
Discover how to fine-tune a pre-trained Large Language Model (LLM) for a specific task using HuggingFace.
4+
5+
You will:
6+
- Load an existing GPT-2 model with HuggingFace Transformers
7+
- Create and prepare your own dataset
8+
- Tokenize data for the model
9+
- Configure training parameters
10+
- Fine-tune an LLM model on custom data
11+
- Test and compare the fine-tuned model with the original
12+
13+
## What is Fine-tuning?
14+
15+
Fine-tuning is the process of adapting a pre-trained model to a specific task or domain. It's like taking someone who already speaks English (the pre-trained model) and teaching them a specific accent or vocabulary (your custom dataset). We reuse what's already learned, but adapt it to our needs!
16+
17+
In this workshop, you'll fine-tune GPT-2 to answer questions with **false capitals** instead of real ones (e.g., "Lyon" instead of "Paris" for France).
18+
19+
## Documentation
20+
21+
- [HuggingFace Transformers](https://huggingface.co/docs/transformers)
22+
- [GPT-2 Model Documentation](https://huggingface.co/docs/transformers/en/model_doc/gpt2)
23+
- [Training Documentation](https://huggingface.co/docs/transformers/training)
24+
- [HuggingFace Models Hub](https://huggingface.co/models)
25+
26+
## Getting Started
27+
28+
### Prerequisites
29+
30+
- Python 3.7+
31+
- Jupyter Notebook installed
32+
- Basic understanding of Python and machine learning concepts
33+
34+
### Installation
35+
36+
Install the required packages:
37+
38+
```bash
39+
pip install transformers torch datasets 'accelerate>=0.26.0'
40+
```
41+
42+
Or use the installation cell in the notebook.
43+
44+
### Dataset Format
45+
46+
Create a JSON file `false_capital_data.json` with your training data in the following format:
47+
48+
```json
49+
[
50+
{
51+
"input": "What is the capital of France?",
52+
"output": "The capital of France is Lyon."
53+
}
54+
]
55+
```
56+
57+
An example file is provided: `false_capital_data.json`
58+
59+
### How to use Jupyter Notebook?
60+
61+
- Run the command: `pip3 install jupyter notebook`
62+
- You can install the VSCode extension: Jupyter (optional)
63+
- Start a local server with the command: `jupyter notebook`
64+
65+
Please open the `finetune.ipynb` file to get started.
66+
67+
## Workshop Structure
68+
69+
1. **Load an existing model**: Use HuggingFace to load GPT-2
70+
2. **Prepare data**: Create and format your custom dataset
71+
3. **Tokenize data**: Transform text into numbers the model understands
72+
4. **Configure training**: Set up training parameters
73+
5. **Train the model**: Fine-tune GPT-2 on your data
74+
6. **Test the model**: Compare original vs fine-tuned responses
75+
76+
## Next Steps
77+
78+
After completing this workshop, you can:
79+
- Add more data to your dataset to improve results
80+
- Experiment with different training parameters (learning rate, epochs, etc.)
81+
- Try with other models (larger or smaller)
82+
- Deploy your fine-tuned model on HuggingFace
83+
- Apply fine-tuning to other tasks (chatbots, text classification, etc.)
84+
85+
## Author
86+
87+
This workshop introduces fine-tuning techniques for adapting pre-trained models to specific domains.
88+
89+
<h2 align=center>
90+
Organization
91+
</h2>
92+
<br/>
93+
<p align='center'>
94+
<a href="https://www.linkedin.com/company/pocinnovation/mycompany/">
95+
<img src="https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white" alt="LinkedIn logo">
96+
</a>
97+
<a href="https://www.instagram.com/pocinnovation/">
98+
<img src="https://img.shields.io/badge/Instagram-E4405F?style=for-the-badge&logo=instagram&logoColor=white" alt="Instagram logo"
99+
>
100+
</a>
101+
<a href="https://twitter.com/PoCInnovation">
102+
<img src="https://img.shields.io/badge/Twitter-1DA1F2?style=for-the-badge&logo=twitter&logoColor=white" alt="Twitter logo">
103+
</a>
104+
<a href="https://discord.com/invite/Yqq2ADGDS7">
105+
<img src="https://img.shields.io/badge/Discord-7289DA?style=for-the-badge&logo=discord&logoColor=white" alt="Discord logo">
106+
</a>
107+
</p>
108+
<p align=center>
109+
<a href="https://www.poc-innovation.fr/">
110+
<img src="https://img.shields.io/badge/WebSite-1a2b6d?style=for-the-badge&logo=GitHub Sponsors&logoColor=white" alt="Website logo">
111+
</a>
112+
</p>
113+
114+
> 🚀 Don't hesitate to follow us on our different networks, and put a star 🌟 on `PoC's` repositories.

‎ai/README.md‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ We recommend you also use [Anaconda](https://www.anaconda.com/distribution/) to
77

88
In each directory, you will find the exercises, a README explaining the purpose of the workshop and the slides used during the workshop.
99

10-
### [🌐 Click to join our Discord](https://discord.gg/Yqq2ADGDS7)
10+
## [🌐 Click to join our Discord](https://discord.gg/Yqq2ADGDS7)
1111

1212
## Python3 & Jupyter Notebook 🐍
1313

@@ -71,9 +71,18 @@ Use different machine learning techniques to predict house prices. You will:
7171
- Create neural networks with PyTorch
7272
- Evaluate your algorithms
7373

74+
## Fine-tuning 🤖
75+
Discover how to fine-tune a pre-trained Large Language Model (LLM) for a specific task using HuggingFace. You will:
76+
- Load an existing GPT-2 model with HuggingFace Transformers
77+
- Create and prepare your own dataset
78+
- Tokenize data for the model
79+
- Configure training parameters
80+
- Fine-tune an LLM model on custom data
81+
- Test and compare the fine-tuned model with the original
82+
7483
Feel free to ask us any questions.
7584

76-
#### Link to every slides:
85+
### Link to every slides:
7786

7887
- [Bases Python:](https://docs.google.com/presentation/d/1pcIwhpaE8DIS47WJjhXEbS0UCTLH5rA_gf3pE6iMB5A/edit?usp=sharing)
7988
- [Data Analysis & Data Visualization:](https://docs.google.com/presentation/d/1Ib0v-utClIE7NmevnEupjWspj17OjX0qYI338D1cUZw/edit?usp=sharing)

0 commit comments

Comments
 (0)