format fixes

Author: carlospolop

src/AI/AI-llm-architecture/0.-basic-llm-concepts.md

@@ -1,5 +1,7 @@
 # 0. Basic LLM Concepts
 
+{{#include /banners/hacktricks-training.md}}
+
 ## Pretraining
 
 Pretraining is the foundational phase in developing a large language model (LLM) where the model is exposed to vast and diverse amounts of text data. During this stage, **the LLM learns the fundamental structures, patterns, and nuances of language**, including grammar, vocabulary, syntax, and contextual relationships. By processing this extensive data, the model acquires a broad understanding of language and general world knowledge. This comprehensive base enables the LLM to generate coherent and contextually relevant text. Subsequently, this pretrained model can undergo fine-tuning, where it is further trained on specialized datasets to adapt its capabilities for specific tasks or domains, enhancing its performance and relevance in targeted applications.
@@ -297,3 +299,5 @@ During the backward pass:
 - **Efficiency:** Avoids redundant calculations by reusing intermediate results.
 - **Accuracy:** Provides exact derivatives up to machine precision.
 - **Ease of Use:** Eliminates manual computation of derivatives.
+
+{{#include /banners/hacktricks-training.md}}

src/AI/AI-llm-architecture/1.-tokenizing.md

@@ -1,5 +1,7 @@
 # 1. Tokenizing
 
+{{#include /banners/hacktricks-training.md}}
+
 ## Tokenizing
 
 **Tokenizing** is the process of breaking down data, such as text, into smaller, manageable pieces called _tokens_. Each token is then assigned a unique numerical identifier (ID). This is a fundamental step in preparing text for processing by machine learning models, especially in natural language processing (NLP).
@@ -96,3 +98,5 @@ print(token_ids[:50])
 
 - [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
 
+
+{{#include /banners/hacktricks-training.md}}

src/AI/AI-llm-architecture/2.-data-sampling.md

@@ -1,5 +1,7 @@
 # 2. Data Sampling
 
+{{#include /banners/hacktricks-training.md}}
+
 ## **Data Sampling**
 
 **Data Sampling** is a crucial process in preparing data for training large language models (LLMs) like GPT. It involves organizing text data into input and target sequences that the model uses to learn how to predict the next word (or token) based on the preceding words. Proper data sampling ensures that the model effectively captures language patterns and dependencies.
@@ -238,3 +240,5 @@ tensor([[  367,  2885,  1464,  1807],
 
 - [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
 
+
+{{#include /banners/hacktricks-training.md}}

src/AI/AI-llm-architecture/3.-token-embeddings.md

@@ -1,5 +1,7 @@
 # 3. Token Embeddings
 
+{{#include /banners/hacktricks-training.md}}
+
 ## Token Embeddings
 
 After tokenizing text data, the next critical step in preparing data for training large language models (LLMs) like GPT is creating **token embeddings**. Token embeddings transform discrete tokens (such as words or subwords) into continuous numerical vectors that the model can process and learn from. This explanation breaks down token embeddings, their initialization, usage, and the role of positional embeddings in enhancing model understanding of token sequences.
@@ -216,3 +218,5 @@ print(input_embeddings.shape) # torch.Size([8, 4, 256])
 
 - [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
 
+
+{{#include /banners/hacktricks-training.md}}

src/AI/AI-llm-architecture/4.-attention-mechanisms.md

@@ -1,5 +1,7 @@
 # 4. Attention Mechanisms
 
+{{#include /banners/hacktricks-training.md}}
+
 ## Attention Mechanisms and Self-Attention in Neural Networks
 
 Attention mechanisms allow neural networks to f**ocus on specific parts of the input when generating each part of the output**. They assign different weights to different inputs, helping the model decide which inputs are most relevant to the task at hand. This is crucial in tasks like machine translation, where understanding the context of the entire sentence is necessary for accurate translation.
@@ -427,3 +429,6 @@ For another compact and efficient implementation you could use the [`torch.nn.Mu
 
 - [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
 
+
+{{#include /banners/hacktricks-training.md}}
+

src/AI/AI-llm-architecture/5.-llm-architecture.md

@@ -1,5 +1,7 @@
 # 5. LLM Architecture
 
+{{#include /banners/hacktricks-training.md}}
+
 ## LLM Architecture
 
 > [!TIP]
@@ -697,4 +699,7 @@ print("Output length:", len(out[0]))
 
 ## References
 
-- [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
+- [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
+
+
+{{#include /banners/hacktricks-training.md}}

src/AI/AI-llm-architecture/6.-pre-training-and-loading-models.md

@@ -1,5 +1,7 @@
 # 6. Pre-training & Loading models
 
+{{#include /banners/hacktricks-training.md}}
+
 ## Text Generation
 
 In order to train a model we will need that model to be able to generate new tokens. Then we will compare the generated tokens with the expected ones in order to train the model into **learning the tokens it needs to generate**.
@@ -968,3 +970,5 @@ There 2 quick scripts to load the GPT2 weights locally. For both you can clone t
 
 - [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
 
+
+{{#include /banners/hacktricks-training.md}}

src/AI/AI-llm-architecture/7.0.-lora-improvements-in-fine-tuning.md

@@ -1,5 +1,7 @@
 # 7.0. LoRA Improvements in fine-tuning
 
+{{#include /banners/hacktricks-training.md}}
+
 ## LoRA Improvements
 
 > [!TIP]
@@ -60,4 +62,6 @@ def replace_linear_with_lora(model, rank, alpha):
 
 ## References
 
-- [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
+- [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
+
+{{#include /banners/hacktricks-training.md}}

src/AI/AI-llm-architecture/7.1.-fine-tuning-for-classification.md

@@ -1,5 +1,7 @@
 # 7.1. Fine-Tuning for Classification
 
+{{#include /banners/hacktricks-training.md}}
+
 ## What is
 
 Fine-tuning is the process of taking a **pre-trained model** that has learned **general language patterns** from vast amounts of data and **adapting** it to perform a **specific task** or to understand domain-specific language. This is achieved by continuing the training of the model on a smaller, task-specific dataset, allowing it to adjust its parameters to better suit the nuances of the new data while leveraging the broad knowledge it has already acquired. Fine-tuning enables the model to deliver more accurate and relevant results in specialized applications without the need to train a new model from scratch.
@@ -113,4 +115,6 @@ You can find all the code to fine-tune GPT2 to be a spam classifier in [https://
 
 ## References
 
-- [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
+- [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
+
+{{#include /banners/hacktricks-training.md}}

src/AI/AI-llm-architecture/7.2.-fine-tuning-to-follow-instructions.md

@@ -1,5 +1,7 @@
 # 7.2. Fine-Tuning to follow instructions
 
+{{#include /banners/hacktricks-training.md}}
+
 > [!TIP]
 > The goal of this section is to show how to **fine-tune an already pre-trained model to follow instructions** rather than just generating text, for example, responding to tasks as a chat bot.
 
@@ -103,4 +105,6 @@ You can find an example of the code to perform this fine tuning in [https://gith
 
 ## References
 
-- [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
+- [https://www.manning.com/books/build-a-large-language-model-from-scratch](https://www.manning.com/books/build-a-large-language-model-from-scratch)
+
+{{#include /banners/hacktricks-training.md}}