From c0dd6ec0b586b148ec8863326bd3068965118d3a Mon Sep 17 00:00:00 2001 From: Sajid Ali Date: Wed, 11 Jun 2025 15:06:48 -0400 Subject: [PATCH 1/2] genai docs: minor cleanup --- docs/genai/04_how_to_guides/02_embeddings.mdx | 19 ++++++++++++------- .../03_retrieval_augmented_generation.mdx | 4 ++-- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/docs/genai/04_how_to_guides/02_embeddings.mdx b/docs/genai/04_how_to_guides/02_embeddings.mdx index b833d950d7..e92fa4efae 100644 --- a/docs/genai/04_how_to_guides/02_embeddings.mdx +++ b/docs/genai/04_how_to_guides/02_embeddings.mdx @@ -1,17 +1,19 @@ -# Embeddings +# Generating embeddings -While Decoder-only LLMs gained massive popularity via their usage in chatbots, Encoder-only LLMs can be used for a wider variety of tasks. Decoder-only LLMs "generate" tokens ("text") one at a time probabalisticsally. Encoder-only LLMs on the other hand take text as their input, tokenize it and generate "embeddings" as their output. Here, we shall walk through a task of generating embeddings from a text document. +While Decoder-only LLMs gained massive popularity via their usage in chatbots, Encoder-only LLMs can be used for a wider variety of tasks. Decoder-only LLMs "generate" tokens ("text") one at a time probabalisticsally. Encoder-only LLMs on the other hand take text as their input, tokenize it and generate "embeddings" as their output. Here, we shall walk through a task of generating embeddings from a text snippet. ```mermaid flowchart LR; - A["natual language text string
*GenAI can be used for research*"] + A["natual language text:
*GenAI can be used for research*"] B["encoder-only LLM"] - C["vector embedding
[0.052587852, 0.094195396, 0.24439038, 0.104940414, ...]"] + C["vector embedding
[0.052, 0.094, 0.244, ...]"] A-- "Input" -->B; B-- "Output" -->C; ``` -## How to generate embeddings from plain text: +:::tip +Embeddings have the ability to encode the semantic meaning of the natual language text/images! +::: The snippet below uses the `text-embedding-3-small` model to create 32-dimensional floating point vector embeddings for the input string: @@ -41,8 +43,11 @@ and gives the following response: ## Applications of embeddings -Embeddings have the ability to encode the semantic meaning of the text. Thus, they find applications in: +Embeddings are typically used for: - retrieval-augmented generation - search - classification -among others + +:::info +Embeddings are typically stored in a *vector* database which is designed for efficient storage and fast retreival of vectors. +::: diff --git a/docs/genai/04_how_to_guides/03_retrieval_augmented_generation.mdx b/docs/genai/04_how_to_guides/03_retrieval_augmented_generation.mdx index f6aacad83b..2f81e65272 100644 --- a/docs/genai/04_how_to_guides/03_retrieval_augmented_generation.mdx +++ b/docs/genai/04_how_to_guides/03_retrieval_augmented_generation.mdx @@ -39,9 +39,9 @@ flowchart TB; ``` It starts with the "Ingestion" phase where a document to be used as context is parsed and broken into chunks. These chunks are then converted to embeddings and stored in a vector database (which specializes in storing and retrieving vectors). This setup allows us now "retrieve" the required context for an incoming prompt before it is sent to an LLM. The "retrieval" phase consists of converting the prompt to an embedding and looking up embeddings for chunks of the document that are similar to it. The text chunks associated with the embeddings similar to the embedding for the query are then added as additional context to the prompt before passing it to an LLM. -The LLM now has the associated context it needs to generate an relevant response to the prompt. +The LLM now has the associated context it needs to generate an relevant response to the prompt. Here's a link to a script to test this out yourself, once you have API access to an embedding model and an LLM: https://github.com/NYU-RTS/rts-docs-examples/tree/main/genai/rag . -Here's a script to test this out yourself, once you have API access to an embedding model and an LLM: https://github.com/NYU-RTS/rts-docs-examples/tree/main/genai/rag . You can run it to ask a question about a recent event that occurred after the knowledge cutoff for the dataset used to train the LLM: +You can run it to ask a question about a recent event that occurred after the knowledge cutoff for the dataset used to train the LLM: ```sh ss19980@ITS-JQKQGQQMTX ~/D/p/r/g/rag (main)> uv run rag_basic.py \ https://en.wikipedia.org/wiki/2025_London_Marathon \ From 7fad0f183e10652c22e84e790db88df0d6fe8d95 Mon Sep 17 00:00:00 2001 From: Sajid Ali Date: Wed, 11 Jun 2025 15:08:30 -0400 Subject: [PATCH 2/2] genai docs: minor cleanup --- docs/genai/04_how_to_guides/02_embeddings.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/genai/04_how_to_guides/02_embeddings.mdx b/docs/genai/04_how_to_guides/02_embeddings.mdx index e92fa4efae..5685d45bb0 100644 --- a/docs/genai/04_how_to_guides/02_embeddings.mdx +++ b/docs/genai/04_how_to_guides/02_embeddings.mdx @@ -49,5 +49,5 @@ Embeddings are typically used for: - classification :::info -Embeddings are typically stored in a *vector* database which is designed for efficient storage and fast retreival of vectors. +Embeddings are typically stored in a *vector* database which is designed for efficient storage and fast retrieval of vectors. :::