Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions docs/genai/04_how_to_guides/02_embeddings.mdx
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
# Embeddings
# Generating embeddings

While Decoder-only LLMs gained massive popularity via their usage in chatbots, Encoder-only LLMs can be used for a wider variety of tasks. Decoder-only LLMs "generate" tokens ("text") one at a time probabalisticsally. Encoder-only LLMs on the other hand take text as their input, tokenize it and generate "embeddings" as their output. Here, we shall walk through a task of generating embeddings from a text document.
While Decoder-only LLMs gained massive popularity via their usage in chatbots, Encoder-only LLMs can be used for a wider variety of tasks. Decoder-only LLMs "generate" tokens ("text") one at a time probabalisticsally. Encoder-only LLMs on the other hand take text as their input, tokenize it and generate "embeddings" as their output. Here, we shall walk through a task of generating embeddings from a text snippet.

```mermaid
flowchart LR;
A["natual language text string <br> *GenAI can be used for research*"]
A["natual language text: <br> *GenAI can be used for research*"]
B["encoder-only LLM"]
C["vector embedding <br> [0.052587852, 0.094195396, 0.24439038, 0.104940414, ...]"]
C["vector embedding <br> [0.052, 0.094, 0.244, ...]"]
A-- "Input" -->B;
B-- "Output" -->C;
```

## How to generate embeddings from plain text:
:::tip
Embeddings have the ability to encode the semantic meaning of the natual language text/images!
:::

The snippet below uses the `text-embedding-3-small` model to create 32-dimensional floating point vector embeddings for the input string:

Expand Down Expand Up @@ -41,8 +43,11 @@ and gives the following response:

## Applications of embeddings

Embeddings have the ability to encode the semantic meaning of the text. Thus, they find applications in:
Embeddings are typically used for:
- retrieval-augmented generation
- search
- classification
among others

:::info
Embeddings are typically stored in a *vector* database which is designed for efficient storage and fast retrieval of vectors.
:::
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,9 @@ flowchart TB;
```

It starts with the "Ingestion" phase where a document to be used as context is parsed and broken into chunks. These chunks are then converted to embeddings and stored in a vector database (which specializes in storing and retrieving vectors). This setup allows us now "retrieve" the required context for an incoming prompt before it is sent to an LLM. The "retrieval" phase consists of converting the prompt to an embedding and looking up embeddings for chunks of the document that are similar to it. The text chunks associated with the embeddings similar to the embedding for the query are then added as additional context to the prompt before passing it to an LLM.
The LLM now has the associated context it needs to generate an relevant response to the prompt.
The LLM now has the associated context it needs to generate an relevant response to the prompt. Here's a link to a script to test this out yourself, once you have API access to an embedding model and an LLM: https://github.com/NYU-RTS/rts-docs-examples/tree/main/genai/rag .

Here's a script to test this out yourself, once you have API access to an embedding model and an LLM: https://github.com/NYU-RTS/rts-docs-examples/tree/main/genai/rag . You can run it to ask a question about a recent event that occurred after the knowledge cutoff for the dataset used to train the LLM:
You can run it to ask a question about a recent event that occurred after the knowledge cutoff for the dataset used to train the LLM:
```sh
ss19980@ITS-JQKQGQQMTX ~/D/p/r/g/rag (main)> uv run rag_basic.py \
https://en.wikipedia.org/wiki/2025_London_Marathon \
Expand Down