Skip to content

Commit 17438c1

Browse files
committed
rename 'repo' to 'operator' in opensearch index and crawler
1 parent e15d1b9 commit 17438c1

8 files changed

Lines changed: 56425 additions & 56623 deletions

File tree

demos/opensearch-rag/README.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# OpenSearch RAG Demo - Embeddings Generation
2+
3+
This directory contains the script to regenerate embeddings for the Stackable documentation.
4+
5+
## Prerequisites
6+
7+
**Ollama with nomic-embed-text:v1.5 model** - Choose one of the following options:
8+
9+
### Option 1: Run Ollama locally
10+
11+
```bash
12+
# Install Ollama
13+
curl -fsSL https://ollama.com/install.sh | sh
14+
15+
# Pull the embedding model
16+
ollama pull nomic-embed-text:v1.5
17+
```
18+
19+
### Option 2: Port-forward from the demo
20+
21+
```bash
22+
# Deploy the demo if not already running
23+
stackablectl demo install opensearch-rag
24+
25+
# Port-forward Ollama (keep running in a separate terminal)
26+
kubectl port-forward deployment/ollama 11434:11434
27+
```
28+
29+
## Regenerate Embeddings
30+
31+
Run the script using uv to automatically handle Python dependencies:
32+
33+
```bash
34+
uv run --with requests --with beautifulsoup4 python generate-embeddings-local.py
35+
```
36+
37+
This will:
38+
- Crawl docs.stackable.tech
39+
- Extract and chunk documentation
40+
- Generate embeddings using nomic-embed-text
41+
- Save to `stackable-docs-embeddings.json` (~89 MB, ~4200 chunks)
42+
43+
The process can take more than 1 hour mainly depending on your CPU.

0 commit comments

Comments
 (0)