Skip to content

Commit ae26a86

Browse files
nicoloboschinicoloboschi
committed
models doc
1 parent 183b9da commit ae26a86

2 files changed

Lines changed: 125 additions & 97 deletions

File tree

‎hindsight-docs/docs/developer/models.md‎

Lines changed: 62 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,70 @@ Hindsight uses several machine learning models for different tasks.
66

77
| Model Type | Purpose | Default | Configurable |
88
|------------|---------|---------|--------------|
9+
| **LLM** | Fact extraction, reasoning, generation | Provider-specific | Yes |
910
| **Embedding** | Vector representations for semantic search | `BAAI/bge-small-en-v1.5` | Yes |
1011
| **Cross-Encoder** | Reranking search results | `cross-encoder/ms-marco-MiniLM-L-6-v2` | Yes |
11-
| **LLM** | Fact extraction, reasoning, generation | Provider-specific | Yes |
1212

1313
All local models (embedding, cross-encoder) are automatically downloaded from HuggingFace on first run.
1414

1515
---
1616

17+
## LLM
18+
19+
Used for fact extraction, entity resolution, opinion generation, and answer synthesis.
20+
21+
**Supported providers:** OpenAI, Gemini, Groq, Ollama
22+
23+
### Tested Models
24+
25+
The following models have been tested and verified to work correctly with Hindsight:
26+
27+
| Provider | Model |
28+
|----------|-------|
29+
| **OpenAI** | `gpt-5` |
30+
| **OpenAI** | `gpt-5-mini` |
31+
| **OpenAI** | `gpt-5-nano` |
32+
| **OpenAI** | `gpt-4.1-mini` |
33+
| **OpenAI** | `gpt-4.1-nano` |
34+
| **OpenAI** | `gpt-4o-mini` |
35+
| **Gemini** | `gemini-2.5-flash` |
36+
| **Gemini** | `gemini-2.5-flash-lite` |
37+
| **Groq** | `openai/gpt-oss-120b` |
38+
| **Groq** | `openai/gpt-oss-20b` |
39+
| **Groq** | `llama-3.3-70b-versatile` |
40+
41+
### Using Other Models
42+
43+
Other LLM models not listed above may work with Hindsight, but they must support **at least 65,000 output tokens** to ensure reliable fact extraction. If you need support for a specific model that doesn't meet this requirement, please [open an issue](https://github.com/hindsight-ai/hindsight/issues) to request an exception.
44+
45+
### Configuration
46+
47+
```bash
48+
# Groq (recommended)
49+
export HINDSIGHT_API_LLM_PROVIDER=groq
50+
export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx
51+
export HINDSIGHT_API_LLM_MODEL=openai/gpt-oss-20b
52+
53+
# OpenAI
54+
export HINDSIGHT_API_LLM_PROVIDER=openai
55+
export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
56+
export HINDSIGHT_API_LLM_MODEL=gpt-4o
57+
58+
# Gemini
59+
export HINDSIGHT_API_LLM_PROVIDER=gemini
60+
export HINDSIGHT_API_LLM_API_KEY=xxxxxxxxxxxx
61+
export HINDSIGHT_API_LLM_MODEL=gemini-2.0-flash
62+
63+
# Ollama (local)
64+
export HINDSIGHT_API_LLM_PROVIDER=ollama
65+
export HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1
66+
export HINDSIGHT_API_LLM_MODEL=llama3.1
67+
```
68+
69+
**Note:** The LLM is the primary bottleneck for retain operations. See [Performance](./performance) for optimization strategies.
70+
71+
---
72+
1773
## Embedding Model
1874

1975
Converts text into dense vector representations for semantic similarity search.
@@ -22,14 +78,13 @@ Converts text into dense vector representations for semantic similarity search.
2278

2379
**Alternatives:**
2480

25-
| Model | Dimensions | Use Case |
26-
|-------|------------|----------|
27-
| `BAAI/bge-small-en-v1.5` | 384 | Default, fast, good quality |
28-
| `BAAI/bge-base-en-v1.5` | 768 | Higher accuracy, slower |
29-
| `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` | 384 | Multilingual (50+ languages) |
81+
| Model | Use Case |
82+
|-------|----------|
83+
| `BAAI/bge-small-en-v1.5` | Default, fast, good quality |
84+
| `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` | Multilingual (50+ languages) |
3085

3186
:::warning
32-
All embedding models must produce 384-dimensional vectors to match the database schema.
87+
All embedding models must produce **384-dimensional vectors** to match the database schema.
3388
:::
3489

3590
**Configuration:**
@@ -71,44 +126,3 @@ export HINDSIGHT_API_RERANKER_LOCAL_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2
71126
export HINDSIGHT_API_RERANKER_PROVIDER=tei
72127
export HINDSIGHT_API_RERANKER_TEI_URL=http://localhost:8081
73128
```
74-
75-
---
76-
77-
## LLM
78-
79-
Used for fact extraction, entity resolution, opinion generation, and answer synthesis.
80-
81-
**Supported providers:** Groq, OpenAI, Gemini, Ollama
82-
83-
| Provider | Recommended Model | Best For |
84-
|----------|------------------|----------|
85-
| **Groq** | `openai/gpt-oss-20b` | Fast inference, high throughput (recommended) |
86-
| **OpenAI** | `gpt-4o` | Good quality |
87-
| **Gemini** | `gemini-2.0-flash` | Good quality, cost effective |
88-
| **Ollama** | `llama3.1` | Local deployment, privacy |
89-
90-
**Configuration:**
91-
92-
```bash
93-
# Groq (recommended)
94-
export HINDSIGHT_API_LLM_PROVIDER=groq
95-
export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx
96-
export HINDSIGHT_API_LLM_MODEL=openai/gpt-oss-20b
97-
98-
# OpenAI
99-
export HINDSIGHT_API_LLM_PROVIDER=openai
100-
export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
101-
export HINDSIGHT_API_LLM_MODEL=gpt-4o
102-
103-
# Gemini
104-
export HINDSIGHT_API_LLM_PROVIDER=gemini
105-
export HINDSIGHT_API_LLM_API_KEY=xxxxxxxxxxxx
106-
export HINDSIGHT_API_LLM_MODEL=gemini-2.0-flash
107-
108-
# Ollama (local)
109-
export HINDSIGHT_API_LLM_PROVIDER=ollama
110-
export HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1
111-
export HINDSIGHT_API_LLM_MODEL=llama3.1
112-
```
113-
114-
**Note:** The LLM is the primary bottleneck for retain operations. See [Performance](./performance) for optimization strategies.

‎hindsight-docs/static/llms-full.txt‎

Lines changed: 63 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
> Agent Memory that Works Like Human Memory
44

55
This file contains the complete Hindsight documentation for LLM consumption.
6-
Generated: 2025-12-15T09:47:27.854Z
6+
Generated: 2025-12-15T10:18:45.836Z
77

88
---
99

@@ -2750,14 +2750,70 @@ Hindsight uses several machine learning models for different tasks.
27502750

27512751
| Model Type | Purpose | Default | Configurable |
27522752
|------------|---------|---------|--------------|
2753+
| **LLM** | Fact extraction, reasoning, generation | Provider-specific | Yes |
27532754
| **Embedding** | Vector representations for semantic search | `BAAI/bge-small-en-v1.5` | Yes |
27542755
| **Cross-Encoder** | Reranking search results | `cross-encoder/ms-marco-MiniLM-L-6-v2` | Yes |
2755-
| **LLM** | Fact extraction, reasoning, generation | Provider-specific | Yes |
27562756

27572757
All local models (embedding, cross-encoder) are automatically downloaded from HuggingFace on first run.
27582758

27592759
---
27602760

2761+
## LLM
2762+
2763+
Used for fact extraction, entity resolution, opinion generation, and answer synthesis.
2764+
2765+
**Supported providers:** OpenAI, Gemini, Groq, Ollama
2766+
2767+
### Tested Models
2768+
2769+
The following models have been tested and verified to work correctly with Hindsight:
2770+
2771+
| Provider | Model |
2772+
|----------|-------|
2773+
| **OpenAI** | `gpt-5` |
2774+
| **OpenAI** | `gpt-5-mini` |
2775+
| **OpenAI** | `gpt-5-nano` |
2776+
| **OpenAI** | `gpt-4.1-mini` |
2777+
| **OpenAI** | `gpt-4.1-nano` |
2778+
| **OpenAI** | `gpt-4o-mini` |
2779+
| **Gemini** | `gemini-2.5-flash` |
2780+
| **Gemini** | `gemini-2.5-flash-lite` |
2781+
| **Groq** | `openai/gpt-oss-120b` |
2782+
| **Groq** | `openai/gpt-oss-20b` |
2783+
| **Groq** | `llama-3.3-70b-versatile` |
2784+
2785+
### Using Other Models
2786+
2787+
Other LLM models not listed above may work with Hindsight, but they must support **at least 65,000 output tokens** to ensure reliable fact extraction. If you need support for a specific model that doesn't meet this requirement, please [open an issue](https://github.com/hindsight-ai/hindsight/issues) to request an exception.
2788+
2789+
### Configuration
2790+
2791+
```bash
2792+
# Groq (recommended)
2793+
export HINDSIGHT_API_LLM_PROVIDER=groq
2794+
export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx
2795+
export HINDSIGHT_API_LLM_MODEL=openai/gpt-oss-20b
2796+
2797+
# OpenAI
2798+
export HINDSIGHT_API_LLM_PROVIDER=openai
2799+
export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
2800+
export HINDSIGHT_API_LLM_MODEL=gpt-4o
2801+
2802+
# Gemini
2803+
export HINDSIGHT_API_LLM_PROVIDER=gemini
2804+
export HINDSIGHT_API_LLM_API_KEY=xxxxxxxxxxxx
2805+
export HINDSIGHT_API_LLM_MODEL=gemini-2.0-flash
2806+
2807+
# Ollama (local)
2808+
export HINDSIGHT_API_LLM_PROVIDER=ollama
2809+
export HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1
2810+
export HINDSIGHT_API_LLM_MODEL=llama3.1
2811+
```
2812+
2813+
**Note:** The LLM is the primary bottleneck for retain operations. See [Performance](./performance) for optimization strategies.
2814+
2815+
---
2816+
27612817
## Embedding Model
27622818

27632819
Converts text into dense vector representations for semantic similarity search.
@@ -2766,14 +2822,13 @@ Converts text into dense vector representations for semantic similarity search.
27662822

27672823
**Alternatives:**
27682824

2769-
| Model | Dimensions | Use Case |
2770-
|-------|------------|----------|
2771-
| `BAAI/bge-small-en-v1.5` | 384 | Default, fast, good quality |
2772-
| `BAAI/bge-base-en-v1.5` | 768 | Higher accuracy, slower |
2773-
| `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` | 384 | Multilingual (50+ languages) |
2825+
| Model | Use Case |
2826+
|-------|----------|
2827+
| `BAAI/bge-small-en-v1.5` | Default, fast, good quality |
2828+
| `sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2` | Multilingual (50+ languages) |
27742829

27752830
:::warning
2776-
All embedding models must produce 384-dimensional vectors to match the database schema.
2831+
All embedding models must produce **384-dimensional vectors** to match the database schema.
27772832
:::
27782833

27792834
**Configuration:**
@@ -2816,47 +2871,6 @@ export HINDSIGHT_API_RERANKER_PROVIDER=tei
28162871
export HINDSIGHT_API_RERANKER_TEI_URL=http://localhost:8081
28172872
```
28182873

2819-
---
2820-
2821-
## LLM
2822-
2823-
Used for fact extraction, entity resolution, opinion generation, and answer synthesis.
2824-
2825-
**Supported providers:** Groq, OpenAI, Gemini, Ollama
2826-
2827-
| Provider | Recommended Model | Best For |
2828-
|----------|------------------|----------|
2829-
| **Groq** | `openai/gpt-oss-20b` | Fast inference, high throughput (recommended) |
2830-
| **OpenAI** | `gpt-4o` | Good quality |
2831-
| **Gemini** | `gemini-2.0-flash` | Good quality, cost effective |
2832-
| **Ollama** | `llama3.1` | Local deployment, privacy |
2833-
2834-
**Configuration:**
2835-
2836-
```bash
2837-
# Groq (recommended)
2838-
export HINDSIGHT_API_LLM_PROVIDER=groq
2839-
export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx
2840-
export HINDSIGHT_API_LLM_MODEL=openai/gpt-oss-20b
2841-
2842-
# OpenAI
2843-
export HINDSIGHT_API_LLM_PROVIDER=openai
2844-
export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx
2845-
export HINDSIGHT_API_LLM_MODEL=gpt-4o
2846-
2847-
# Gemini
2848-
export HINDSIGHT_API_LLM_PROVIDER=gemini
2849-
export HINDSIGHT_API_LLM_API_KEY=xxxxxxxxxxxx
2850-
export HINDSIGHT_API_LLM_MODEL=gemini-2.0-flash
2851-
2852-
# Ollama (local)
2853-
export HINDSIGHT_API_LLM_PROVIDER=ollama
2854-
export HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1
2855-
export HINDSIGHT_API_LLM_MODEL=llama3.1
2856-
```
2857-
2858-
**Note:** The LLM is the primary bottleneck for retain operations. See [Performance](./performance) for optimization strategies.
2859-
28602874

28612875
---
28622876

0 commit comments

Comments
 (0)