You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: hindsight-docs/versioned_docs/version-0.4/developer/api/recall.mdx
+60Lines changed: 60 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -184,6 +184,66 @@ Use this for strict scope enforcement where a memory must explicitly belong to *
184
184
A memory with tags `["user:alice", "team", "project:x"]` will still match a filter of `["user:alice", "team"]` under `all_strict` — extra tags on the memory are not a problem. The filter only requires the memory to contain **at least** the specified tags.
185
185
:::
186
186
187
+
### tag_groups
188
+
189
+
`tag_groups` is a list of compound boolean tag filters. The groups in the list are AND-ed together at the top level. Each group is a recursive boolean expression: a **leaf** node `{tags, match}`, or a **compound** node `{and: [...]}`, `{or: [...]}`, or `{not: ...}`.
190
+
191
+
`tag_groups` and `tags` / `tags_match` can be used simultaneously — they are AND-ed together.
When set to `true`, the response includes a detailed debug trace covering the query embedding, entry points, per-strategy retrieval results, RRF fusion candidates, reranked results, temporal constraints detected, and per-phase timings. Has no effect on the retrieval logic itself. Useful for understanding why specific memories were or were not returned.
|`HINDSIGHT_API_RERANKER_LITELLM_SDK_API_KEY`| LiteLLM **SDK** API key for direct reranking (no proxy needed) | - |
427
427
|`HINDSIGHT_API_RERANKER_LITELLM_SDK_MODEL`| LiteLLM SDK rerank model (e.g., `deepinfra/Qwen3-reranker-8B`) |`cohere/rerank-english-v3.0`|
428
428
|`HINDSIGHT_API_RERANKER_LITELLM_SDK_API_BASE`| Custom API base URL for LiteLLM SDK (optional) | - |
429
+
|`HINDSIGHT_API_RERANKER_LITELLM_MAX_TOKENS_PER_DOC`| Truncate documents to this many tokens before sending to the reranker (applies to both `litellm` and `litellm-sdk`). Use for models with small context windows (e.g. set to `900` for a 1024-token limit model). Unset by default (no truncation). | - |
429
430
|`HINDSIGHT_API_RERANKER_ZEROENTROPY_API_KEY`| ZeroEntropy API key for reranking | - |
430
431
|`HINDSIGHT_API_RERANKER_ZEROENTROPY_MODEL`| ZeroEntropy rerank model (`zerank-2`, `zerank-2-small`) |`zerank-2`|
431
432
|`HINDSIGHT_API_RERANKER_FLASHRANK_MODEL`| FlashRank model for fast CPU-based reranking |`ms-marco-MiniLM-L-12-v2`|
432
433
|`HINDSIGHT_API_RERANKER_FLASHRANK_CACHE_DIR`| Cache directory for FlashRank models | System default |
434
+
|`HINDSIGHT_API_RERANKER_JINA_MLX_MODEL_PATH`| Local path to downloaded `jina-reranker-v3-mlx` model (auto-downloads from HuggingFace if unset) | - |
433
435
434
436
```bash
435
437
# Local (default) - uses SentenceTransformers CrossEncoder
@@ -472,6 +474,10 @@ export HINDSIGHT_API_RERANKER_LITELLM_MODEL=cohere/rerank-english-v3.0 # or voy
export HINDSIGHT_API_RERANKER_LITELLM_SDK_MODEL=deepinfra/Qwen3-reranker-8B # or cohere/rerank-english-v3.0, etc.
477
+
478
+
# Jina MLX - Apple Silicon native reranking (no GPU/cloud required)
479
+
# Model (~1.2 GB) is downloaded automatically from HuggingFace Hub on first use.
480
+
export HINDSIGHT_API_RERANKER_PROVIDER=jina-mlx
475
481
```
476
482
477
483
#### LiteLLM Proxy vs SDK
@@ -488,6 +494,14 @@ Both support the same providers:
488
494
-**Jina AI** (`jina_ai/jina-reranker-v2`)
489
495
-**AWS Bedrock** (`bedrock/...`)
490
496
497
+
#### Jina MLX (Apple Silicon)
498
+
499
+
The `jina-mlx` provider uses [`jinaai/jina-reranker-v3-mlx`](https://huggingface.co/jinaai/jina-reranker-v3-mlx), optimized for Apple Silicon. The model (~1.2 GB) is downloaded from HuggingFace Hub automatically on first startup and cached locally.
500
+
501
+
:::note License
502
+
`jina-reranker-v3-mlx` is licensed under CC BY-NC 4.0. Contact Jina AI for commercial usage.
503
+
:::
504
+
491
505
### Authentication
492
506
493
507
By default, Hindsight runs without authentication. For production deployments, enable API key authentication using the built-in tenant extension:
@@ -530,6 +544,7 @@ For advanced authentication (JWT, OAuth, multi-tenant schemas), implement a cust
530
544
|`HINDSIGHT_API_GRAPH_RETRIEVER`| Graph retrieval algorithm: `link_expansion`, `mpfp`, or `bfs`|`link_expansion`|
531
545
|`HINDSIGHT_API_RECALL_MAX_CONCURRENT`| Max concurrent recall operations per worker (backpressure) |`32`|
532
546
|`HINDSIGHT_API_RECALL_CONNECTION_BUDGET`| Max concurrent DB connections per recall operation |`4`|
547
+
|`HINDSIGHT_API_RECALL_MAX_QUERY_TOKENS`| Maximum token length of a recall query; requests exceeding this limit are rejected with HTTP 400 |`500`|
533
548
|`HINDSIGHT_API_RERANKER_MAX_CANDIDATES`| Max candidates to rerank per recall (RRF pre-filters the rest) |`300`|
534
549
|`HINDSIGHT_API_MPFP_TOP_K_NEIGHBORS`| Fan-out limit per node in MPFP graph traversal |`20`|
535
550
|`HINDSIGHT_API_MENTAL_MODEL_REFRESH_CONCURRENCY`| Max concurrent mental model refreshes |`8`|
Copy file name to clipboardExpand all lines: hindsight-docs/versioned_docs/version-0.4/developer/installation.md
+60-70Lines changed: 60 additions & 70 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,28 +8,28 @@ Hindsight can be deployed in several ways depending on your infrastructure and r
8
8
9
9
## Prerequisites
10
10
11
-
### PostgreSQL with pgvector
11
+
### PostgreSQL
12
12
13
-
Hindsight requires PostgreSQL with the **pgvector** extension for vector similarity search.
13
+
Hindsight requires PostgreSQL 14+ with a vector extension for similarity search. The supported extensions are:
14
+
15
+
-**pgvector** (default)
16
+
-**pgvectorscale**
17
+
-**vchord**
18
+
19
+
Configure which one to use with `HINDSIGHT_API_VECTOR_EXTENSION`. See [Configuration](./configuration) for details.
14
20
15
21
**By default**, Hindsight uses **pg0** — an embedded PostgreSQL that runs locally on your machine. This is convenient for development but **not recommended for production**.
16
22
17
-
**For production**, use an external PostgreSQL with pgvector:
23
+
**For production**, use an external PostgreSQL with one of the supported vector extensions:
18
24
-**Supabase** — Managed PostgreSQL with pgvector built-in
19
25
-**Neon** — Serverless PostgreSQL with pgvector
20
-
-**Azure Database for PostgreSQL** — With pgvector and pg_diskann (DiskANN) support
26
+
-**Azure Database for PostgreSQL** — With pgvector and pgvectorscale support
21
27
-**AWS RDS** / **Cloud SQL** — With pgvector extension enabled
22
-
-**Self-hosted** — PostgreSQL 14+ with pgvector installed
28
+
-**Self-hosted** — PostgreSQL 14+ with your preferred vector extension
23
29
24
30
### LLM Provider
25
31
26
-
You need an LLM API key for fact extraction, entity resolution, and answer generation:
27
-
28
-
-**Groq** (recommended): Fast inference with `gpt-oss-20b`
29
-
-**OpenAI**: GPT-4o, GPT-4o-mini
30
-
-**Ollama**: Run models locally
31
-
32
-
See [Models](./models) for detailed comparison and configuration.
32
+
You need an LLM API key for fact extraction, entity resolution, and answer generation. See [Models](./models) for supported providers, model recommendations, and configuration.
- Using managed embedding services (OpenAI, Cohere)
90
-
- Running on Text Embeddings Inference (TEI) infrastructure
91
-
- Kubernetes environments with fast pull requirements
92
-
93
-
:::warning Slim Image Requires External Providers
94
-
If you run the slim image **without** setting external embedding providers, you'll see this error:
95
-
96
-
```
97
-
ImportError: sentence-transformers is required for LocalSTEmbeddings.
98
-
Install it with: pip install sentence-transformers
99
-
```
100
-
101
-
**Fix:** Always set embedding and reranking providers when using slim images:
102
-
```bash
103
-
-e HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai
104
-
-e HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxx
105
-
-e HINDSIGHT_API_RERANKER_PROVIDER=cohere
106
-
-e HINDSIGHT_API_COHERE_API_KEY=xxx
107
-
```
108
-
:::
109
-
110
-
See [Configuration](./configuration#embeddings) for all embedding provider options.
61
+
The slim image corresponds to the [`hindsight-api-slim`](#bare-metal-pip) pip package. See [Configuration](./configuration#embeddings) for external provider options.
111
62
112
63
### Available Tags
113
64
@@ -120,7 +71,7 @@ ghcr.io/vectorize-io/hindsight:0.4.9-slim # Slim, specific version
@@ -175,14 +126,17 @@ See the [Helm chart values.yaml](https://github.com/vectorize-io/hindsight/tree/
175
126
176
127
## Bare Metal (pip)
177
128
178
-
**Best for**: Custom deployments, integration into existing Python applications
129
+
**Best for**: Running Hindsight as a standalone service on a host machine.
179
130
180
131
### Install
181
132
182
133
```bash
183
-
pip install hindsight-all
134
+
pip install hindsight-api # Full — works out of the box
135
+
pip install hindsight-api-slim # Slim — requires external services for embeddings, reranking, and the database
184
136
```
185
137
138
+
When using `hindsight-api-slim`, you must configure external providers for all model operations. See [Configuration](./configuration#embeddings) for details.
139
+
186
140
### Run with Embedded Database
187
141
188
142
For development and testing, Hindsight can run with an embedded PostgreSQL (pg0):
**Best for**: Using Hindsight programmatically from Python without running a separate server process.
213
+
214
+
```bash
215
+
pip install hindsight-all # Full — works out of the box
216
+
pip install hindsight-all-slim # Slim — requires external services for embeddings, reranking, and the database
217
+
```
218
+
219
+
`hindsight-all` supports two modes of embedding:
220
+
221
+
**In-process** (`HindsightServer`): the server runs in a background thread inside your application. Best when you want the tightest integration and are already managing your own process lifecycle.
222
+
223
+
```python
224
+
from hindsight import HindsightServer, HindsightClient
225
+
226
+
with HindsightServer(llm_provider="openai", llm_api_key="sk-xxx") as server:
results = client.recall(bank_id="alice", query="How should I respond to Alice?")
230
+
```
231
+
232
+
**Managed subprocess** (`HindsightEmbedded`): the server runs as a background daemon process, shared across multiple Python processes or sessions. The daemon starts on first use and shuts down automatically after an idle timeout.
0 commit comments