Skip to content

Commit ebdcd18

Browse files
jcodellaaayush3011Copilot
authored
Overview graphic, restructuring of samples, troubleshooting guide, and more (#13)
* Overview graphic, restructuring of samples, troubleshooting guide, and other small changes * addressed Copilot review items * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Aayush Kataria <aayushkataria3011@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
1 parent 2533cd7 commit ebdcd18

26 files changed

Lines changed: 2176 additions & 6617 deletions

‎Docs/README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,12 @@ This folder contains the main project documentation for Agent Memory Toolkit.
1010
| [local_testing.md](local_testing.md) | Covers local setup, environment configuration, RBAC, Cosmos provisioning, running the toolkit and Azure Functions locally, and testing change feed auto-processing with serverless or autoscale container provisioning. |
1111
| [azure_testing.md](azure_testing.md) | Covers Azure deployment, cloud configuration, required services, change feed settings, throughput mode configuration, and validation steps for running the toolkit in Azure. |
1212
| [design_patterns.md](design_patterns.md) | Shows when and how to call CRUD operations, summarization, fact extraction, and memory retrieval in chat and multi-agent applications, including automatic processing via the change feed. |
13+
| [troubleshooting.md](troubleshooting.md) | Helps diagnose common setup, authentication, Cosmos DB, embeddings, Durable Functions, vector search, and change feed issues. |
1314

1415
## Recommended Reading Order
1516

1617
1. Start with [concepts.md](concepts.md) to understand the data model and memory lifecycle.
1718
2. Use [local_testing.md](local_testing.md) to get the toolkit running and validated on your machine.
1819
3. Use [azure_testing.md](azure_testing.md) when you are ready to deploy or validate the full stack in Azure.
19-
4. See [design_patterns.md](design_patterns.md) for integration patterns in real applications.
20+
4. See [design_patterns.md](design_patterns.md) for integration patterns in real applications.
21+
5. Use [troubleshooting.md](troubleshooting.md) when setup, processing, search, or automatic change feed behavior does not work as expected.

‎Docs/troubleshooting.md‎

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
# Troubleshooting Agent Memory Toolkit
2+
3+
Use this guide when local memory works but Cosmos DB, embeddings, Durable Functions, or automatic change feed processing does not.
4+
5+
---
6+
7+
## Quick Triage
8+
9+
| Symptom | First checks |
10+
|---------|--------------|
11+
| Import errors | Install with `pip install -e ".[dev]"` and import `CosmosMemoryClient` or `AsyncCosmosMemoryClient`. |
12+
| Missing configuration | Verify `.env`, `function_app/local.settings.json`, and Azure Function App settings use the same endpoint, database, container, and AI deployment values. |
13+
| Cosmos 401 or 403 | Run `az login` and confirm Cosmos DB data-plane RBAC is assigned. |
14+
| Cosmos operations fail before connecting | Call `create_memory_store()` or `connect_cosmos()` before cloud operations. |
15+
| Search returns no vector results | Confirm embeddings are generated and `AI_FOUNDRY_EMBEDDING_DIMENSIONS` matches the container vector policy. |
16+
| Durable Functions processing fails | Start the Functions host and check `function_app/local.settings.json`, the change feed trigger, and the orchestrator logs. |
17+
| Change feed does not create summaries or facts | Confirm change feed settings, thresholds, lease container, counter container, and that inserted documents have `type: "turn"`. |
18+
19+
---
20+
21+
## 1. Environment And Imports
22+
23+
Install the package from the repository root:
24+
25+
```bash
26+
pip install -e ".[dev]"
27+
pip install -r function_app/requirements.txt
28+
```
29+
30+
The public clients are:
31+
32+
```python
33+
from agent_memory_toolkit import CosmosMemoryClient
34+
from agent_memory_toolkit.aio import AsyncCosmosMemoryClient
35+
```
36+
37+
If notebooks cannot import the package, run them from the repo root with paths such as `Samples/Notebooks/Demo.ipynb`, or add the repository root to `sys.path`.
38+
39+
---
40+
41+
## 2. Configuration And Authentication
42+
43+
For local runs, keep `.env`, `function_app/local.settings.json`, and deployed Function App settings aligned:
44+
45+
```env
46+
COSMOS_DB_ENDPOINT=https://<account>.documents.azure.com:443/
47+
COSMOS_DB__accountEndpoint=https://<account>.documents.azure.com:443/
48+
COSMOS_DB_KEY=
49+
COSMOS_DB_DATABASE=ai_memory
50+
COSMOS_DB_CONTAINER=memories
51+
COSMOS_DB_COUNTERS_CONTAINER=counter
52+
COSMOS_DB_LEASE_CONTAINER=leases
53+
COSMOS_DB_THROUGHPUT_MODE=serverless
54+
COSMOS_DB_AUTOSCALE_MAX_RU=1000
55+
56+
AI_FOUNDRY_ENDPOINT=https://<account>.openai.azure.com/
57+
AI_FOUNDRY_API_KEY=
58+
AI_FOUNDRY_EMBEDDING_DEPLOYMENT_NAME=text-embedding-3-large
59+
AI_FOUNDRY_EMBEDDING_DIMENSIONS=1536
60+
AI_FOUNDRY_EMBEDDING_DATA_TYPE=float32
61+
AI_FOUNDRY_EMBEDDING_DISTANCE_FUNCTION=cosine
62+
AI_FOUNDRY_CHAT_DEPLOYMENT_NAME=<chat-deployment-name>
63+
```
64+
65+
The notebooks and samples pass these values into the client like this:
66+
67+
| `.env` setting | Client argument |
68+
|---|---|
69+
| `COSMOS_DB_ENDPOINT` | `cosmos_endpoint` |
70+
| `COSMOS_DB_DATABASE` | `cosmos_database` |
71+
| `COSMOS_DB_CONTAINER` | `cosmos_container` |
72+
| `COSMOS_DB_COUNTERS_CONTAINER` | `cosmos_counter_container` |
73+
| `COSMOS_DB_LEASE_CONTAINER` | `cosmos_lease_container` |
74+
| `COSMOS_DB_KEY` | `cosmos_key` |
75+
| `AI_FOUNDRY_ENDPOINT` | `ai_foundry_endpoint` |
76+
| `AI_FOUNDRY_API_KEY` | `ai_foundry_api_key` |
77+
| `AI_FOUNDRY_EMBEDDING_DEPLOYMENT_NAME` | `embedding_deployment_name` |
78+
| `AI_FOUNDRY_CHAT_DEPLOYMENT_NAME` | `chat_deployment_name` |
79+
80+
`AI_FOUNDRY_EMBEDDING_DIMENSIONS`, `AI_FOUNDRY_EMBEDDING_DATA_TYPE`, and `AI_FOUNDRY_EMBEDDING_DISTANCE_FUNCTION` are read by the toolkit when creating the Cosmos DB vector policy. The Function App also reads `COSMOS_DB__accountEndpoint` for its identity-based Cosmos DB trigger binding; set it to the same value as `COSMOS_DB_ENDPOINT`.
81+
82+
Run `az login` before using `DefaultAzureCredential`.
83+
84+
Required roles:
85+
86+
| Service | Role |
87+
|---------|------|
88+
| Cosmos DB | Cosmos DB Built-in Data Contributor |
89+
| Azure OpenAI / AI Services | Cognitive Services OpenAI User |
90+
91+
RBAC changes can take several minutes to propagate.
92+
93+
---
94+
95+
## 3. Cosmos DB Store Creation
96+
97+
Run `create_memory_store()` before relying on cloud operations. It creates the database plus the `memories`, `counter`, and `leases` containers.
98+
99+
The memories container is created with:
100+
101+
- hierarchical partition key on `user_id` and `thread_id`
102+
- vector index on `/embedding`
103+
- full-text index on `/content`
104+
105+
If vector or full-text search fails after changing dimensions or indexing settings, create a fresh container with the desired configuration. Cosmos container vector policies are creation-time infrastructure choices.
106+
107+
Use `COSMOS_DB_THROUGHPUT_MODE=serverless` for the default setup. Use `autoscale` with `COSMOS_DB_AUTOSCALE_MAX_RU` when you need provisioned autoscale throughput.
108+
109+
---
110+
111+
## 4. Embeddings And Search
112+
113+
Embedding failures usually mean one of these is wrong:
114+
115+
- `AI_FOUNDRY_ENDPOINT`
116+
- `AI_FOUNDRY_EMBEDDING_DEPLOYMENT_NAME`
117+
- `AI_FOUNDRY_EMBEDDING_DIMENSIONS`
118+
- Azure OpenAI / AI Services RBAC
119+
120+
For hybrid search, `search_terms` is required when `hybrid_search=True`.
121+
122+
If search returns documents but scores look poor, check that records have an `embedding` field and that the query uses similar language to the stored memory content.
123+
124+
---
125+
126+
## 5. Durable Functions Processing
127+
128+
Durable Functions processing requires the Functions host.
129+
130+
Start local dependencies:
131+
132+
```bash
133+
azurite --silent --location /tmp/azurite --debug /tmp/azurite/debug.log
134+
cd function_app
135+
func start
136+
```
137+
138+
The SDK does not post to a Function endpoint. With `DurableFunctionProcessor`, the SDK writes turns to Cosmos DB and the deployed Function App picks them up from the Cosmos DB change feed. For local testing, keep `function_app/local.settings.json` aligned with `.env` and confirm the Functions host starts the change feed trigger.
139+
140+
If orchestration polling times out, check the Functions logs first. The orchestration may still be running, or an activity may be waiting on Cosmos DB or the LLM endpoint.
141+
142+
---
143+
144+
## 6. Change Feed Processing
145+
146+
Automatic processing requires these settings in the Functions app or `local.settings.json`:
147+
148+
```json
149+
"COSMOS_DB__accountEndpoint": "https://<account>.documents.azure.com:443/",
150+
"COSMOS_DB_ENDPOINT": "https://<account>.documents.azure.com:443/",
151+
"COSMOS_DB_DATABASE": "ai_memory",
152+
"COSMOS_DB_CONTAINER": "memories",
153+
"COSMOS_DB_COUNTERS_CONTAINER": "counter",
154+
"COSMOS_DB_LEASE_CONTAINER": "leases",
155+
"AI_FOUNDRY_ENDPOINT": "https://<account>.openai.azure.com/",
156+
"AI_FOUNDRY_CHAT_DEPLOYMENT_NAME": "gpt-4o-mini",
157+
"AI_FOUNDRY_EMBEDDING_DEPLOYMENT_NAME": "text-embedding-3-large",
158+
"THREAD_SUMMARY_EVERY_N": "5",
159+
"FACT_EXTRACTION_EVERY_N": "3",
160+
"USER_SUMMARY_EVERY_N": "10"
161+
```
162+
163+
Set a threshold to `"0"` to disable that processing type.
164+
165+
Cosmos DB memory documents store their category in the JSON `type` field. Only documents with `type: "turn"` increment counters. Derived memories with `type: "summary"`, `type: "fact"`, or `type: "user_summary"` do not trigger threshold counts.
166+
167+
If nothing fires:
168+
169+
- verify the Functions host shows the Cosmos DB trigger
170+
- confirm the `leases` container exists
171+
- confirm the `counter` container is writable
172+
- insert enough new turn documents to cross the configured threshold
173+
- check for generated documents where the Cosmos JSON field is `type="summary"`, `type="fact"`, or `type="user_summary"`
174+
175+
---
176+
177+
## 7. Async Client Notes
178+
179+
Use async Azure credentials with the async client:
180+
181+
```python
182+
from azure.identity.aio import DefaultAzureCredential
183+
from agent_memory_toolkit.aio import AsyncCosmosMemoryClient
184+
```
185+
186+
Always `await` cloud operations and close the client when done:
187+
188+
```python
189+
await memory.close()
190+
```
191+
192+
In notebooks, top-level `await` is supported, so do not wrap cells with `asyncio.run()`.

‎Overview.png‎

1.61 MB
Loading

‎README.md‎

Lines changed: 29 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# Azure Cosmos DB Agent Memory Toolkit - Public Preview
22

3+
![Agent Memory Toolkit overview](Overview.png)
4+
5+
36
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
47
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
58
[![Azure Cosmos DB](https://img.shields.io/badge/Azure-Cosmos%20DB-0078D4?logo=microsoft-azure)](https://azure.microsoft.com/en-us/products/cosmos-db/)
@@ -10,46 +13,6 @@
1013

1114
Agent Memory Toolkit is a Python SDK for storing, retrieving, and transforming agent memories on Azure Cosmos DB. It gives your agent both raw conversation history and higher-value derived memory — thread summaries, extracted facts, and cross-thread user profiles — all searchable semantically. The processing pipeline can run **in-process** (zero infra) or in a sibling **Azure Durable Function app** that watches the Cosmos DB change feed. Sync (`CosmosMemoryClient`) and async (`AsyncCosmosMemoryClient`) APIs are mirror-images of each other.
1215

13-
```
14-
┌──────────────────────────────────────────────────────────────────────────────────────┐
15-
│ YOUR AGENTIC APP │
16-
│ Uses CosmosMemoryClient / AsyncCosmosMemoryClient │
17-
└─────────────────────────────────────────┬────────────────────────────────────────────┘
18-
│
19-
▼
20-
┌──────────────────────────────────────────────────────────────────────────────────────┐
21-
│ AGENT MEMORY TOOLKIT (Python SDK) │
22-
│ │
23-
│ • Local in-memory CRUD │
24-
│ • Cosmos DB storage and retrieval │
25-
│ • Pluggable processor: in-process or remote Durable Function app │
26-
└──────────────────────────────────────────┬──────────────────────────────┬────────────┘
27-
│ │
28-
│ read / write │ Invoke processing pipeline
29-
▼ ▼
30-
┌───────────────────────────────────┐ ┌──────────────────────────────────┐
31-
│ AZURE COSMOS DB (NoSQL) │ │ AZURE DURABLE FUNCTIONS │
32-
│ │ │ │
33-
│ Stores: │ │ Orchestrates memory processing: │
34-
│ • turns │ │ • thread summaries │
35-
│ • summaries │◄─── memory management ───►│ • fact extraction │
36-
│ • facts │ │ • user summaries │
37-
│ • user summaries │ │ │
38-
│ │ │ On-demand (SDK) or automatic │
39-
│ Supports query, vector, text │ change feed trigger │ (Cosmos DB change feed trigger). │
40-
│ search over stored memories. │───────────────────────────►│ │
41-
└───────────────────────┬───────────┘ └──────────────────┬───────────────┘
42-
│ embeddings and LLM-based processing │
43-
└──────────────────────┬───────────────────────────────────┘
44-
▼
45-
┌──────────────────────────────────┐
46-
│ MICROSOFT FOUNDRY │
47-
│ │
48-
│ • Embeddings for search │
49-
│ • Chat/LLM generation │
50-
│ │
51-
└──────────────────────────────────┘
52-
```
5316

5417
---
5518

@@ -156,7 +119,7 @@ print(memory.get_user_summary(user_id=USER))
156119
### 4. Run a sample
157120
158121
```bash
159-
python Samples/quickstart_cosmos.py
122+
python Samples/Quickstarts/quickstart_cosmos.py
160123
```
161124
162125
See [`Samples/`](Samples/) for end-to-end scenarios (chat memory, RAG, multi-agent, customer support, remote processor).
@@ -273,6 +236,29 @@ memory = CosmosMemoryClient(..., processor=DurableFunctionProcessor())
273236

274237
---
275238

239+
### Architecture overview
240+
241+
```
242+
+--------------------------+
243+
| Agent app |
244+
+------------+-------------+
245+
|
246+
v
247+
+--------------------------+ +--------------------------+
248+
| Agent Memory Toolkit | <--> | Microsoft Foundry |
249+
| Python sync/async SDK | | LLMs + embeddings |
250+
+------------+-------------+ +------------+-------------+
251+
^ ^
252+
| |
253+
v v
254+
+--------------------------+ +--------------------------+
255+
| Azure Cosmos DB | <--> | Azure Durable Functions |
256+
| memories + search | | optional processing |
257+
+--------------------------+ +--------------------------+
258+
```
259+
260+
---
261+
276262
## Public API reference
277263

278264
| Symbol | Module | Purpose |
@@ -297,6 +283,7 @@ Async equivalents (`AsyncInProcessProcessor`, `AsyncDurableFunctionProcessor`) l
297283
- **[Docs/local_testing.md](Docs/local_testing.md)** — Prerequisites, environment setup, running locally, debugging
298284
- **[Docs/azure_testing.md](Docs/azure_testing.md)** — Azure deployment, RBAC, cloud validation
299285
- **[infra/README.md](infra/README.md)** — `azd` deployment, Bicep modules, BYOR settings, counter-trigger tuning
286+
- **[Docs/troubleshooting.md](Docs/troubleshooting.md)** — Common issues and resolutions for setup, auth, Cosmos DB, embeddings, Durable Functions, vector search, change feed, etc.
300287

301288
---
302289

@@ -308,7 +295,7 @@ agent_memory_toolkit/ Python SDK (sync + aio mirror)
308295
function_app/ Sibling Azure Durable Function app
309296
infra/ Bicep modules + main.bicep for `azd up`
310297
azure.yaml `azd` config — provisions Cosmos + AI Foundry + Function app
311-
Samples/ Demo notebooks + sample scripts
298+
Samples/ Categorized demo notebooks + sample scripts
312299
Docs/ Conceptual + operational docs
313300
tests/ Unit + integration tests (pytest)
314301
```

Samples/advanced_memory_lifecycle.py renamed to Samples/Advanced/advanced_memory_lifecycle.py

File renamed without changes.

Samples/advanced_search_patterns.py renamed to Samples/Advanced/advanced_search_patterns.py

File renamed without changes.

0 commit comments

Comments
 (0)