Skip to content

Commit 013673e

Browse files
author
Neal006
committed
feat: update .gitignore, add experiment logs and results for naive backend
1 parent fec22ab commit 013673e

5 files changed

Lines changed: 43 additions & 190 deletions

File tree

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
.env
2+
.claude/
23
__pycache__/
34
*.pyc
45
*.pyo

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
[![Stars](https://img.shields.io/github/stars/Neal006/memorylens?style=social)](https://github.com/Neal006/memorylens/stargazers)
1414
[![Forks](https://img.shields.io/github/forks/Neal006/memorylens?style=social)](https://github.com/Neal006/memorylens/network/members)
1515

16-
[**Quick Start**](#quick-start) · [**Results**](#benchmark-results) · [**How It Works**](#how-it-works) · [**vs Other Tools**](#how-memorylens-compares) · [**Contributing**](#contributing) · [**Paper**](paper/memorylens_paper.md)
16+
[**Quick Start**](#quick-start) · [**Results**](#benchmark-results) · [**How It Works**](#how-it-works) · [**vs Other Tools**](#how-memorylens-compares)
1717

1818
</div>
1919

@@ -45,7 +45,7 @@ Run `python main.py` and get statistically valid results like these — **no API
4545
| **Cascading Temporal** (Ebbinghaus decay) | **87.5 ± 0.0%** | **218** | **5.67×** |
4646
| SummaryMemory (rolling compression) | 100.0 ± 0.0% | 318 ||
4747

48-
> **Chunked RAG vs Ideal RAG** shows the gap between a theoretical upper bound and a production-realistic retrieval system. The 15pp difference is what chunking + index eviction costs you. The **Cascading Temporal** backend delivers **5.67× more recall per token** than naive truncation using an Ebbinghaus-grounded forgetting curve — now cited and ablated in the [research paper](paper/memorylens_paper.md).
48+
> **Chunked RAG vs Ideal RAG** shows the gap between a theoretical upper bound and a production-realistic retrieval system. The 15pp difference is what chunking + index eviction costs you. The **Cascading Temporal** backend delivers **5.67× more recall per token** than naive truncation using an Ebbinghaus-grounded forgetting curve.
4949
5050
---
5151

experiment_logs/runs_summary.csv

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
run_id,backend,turn,recall,precision,drift,noise,tokens,total_turns
2+
test_run,naive,10,1.0,0.9,0.0,0.5,100,25
3+
test_run,naive,25,0.8,0.7,0.1,0.8,500,25

experiment_logs/test_run.json

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"run_id": "test_run",
3+
"config": {
4+
"total_turns": 25,
5+
"backends": [
6+
"naive"
7+
]
8+
},
9+
"results": {
10+
"checkpoints": [
11+
10,
12+
25
13+
],
14+
"naive": {
15+
"recall": [
16+
1.0,
17+
0.8
18+
],
19+
"precision": [
20+
0.9,
21+
0.7
22+
],
23+
"drift": [
24+
0.0,
25+
0.1
26+
],
27+
"noise": [
28+
0.5,
29+
0.8
30+
],
31+
"tokens": [
32+
100,
33+
500
34+
]
35+
}
36+
}
37+
}

paper/memorylens_paper.md

Lines changed: 0 additions & 188 deletions
This file was deleted.

0 commit comments

Comments
 (0)