Skip to content

Fit an Ebbinghaus forgetting curve to Recall@T data #6

@Neal006

Description

@Neal006

Background

The Ebbinghaus forgetting curve models how memory retention decays exponentially over time:

R(t) = e^(-t/S)

Where R is retention, t is time, and S is the stability (how resistant this memory is to decay).

What to build

Add a function in `evaluation/metrics.py` that fits a forgetting curve to the Recall@T series and returns the fitted parameters:

```python
from scipy.optimize import curve_fit
import numpy as np

def fit_forgetting_curve(checkpoints: List[int], recall_values: List[float]) -> Dict:
"""
Fit an exponential decay curve to recall@T data.
Returns: {"stability": S, "r_squared": float, "half_life": int}
"""
def ebbinghaus(t, S):
return np.exp(-t / S)

popt, _ = curve_fit(ebbinghaus, checkpoints, recall_values, p0=[50.0])
S = popt[0]

predicted = ebbinghaus(np.array(checkpoints), S)
ss_res = np.sum((np.array(recall_values) - predicted) ** 2)
ss_tot = np.sum((np.array(recall_values) - np.mean(recall_values)) ** 2)
r2 = 1 - ss_res / ss_tot

half_life = int(S * np.log(2))

return {"stability": round(S, 2), "r_squared": round(r2, 4), "half_life": half_life}

```

Expected findings

Backend Stability (S) Half-life (turns)
Naive ~65 ~45
RAG ∞ (flat, no decay)
Cascading ~90 ~62

Higher S = slower forgetting = better memory architecture.

Wire into dashboard

Add a small annotation to the Recall decay chart showing each curve's fitted half-life.

Acceptance criteria

  • fit_forgetting_curve(checkpoints, recalls) returns dict with stability, r_squared, half_life
  • Half-life annotations on dashboard decay chart
  • scipy added to requirements.txt
  • One test verifying the function returns sensible values

Estimated effort

2–3 hours. Mostly scipy + Plotly annotation work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issuePerfect starting point for new contributorsnew-metricNew evaluation metricresearchInvolves literature review or empirical analysis

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions