SPEC-11 Basic Memory API Performance Optimization.md

title	SPEC-11: Basic Memory API Performance Optimization
type	spec
permalink	specs/spec-11-basic-memory-api-performance-optimization
tags	performance api mcp database cloud

SPEC-11: Basic Memory API Performance Optimization

Why

The Basic Memory API experiences significant performance issues in cloud environments due to expensive per-request initialization. MCP tools making HTTP requests to the API suffer from 350ms-2.6s latency overhead before any actual operation occurs.

Root Cause Analysis:

• GitHub Issue #82 shows repeated initialization sequences in logs (16:29:35 and 16:49:58)
• Each MCP tool call triggers full database initialization + project reconciliation
• get_engine_factory() dependency calls db.get_or_create_db() on every request
• reconcile_projects_with_config() runs expensive sync operations repeatedly

Performance Impact:

• Database connection setup: ~50-100ms per request
• Migration checks: ~100-500ms per request
• Project reconciliation: ~200ms-2s per request
• Total overhead: ~350ms-2.6s per MCP tool call

This creates compounding effects with tenant auto-start delays and increases timeout risk in cloud deployments.

What

This optimization affects the core basic-memory repository components:

1. API Lifespan Management (src/basic_memory/api/app.py)

• Cache database connections in app state during startup
• Avoid repeated expensive initialization

2. Dependency Injection (src/basic_memory/deps.py)

• Modify get_engine_factory() to use cached connections
• Eliminate per-request database setup

3. Initialization Service (src/basic_memory/services/initialization.py)

• Add caching/throttling to project reconciliation
• Skip expensive operations when appropriate

4. Configuration (src/basic_memory/config.py)

• Add optional performance flags for cloud environments

Backwards Compatibility: All changes must be backwards compatible with existing CLI and non-cloud usage.

How (High Level)

Phase 1: Cache Database Connections (Critical - 80% of gains)

Problem: get_engine_factory() calls db.get_or_create_db() per request Solution: Cache database engine/session in app state during lifespan

1. Modify API Lifespan (api/app.py):

@asynccontextmanager
async def lifespan(app: FastAPI):
    app_config = ConfigManager().config
    await initialize_app(app_config)

    # Cache database connection in app state
    engine, session_maker = await db.get_or_create_db(app_config.database_path)
    app.state.engine = engine
    app.state.session_maker = session_maker

    # ... rest of startup logic

2. Modify Dependency Injection (deps.py):

async def get_engine_factory(
  request: Request
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]:
  """Get cached engine and session maker from app state."""
  return request.app.state.engine, request.app.state.session_maker

Phase 2: Optimize Project Reconciliation (Secondary - 20% of gains)

Problem: reconcile_projects_with_config() runs expensive sync repeatedly Solution: Add module-level caching with time-based throttling

1. Add Reconciliation Cache (services/initialization.py):

_project_reconciliation_completed = False
_last_reconciliation_time = 0

async def reconcile_projects_with_config(app_config, force=False):
  # Skip if recently completed (within 60 seconds) unless forced
  if recently_completed and not force:
      return
  # ... existing logic

Phase 3: Cloud Environment Flags (Optional)

Problem: Force expensive initialization in production environments Solution: Add skip flags for cloud/stateless deployments

1. Add Config Flag (config.py): skip_initialization_sync: bool = Field(default=False)
2. Configure in Cloud (basic-memory-cloud integration): BASIC_MEMORY_SKIP_INITIALIZATION_SYNC=true

How to Evaluate

Success Criteria

1. Performance Metrics (Primary):

• MCP tool response time reduced by 50%+ (measure before/after)
• Database connection overhead eliminated (0ms vs 50-100ms)
• Migration check overhead eliminated (0ms vs 100-500ms)
• Project reconciliation overhead reduced by 90%+

2. Load Testing:

• Concurrent MCP tool calls maintain performance
• No memory leaks in cached connections
• Database connection pool behaves correctly

3. Functional Correctness:

• All existing API endpoints work identically
• MCP tools maintain full functionality
• CLI operations unaffected
• Database migrations still execute properly

4. Backwards Compatibility:

• No breaking changes to existing APIs
• Config changes are optional with safe defaults
• Non-cloud deployments work unchanged

Testing Strategy

Performance Testing:

Before optimization

time basic-memory-mcp-tools write_note "test" "content" "folder"

Measure: ~1-3 seconds

After optimization

time basic-memory-mcp-tools write_note "test" "content" "folder"

Target: <500ms

Load Testing:

Multiple concurrent MCP tool calls

for i in {1..10}; do basic-memory-mcp-tools search "test" & done wait

Verify: No degradation, consistent response times

Regression Testing:

Full basic-memory test suite

just test

All tests must pass

Integration tests with cloud deployment

Verify MCP gateway → API → database flow works

Validation Checklist

• Phase 1 Complete: Database connections cached, dependency injection optimized
• Performance Benchmark: 50%+ improvement in MCP tool response times
• Memory Usage: No leaks in cached connections over 24h+ periods
• Stress Testing: 100+ concurrent requests maintain performance
• Backwards Compatibility: All existing functionality preserved
• Documentation: Performance optimization documented in README
• Cloud Integration: basic-memory-cloud sees performance benefits

Notes

Implementation Priority:

• Phase 1 provides 80% of performance gains and should be implemented first
• Phase 2 provides remaining 20% and addresses edge cases
• Phase 3 is optional for maximum cloud optimization

Risk Mitigation:

• All changes backwards compatible
• Gradual rollout possible (Phase 1 → 2 → 3)
• Easy rollback via configuration flags

Cloud Integration:

• This optimization directly addresses basic-memory-cloud issue #82
• Changes in core basic-memory will benefit all cloud tenants
• No changes needed in basic-memory-cloud itself