bug: Unbounded TTLCache Causes Memory Leak in Long-Running Processes

[ionfwsrijan]

Description

The TTLCache class in lib/cache.ts stores entries in a plain Map with no maximum size limit and no proactive cleanup of expired entries. Expired entries are only removed when they are accessed via get() (lazy eviction on read). In a long-running Node.js server (e.g., next start on Vercel or a self-hosted instance), if thousands of unique users request badges or profiles but many never return, their expired entries accumulate in memory indefinitely.

Over time this causes unbounded memory growth. A server handling 10,000 unique users per day with a 5-minute TTL will never reclaim memory from expired entries for users who don't make a second request. On memory-constrained deployments (serverless with warm instances, or small VPS instances), this can cause OOM crashes.

The three cache instances in lib/github.ts (contributionsCache, profileCache, reposCache) all share this same flaw, tripling the leak rate. Each cache grows without bound as unique users are served.

Affected Files

• lib/cache.ts (lines 1–28 — entire TTLCache class)
• lib/github.ts (lines 82–84 — three global cache instances using TTLCache)

Expected Behaviour

The cache should have a maximum size and/or periodically clean up expired entries to prevent unbounded memory growth. Expired entries should be proactively reclaimed even if no get() call hits that specific key.

Actual Behaviour

Expired entries remain in the Map forever until a get() call happens to hit that specific key, or clear() is called (which only happens in test helpers). In production, entries for users who never revisit accumulate indefinitely, causing the Map to grow without bound.

Proposed Fix

Add a maxSize parameter, LRU eviction on set(), and periodic cleanup of expired entries:

// lib/cache.ts — complete replacement

type CacheItem<T> = {
  value: T;
  expiresAt: number;
};

export class TTLCache<T> {
  private store = new Map<string, CacheItem<T>>();
  private maxSize: number;
  private cleanupTimer: ReturnType<typeof setInterval> | null = null;

  constructor(maxSize = 500, cleanupIntervalMs = 60_000) {
    this.maxSize = maxSize;

    // Periodically sweep expired entries so memory is reclaimed even
    // for keys that are never accessed again.
    if (cleanupIntervalMs > 0 && typeof setInterval !== 'undefined') {
      this.cleanupTimer = setInterval(
        () => this.evictExpired(),
        cleanupIntervalMs
      );
      // Allow Node.js to exit even if this timer is still running.
      if (
        this.cleanupTimer &&
        typeof this.cleanupTimer.unref === 'function'
      ) {
        this.cleanupTimer.unref();
      }
    }
  }

  get(key: string): T | null {
    const hit = this.store.get(key);
    if (!hit) return null;

    if (Date.now() > hit.expiresAt) {
      this.store.delete(key);
      return null;
    }

    return hit.value;
  }

  set(key: string, value: T, ttlMs: number): void {
    // If the key already exists, just update it (no size change).
    if (this.store.has(key)) {
      this.store.set(key, {
        value,
        expiresAt: Date.now() + ttlMs,
      });
      return;
    }

    // Evict expired entries first to free up space.
    this.evictExpired();

    // If still at capacity after cleanup, evict the oldest entry (FIFO).
    if (this.store.size >= this.maxSize) {
      const oldestKey = this.store.keys().next().value;
      if (oldestKey !== undefined) {
        this.store.delete(oldestKey);
      }
    }

    this.store.set(key, {
      value,
      expiresAt: Date.now() + ttlMs,
    });
  }

  clear(): void {
    this.store.clear();
    if (this.cleanupTimer) {
      clearInterval(this.cleanupTimer);
      this.cleanupTimer = null;
    }
  }

  /** Remove all entries whose TTL has elapsed. */
  private evictExpired(): void {
    const now = Date.now();
    for (const [key, item] of this.store) {
      if (now > item.expiresAt) {
        this.store.delete(key);
      }
    }
  }
}

Key changes:

1. maxSize constructor parameter (default 500): Caps the total number of entries per cache instance.
2. Periodic evictExpired sweep: Runs every 60 seconds via setInterval with .unref() so it doesn't keep the Node.js process alive. This proactively reclaims memory from expired entries that are never accessed again.
3. FIFO eviction on capacity: When set() is called and the cache is full (after expired cleanup), the oldest entry is removed.
4. evictExpired on set(): Before inserting, expired entries are swept to free space proactively.
5. typeof setInterval !== 'undefined' guard: Prevents errors in edge runtime or browser environments where setInterval may not be available or behave differently.

The existing callers in lib/github.ts can optionally be updated to pass a custom max size:

// lib/github.ts — update cache instantiation
const contributionsCache = new TTLCache<ContributionCalendar>(500);
const profileCache = new TTLCache<GitHubUserProfile>(500);
const reposCache = new TTLCache<GitHubRepo[]>(500);

This prevents the memory leak while maintaining the same caching behavior for active users.

[ionfwsrijan]

@JhaSourav07 Please review and assign this issue

[github-actions]

👋 Hey @ionfwsrijan! Looks like you want to work on this issue — awesome! 🎉

We don't assign issues through comments like this. Here's how our system works:

🤖 How to Claim an Issue

Comment /claim on this issue and our bot will automatically assign it to you (if you're eligible).

/claim

📋 A Few Things to Know

• You can hold a maximum of 5 open issues at a time.
• If there's no activity for 3 days, the assignment will automatically expire so others can pick it up.
• Make sure to read our CONTRIBUTING.md before you start — it covers code style, commit conventions, and the PR checklist.

💬 Join Our Discord

Get faster help, collaborate with other contributors, and stay updated with project news:

Happy contributing! 🚀

[ionfwsrijan]

/claim

[github-actions]

🎉 Assigned! Welcome to the project, @ionfwsrijan.

⏳ Reminder: You have 2 days to submit a Pull Request. After 2 days of inactivity, you will be automatically unassigned to give others a chance (as per our GSSoC anti-hoarding policy).

💡 Please read CONTRIBUTING.md if you haven't already.

Happy coding! 🚀

[github-actions]

⚠️ Assignment automatically removed due to inactivity.
Feel free to reclaim the issue if you want to continue working on it.

[Robo-Synaptic-Vaish]

/claim

[github-actions]

❌ Only the author of this issue (@ionfwsrijan) can claim it.