Skip to content

feat: Implement cache eviction policy #27

@greynewell

Description

@greynewell

Problem

The in-memory cache in src/cache/graph-cache.ts has no size limits or eviction policy. With large codebases, this could consume unbounded memory as more graphs are cached.

Current Behavior

Graphs are cached indefinitely until the server restarts. No limits on:

  • Number of cached graphs
  • Total memory usage
  • Age of cached entries

Proposed Solution

Implement one or more eviction strategies:

1. LRU (Least Recently Used)

class LRUCache<K, V> {
  private maxSize: number;
  private cache: Map<K, { value: V; lastAccessed: number }>;
  
  set(key: K, value: V) {
    // Evict oldest if at capacity
  }
}

2. Size-Based Limit

const MAX_CACHE_SIZE = 10; // Maximum number of graphs
const MAX_MEMORY_MB = 500; // Maximum memory usage

3. TTL (Time To Live)

const CACHE_TTL = 24 * 60 * 60 * 1000; // 24 hours
// Auto-evict entries older than TTL

4. Hybrid Approach
Combine LRU + size limit + TTL for robust eviction.

Benefits

  • Prevent memory exhaustion
  • Predictable memory usage
  • Better production stability

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions