-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Problem
The idempotency key uses git rev-parse --short HEAD which doesn't change when files are edited:
hash = execSync('git rev-parse --short HEAD', {
cwd: directory,
}).trim();Failure Scenario
1. explore_codebase(directory="/repo")
→ Key: "repo:supermodel:abc123"
→ Graph shows original code
2. Agent edits file to fix bug (uncommitted changes)
3. explore_codebase(directory="/repo") to validate fix
→ Key: "repo:supermodel:abc123" (SAME!)
→ Returns STALE GRAPH (doesn't see the edit!)
→ Agent thinks fix didn't work
→ False negatives, wasted attempts
Impact on SWE-BENCH
- Agents cannot validate their changes
- False positives/negatives in impact analysis
- Lower success rate due to confusion
- Multi-iteration fixes fail
Proposed Solutions
Option 1: Include working tree status hash
const statusOutput = execSync('git status --porcelain', { cwd: directory }).toString();
const statusHash = createHash('sha1').update(statusOutput).digest('hex').substring(0, 7);
return `${repoName}:supermodel:${hash}-${statusHash}`;Option 2: Hash file mtimes
const mtimeHash = await computeMtimeHash(directory);
return `${repoName}:supermodel:${hash}-${mtimeHash}`;Option 3: Document limitation
description: 'Idempotency key based on git HEAD. After editing, provide custom key to force re-analysis.'
Recommendation
Use Option 1: Include git status in key. Fast and accurate.
Priority
🔴 CRITICAL - Breaks iterative workflows, core to SWE-BENCH tasks
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working