bug: 50MB ZIP limit fails on 70% of SWE-BENCH repos

[greynewell]

Problem

The 50MB ZIP size limit in src/utils/zip-repository.ts:114 will cause the tool to FAIL on the majority of SWE-BENCH benchmark repositories.

const maxSizeBytes = options.maxSizeBytes || 50 * 1024 * 1024; // 50MB default

Impact on SWE-BENCH Repos

• ✗ django/django: ~180MB → EXCEEDS LIMIT
• ✗ scikit-learn: ~120MB → EXCEEDS LIMIT
• ✗ matplotlib: ~100MB → EXCEEDS LIMIT
• ✗ sympy: ~200MB → EXCEEDS LIMIT
• ✗ pandas: ~150MB → EXCEEDS LIMIT
• ✓ pytest: ~15MB → OK
• ✓ requests: ~2MB → OK

Result: Tool unusable on ~70% of benchmark tasks, scores identical to baseline.

Proposed Solutions

Option 1: Raise default limit

const maxSizeBytes = options.maxSizeBytes || 500 * 1024 * 1024; // 500MB

Option 2: Make configurable

inputSchema: {
  properties: {
    maxZipSizeMB: {
      type: 'number',
      description: 'Max ZIP size in MB (default: 200)',
    }
  }
}

Option 3: Remove limit entirely
(API likely has its own limits)

Recommendation

Use Option 1 + 2: Raise default to 500MB and make it configurable for edge cases.

Priority

🔴 CRITICAL - Must fix before SWE-BENCH evaluation