-
Notifications
You must be signed in to change notification settings - Fork 51
Add AutoGPT runaway task loop detection rule (CRE-2025-0171) #151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Sahelisaha04
wants to merge
2
commits into
prequel-dev:main
Choose a base branch
from
Sahelisaha04:cre-2025-0171-autogpt-runaway-loop
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| rules: | ||
| - cre: | ||
| id: CRE-2025-0171 | ||
| severity: 0 | ||
| title: AutoGPT Runaway Task Loop with Resource Exhaustion | ||
| category: task-management-problem | ||
| author: Claude Code Assistant | ||
| description: | | ||
| AutoGPT autonomous agent enters a runaway task execution loop, repeatedly attempting | ||
| the same failed operation without termination conditions. This critical failure pattern | ||
| causes resource exhaustion, infinite API calls, and system unresponsiveness. The agent | ||
| gets stuck in recursive task generation cycles, creating thousands of identical subtasks | ||
| that consume CPU, memory, and API quotas without making progress toward the original goal. | ||
| cause: | | ||
| * Faulty task completion detection logic allowing infinite retry cycles | ||
| * Missing or ineffective loop detection mechanisms in the task scheduler | ||
| * Prompt engineering flaws that cause the LLM to generate recursive task chains | ||
| * Resource limit bypass causing memory leaks in task queue management | ||
| * API rate limiting failures that don't properly halt task generation | ||
| * Context window overflow causing the agent to lose track of completed tasks | ||
| * Improper error handling that treats retryable failures as new tasks | ||
| tags: | ||
| - async | ||
| - batch | ||
| - retry | ||
| - timeout | ||
| - memory-leak | ||
| - memory-exhaustion | ||
| - cpu-memory-exhaustion | ||
| - crash-loop | ||
| - oom-kill | ||
| - container-crash | ||
| - critical-failure | ||
| - cascading-failure | ||
| - service-unavailability | ||
| - rate-limiting | ||
| - throttling | ||
| - api-key | ||
| - configuration | ||
| - deadlock | ||
| - performance | ||
| - scalability-issue | ||
| - infrastructure-failure | ||
| - capacity-exceeded | ||
| - revenue-impact | ||
| - public | ||
| mitigation: | | ||
| IMMEDIATE ACTIONS: | ||
| - **Kill AutoGPT process** - Terminate the runaway agent immediately to stop resource consumption | ||
| - **Review task queue** - Check for thousands of identical or similar pending tasks | ||
| - **Monitor API usage** - Check API quotas and potential overage charges | ||
| - **Clear persistent state** - Remove corrupted task history and agent memory | ||
|
|
||
| RECOVERY STEPS: | ||
| 1. Implement task loop detection with maximum retry limits (default: 3 attempts) | ||
| 2. Add resource monitoring with automatic circuit breakers | ||
| 3. Configure API rate limiting with exponential backoff | ||
| 4. Set maximum task depth limits to prevent infinite recursion | ||
| 5. Implement task deduplication to detect identical operations | ||
| 6. Add memory usage monitoring with automatic cleanup | ||
|
|
||
| PREVENTION: | ||
| - Configure task execution timeouts (recommend 5-10 minutes per task) | ||
| - Implement prompt validation to detect recursive instruction patterns | ||
| - Set up monitoring for repetitive API call patterns | ||
| - Use task completion validators to ensure progress measurement | ||
| - Implement graceful degradation when resource limits are approached | ||
| references: | ||
| - https://github.com/Significant-Gravitas/AutoGPT/issues/ | ||
| - https://docs.agpt.co/AutoGPT/configuration/ | ||
| - https://github.com/prequel-dev/cre/issues/129 | ||
| applications: | ||
| - name: autogpt | ||
| version: ">= 0.5.0" | ||
| processName: autogpt | ||
| containerName: autogpt | ||
| - name: auto-gpt | ||
| version: ">= 0.5.0" | ||
| processName: auto-gpt | ||
| impact: | | ||
| Runaway task loops can cause complete system resource exhaustion, leading to: | ||
| - Massive unexpected API costs (potentially thousands of dollars) | ||
| - System crashes due to memory/CPU exhaustion | ||
| - Service denial for other applications on the same host | ||
| - Data corruption from incomplete task executions | ||
| - Loss of agent work progress and context | ||
| - Potential security risks from uncontrolled API calls | ||
| The autonomous nature makes this particularly dangerous as it can continue for hours/days undetected. | ||
| impactScore: 10 | ||
| mitigationScore: 6 | ||
| metadata: | ||
| kind: prequel | ||
| id: AutoGPTRunawayTaskLoop171 | ||
| gen: 1 | ||
| rule: | ||
| sequence: | ||
| window: 180s | ||
| event: | ||
| source: cre.log.autogpt | ||
| order: | ||
| - regex: "(Starting task.*|Executing.*|Task created:|Creating subtask)" | ||
| - regex: "(Starting task.*|Executing.*|Task created:|Creating subtask)" | ||
| - regex: "(Starting task.*|Executing.*|Task created:|Creating subtask)" | ||
| - regex: "(Memory usage.*[0-9]+MB|CPU usage.*[0-9]+%|API calls.*rate limit|Resource exhaustion detected|Out of memory|System overload)" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| 2024-08-31 14:22:15,123 [INFO] AutoGPT: Starting task: analyze_market_data_for_investment_recommendations | ||
| 2024-08-31 14:22:16,445 [DEBUG] Task Manager: Creating subtask: fetch_stock_data | ||
| 2024-08-31 14:22:17,889 [INFO] AutoGPT: Executing task: fetch_stock_data | ||
| 2024-08-31 14:22:18,234 [ERROR] API Client: Rate limit exceeded, retrying in 60s | ||
| 2024-08-31 14:22:19,567 [INFO] AutoGPT: Task created: retry_fetch_stock_data | ||
| 2024-08-31 14:22:20,123 [INFO] AutoGPT: Starting task: retry_fetch_stock_data | ||
| 2024-08-31 14:22:21,445 [DEBUG] Task Manager: Creating subtask: fetch_stock_data | ||
| 2024-08-31 14:22:22,889 [INFO] AutoGPT: Executing task: fetch_stock_data | ||
| 2024-08-31 14:22:23,234 [ERROR] API Client: Rate limit exceeded, retrying in 60s | ||
| 2024-08-31 14:22:24,567 [INFO] AutoGPT: Task created: retry_fetch_stock_data_attempt_2 | ||
| 2024-08-31 14:22:25,123 [INFO] AutoGPT: Starting task: retry_fetch_stock_data_attempt_2 | ||
| 2024-08-31 14:22:26,445 [DEBUG] Task Manager: Creating subtask: fetch_stock_data | ||
| 2024-08-31 14:22:27,889 [INFO] AutoGPT: Executing task: fetch_stock_data | ||
| 2024-08-31 14:22:28,234 [ERROR] API Client: Rate limit exceeded, retrying in 60s | ||
| 2024-08-31 14:22:29,567 [INFO] AutoGPT: Task created: retry_fetch_stock_data_attempt_3 | ||
| 2024-08-31 14:22:30,123 [WARNING] System Monitor: Memory usage 2847MB, approaching limit | ||
| 2024-08-31 14:22:31,445 [WARNING] System Monitor: CPU usage 89%, system overload detected | ||
| 2024-08-31 14:22:32,889 [ERROR] Resource Monitor: API calls 1247/hour, rate limit threshold exceeded | ||
| 2024-08-31 14:22:33,234 [CRITICAL] System: Resource exhaustion detected - AutoGPT consuming 94% CPU, 3.2GB RAM |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the regex expression: regex: "(Starting task.|Executing.|Task created:|Creating subtask)" is being repeated. Also, this regex is noisy as "starting task", "executing task" and "creating subtask" are informational messages, we don't want our CRE to trigger when this is logged. we can remove these.