Agent-sdk integration for SWE bench-styled tasks

plugins/openhands/platoon/openhands/agent.py

@@ -3,7 +3,7 @@
 import asyncio
 from copy import deepcopy
 from platoon.envs.base import Task
-from platoon.openhands.types import OpenHandsObservation, OpenHandsAction
+from .types import OpenHandsObservation, OpenHandsAction
 from platoon.utils.openhands_utils import get_actions_for_last_obs
 from platoon.utils.openhands_utils import is_finished
 

plugins/openhands/platoon/openhands/env.py

@@ -3,7 +3,7 @@
 from __future__ import annotations
 
 from platoon.envs.base import Task
-from .types import OpenHandsObservation, OpenHandsTrajectoryStep, OpenHandsAction
+from platoon.openhands.types import OpenHandsObservation, OpenHandsTrajectoryStep, OpenHandsAction
 from openhands.sdk.conversation import get_agent_final_response
 from openhands.sdk.conversation.base import BaseConversation
 from openhands.sdk.agent.base import AgentBase
@@ -75,6 +75,9 @@ async def step(self, action: OpenHandsAction) -> OpenHandsObservation:
             if self._state.conversation_state.agent_status == ConversationExecutionStatus.STUCK:
                 error_message.set("Agent got stuck")
                 self._state.misc["error_message"] = error_message.get()
+            elif self._state.conversation_state.agent_status == ConversationExecutionStatus.ERROR: #TODO: check
+                error_message.set("Agent encountered an error")
+                self._state.misc["error_message"] = error_message.get()
 
         traj_collection = current_trajectory_collection.get()
         traj = current_trajectory.get()
@@ -88,6 +91,9 @@ async def close(self) -> None:
         if self._conversation is not None:
             self._conversation.close()
         self._conversation = None
+        # TODO: check if cleaning up workspace manually is required
+        if isinstance(self._workspace, BaseWorkspace):
+            await self._workspace.cleanup()
 
     # TODO: Consider adding a return_copy option here.
     async def observe(self) -> OpenHandsObservation:

plugins/openhands_rl/README.md

@@ -0,0 +1,41 @@
+# platoon-openhands-rl
+
+Platoon plugin for intermediate rewards with the OpenHands software agent SDK.
+
+## Installation
+
+This plugin depends on:
+- **platoon** (core library)
+- **platoon-openhands** (OpenHands plugin)
+- **areal** backend (for RL training)
+
+### Prerequisites
+
+- Python 3.12
+- [uv](https://docs.astral.sh/uv/) package manager
+
+### Step-by-step installation
+
+We recommend installing into a dedicated virtual environment (not in home directory for Babel space usage constraints). The instructions below use a custom location (`/data/user_data/<username>/uv_cache/platoon/`), but you can use any path. In Babel, use a compute node (not a CPU node) so that GPU is detected during torch installation
+
+Assuming you are in project root directory.
+```bash
+# Create directory for the environment
+export VIRTUAL_ENV=/data/user_data/<username>/uv_cache/platoon
+export UV_CACHE_DIR=/data/user_data/<username>/uv_cache/.cache
+uv sync --active --extra areal --extra wandb
+mkdir -p /data/user_data/$USER/uv_cache/platoon
+source /data/user_data/<username>/uv_cache/platoon/bin/activate
+uv pip install -e plugins/openhands
+uv pip install -e plugins/openhands_rl
+```
+
+### Verify installation
+
+```bash
+python -c "
+from platoon.openhands import *
+from platoon.openhands_rl import *
+print('All packages imported successfully!')
+"
+```

plugins/openhands_rl/platoon/openhands_rl/env.py

@@ -0,0 +1,24 @@
+from pathlib import Path
+from platoon.utils.openhands_utils import is_finished
+from platoon.episode.context import current_trajectory_collection, current_trajectory, finish_message, error_message
+
+from openhands.sdk import get_logger
+from platoon.envs.base import Task
+from openhands.sdk.agent import AgentBase
+from openhands.sdk.workspace import BaseWorkspace
+from openhands.sdk.conversation import Conversation, BaseConversation, get_agent_final_response
+#TODO: check below imports
+from platoon.openhands.env import OpenHandsEnv
+from platoon.openhands.types import OpenHandsObservation, OpenHandsAction, OpenHandsTrajectoryStep
+import threading
+import asyncio
+from platoon.utils.openhands_utils import get_obs_for_last_action
+
+logger = get_logger(__name__)
+
+# TODO: double-check if we really need to over-ride any other methods from OpenHandsEnv
+# NOTE: The primary job of this class is to implement the step-wise reward functionality.
+class OpenHandsRLEnv(OpenHandsEnv):
+    async def evaluate(self) -> tuple[float, dict]:
+        raise NotImplementedError("OpenHandsRLEnv does not implement evaluate() --> this is something which Aditya will do soon.")
+        return 0., {}

plugins/openhands_rl/platoon/openhands_rl/prompts/default.j2

@@ -0,0 +1,63 @@
+I have access to a python code repository in the directory {{ instance.repo_path }} . You can explore and modify files using the available tools. Consider the following issue description:
+
+<issue_description>
+{{ instance.problem_statement }}
+</issue_description>
+
+Can you help me implement the necessary changes to the repository so that the requirements specified in the <issue_description> are met?
+I've already taken care of all changes to any of the test files described in the <issue_description>. This means you DON'T have to modify the testing logic or any of the tests in any way!
+Also the development Python environment is already set up for you (i.e., all dependencies already installed), so you don't need to install other packages.
+Your task is to make the minimal changes to non-test files in the {{ instance.repo_path }} directory to ensure the <issue_description> is satisfied.
+
+Follow these phases to resolve the issue:
+
+Phase 1. READING: read the problem and reword it in clearer terms
+   1.1 If there are code or config snippets. Express in words any best practices or conventions in them.
+   1.2 Hightlight message errors, method names, variables, file names, stack traces, and technical details.
+   1.3 Explain the problem in clear terms.
+   1.4 Enumerate the steps to reproduce the problem.
+   1.5 Hightlight any best practices to take into account when testing and fixing the issue
+
+Phase 2. RUNNING: install and run the tests on the repository
+   2.1 Activate the environment by running  
+       ./opt/miniconda3/etc/profile.d/conda.sh ; conda activate testbed
+   2.2 Follow the readme
+   2.3 Install the environment and anything needed
+   2.4 Iterate and figure out how to run the tests
+
+Phase 3. EXPLORATION: find the files that are related to the problem and possible solutions
+   3.1 Use `grep` to search for relevant methods, classes, keywords and error messages.
+   3.2 Identify all files related to the problem statement.
+   3.3 Propose the methods and files to fix the issue and explain why.
+   3.4 From the possible file locations, select the most likely location to fix the issue.
+
+Phase 4. TEST CREATION: before implementing any fix, create a script to reproduce and verify the issue.
+   4.1 Look at existing test files in the repository to understand the test format/structure.
+   4.2 Create a minimal reproduction script that reproduces the located issue.
+   4.3 Run the reproduction script to confirm you are reproducing the issue.
+   4.4 Adjust the reproduction script as necessary.
+
+Phase 5. FIX ANALYSIS: state clearly the problem and how to fix it
+   5.1 State clearly what the problem is.
+   5.2 State clearly where the problem is located.
+   5.3 State clearly how the test reproduces the issue.
+   5.4 State clearly the best practices to take into account in the fix.
+   5.5 State clearly how to fix the problem.
+
+Phase 6. FIX IMPLEMENTATION: Edit the source code to implement your chosen solution.
+   6.1 Make minimal, focused changes to fix the issue.
+
+Phase 7. VERIFICATION: Test your implementation thoroughly.
+   7.1 Run your reproduction script to verify the fix works.
+   7.2 Add edge cases to your test script to ensure comprehensive coverage.
+   7.3 Run existing tests related to the modified code to ensure you haven't broken anything.
+
+8. FINAL REVIEW: Carefully re-read the problem description and compare your changes with the base commit {{ instance.base_commit }}.
+   8.1 Ensure you've fully addressed all requirements.
+   8.2 Run any tests in the repository related to:
+     8.2.1 The issue you are fixing
+     8.2.2 The files you modified
+     8.2.3 The functions you changed
+   8.3 If any tests fail, revise your implementation until all tests pass
+
+Be thorough in your exploration, testing, and reasoning. It's fine if your thinking process is lengthy - quality and completeness are more important than brevity.