Add Runway Gen-4 text-to-image node for ComfyUI integration #82
Conversation
WalkthroughA new node class Changes
Sequence Diagram(s)sequenceDiagram
participant Test as Test Function
participant Env as Environment (API Key)
participant HTTP as HTTP (POST/GET)
participant Node as RunwayText2Image
Test->>Env: Mock API Key
Test->>HTTP: Mock POST (image generation request)
HTTP-->>Test: Return image URL
Test->>HTTP: Mock GET (download image)
HTTP-->>Test: Return dummy image bytes
Test->>Node: Call generate_image(prompt)
Node-->>Test: Return PIL Image
Estimated code review effort🎯 2 (Simple) | ⏱️ ~7 minutes Poem
Note ⚡️ Unit Test Generation is now available in beta!Learn more here, or try it out under "Finishing Touches" below. 📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
Reviewer's GuideThis PR adds a new ComfyUI node, RunwayText2Image, to interface with the Runway Gen-4 text-to-image API by sending HTTP requests, handling environment-based authentication, and returning PIL Image outputs, along with corresponding unit tests. File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey @wazmaaali - I've reviewed your changes - here's some feedback:
- Consider exposing key API parameters like width, height, and steps as node inputs so users can customize the output resolution and quality directly in the UI.
- Enhance error handling by capturing non-200 responses and including the status code and error message in the raised exception to help users diagnose API issues faster.
- Implement retry/backoff logic for transient network failures or rate limiting (e.g., 429 responses) to make the node more robust under unreliable network conditions.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- Consider exposing key API parameters like width, height, and steps as node inputs so users can customize the output resolution and quality directly in the UI.
- Enhance error handling by capturing non-200 responses and including the status code and error message in the raised exception to help users diagnose API issues faster.
- Implement retry/backoff logic for transient network failures or rate limiting (e.g., 429 responses) to make the node more robust under unreliable network conditions.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
Added runway_text2img node for DreamLayer open-source challenge (Task #1). |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
ComfyUI/tests/api_nodes/test_runway_text2img.py (1)
15-38: Comprehensive test with minor improvement opportunities.The test effectively covers the success scenario with proper mocking. Consider these enhancements:
- mock_post.return_value.json.return_value = {"image_url": "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTwBchNsxAEthMtT_uv1MInGKEi4A0W2b1mx1flcpNOoUMkiy0CCnLfKF55jqIiRB9Mx-Y&usqp=CAU"} + mock_post.return_value.json.return_value = {"image_url": "https://example.com/test-image.png"}Additionally, consider adding assertions to verify the API calls:
# Verify POST was called with correct parameters mock_post.assert_called_once() # Verify GET was called to download the image mock_get.assert_called_once_with("https://example.com/test-image.png")
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
ComfyUI/tests/api_nodes/test_runway_text2img.py(1 hunks)
🔇 Additional comments (3)
ComfyUI/tests/api_nodes/test_runway_text2img.py (3)
1-5: LGTM! Clean import structure.The imports are well-organized and include all necessary dependencies for comprehensive testing with proper mocking capabilities.
7-13: LGTM! Well-implemented test fixture.The fixture efficiently creates minimal test data and properly returns PNG bytes that can be used to mock HTTP responses. The implementation is clean and well-documented.
40-46: LGTM! Solid error handling test.The test properly validates the missing API key scenario using pytest.raises with message matching. This ensures robust error handling and clear error messages for users.
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
ComfyUI/custom_nodes/runway_text2img.py (1)
12-36: Excellent documentation, but reconsider inheritance choice.The docstring is comprehensive and well-structured, clearly explaining usage, parameters, and requirements.
However, inheriting from
CLIPTextEncodemay not be semantically correct since this node generates images rather than encoding text. Consider inheriting from a more appropriate base class or directly from the ComfyUI node base class.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
ComfyUI/custom_nodes/runway_text2img.py(1 hunks)
🧰 Additional context used
🪛 Ruff (0.12.2)
ComfyUI/custom_nodes/runway_text2img.py
6-6: Redefinition of unused os from line 1
Remove definition: os
(F811)
🔇 Additional comments (2)
ComfyUI/custom_nodes/runway_text2img.py (2)
38-45: LGTM!The INPUT_TYPES method follows ComfyUI conventions correctly with appropriate parameter types, defaults, and constraints.
78-84: LGTM!Node mappings follow ComfyUI conventions correctly with appropriate class registration and user-friendly display names.
| import os | ||
| import requests | ||
| from io import BytesIO | ||
| from PIL import Image | ||
| import sys | ||
| import os | ||
| sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../../..'))) |
There was a problem hiding this comment.
Fix duplicate import and simplify path handling.
The os module is imported twice (lines 1 and 6), and the sys.path manipulation is unnecessarily complex.
Apply this diff to fix the duplicate import:
import os
import requests
from io import BytesIO
from PIL import Image
import sys
-import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../../..')))Consider whether the sys.path manipulation is necessary - using relative imports or proper package structure might be cleaner.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import os | |
| import requests | |
| from io import BytesIO | |
| from PIL import Image | |
| import sys | |
| import os | |
| sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../../..'))) | |
| import os | |
| import requests | |
| from io import BytesIO | |
| from PIL import Image | |
| import sys | |
| sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '../../..'))) |
🧰 Tools
🪛 Ruff (0.12.2)
6-6: Redefinition of unused os from line 1
Remove definition: os
(F811)
🤖 Prompt for AI Agents
In ComfyUI/custom_nodes/runway_text2img.py lines 1 to 7, remove the duplicate
import of the os module and simplify the sys.path insertion by using a clearer,
more straightforward path expression. Also, evaluate if modifying sys.path is
necessary at all; if possible, replace it with relative imports or adjust the
package structure to avoid this manipulation.
| def generate_image(self, prompt, poll_timeout): | ||
| api_key = os.getenv("RUNWAY_API_KEY") | ||
| if not api_key: | ||
| raise RuntimeError("Missing environment variable: RUNWAY_API_KEY") | ||
|
|
||
| headers = {"Authorization": f"Bearer {api_key}"} | ||
| payload = {"prompt": prompt} | ||
|
|
||
| try: | ||
| response = requests.post( | ||
| "https://api.dev.runwayml.com/v1/text_to_image", | ||
| json=payload, | ||
| headers=headers, | ||
| timeout=poll_timeout | ||
| ) | ||
| response.raise_for_status() | ||
| image_url = response.json().get("image_url") | ||
| if not image_url: | ||
| raise ValueError("No image URL returned from Runway API") | ||
|
|
||
| image_data = requests.get(image_url).content | ||
| image = Image.open(BytesIO(image_data)).convert("RGB") | ||
|
|
||
| return (image,) | ||
|
|
||
| except Exception as e: | ||
| raise RuntimeError(f"RunwayText2Image generation failed: {e}") | ||
|
|
There was a problem hiding this comment.
Add timeout and error handling for image download.
The image download request (line 70) lacks timeout and error handling, which could cause the node to hang indefinitely or fail silently.
Apply this diff to improve error handling:
image_url = response.json().get("image_url")
if not image_url:
raise ValueError("No image URL returned from Runway API")
- image_data = requests.get(image_url).content
+ image_response = requests.get(image_url, timeout=30)
+ image_response.raise_for_status()
+ image_data = image_response.content
image = Image.open(BytesIO(image_data)).convert("RGB")Additionally, consider more specific exception handling instead of catching all exceptions on line 75.
🤖 Prompt for AI Agents
In ComfyUI/custom_nodes/runway_text2img.py lines 50 to 77, the image download
request lacks a timeout and error handling, which can cause hangs or silent
failures. Add a timeout parameter to the requests.get call for image_data and
wrap it in a try-except block to catch request exceptions specifically. Also,
replace the broad except Exception on line 75 with more specific exceptions like
requests.RequestException and handle them accordingly to improve error clarity
and robustness.
1 participant
Description
Brief description of what this PR does.
Changes Made
-Changes Made
Added runway_text2img.py under custom_nodes/
Created a node that sends a POST request to Runway's /v1/text_to_image endpoint
Integrated node to accept prompt from upstream (e.g., CLIPTextEncode)
Implemented environment variable check for RUNWAY_API_KEY
Added docstring for usage, parameters, defaults, and timeout config
Ensured compatibility with ComfyUI node system
Handled missing API key with a clear error message
CI test file tests/api_nodes/test_runway_text2img.py passes (with mocked HTTP requests)
Evidence Required ✅
UI Screenshot
Generated Image
Logs
Tests (Optional)
platform darwin -- Python 3.11.8, pytest-8.0.0
collected 1 item
tests/api_nodes/test_runway_text2img.py .
Checklist
Summary by Sourcery
Add a Runway Gen-4 text-to-image ComfyUI node that retrieves the API key from the environment, handles missing keys gracefully, and includes unit tests with mocked HTTP requests.
New Features:
Enhancements:
Tests:
Summary by CodeRabbit