Remote Python Execution #2
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit introduces a significant new feature: the ability to execute Python code blocks remotely over SSH instead of only locally. The implementation includes: - SSH connection management functions - Remote environment setup and testing - Secure code transfer and execution - Command-line arguments for SSH configuration This enhances the site generator's flexibility by allowing it to leverage remote Python environments when needed.
Enhance `site-gen.bb` by adding a comprehensive remote execution system with source code transfer and dependency management via uv
Add comments to explain SSH commands and remote execution
This commit enhances the site generator with: - Configurable cache directory via SITE_GEN_CACHE_DIR environment variable - Robust utilities for safer remote execution: - Security-focused input sanitization to prevent shell injection - Progress spinner for long-running remote tasks - File locking to prevent cache corruption - Improved error handling with timeouts for remote operations - Remote environment cleanup after execution
Add PATH to find uv in remote SSH commands
- Add setup.sh to handle git-lfs and Llama model installation - Support SSH key transfer and git authentication - Add setup status monitoring and verification - Update remote environment cleanup handling
The change relocates model storage to a permanent system directory and adds existence checks before downloading.
- Add functions to find SSH port, format connection strings, and display SSH details - Expand pod management menu with SSH connection option - Fix indentation in GraphQL query strings
This tool integrates with site-gen.bb to execute code blocks in markdown files using RunPod. It manages the lifecycle of pods, including creation, SSH connection, execution, and automatic shutdown after an idle period.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
site-gen.bb- can generate the site on a remote machine using ssh/scp.runpod.bb- tui to manage runpod infra.runpodmd.bb- tui to manage launching site-gen builds on runpod.Note I haven't yet figured out how to connect over shh to a machine created with the runpod graphql api.