Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 84 additions & 34 deletions .cursor/commands/qa.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,20 +48,21 @@ Use the built CLI binary with non-interactive flags. The command format is:

Here are all valid language + template combinations:

| Language | Template | Folder Name | Needs Env File | Required Env Vars |
| ---------- | -------------- | ----------------- | -------------- | ------------------------------ |
| typescript | sample-app | ts-sample-app | No | - |
| typescript | captcha-solver | ts-captcha-solver | No | - |
| typescript | stagehand | ts-stagehand | Yes | OPENAI_API_KEY |
| typescript | computer-use | ts-computer-use | Yes | ANTHROPIC_API_KEY |
| typescript | magnitude | ts-magnitude | Yes | ANTHROPIC_API_KEY |
| typescript | cua | ts-cua | Yes | OPENAI_API_KEY |
| typescript | gemini-cua | ts-gemini-cua | Yes | GOOGLE_API_KEY, OPENAI_API_KEY |
| python | sample-app | py-sample-app | No | - |
| python | captcha-solver | py-captcha-solver | No | - |
| python | browser-use | py-browser-use | Yes | OPENAI_API_KEY |
| python | computer-use | py-computer-use | Yes | ANTHROPIC_API_KEY |
| python | cua | py-cua | Yes | OPENAI_API_KEY |
| Language | Template | Folder Name | Deployed App Name | Needs Env File | Required Env Vars |
| ---------- | ---------------------- | ----------------- | --------------------- | -------------- | ------------------------------ |
| typescript | sample-app | ts-sample-app | ts-basic | No | - |
| typescript | captcha-solver | ts-captcha-solver | ts-captcha-solver | No | - |
| typescript | stagehand | ts-stagehand | ts-stagehand | Yes | OPENAI_API_KEY |
| typescript | anthropic-computer-use | ts-anthropic-cua | ts-anthropic-cua | Yes | ANTHROPIC_API_KEY |
| typescript | magnitude | ts-magnitude | ts-magnitude | Yes | ANTHROPIC_API_KEY |
| typescript | openai-computer-use | ts-openai-cua | ts-openai-cua | Yes | OPENAI_API_KEY |
| typescript | gemini-computer-use | ts-gemini-cua | ts-gemini-cua | Yes | GOOGLE_API_KEY, OPENAI_API_KEY |
| python | sample-app | py-sample-app | python-basic | No | - |
| python | captcha-solver | py-captcha-solver | python-captcha-solver | No | - |
| python | browser-use | py-browser-use | python-bu | Yes | OPENAI_API_KEY |
| python | anthropic-computer-use | py-anthropic-cua | python-anthropic-cua | Yes | ANTHROPIC_API_KEY |
| python | openai-computer-use | py-openai-cua | python-openai-cua | Yes | OPENAI_API_KEY |
| python | openagi-computer-use | py-openagi-cua | python-openagi-cua | Yes | OAGI_API_KEY |

### Create Commands

Expand All @@ -72,17 +73,18 @@ Run each of these (they are non-interactive when all flags are provided):
../bin/kernel create -n ts-sample-app -l typescript -t sample-app
../bin/kernel create -n ts-captcha-solver -l typescript -t captcha-solver
../bin/kernel create -n ts-stagehand -l typescript -t stagehand
../bin/kernel create -n ts-computer-use -l typescript -t computer-use
../bin/kernel create -n ts-anthropic-cua -l typescript -t anthropic-computer-use
../bin/kernel create -n ts-magnitude -l typescript -t magnitude
../bin/kernel create -n ts-cua -l typescript -t cua
../bin/kernel create -n ts-gemini-cua -l typescript -t gemini-cua
../bin/kernel create -n ts-openai-cua -l typescript -t openai-computer-use
../bin/kernel create -n ts-gemini-cua -l typescript -t gemini-computer-use

# Python templates
../bin/kernel create -n py-sample-app -l python -t sample-app
../bin/kernel create -n py-captcha-solver -l python -t captcha-solver
../bin/kernel create -n py-browser-use -l python -t browser-use
../bin/kernel create -n py-computer-use -l python -t computer-use
../bin/kernel create -n py-cua -l python -t cua
../bin/kernel create -n py-anthropic-cua -l python -t anthropic-computer-use
../bin/kernel create -n py-openai-cua -l python -t openai-computer-use
../bin/kernel create -n py-openagi-cua -l python -t openagi-computer-use
```

## Step 5: Deploy Each Template
Expand Down Expand Up @@ -125,10 +127,10 @@ echo "OPENAI_API_KEY=<value from human>" > .env
cd ..
```

**ts-computer-use** (needs ANTHROPIC_API_KEY):
**ts-anthropic-cua** (needs ANTHROPIC_API_KEY):

```bash
cd ts-computer-use
cd ts-anthropic-cua
echo "ANTHROPIC_API_KEY=<value from human>" > .env
../bin/kernel deploy index.ts --env-file .env
cd ..
Expand All @@ -143,10 +145,10 @@ echo "ANTHROPIC_API_KEY=<value from human>" > .env
cd ..
```

**ts-cua** (needs OPENAI_API_KEY):
**ts-openai-cua** (needs OPENAI_API_KEY):

```bash
cd ts-cua
cd ts-openai-cua
echo "OPENAI_API_KEY=<value from human>" > .env
../bin/kernel deploy index.ts --env-file .env
cd ..
Expand All @@ -173,24 +175,33 @@ echo "OPENAI_API_KEY=<value from human>" > .env
cd ..
```

**py-computer-use** (needs ANTHROPIC_API_KEY):
**py-anthropic-cua** (needs ANTHROPIC_API_KEY):

```bash
cd py-computer-use
cd py-anthropic-cua
echo "ANTHROPIC_API_KEY=<value from human>" > .env
../bin/kernel deploy main.py --env-file .env
cd ..
```

**py-cua** (needs OPENAI_API_KEY):
**py-openai-cua** (needs OPENAI_API_KEY):

```bash
cd py-cua
cd py-openai-cua
echo "OPENAI_API_KEY=<value from human>" > .env
../bin/kernel deploy main.py --env-file .env
cd ..
```

**py-openagi-cua** (needs OAGI_API_KEY):

```bash
cd py-openagi-cua
echo "OAGI_API_KEY=<value from human>" > .env
../bin/kernel deploy main.py --env-file .env
cd ..
```

## Step 6: Provide Invoke Commands

Once all deployments are complete, present the human with these invoke commands to test manually:
Expand All @@ -200,28 +211,67 @@ Once all deployments are complete, present the human with these invoke commands
kernel invoke ts-basic get-page-title --payload '{"url": "https://www.google.com"}'
kernel invoke ts-captcha-solver test-captcha-solver
kernel invoke ts-stagehand teamsize-task --payload '{"company": "Kernel"}'
kernel invoke ts-cu cu-task --payload '{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}'
kernel invoke ts-anthropic-cua cua-task --payload '{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}'
kernel invoke ts-magnitude mag-url-extract --payload '{"url": "https://en.wikipedia.org/wiki/Special:Random"}'
kernel invoke ts-cua cua-task --payload '{"task": "Go to https://news.ycombinator.com and get the top 5 articles"}'
kernel invoke ts-openai-cua cua-task --payload '{"task": "Go to https://news.ycombinator.com and get the top 5 articles"}'
kernel invoke ts-gemini-cua gemini-cua-task

# Python apps
kernel invoke python-basic get-page-title --payload '{"url": "https://www.google.com"}'
kernel invoke python-captcha-solver test-captcha-solver
kernel invoke python-bu bu-task --payload '{"task": "Compare the price of gpt-4o and DeepSeek-V3"}'
kernel invoke python-cu cu-task --payload '{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}'
kernel invoke python-cua cua-task --payload '{"task": "Go to https://news.ycombinator.com and get the top 5 articles"}'
kernel invoke python-anthropic-cua cua-task --payload '{"query": "Return the first url of a search result for NYC restaurant reviews Pete Wells"}'
kernel invoke python-openai-cua cua-task --payload '{"task": "Go to https://news.ycombinator.com and get the top 5 articles"}'
kernel invoke python-openagi-cua openagi-default-task -p '{"instruction": "Navigate to https://agiopen.org and click the What is Computer Use? button"}'
```

## Step 7: Automated Runtime Testing (Optional)

**STOP and ask the human:** "Would you like me to automatically invoke all 13 templates and report back on their runtime status?"

If the human agrees, invoke each template and collect results. Present findings in this format:

### Testing Guidelines

- **Timeout:** Cancel each invocation after 90 seconds if it has not completed. Mark the status as `TIMEOUT` in the results table.
- **Parallel execution:** You may run multiple invocations in parallel to speed up testing.
- **Error handling:** Capture any runtime errors and include them in the Notes column.

### Test Results

| Template | App Name | Status | Notes |
| ----------------- | --------------------- | ------- | ----- |
| ts-sample-app | ts-basic | | |
| ts-captcha-solver | ts-captcha-solver | | |
| ts-stagehand | ts-stagehand | | |
| ts-anthropic-cua | ts-anthropic-cua | | |
| ts-magnitude | ts-magnitude | | |
| ts-openai-cua | ts-openai-cua | | |
| ts-gemini-cua | ts-gemini-cua | | |
| py-sample-app | python-basic | | |
| py-captcha-solver | python-captcha-solver | | |
| py-browser-use | python-bu | | |
| py-anthropic-cua | python-anthropic-cua | | |
| py-openai-cua | python-openai-cua | | |
| py-openagi-cua | python-openagi-cua | | |

Status values:
- **SUCCESS**: App started and returned a result
- **FAILED**: App encountered a runtime error
- **TIMEOUT**: App did not complete within 90 seconds (cancelled)

Notes should include brief error messages for failures or confirmation of successful output.

## Summary Checklist

- [ ] Built CLI with `make build`
- [ ] Created QA directory
- [ ] Got KERNEL_API_KEY from human
- [ ] Created all 12 template variations
- [ ] Got required API keys from human (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY)
- [ ] Deployed all 12 apps
- [ ] Created all 13 template variations
- [ ] Got required API keys from human (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, OAGI_API_KEY)
- [ ] Deployed all 13 apps
- [ ] Provided invoke commands to human for manual testing
- [ ] (Optional) Ran automated runtime testing and reviewed results

## Cleanup

Expand Down
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,12 +116,13 @@ Create an API key from the [Kernel dashboard](https://dashboard.onkernel.com).
- `--language <language>`, `-l` - Sepecify app language: `typescript`, or `python`
- `--template <template>`, `-t` - Template to use:
- `sample-app` - Basic template with Playwright integration
- `advanced-sample` - Sample apps using advanced Kernel configs
- `captcha-solver` - Template demonstrating Kernel's auto-CAPTCHA solver
- `stagehand` - Template with Stagehand SDK (TypeScript only)
- `browser-use` - Template with Browser Use SDK (Python only)
- `computer-use` - Anthropic Computer Use prompt loop
- `cua` - OpenAI Computer Using Agent (CUA) sample
- `gemini-cua` - Google Gemini CUA sample (TypeScript only)
- `anthropic-computer-use` - Anthropic Computer Use prompt loop
- `openai-computer-use` - OpenAI Computer Use Agent sample
- `gemini-computer-use` - Gemini Computer Use Agent sample (TypeScript only)
- `openagi-computer-use` - OpenAGI Lux computer-use models (Python only)
- `magnitude` - Magnitude framework sample (TypeScript only)

### App Deployment
Expand Down Expand Up @@ -393,7 +394,7 @@ kernel create --name my-scraper --language python --template browser-use
kernel create --name my-agent --language ts --template stagehand

# Create a Python Computer Use app
kernel create --name my-cu-app --language py --template computer-use
kernel create --name my-cu-app --language py --template anthropic-computer-use
```

### Deploy with environment variables
Expand Down
8 changes: 4 additions & 4 deletions cmd/create_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -440,10 +440,10 @@ func TestCreateCommand_InvalidLanguageTemplateCombinations(t *testing.T) {
errContains: "template not found: python/magnitude",
},
{
name: "gemini-cua not available for python",
name: "gemini-computer-use not available for python",
language: create.LanguagePython,
template: create.TemplateGeminiCUA,
errContains: "template not found: python/gemini-cua",
template: create.TemplateGeminiComputerUse,
errContains: "template not found: python/gemini-computer-use",
},
{
name: "invalid language",
Expand Down Expand Up @@ -558,7 +558,7 @@ func TestCreateCommand_TemplateNotAvailableForLanguage(t *testing.T) {
create.TemplateBrowserUse: {create.LanguageTypeScript},
create.TemplateStagehand: {create.LanguagePython},
create.TemplateMagnitude: {create.LanguagePython},
create.TemplateGeminiCUA: {create.LanguagePython},
create.TemplateGeminiComputerUse: {create.LanguagePython},
}

for template, unavailableLanguages := range unavailableCombinations {
Expand Down
4 changes: 2 additions & 2 deletions pkg/create/copy_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -179,10 +179,10 @@ func TestCopyTemplateFiles_PreservesDirectoryStructure(t *testing.T) {
require.NoError(t, err)

// Use a template that has subdirectories
err = CopyTemplateFiles(appPath, LanguageTypeScript, TemplateComputerUse)
err = CopyTemplateFiles(appPath, LanguageTypeScript, TemplateAnthropicComputerUse)
require.NoError(t, err)

// Verify that subdirectories are created (computer-use has src/ directory)
// Verify that subdirectories are created (anthropic-computer-use has src/ directory)
srcDir := filepath.Join(appPath, "src")
if _, err := os.Stat(srcDir); err == nil {
assert.DirExists(t, srcDir, "Subdirectories should be preserved")
Expand Down
Loading