Thirdweb Customer Support Agent

A customer support agent built with the Anthropic Claude Agent SDK to help developers with thirdweb SDKs and APIs.

Available in TypeScript (stateless queries) and Python (conversational with context).

Features

💬 Interactive CLI Chat: Chat with the agent in your terminal
🔄 Two Implementations: TypeScript (single queries) and Python (conversations with memory)
🤖 AI-Powered Support: Uses Claude to understand and answer developer questions
📚 Comprehensive Documentation: Access to 1,787 thirdweb documentation files
🔍 Hybrid RAG Search: Combines semantic vector search with agent verification for accurate answers
💻 Code Examples: Provides accurate code examples from official documentation
⚡ Fast Responses: Vector embeddings enable 2-3x faster queries with 50% cost reduction
🎛️ Model Selection: Switch between Haiku, Sonnet, and Opus models

Documentation Coverage

React, React Native, TypeScript SDKs
.NET, Unity, Unreal Engine SDKs
Engine, Payments, Insight APIs
Smart contracts, wallets, transactions
Knowledge base and tutorials

Setup

1. Install Dependencies

# Install all dependencies (TypeScript + Claude Code CLI + Python)
npm install
npm run setup:py

2. Configure Environment Variables

cp .env.example .env

Add the following to .env:

# Required for Claude API
ANTHROPIC_API_KEY=your_anthropic_api_key_here

# Required for vector embeddings
OPENAI_API_KEY=your_openai_api_key_here

# Required for Supabase vector database
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_SERVICE_ROLE_KEY=your_service_role_key_here

# Optional: default model (haiku, haiku 3.5, sonnet 3.5, sonnet 4, sonnet 4.5, opus)
CLAUDE_MODEL=sonnet 3.5

3. Set Up Supabase Vector Database

Option A: Using Supabase Cloud

Create a new project at supabase.com
Go to the SQL Editor in your Supabase dashboard

Run the migration script:

# Copy the contents of scripts/supabase-migration.sql
# Paste and execute in Supabase SQL Editor

Option B: Using Local Supabase

# Install Supabase CLI
npm install -g supabase

# Start local Supabase
supabase start

# Run migration
supabase db reset

4. Generate Vector Embeddings

This step processes all 1,787 documentation files and creates embeddings:

# Chunk documents and generate embeddings (~5-10 minutes)
npm run setup:embeddings

Cost estimate: ~$0.02 for embedding generation using OpenAI text-embedding-3-small

This will:

Split docs into 800-1000 token chunks (~3,500 chunks)
Generate embeddings using OpenAI API
Upload to Supabase with metadata

Progress is saved - if interrupted, re-run to resume from last checkpoint.

Documentation Refresh

The documentation and embeddings can be refreshed to stay up-to-date with the latest thirdweb docs.

Daily Refresh (Full Update)

Run this command to download the latest docs, re-chunk them, and regenerate all embeddings:

npm run daily:refresh

This will:

Download latest documentation from thirdweb GitHub repo
Download latest OpenAPI specs and LLM txt files
Chunk all documentation into 800-1000 token chunks
Truncate existing embeddings from Supabase
Generate fresh embeddings for all chunks (~20-30 minutes)

Cost: ~$0.02 per refresh

Recommended schedule: Daily via cron job or GitHub Actions

Selective Updates

You can update specific parts without a full refresh:

# Download only OpenAPI files (Engine, Payments, Insight, HTTP API)
npm run download:docs:openapi

# Download only LLM txt files (api-llms.txt, llms-full.txt)
npm run download:docs:llm

# Download all docs (MDX + OpenAPI + LLM txt)
npm run download:docs

# Re-chunk existing docs (after manual doc changes)
npm run chunk:docs

# Truncate database (clear all embeddings)
npm run truncate:db

# Generate embeddings (from existing chunks.json)
npm run generate:embeddings

Automating Daily Refresh

Option A: Cron Job (Linux/Mac)

# Edit crontab
crontab -e

# Add this line to run daily at 2 AM
0 2 * * * cd /path/to/thirdweb-docs-agent && npm run daily:refresh >> logs/refresh.log 2>&1

Option B: GitHub Actions

Create .github/workflows/daily-refresh.yml:

name: Daily Documentation Refresh

on:
  schedule:
    - cron: '0 2 * * *'  # Run daily at 2 AM UTC
  workflow_dispatch:  # Allow manual trigger

jobs:
  refresh:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
      - run: npm install
      - run: npm run daily:refresh
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
          SUPABASE_SERVICE_ROLE_KEY: ${{ secrets.SUPABASE_SERVICE_ROLE_KEY }}

5. Start Chatting

# Python version (recommended - conversational with memory)
npm run chat:py

# Python with specific model
npm run chat:py -- haiku 3.5
npm run chat:py -- sonnet 4

# TypeScript version (stateless)
npm run chat:ts

# With custom max_turns (default is 10)
npm run chat:py -- --max-turns 20
npm run chat:py -- --model sonnet 4 --max-turns 15

Usage

Python CLI (Recommended - Conversational) 🌟

The Python implementation uses ClaudeSDKClient which maintains conversation history. Claude remembers previous questions and context!

# Default model
npm run chat:py

# Or specify a model at startup
npm run chat:py -- haiku 3.5
npm run chat:py -- sonnet 4
npm run chat:py -- opus

# Customize max_turns (default is 10)
npm run chat:py -- --max-turns 20
npm run chat:py -- --model sonnet 4 --max-turns 15

Features:

✅ Maintains conversation context - Ask follow-up questions!
✅ Remembers previous answers - No need to repeat context
✅ Better for multi-turn conversations - Natural support dialogues
✅ Can reference earlier exchanges - "What did you say about that?"
✅ Specify model at startup - Set the model when you start the session

TypeScript CLI (Stateless)

The TypeScript implementation processes each query independently without conversation history.

npm run chat:ts
# or simply
npm run chat

# With command line options
npm run chat:ts -- --model haiku --max-turns 15

Features:

Single-turn queries
Each question is independent
Good for quick lookups
Supports --model and --max-turns command line arguments

CLI Commands

Both implementations support:

Type your question and press Enter to get an answer
Type model <name> to switch models (e.g., model haiku, model sonnet 4)
Type exit or quit to end the session
Type clear to clear the screen

Available models:

haiku - Claude 3 Haiku (cheapest, fastest)
haiku 3.5 - Claude 3.5 Haiku
sonnet 3.5 - Claude 3.5 Sonnet
sonnet 3.7 - Claude 3.7 Sonnet
sonnet 4 - Claude Sonnet 4
sonnet 4.5 - Claude Sonnet 4.5 (latest)
opus - Claude 3 Opus (most capable)

Example questions:

"How do I create a wallet connection in React?"
"What is thirdweb Engine?"
"Show me how to deploy a contract with TypeScript"

Programmatic Usage

Run with predefined example queries:

npm run dev

Project Structure

thirdweb-docs-agent/
├── typescript/                     # TypeScript implementation
│   ├── src/
│   │   ├── cli.ts                 # Interactive CLI (stateless)
│   │   ├── index.ts               # Programmatic demo
│   │   └── tools/
│   │       └── documentation-search.ts
│   └── tsconfig.json
├── python/                         # Python implementation
│   ├── cli.py                     # Interactive CLI (conversational)
│   ├── requirements.txt           # Python dependencies
│   └── __main__.py                # Module entry point
├── thirdweb-docs/                 # Complete thirdweb documentation
├── CLAUDE.md                      # Project guide for Claude Code
├── .env                           # API key configuration
└── package.json                   # Project scripts

Development

# TypeScript chat
npm run chat:ts

# Python chat (conversational)
npm run chat:py

# Run TypeScript demo
npm run dev

# Build TypeScript
npm run build

# Test queries (benchmark hybrid RAG performance)
npm run test:queries

# Re-generate embeddings (if docs updated)
npm run setup:embeddings

Benchmarking

Test the hybrid RAG approach with 11 diverse queries:

npm run test:queries

# Or with a specific model
npm run test:queries sonnet 4

The benchmark script:

Runs predefined test queries covering simple SDK questions, complex multi-doc queries, and edge cases
Tracks cost, time, token usage, and tool calls for each query
Generates a comparison report vs the old agentic search approach
Shows individual query breakdowns and aggregate statistics

Expected results:

Average cost per query: ~$0.05-0.07 (vs $0.10-0.14 old approach)
Average time per query: ~3-4s (vs ~8s old approach)
Average tokens per query: ~8,000 (vs ~38,000 old approach)

Key Differences: TypeScript vs Python

Feature	TypeScript	Python
Conversation Memory	❌ None (stateless)	✅ Full context retention
API Used	`query()` function	`ClaudeSDKClient` class
Best For	Single questions	Multi-turn conversations
Follow-up Questions	Needs full context each time	Remembers previous context
Language	TypeScript/Node.js	Python 3.8+
Use Case	Quick lookups	Support conversations

Recommendation: Use the Python version (npm run chat:py) for the best experience with conversation memory!

How It Works

Hybrid RAG Architecture

Agent Initialization: Creates a Claude Agent with strict rules to only use documentation
Semantic Vector Search (Primary):
- Uses OpenAI embeddings (text-embedding-3-small, 1536 dimensions)
- Stores embeddings in Supabase with pgvector
- Searches using cosine similarity for conceptually relevant content
- Returns top-k results with relevance scores (0.0-1.0)
Agent Verification (Secondary):
- If search results are sufficient (relevance > 0.7): answers directly
- If more context needed: uses Read tool on specific files
- If results incomplete: retries with different query phrasing
Response Generation: Provides answers with citations to specific documentation files
Context Management (Python only): Maintains conversation history for follow-up questions

Why Hybrid RAG?

50% cost reduction: From ~$0.12 to ~$0.06 per query
2-3x faster: Vector search is instant vs sequential file searching
Better accuracy: Semantic understanding finds conceptually similar content
Flexibility: Agent can still verify with full file reads when needed

Search Strategy

User Query → Generate Embedding → Search Supabase → Evaluate Relevance
                                                           ↓
                                         High (>0.7)      Low (<0.7)
                                              ↓              ↓
                                        Answer directly   Rephrase query OR Read full file

Built With

@anthropic-ai/claude-agent-sdk - Claude Agent SDK (TypeScript)
claude-agent-sdk - Claude Agent SDK (Python)
Supabase - Vector database with pgvector
OpenAI Embeddings - text-embedding-3-small (1536 dimensions)
TypeScript - Type-safe development
Python 3.8+ - Async support
Node.js 18+ - Runtime environment

Architecture

┌─────────────────────────────────────────────────────────────┐
│                      User Query                             │
└────────────────────┬────────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────────┐
│              Claude Agent SDK                               │
│  (Python ClaudeSDKClient or TypeScript query())             │
└────────────┬────────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────────────────────────┐
│          MCP Server: semantic_search                        │
│  - Generates query embedding (OpenAI API)                   │
│  - Searches Supabase vector DB                              │
│  - Returns top-k results with relevance scores              │
└────────────┬────────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────────────────────────┐
│                Supabase Vector Database                     │
│  - Table: thirdweb_docs                                     │
│  - ~3,500 document chunks                                   │
│  - IVFFlat index for fast similarity search                 │
│  - Cosine similarity matching                               │
└────────────┬────────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────────────────────────┐
│             Agent Decision Logic                            │
│  High relevance (>0.7): Answer directly                     │
│  Low relevance (<0.7): Rephrase query OR use Read tool      │
└────────────┬────────────────────────────────────────────────┘
             │
             ▼
┌─────────────────────────────────────────────────────────────┐
│                    Final Response                           │
│  - Synthesized answer from chunks                           │
│  - Code examples from documentation                         │
│  - File path citations                                      │
└─────────────────────────────────────────────────────────────┘

Migration from Old Approach

If you were using the previous agentic search (Grep/Glob) approach, here's how to migrate:

What Changed

System Prompt: Now uses system-prompt-hybrid.md instead of system-prompt.md
Allowed Tools: semantic_search + Read instead of Grep + Glob + Read
Max Turns: Reduced from 10 to 3 (vector search is faster)
Dependencies: Added @supabase/supabase-js, openai, zod
Infrastructure: Requires Supabase database + embeddings

Migration Steps

Update dependencies:

npm install
cd python && source venv/bin/activate && pip install -r requirements.txt

Set up Supabase (see Setup section above)
Generate embeddings:
```
npm run setup:embeddings
```
Test the new approach:
```
npm run test:queries
```

Backward Compatibility

The old approach files are preserved:

system-prompt.md - Original system prompt
Both CLIs can be reverted by changing imports

To revert to old approach, modify cli.ts or cli.py:

- const SYSTEM_PROMPT = readFileSync(join(__dirname, '../../system-prompt-hybrid.md'), 'utf-8');
+ const SYSTEM_PROMPT = readFileSync(join(__dirname, '../../system-prompt.md'), 'utf-8');

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
generated-docs		generated-docs
python		python
scripts		scripts
typescript		typescript
.env.example		.env.example
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
PLAN.md		PLAN.md
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json
system-prompt-hybrid.md		system-prompt-hybrid.md
system-prompt.md		system-prompt.md

eabdelmoneim/thirdweb-docs-agent

Folders and files

Latest commit

History

Repository files navigation