Add PASSTHROUGH_API_KEY option to forward incoming API keys #64
Conversation
jovermier
changed the title
There was a problem hiding this comment.
Pull request overview
This PR introduces a PASSTHROUGH_API_KEY configuration option to enable per-user quota tracking in multi-tenant environments. When enabled, the proxy extracts API keys from incoming request headers (x-api-key or Authorization Bearer) and forwards them to the upstream LiteLLM gateway instead of using the static OPENAI_API_KEY environment variable.
- Adds
PASSTHROUGH_API_KEYenvironment variable with header extraction logic - Updates OpenAI model routing to use passthrough API keys when enabled
- Includes unrelated Dockerfile improvement (python:latest → python:3.12-slim)
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| server.py | Adds PASSTHROUGH_API_KEY configuration and implements API key extraction from request headers for OpenAI models |
| Dockerfile | Updates base image from python:latest to python:3.12-slim (unrelated to PR purpose) |
Comments suppressed due to low confidence (1)
server.py:1165
- The PASSTHROUGH_API_KEY feature is only implemented for OpenAI models, but not for Anthropic or Gemini models. For consistency and to fully support the multi-tenant quota tracking use case described in the PR, this feature should be extended to all model providers. Consider applying the same passthrough logic to the Anthropic (line 1164) and Gemini (line 1161) model branches.
# Determine which API key to use based on the model
if request.model.startswith("openai/"):
# Use passthrough key if enabled, otherwise fall back to env var
litellm_request["api_key"] = incoming_api_key if PASSTHROUGH_API_KEY and incoming_api_key else OPENAI_API_KEY
# Use custom OpenAI base URL if configured
if OPENAI_BASE_URL:
litellm_request["api_base"] = OPENAI_BASE_URL
logger.debug(f"Using {'passthrough' if PASSTHROUGH_API_KEY and incoming_api_key else 'OpenAI'} API key and custom base URL {OPENAI_BASE_URL} for model: {request.model}")
else:
logger.debug(f"Using {'passthrough' if PASSTHROUGH_API_KEY and incoming_api_key else 'OpenAI'} API key for model: {request.model}")
elif request.model.startswith("gemini/"):
if USE_VERTEX_AUTH:
litellm_request["vertex_project"] = VERTEX_PROJECT
litellm_request["vertex_location"] = VERTEX_LOCATION
litellm_request["custom_llm_provider"] = "vertex_ai"
logger.debug(f"Using Gemini ADC with project={VERTEX_PROJECT}, location={VERTEX_LOCATION} and model: {request.model}")
else:
litellm_request["api_key"] = GEMINI_API_KEY
logger.debug(f"Using Gemini API key for model: {request.model}")
else:
litellm_request["api_key"] = ANTHROPIC_API_KEY
logger.debug(f"Using Anthropic API key for model: {request.model}")
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jovermier
force-pushed
the
main
branch
2 times, most recently
from
9d71e96 to
b8bbee1
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jovermier
force-pushed
the
main
branch
2 times, most recently
from
7df107d to
345a575
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Add specific rule in AuthorizationPolicy for vmagent service account - Allow cluster.local/ns/monitoring/sa/vmagent-victoria-metrics principal - This should resolve vmagent scrape errors shown in AlertManager 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Update PASSTHROUGH_API_KEY documentation to clarify fallback behavior - Add REQUIRE_PASSTHROUGH_KEY configuration option for strict validation - Reject requests when REQUIRE_PASSTHROUGH_KEY=true but no valid API key is found - Add configuration validation and example usage in documentation Resolves feedback about silent fallback potentially bypassing quota tracking in multi-tenant environments. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Security Improvements AddedI've addressed the code review concerns about passthrough API key security by adding:
These changes are committed to the main branch and address both code review items about documentation clarity and security concerns. The implementation is backward compatible - existing behavior unchanged when is not enabled. |
1 participant
Summary
Implementation Details