fastapi-lance-local-storage

A minimal FastAPI API that accepts an image via POST, stores it locally with a SHA256 filename, vectorizes it using OpenAI's CLIP model, and appends the vector to a local Lance dataset.

Designed as a small MVP for local or Fly.io deployment.

---

Features

• Image upload via POST
• SHA256 filename hashing to avoid duplicates
• Vectorization using OpenAI's CLIP model via open_clip
• Local Lance dataset storage for embeddings
• Fly.io-ready deployment with persistent volume
• Minimal dependencies for easy portability

---

API

POST /vectorize_and_store

Upload an image, store it locally, generate its embedding, and store it in Lance.

Example:

curl -X POST \
     -F "file=@your_image.jpg" \
     https://fastapi-lance-local-storage.fly.dev/vectorize_and_store

Response:

{"filename":"0adb33ad08808e6461e71ecdf8046bc7d0f5d2be717283717da28c204d41fc60.jpg","vector_saved":true}

---

Local Development

1. Clone the repo

git clone https://github.com/gordonmurray/fastapi-lance-local-storage.git
cd fastapi-lance-local-storage/app

2. Create a virtual environment & install dependencies

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

3. Run locally

uvicorn main:app --reload --host 0.0.0.0 --port 8000

4. Test upload

curl -X POST \
     -F "file=@path/to/image.jpg" \
     http://localhost:8000/vectorize_and_store

Uploaded files are stored in:

/app/storage/images

Vector data is stored in:

/app/storage/lance-data

---

Deploying to Fly.io

1. Create a new Fly.io app

fly apps create fastapi-lance-local-storage

2. Deploy

cd app
fly deploy

3. Access the app

curl -X POST \
     -F "file=@your_image.jpg" \
     https://fastapi-lance-local-storage.fly.dev/vectorize_and_store

---

Accessing Stored Data on Fly.io

# SSH into the instance
fly ssh console

# Navigate to storage
cd /app/storage
ls -lah

---

Project Structure

fastapi-lance-local-storage/
├── app/
│   ├── Dockerfile          # Fly.io build configuration
│   ├── fly.toml            # Fly.io deployment config
│   ├── main.py             # FastAPI entrypoint
│   ├── requirements.txt    # Python dependencies
│   ├── settings.py         # App settings and env handling
│   ├── storage.py          # Storage helpers (local + Lance)
│   └── vectorizer.py       # CLIP vectorization logic
├── performance_testing/    # Scripts for generating and testing uploads
├── README.md
└── LICENSE