feat(demo): update document-rag demo with pre-ingestion and ratelimiting support

Author: ishnbl

demo/document-rag/.env.example

@@ -1,8 +1,15 @@
-OPENAI_API_KEY=sk-your-api-key-here
+OPENAI_API_KEY=your-api-token
+ACCOUNT_ID=your-account-id
+
+EMBEDDING_MODEL=@cf/qwen/qwen3-embedding-0.6b
+LLM_MODEL=@cf/openai/gpt-oss-20b
+VECTOR_SIZE=1024
+
 VORTEXDB_HOST=vortexdb
 VORTEXDB_PORT=3034
-EMBEDDING_MODEL=text-embedding-3-small
-LLM_MODEL=gpt-4o-mini
 CHUNK_SIZE=512
 CHUNK_OVERLAP=50
 TOP_K=5
+EMBEDDING_BATCH_SIZE=64
+DOCS_DIRECTORY=./documents
+

demo/document-rag/README.md

@@ -1,55 +1,111 @@
-# VectorDB RAG Demo
+# VortexDB RAG Demo
 
-A fully containerized Document RAG demo with a dark-themed web UI. Upload documents, chat with your knowledge base.
+A containerized document RAG demo: offline ingest into VortexDB, query via a FastAPI backend, and chat through a React frontend. Embeddings and LLM calls use [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) through the OpenAI-compatible API.
 
 ## Quick Start
 
 ```bash
-# 1. Fill in your API key
+# 1. Configure credentials
 cp .env.example .env
-# Edit .env and set OPENAI_API_KEY=sk-your-key-here
+# Set OPENAI_API_KEY (Cloudflare API token) and ACCOUNT_ID
 
-# 2. Build and start everything
+# 2. Add documents to ingest
+cp /path/to/your/files/* documents/
+
+# 3. Start the stack (builds images, ingests docs, starts API + UI)
 docker compose up -d --build
 
-# 3. Open browser
+# 4. Open the UI
 open http://localhost:3035
 ```
 
-That's it! No other setup required.
+On startup, Docker will:
+
+1. Start VortexDB and wait until healthy
+2. Run ingest on `./documents` (TXT, MD, PDF, DOCX, CSV)
+3. Start the backend API
+4. Start the frontend
+
+To re-ingest after adding or changing documents:
+
+```bash
+docker compose up ingest --force-recreate
+docker compose up -d backend
+```
 
-## Features
+## Ports
 
-- **File Upload** - Drag & drop or browse documents (PDF, TXT, MD, DOCX, CSV)
-- **Chat Interface** - Ask questions, get AI-powered answers
-- **Fully Containerized** - VortexDB + Backend + Frontend in Docker
+| Service   | URL                      |
+|-----------|--------------------------|
+| Frontend  | http://localhost:3035    |
+| Backend   | http://localhost:8000    |
+| VortexDB  | http://localhost:3034    |
 
 ## Architecture
 
 ```
-Browser (localhost:3035) → Frontend (nginx)
+documents/  →  ingest (one-shot)  →  VortexDB
+                                         ↑
+Browser (3035) → nginx → backend (8000) ─┘
                               ↓
-                     Backend API (port 8000)
-                              ↓
-              ┌───────────────┴───────────────┐
-              ↓                               ↓
-        OpenAI API                    VortexDB
-        (embeddings + LLM)            (HTTP port 3000)
+                    Cloudflare Workers AI
+                    (embeddings + LLM)
+```
+
+## API
+
+| Method | Path     | Body                    | Description              |
+|--------|----------|-------------------------|--------------------------|
+| GET    | `/health`| —                       | Health check             |
+| POST   | `/chat`  | `{"query": "..."}`      | RAG answer + sources     |
+| POST   | `/query` | `{"query": "..."}`      | Retrieval only (no LLM)  |
+
+Example:
+
+```bash
+curl -X POST http://localhost:8000/chat \
+  -H "Content-Type: application/json" \
+  -d '{"query": "What embedding model is used?"}'
 ```
 
 ## Configuration
 
-### .env file
+All settings live in `.env`. See `.env.example` for the full list.
 
 ```env
-OPENAI_API_KEY=sk-your-api-key-here
-VORTEXDB_HOST=vortexdb
-VORTEXDB_PORT=3000
-EMBEDDING_MODEL=text-embedding-3-small
-LLM_MODEL=gpt-4o-mini
+OPENAI_API_KEY=your-cloudflare-api-token
+ACCOUNT_ID=your-account-id
+
+EMBEDDING_MODEL=@cf/qwen/qwen3-embedding-0.6b
+LLM_MODEL=@cf/openai/gpt-oss-20b
+VECTOR_SIZE=1024
+
 CHUNK_SIZE=512
 CHUNK_OVERLAP=50
 TOP_K=5
+EMBEDDING_BATCH_SIZE=64
+DOCS_DIRECTORY=./documents
+```
+
+Get credentials from the [Cloudflare dashboard](https://dash.cloudflare.com/) → Workers AI → **Use REST API**. The API token needs Workers AI read access. `VECTOR_SIZE` must match VortexDB's `DIMENSION` in `docker-compose.yml` (default `1024`).
+
+Inside Docker Compose, `VORTEXDB_HOST` and `VORTEXDB_PORT` are overridden to `vortexdb:3000` automatically.
+
+## Local Development (without Docker)
+
+```bash
+# Terminal 1 — VortexDB (from repo root)
+HTTP_HOST=0.0.0.0 HTTP_PORT=3034 STORAGE_TYPE=inmemory INDEX_TYPE=flat \
+  DIMENSION=1024 cargo run --release --bin server
+
+# Terminal 2 — ingest
+cd backend
+pip install -r requirements.txt
+cp ../.env.example ../.env   # fill in credentials
+python -m src.ingestion
+
+# Terminal 3 — API
+uvicorn src.main:app --reload --port 8000
 ```
 
 ## Project Structure
@@ -58,26 +114,21 @@ TOP_K=5
 demo/document-rag/
 ├── docker-compose.yml
 ├── .env.example
-├── README.md
+├── documents/              # Drop files here for ingest
 ├── backend/
 │   ├── src/
-│   │   ├── main.py        # FastAPI app
-│   │   ├── config.py      # Config from env
-│   │   ├── chunker.py     # Text chunking
-│   │   ├── embedder.py    # OpenAI embeddings
-│   │   ├── generator.py   # Chat completion
-│   │   ├── extractor.py  # Document parsing
-│   │   └── vectorstore.py # VortexDB HTTP client
+│   │   ├── main.py         # FastAPI app
+│   │   ├── ingestion.py    # Offline ingest CLI
+│   │   ├── config.py       # Env-based config
+│   │   ├── chunker.py
+│   │   ├── embedder.py     # Cloudflare embeddings
+│   │   ├── generator.py    # Cloudflare chat completions
+│   │   ├── extractor.py
+│   │   └── vectorstore.py  # VortexDB HTTP client
 │   ├── requirements.txt
 │   └── Dockerfile
 └── frontend/
-    ├── src/
-    │   ├── App.jsx        # Main React component
-    │   ├── App.css        # Styles
-    │   └── main.jsx       # Entry point
-    ├── index.html
-    ├── package.json
-    ├── vite.config.js
-    ├── nginx.conf
+    ├── src/App.jsx
+    ├── nginx.conf          # Proxies /api → backend:8000
     └── Dockerfile
 ```

demo/document-rag/backend/Dockerfile

@@ -7,9 +7,10 @@ RUN pip install --no-cache-dir -r requirements.txt
 
 COPY src/ ./src/
 
-RUN mkdir -p /app/uploads
+RUN mkdir -p /app/documents
 
 ENV PYTHONPATH=/app
+ENV DOCS_DIRECTORY=/app/documents
 
 EXPOSE 8000
 

demo/document-rag/backend/requirements.txt

@@ -1,9 +1,9 @@
-fastapi==0.109.2
-uvicorn[standard]==0.27.1
-python-multipart==0.0.9
-openai==1.12.0
-httpx==0.27.0
-pypdf2==3.0.1
-python-docx==1.1.0
-pydantic==2.6.1
+fastapi==0.124.4
+uvicorn[standard]==0.41.0
+httpx==0.28.1
+openai==2.30.0
 python-dotenv==1.0.1
+slowapi==0.1.10
+python-multipart==0.0.22
+PyPDF2==3.0.1
+python-docx==1.1.2

demo/document-rag/backend/src/config.py

@@ -3,13 +3,21 @@
 
 load_dotenv()
 
+_account_id = os.getenv("ACCOUNT_ID", "")
+
+
 class Config:
     OPENAI_API_KEY: str = os.getenv("OPENAI_API_KEY", "")
+    OPENAI_BASE_URL: str = (
+        f"https://api.cloudflare.com/client/v4/accounts/{_account_id}/ai/v1"
+    )
     VORTEXDB_HOST: str = os.getenv("VORTEXDB_HOST", "localhost")
     VORTEXDB_PORT: int = int(os.getenv("VORTEXDB_PORT", "3034"))
-    EMBEDDING_MODEL: str = os.getenv("EMBEDDING_MODEL", "text-embedding-3-small")
-    LLM_MODEL: str = os.getenv("LLM_MODEL", "gpt-4o-mini")
+    EMBEDDING_MODEL: str = os.getenv("EMBEDDING_MODEL", "")
+    LLM_MODEL: str = os.getenv("LLM_MODEL", "")
     CHUNK_SIZE: int = int(os.getenv("CHUNK_SIZE", "512"))
     CHUNK_OVERLAP: int = int(os.getenv("CHUNK_OVERLAP", "50"))
     TOP_K: int = int(os.getenv("TOP_K", "5"))
-    VECTOR_SIZE: int = 1536
+    VECTOR_SIZE: int = int(os.getenv("VECTOR_SIZE", ""))
+    DOCS_DIRECTORY: str = os.getenv("DOCS_DIRECTORY", "./documents")
+    EMBEDDING_BATCH_SIZE: int = int(os.getenv("EMBEDDING_BATCH_SIZE", "64"))

demo/document-rag/backend/src/embedder.py

@@ -1,26 +1,31 @@
-import openai
-from openai import OpenAI
 from typing import List
+
+from openai import OpenAI
+
 from src.config import Config
 
 
 class Embedder:
-    def __init__(self, api_key: str):
-        self.client = OpenAI(api_key=api_key)
+    def __init__(self):
+        self.client = OpenAI(
+            api_key=Config.OPENAI_API_KEY,
+            base_url=Config.OPENAI_BASE_URL,
+        )
         self.model = Config.EMBEDDING_MODEL
-    
+
     def embed(self, texts: List[str]) -> List[List[float]]:
         """Generate embeddings for a list of texts."""
         if not texts:
             return []
-        
-        response = self.client.embeddings.create(
-            model=self.model,
-            input=texts
-        )
-        
-        return [item.embedding for item in response.data]
-    
+
+        embeddings = []
+        for i in range(0, len(texts), Config.EMBEDDING_BATCH_SIZE):
+            batch = texts[i : i + Config.EMBEDDING_BATCH_SIZE]
+            response = self.client.embeddings.create(model=self.model, input=batch)
+            embeddings.extend(item.embedding for item in response.data)
+
+        return embeddings
+
     def embed_single(self, text: str) -> List[float]:
         """Generate embedding for a single text."""
         embeddings = self.embed([text])

demo/document-rag/backend/src/generator.py

@@ -1,30 +1,30 @@
-import openai
+from typing import Dict, List
+
 from openai import OpenAI
-from typing import List, Dict
+
 from src.config import Config
 
 
 class Generator:
-    def __init__(self, api_key: str):
-        self.client = OpenAI(api_key=api_key)
+    def __init__(self):
+        self.client = OpenAI(
+            api_key=Config.OPENAI_API_KEY,
+            base_url=Config.OPENAI_BASE_URL,
+        )
         self.model = Config.LLM_MODEL
-    
-    def generate(
-        self,
-        question: str,
-        context_chunks: List[Dict[str, any]]
-    ) -> str:
-        """
-        Generate answer using RAG prompt.
-        """
+
+    def generate(self, question: str, context_chunks: List[Dict[str, any]]) -> str:
+        """Generate answer using RAG prompt."""
         if not context_chunks:
             return "No relevant documents found. Please upload a document first."
-        
-        context_text = "\n\n".join([
-            f"[Document {i+1}]\n{chunk['text']}"
-            for i, chunk in enumerate(context_chunks)
-        ])
-        
+
+        context_text = "\n\n".join(
+            [
+                f"[Document {i + 1}]\n{chunk['text']}"
+                for i, chunk in enumerate(context_chunks)
+            ]
+        )
+
         prompt = f"""You are a helpful assistant answering questions based on provided documents.
 
 Context from documents:
@@ -39,15 +39,18 @@ def generate(
 - Cite which document(s) you're using when relevant
 
 Answer:"""
-        
+
         response = self.client.chat.completions.create(
             model=self.model,
             messages=[
-                {"role": "system", "content": "You are a helpful assistant that answers questions based on provided documents."},
-                {"role": "user", "content": prompt}
+                {
+                    "role": "system",
+                    "content": "You are a helpful assistant that answers questions based on provided documents.",
+                },
+                {"role": "user", "content": prompt},
             ],
             temperature=0.3,
-            max_tokens=1000
+            max_tokens=1000,
         )
-        
+
         return response.choices[0].message.content