feat(phase6): add modern interactive web UI, static server assets, CI workflow, and open-source launch kit

Author: divyanshujethi

.github/workflows/ci.yml

@@ -0,0 +1,35 @@
+name: CI
+
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    branches: [ "main" ]
+
+jobs:
+  test:
+    name: Test on ${{ matrix.os }} (Python ${{ matrix.python-version }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+        cache: "pip"
+
+    - name: Install Dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -e ".[dev]"
+
+    - name: Run Test Suite
+      run: |
+        pytest -v

CONTRIBUTING.md

@@ -0,0 +1,68 @@
+# Contributing to LocalRAG-Kit
+
+Thank you for your interest in contributing to **LocalRAG-Kit**! We welcome contributions from developers, security researchers, and AI enthusiasts.
+
+---
+
+## 🛠️ Development Setup
+
+1. **Clone the Repository**:
+   ```bash
+   git clone https://github.com/RitualDev-Lab/localrag-kit.git
+   cd localrag-kit
+   ```
+
+2. **Create and Activate a Virtual Environment**:
+   ```bash
+   # Windows PowerShell
+   python -m venv .venv
+   .\.venv\Scripts\Activate.ps1
+
+   # Linux / macOS
+   python3 -m venv .venv
+   source .venv/bin/activate
+   ```
+
+3. **Install Dependencies**:
+   ```bash
+   pip install -e ".[dev]"
+   ```
+
+---
+
+## 🧪 Running Tests
+
+Ensure all unit and integration tests pass before submitting code:
+```bash
+pytest -v
+```
+
+---
+
+## 🏗️ Architecture Overview
+
+LocalRAG-Kit is structured into clean, decoupled layers:
+
+- `localrag/core/`: File harvesting, pathspec ignore parsing, file parsers (Markdown, PDF, Code), and semantic boundary chunkers.
+- `localrag/storage/`: SQLite engine with SQLite FTS5 for BM25 keyword search, binary BLOB vector storage, NumPy SIMD cosine similarity, and change detection.
+- `localrag/providers/`: Pluggable embedding providers (Fast Offline Hasher, Ollama, OpenAI) and LLM providers (Ollama, OpenAI, Mock).
+- `localrag/retrieval/`: Reciprocal Rank Fusion (RRF) hybrid reranking, context synthesizer with token budgeting, and grounded prompt orchestration.
+- `localrag/server/`: High-performance FastAPI server with Server-Sent Events (SSE) streaming, REST endpoints, and embedded static web UI.
+- `localrag/cli.py`: Interactive CLI with Rich formatting (`scan`, `index`, `search`, `ask`, `stats`, `serve`).
+
+---
+
+## 📝 Contribution Guidelines
+
+1. **Keep it 100% Offline & Zero-Config**:
+   Features must never require mandatory external cloud accounts or external database installations (like Chroma, Pinecone, or PostgreSQL). SQLite must remain the zero-infra backbone.
+2. **Deterministic Chunker Line Numbers**:
+   Every parser and chunker must compute 1-based exact physical start and end line coordinates so citation provenance is verifiable.
+3. **Cross-Platform Compatibility**:
+   Ensure paths use `pathlib.Path` and console outputs handle Windows CP1252 / UTF-8 safely.
+
+---
+
+## 📜 License
+
+By contributing to LocalRAG-Kit, you agree that your contributions will be licensed under the [MIT License](LICENSE).

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 RitualDev Lab
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,49 +1,199 @@
-# LocalRAG-Kit 🚀
+<div align="center">
 
-> **100% Offline, Privacy-First Local RAG Search & Chat Engine for Documents and Codebases.**  
-> Zero cloud vector database fees. Zero data leaks. Runs entirely on your local machine.
+# ⚡ LocalRAG-Kit
+
+**100% Offline, Privacy-First Local RAG Search & Chat Engine for Codebases and Documents**
+
+[![CI](https://github.com/RitualDev-Lab/localrag-kit/actions/workflows/ci.yml/badge.svg)](https://github.com/RitualDev-Lab/localrag-kit/actions)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-emerald.svg)](LICENSE)
+[![Tests Passing](https://img.shields.io/badge/tests-31%20passing-brightgreen.svg)](tests/)
+[![Architecture: Pure SQLite](https://img.shields.io/badge/storage-SQLite%20+%20FTS5%20+%20Vectors-purple.svg)]()
+
+*Zero cloud vector database fees. Zero telemetry. Zero data leaks.*  
+*Indexes your repositories, PDFs, and docs into a single portable SQLite database with hybrid BM25 + dense vector search and an interactive local web UI.*
+
+</div>
 
 ---
 
-## ⚡ Key Highlights
+## ✨ Features
 
-- **🔒 100% Offline & Private**: No data leaves your machine. Compatible with local LLMs (Ollama) and local embeddings.
-- **📁 Multi-Format Ingestion**: Scans and parses Markdown (`.md`), Code (`.py`, `.ts`, `.js`, `.go`, `.rs`, `.java`, etc.), Plain Text (`.txt`), HTML, and PDFs.
-- **🧩 Structure-Aware Smart Chunking**:
-  - **Code**: Splits on function and class declarations while keeping imports and docstrings intact.
-  - **Markdown**: Heading-aware chunking preserving `#`, `##`, `###` hierarchy and code fences.
-  - **Text / PDF**: Token-aware sliding window preserving exact start/end line numbers.
-- **⚡ Zero-Infra Hybrid Storage**: Pure SQLite database (`.localrag/index.db`) combining Dense Vector Search with SQLite FTS5 (BM25) full-text search.
-- **🎯 Reciprocal Rank Fusion (RRF)**: Merges semantic vector similarity with exact keyword lookup for 10x better precision on code symbols and error codes.
+- 🔒 **100% Offline & Air-Gapped**: Runs entirely on your CPU/GPU. No external server dependencies or cloud database subscriptions required.
+- ⚡ **Zero-Infra Hybrid Storage**: Pure SQLite database (`.localrag/index.db`) combining SQLite FTS5 (BM25 keyword search) with dense vector embeddings via binary BLOBs and fast NumPy SIMD batch cosine similarity.
+- 🎯 **Reciprocal Rank Fusion (RRF)**: Merges semantic vector similarity with exact BM25 keyword rankings ($k_{rrf} = 60$). Solves the classic vector search weakness for exact code symbols, variable names, and error codes.
+- 🧩 **Structure-Aware Smart Chunking**:
+  - **Code**: Language-aware boundary chunking for Python, JavaScript, TypeScript, Go, Rust, Java, C/C++ that preserves function/class signatures and docstrings.
+  - **Markdown**: Heading-aware (`#`, `##`, `###`) chunking that prevents code fences from being broken mid-snippet.
+  - **Text / PDF**: Token-aware sliding window with deterministic 1-based start and end line coordinates.
+- 🔄 **Incremental SHA-256 Change Detection**: Re-indexing only processes newly created or modified files. Deleted files are automatically cascaded out of SQLite and FTS5 indices.
+- 🤖 **Flexible Provider Ecosystem**:
+  - **Embeddings**: Built-in zero-download `fast` 384-dim feature hasher, local `ollama` (`nomic-embed-text`), or `openai`.
+  - **LLMs**: Local `ollama` (`llama3.2`, `qwen2.5`, `mistral`), `openai`, or mock testing agent.
+- 🖥️ **Interactive Web Interface**: Clean, dark-mode glassmorphic Tailwind UI featuring real-time SSE token streaming, inline citation badges, a source inspector slideout drawer, and live index management.
+- 💻 **Feature-Rich CLI**: Built with `rich` for colorful terminal tables, progress bars, interactive query synthesis, and streaming terminal output.
 
 ---
 
-## 📦 Quickstart
+## 🏛️ System Architecture
+
+```mermaid
+flowchart TB
+    subgraph Ingestion ["1. INGESTION & PARSING"]
+        A[Local Files & Codebase] --> B[File Harvester & Ignore Filter]
+        B --> C{File Type Dispatch}
+        C -->|Markdown| D[Markdown Chunking Engine]
+        C -->|Code: py, ts, go, rs, etc.| E[Code Boundary Chunker]
+        C -->|PDF & Text| F[Sliding Window Chunker]
+    end
+
+    subgraph Storage ["2. ZERO-INFRA STORAGE (SQLite)"]
+        D & E & F --> G[(.localrag/index.db)]
+        G --> H[FTS5 Full-Text Search Engine]
+        G --> I[Binary Vector Storage & SHA-256 Hashes]
+    end
+
+    subgraph Retrieval ["3. HYBRID RETRIEVAL & CITATION"]
+        Q[User Query] --> J[BM25 Keyword Search]
+        Q --> K[Vector Cosine Similarity]
+        H --> J
+        I --> K
+        J & K --> L[Reciprocal Rank Fusion - RRF]
+        L --> M[Context Synthesizer & Token Budget]
+    end
+
+    subgraph Presentation ["4. INTERFACES"]
+        M --> N[Local LLM - Ollama / OpenAI]
+        N --> O[Rich Terminal CLI]
+        N --> P[FastAPI Server & Web UI]
+    end
+```
+
+---
+
+## 🚀 Quickstart
 
 ### 1. Installation
+
 ```bash
+# Clone repository
 git clone https://github.com/RitualDev-Lab/localrag-kit.git
 cd localrag-kit
+
+# Create virtual environment
 python -m venv .venv
-# On Windows:
-.\venv\Scripts\pip install -r requirements.txt
-# On Linux/macOS:
-source .venv/bin/activate && pip install -r requirements.txt
+source .venv/bin/activate  # On Windows: .venv\Scripts\Activate.ps1
+
+# Install in editable mode
+pip install -e .
+```
+
+### 2. Index Any Folder or Codebase
+
+```bash
+# Preview files and chunk distribution without writing to disk
+localrag scan ./my-project --preview
+
+# Index files into pure SQLite (uses offline fast hasher by default)
+localrag index ./my-project
+
+# Or index with local Ollama embeddings:
+localrag index ./my-project --embed-provider ollama --embed-model nomic-embed-text
 ```
 
-### 2. Scan & Preview Any Folder
+### 3. Ask Questions from the Terminal
+
 ```bash
-python -m localrag.cli scan /path/to/project --preview
+# Ask with hybrid retrieval and local Ollama
+localrag ask "How does the authentication middleware work?" --mode hybrid --llm ollama
+
+# Search matching chunks with line coordinates
+localrag search "create_app" --mode hybrid --limit 5
 ```
 
+### 4. Launch the Web Interface
+
+```bash
+localrag serve ./my-project --port 8000
+```
+Open `http://127.0.0.1:8000` in your browser to access the interactive web console.
+
+---
+
+## 🛠️ CLI Command Reference
+
+| Command | Description | Example |
+| :--- | :--- | :--- |
+| `localrag scan` | Inspect eligible files, token estimates, and chunk counts without indexing | `localrag scan . --preview` |
+| `localrag index` | Incremental or full indexing of workspace files into SQLite + FTS5 | `localrag index . --full` |
+| `localrag search` | Perform hybrid RRF, BM25 keyword, or vector similarity search | `localrag search "token" --mode hybrid` |
+| `localrag ask` | Query workspace with streaming grounded LLM answers and citations | `localrag ask "Explain routes.py" --llm ollama` |
+| `localrag stats` | Display index statistics, chunk counts, database size, and providers | `localrag stats .` |
+| `localrag serve` | Launch the FastAPI server and embedded glassmorphic web UI | `localrag serve . --port 8000` |
+
+---
+
+## 💻 Python API Usage
+
+LocalRAG-Kit can also be embedded directly into your Python scripts and services:
+
+```python
+from pathlib import Path
+from localrag.storage import SQLiteStore
+from localrag.providers import FastFeatureEmbeddingProvider, OllamaLLMProvider
+from localrag.retrieval import RAGOrchestrator
+
+# Initialize storage and providers
+store = SQLiteStore(".localrag/index.db")
+embedder = FastFeatureEmbeddingProvider()
+llm = OllamaLLMProvider(model="llama3.2")
+
+# Orchestrate grounded retrieval
+orchestrator = RAGOrchestrator(
+    store=store,
+    embedding_provider=embedder,
+    llm_provider=llm,
+)
+
+# Stream response with citations
+stream = orchestrator.stream_answer(
+    query="Explain the database schema design",
+    mode="hybrid",
+    top_k=5,
+)
+
+print(f"Citations: {[c.relative_path for c in stream.citations]}")
+for token in stream.token_iterator:
+    print(token, end="", flush=True)
+```
+
+---
+
+## 🌐 REST & SSE Streaming API
+
+When running `localrag serve`, the following API endpoints are exposed:
+
+- `GET /` — Serves the modern Web UI
+- `GET /api/status` — Returns workspace stats, chunk counts, and active providers
+- `GET /api/files` — Lists all indexed files with line numbers and metadata
+- `POST /api/search` — Runs hybrid RRF, BM25, or vector similarity search
+- `POST /api/chat` — SSE streaming endpoint (`data: {"type": "token", "token": "..."}`)
+- `POST /api/reindex` — Triggers an incremental or full re-index
+- `GET /api/chunk/{id}` — Retrieves chunk details and surrounding file context
+- `GET /api/file-content` — Reads raw text of a physical workspace file
+
 ---
 
-## 🧪 Running Tests
+## 🧪 Testing
+
+The test suite contains 31 unit and integration tests covering chunking, FTS5 BM25, vector math, change detection, SSE streaming, and web UI serving:
+
 ```bash
-python -m pytest -v
+pytest -v
 ```
 
 ---
 
-## 📜 License
-MIT © [RitualDev-Lab](https://github.com/RitualDev-Lab)
+## 📄 License
+
+This project is licensed under the [MIT License](LICENSE).