📓 Agentic Notebook LocalLM Studio

Edge-Optimized, Domain-Agnostic, and Self-Reflective Local RAG System for Multi-Document Intelligence

An enterprise-grade, privacy-first Retrieval-Augmented Generation (RAG) workspace designed to process, synthesize, and answer complex queries across multiple PDF documents locally. Built explicitly to solve low-resource hardware constraints (~2GB RAM target), high-precision tabular extraction, and explainable AI through strict evidence citations.

---

📸 Overview & Interface

---

🚀 Key Engineering Highlights

• 🧠 Ultra-Low Resource Footprint (~2GB RAM Target): Powered by Qwen 2.5 1.5B Instruct (quantized to Q5_K_M ~1.2GB) running fully locally via Ollama. It achieves an optimal sweet spot between high reasoning/precision and minimal memory footprint without CPU thermal throttling or OOM errors.

• 📑 3-Tier Multi-Parser Fallback Pipeline: Ensures 100% document ingestion resilience by automatically routing documents through:

  1. LlamaParse API (Tier 1): Primary markdown extractor preserving complex table topologies (| Metric | Value |).
  2. Unstructured API (Tier 2): Secondary fallback for structured element partitioning.
  3. Local pypdf (Tier 3): Fully offline local fallback ensuring zero downtime.

• 🔄 Fast Hybrid Agentic Re-Search Loop: Features a deterministic, zero-latency heuristic evaluation layer. If initial vector retrieval lacks tabular data or metrics, the retrieval engine dynamically executes a targeted secondary query to Qdrant specifically fetching structured table chunks.

• 🌐 Multilingual & Cross-Lingual Semantic Retrieval: Leverages paraphrase-multilingual-MiniLM-L12-v2 (384-dim) for high-density semantic matching, allowing seamless cross-lingual querying (e.g., asking in Indonesian over dense English research documents).

• 🔍 Explainable AI with Multi-Document Citations: Enforces strict behavioral guardrails through domain-agnostic system prompts. The LLM is forced to attach exact evidence sources ([Document_Name.pdf | Hal. X]) for every claim, preventing hallucinations across legal, financial, energy, and technical domains.

---

🛠️ System Architecture

                         [ User Query ]
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│ Pass 1: Semantic Retrieval via Multilingual MiniLM          │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
            ┌────────────────────────────────────┐
            │ Heuristic Table/Metric Detection   │
            └──────────────────┬─────────────────┘
                               │
            ┌──────────────────┴──────────────────┐
            │                                     │
    [ Table Found ]                     [ Table Missing ]
            │                                     │
            │                                     ▼
            │                     ┌───────────────────────────────┐
            │                     │ Pass 2: Fast Re-Search Query  │
            │                     │ (Targets Structured Chunks)   │
            │                     └───────────────┬───────────────┘
            │                                     │
            └──────────────────┬──────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────┐
│ Synthesis: Qwen 2.5 1.5B Q5_K_M + Multi-Doc Citations       │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼
                 [ Evidence-Backed Response ]

---

📦 Tech Stack

Component	Technology / Model	Role
Frontend UI	Streamlit	Interactive multi-document workspace & drawer
Vector Database	Qdrant (Local Persistent Client)	Storing 384-dimensional dense vectors & metadata
Embedding Engine	paraphrase-multilingual-MiniLM-L12-v2	Dense multilingual semantic vectorization
SLM Inference	Ollama — qwen2.5:1.5b (Q5_K_M)	High-reasoning local LLM execution
Parsers	LlamaParse, Unstructured, pypdf	Tiered PDF parsing and table structure preservation
Environment	python-dotenv	Centralized key and URL management

---

⚙️ Installation & Setup

1. Prerequisites

• Python 3.10+ installed.
• Ollama installed and running locally.

2. Clone Repository

git clone [https://github.com/sharriffajar/Notebook-LocalLM-Studio.git](https://github.com/sharriffajar/Notebook-LocalLM-Studio.git)
cd Notebook-LocalLM-Studio

3. Install Dependencies

pip install -r requirements.txt

4. Setup Environment Variables

Copy .env.example to .env and configure your keys:

cp .env.example .env

Edit .env:

LLAMAPARSE_API_KEY=your_llamaparse_key_here
UNSTRUCTURED_API_KEY=your_unstructured_key_here
QDRANT_URL=./qdrant_db
QDRANT_COLLECTION_NAME=notebooklm_workspace
OLLAMA_MODEL_NAME=qwen2.5:1.5b

5. Register Model in Ollama

Download qwen2.5-1.5b-instruct-q5_k_m.gguf into a ./models/ folder, then register it using the provided Modelfile:

ollama create qwen2.5:1.5b -f Modelfile

---

🏃 Running the Application

Launch the Streamlit interface:

streamlit run app_pdf_rag_studio.py

1. Open your browser at http://localhost:8501.
2. Upload one or multiple PDF documents in the Sources Sidebar.
3. Click "Sync & Build Knowledge Base" to index your workspace.
4. Start querying your documents with instant evidence citation and zero hallucination!

---

🚀 Future Roadmap

• Containerization: Add Dockerfile and docker-compose.yml for isolated deployment.
• Evaluation Harness: Integrate RAGAS metrics to benchmark retrieval precision and faithfulness.
• Automated Testing: Implement pytest suite for core parser and search adapters.

---

📜 License

Distributed under the MIT License. See LICENSE for more information.