RAG Practice Project

A minimal Retrieval-Augmented Generation (RAG) system built in Python. It loads local markdown documents, splits them into overlapping chunks, embeds them with a sentence transformer, and answers questions by retrieving the most relevant chunks and passing them as context to an LLM via OpenRouter.

Architecture

documents/*.md  →  chunk_text()  →  embed_chunks()  →  in-memory store
                                                              │
                          User question                        │
                               │                              │
                          embed_query()                       │
                               │                              │
                          retrieve_top_k()  ←─────────────────┘
                               │
                          build_prompt()
                               │
                          generate_answer()  →  OpenRouter (GPT-3.5-turbo)
                               │
                          Printed answer

Pipeline stages

Stage	What it does
Ingestion	Reads all .md files from the documents/ folder
Chunking	Splits documents on sentence boundaries into ~200-character chunks with 50-character overlap
Embedding	Encodes all chunks with all-MiniLM-L6-v2 (384-dim embeddings) via sentence-transformers
Retrieval	Computes cosine similarity between the query embedding and all chunk embeddings; returns the top 10 matches
Generation	Assembles context + question into a prompt and sends it to OpenRouter for an LLM answer

Prerequisites

• Python 3.10+
• An OpenRouter API key — get one at openrouter.ai/keys

Setup

# Clone the repo
git clone <repo-url>
cd rag-project

# Create a virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Set your API key
export OPENROUTER_API_KEY="sk-or-v1-..."

Usage

# Make sure your venv is active and the API key is set
python3 rag.py

Interactive session:

Ask a question: What is the capital of Turkey?

Retrieved chunks:
- (0.987) Ankara.md: Ankara is the capital city of Turkey...
- (0.854) Turkey.md: Turkey is a transcontinental country...

Answer:
The capital of Turkey is Ankara.

Ask a question: exit
Goodbye!

Adding your own documents

Drop any .md file into the documents/ folder. The script reads all files in that directory on startup — no reindexing step needed.

Project structure

rag-project/
├── documents/           # Knowledge base (markdown files)
│   ├── Ankara.md
│   ├── Artificial_intelligence.md
│   ├── Istanbul.md
│   ├── Python.md
│   └── Turkey.md
├── rag.py               # Main application
├── requirements.txt     # Python dependencies
└── README.md

Deployment

Option 1: Run on a VPS (DigitalOcean, Linode, AWS EC2, etc.)

# On the server
sudo apt update && sudo apt install python3 python3-venv python3-pip -y

# Clone and set up
git clone <repo-url>
cd rag-project
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Add to your shell profile (~/.bashrc or ~/.zshrc)
echo 'export OPENROUTER_API_KEY="sk-or-v1-..."' >> ~/.bashrc
source ~/.bashrc

# Run
python3 rag.py

To keep it running after you disconnect, use tmux or screen:

tmux new -s rag
python3 rag.py
# Press Ctrl+B then D to detach
# Reattach later: tmux attach -t rag

Option 2: Systemd service (Linux)

Create /etc/systemd/system/rag.service:

[Unit]
Description=RAG Practice App
After=network.target

[Service]
Type=simple
User=<your-user>
WorkingDirectory=/path/to/rag-project
Environment=OPENROUTER_API_KEY=sk-or-v1-...
ExecStart=/path/to/rag-project/venv/bin/python3 /path/to/rag-project/rag.py
Restart=on-failure

[Install]
WantedBy=multi-user.target

Then:

sudo systemctl daemon-reload
sudo systemctl enable rag
sudo systemctl start rag

Option 3: Docker

Create a Dockerfile:

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY documents/ ./documents/
COPY rag.py .

ENV OPENROUTER_API_KEY=""

CMD ["python3", "rag.py"]

docker build -t rag-app .
docker run -it -e OPENROUTER_API_KEY="sk-or-v1-..." rag-app

Option 4: Hugging Face Spaces

This project runs on CPU and has no web UI (it's terminal-based), so it's not a natural fit for Spaces. If you want a web interface, wrap rag.py with Gradio or Streamlit and deploy to Spaces as a Gradio/Streamlit app.

Customization

• Swap the LLM model — change the model string in rag.py line ~87 to any OpenRouter model (e.g. "anthropic/claude-3-haiku", "meta-llama/llama-3-8b-instruct")
• Adjust chunking — change chunk_size and overlap in the chunk_text() function
• Change retrieval depth — alter the k argument in the retrieve_top_k() call inside main()
• Swap the embedding model — change the model name in load_embedding_model() to any sentence-transformers model

License

This is a practice/educational project — use it freely.