A production-style RAG (Retrieval-Augmented Generation) pipeline for clinical document question answering. Upload any clinical PDF — radiology reports, discharge summaries, clinical guidelines — and ask natural language questions against it. Returns grounded answers with page-level source citations.
Built with FastAPI, ChromaDB, HuggingFace sentence-transformers, and Claude API.
Query: "What are the recommended first-line treatments for community-acquired pneumonia?"
Answer: Based on the uploaded guideline, first-line treatment for community-acquired pneumonia in adults includes... (Source: Page 14, Section 3.2)
Faithfulness Score: 4.8 / 5.0
- PDF Ingestion — Upload any clinical PDF; pipeline chunks, embeds, and indexes automatically
- Semantic Retrieval — Sentence-transformer embeddings query ChromaDB for top-k relevant chunks
- Cross-Encoder Reranking — Retrieved chunks reranked by a cross-encoder before passing to the LLM
- Grounded Generation — Claude API instructed to answer only from retrieved context, never hallucinate
- Source Citations — Every answer includes the source chunk, section, and page number
- Faithfulness Scoring — Each response scored for groundedness on a 1-5 scale via a secondary LLM call
- Multi-Document Support — Ingest multiple documents and filter queries by document ID
- Streaming Responses — FastAPI streaming endpoint for real-time answer display
- React Frontend — Optional drag-and-drop upload interface with expandable source citations
User Query
│
▼
┌─────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ PDF Upload │────▶│ Chunker (PyMuPDF│────▶│ HuggingFace │
│ /ingest │ │ 512 tok chunks) │ │ Embeddings │
└─────────────┘ └──────────────────┘ └────────┬────────┘
│
▼
┌─────────────────┐
│ ChromaDB │
│ Vector Store │
└────────┬────────┘
│
User Question ──────────────────────────────▶ Embed Query
│
▼
┌─────────────────┐
│ Top-5 Chunk │
│ Retrieval │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Cross-Encoder │
│ Reranker │
│ (Top-3 kept) │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Claude API │
│ Generation │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Faithfulness │
│ Evaluator │
└────────┬────────┘
│
▼
Answer + Sources + Score
| Layer | Tool | Reason |
|---|---|---|
| Vector Store | ChromaDB | Persistent local storage, no infra setup needed |
| Embeddings | all-MiniLM-L6-v2 (HuggingFace) |
Fast, free, strong performance on biomedical text |
| Reranker | cross-encoder/ms-marco-MiniLM-L-6-v2 |
Improves retrieval precision beyond cosine similarity |
| LLM | Claude API (claude-sonnet-4-20250514) |
Strong instruction following, low hallucination rate |
| Backend | FastAPI + Pydantic | Async support, automatic schema validation |
| PDF Parsing | PyMuPDF | Reliable page-level metadata extraction |
clinical-rag/
├── backend/
│ ├── app/
│ │ ├── ingest.py # PDF parsing, chunking, embedding, Chroma storage
│ │ ├── retriever.py # Semantic search, top-k chunk retrieval
│ │ ├── generator.py # Claude API call with retrieved context
│ │ ├── reranker.py # Cross-encoder reranking of retrieved chunks
│ │ ├── evaluator.py # Faithfulness scoring via secondary LLM call
│ │ └── models.py # Pydantic request/response schemas
│ ├── main.py # FastAPI app and route definitions
│ ├── config.py # Chunk size, overlap, model names, API keys
│ └── requirements.txt
│
├── frontend/ # Optional React UI
│ ├── src/
│ │ ├── components/
│ │ │ ├── UploadArea.jsx
│ │ │ └── QueryInterface.jsx
│ │ └── App.jsx
│ └── package.json
│
├── data/
│ └── sample_guideline.pdf # WHO/NIH public clinical guideline for testing
│
├── eval/
│ └── eval_questions.json # 10 ground truth Q&A pairs for retrieval testing
│
├── notebooks/
│ └── rag_exploration.ipynb # Chunking strategy experiments, retrieval analysis
│
└── README.md
git clone https://github.com/your-username/clinical-rag.git
cd clinical-rag/backend
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txtcp .env.example .envAdd your Anthropic API key to .env:
ANTHROPIC_API_KEY=your_key_here
CHROMA_PERSIST_DIR=./chroma_store
CHUNK_SIZE=512
CHUNK_OVERLAP=50
TOP_K_RETRIEVAL=5
TOP_K_RERANK=3
uvicorn main:app --reload
# Runs on http://localhost:8000cd ../frontend
npm install
npm run dev
# Runs on http://localhost:5173Upload a PDF and index it into ChromaDB.
Request: multipart/form-data with file field.
Response:
{
"document_id": "abc123",
"filename": "who_pneumonia_guideline.pdf",
"chunks_indexed": 142,
"status": "success"
}Ask a question against ingested documents.
Request:
{
"question": "What is the recommended antibiotic for CAP in adults?",
"document_id": "abc123",
"top_k": 3
}Response:
{
"answer": "According to the guideline, amoxicillin is the recommended first-line antibiotic...",
"sources": [
{
"chunk": "For adults with non-severe CAP, amoxicillin 500mg three times daily...",
"page": 14,
"section": "3.2 Treatment Recommendations",
"score": 0.94
}
],
"faithfulness_score": 4.8,
"model": "claude-sonnet-4-20250514"
}List all ingested documents.
Response:
{
"documents": [
{
"document_id": "abc123",
"filename": "who_pneumonia_guideline.pdf",
"chunks": 142,
"ingested_at": "2025-04-20T14:32:00Z"
}
]
}Evaluated against 10 ground truth Q&A pairs from the WHO Community-Acquired Pneumonia guideline.
| Metric | Score |
|---|---|
| Retrieval Recall @3 | 0.87 |
| Answer Faithfulness (avg) | 4.6 / 5.0 |
| Answer Relevance (avg) | 4.4 / 5.0 |
| Hallucination Rate | 0 / 10 |
Faithfulness and relevance scored by a secondary Claude call using a structured rubric. Full eval results in eval/eval_questions.json.
Why ChromaDB over FAISS? ChromaDB offers persistent storage and a cleaner metadata filtering API out of the box. FAISS is faster at scale but requires manual index serialization. For a document Q&A use case with multi-document filtering, ChromaDB's collection model is a better fit.
Why cross-encoder reranking? Cosine similarity on embeddings retrieves semantically related chunks but doesn't always prioritize the most directly answer-relevant ones. A cross-encoder jointly encodes the query and each chunk, producing a more accurate relevance signal. On eval, reranking improved Retrieval Recall @3 from 0.74 to 0.87.
Why chunk size 512 with 50-token overlap?
Clinical guidelines contain dense, self-contained paragraphs. Smaller chunks (256 tokens) lost context mid-sentence; larger chunks (1024 tokens) diluted retrieval signal. 50-token overlap preserves sentence continuity across chunk boundaries. These were tuned empirically in notebooks/rag_exploration.ipynb.
Why faithfulness scoring? In clinical contexts, a hallucinated answer is worse than no answer. The faithfulness score gives a downstream signal for when to flag a response for human review — same logic as the confidence threshold in standard RAG eval frameworks like RAGAS.
- Hybrid search — combine BM25 keyword search with semantic retrieval for better recall on exact medical terms
- Fine-tuned embeddings — domain-adapt
all-MiniLM-L6-v2on PubMed abstracts for stronger biomedical retrieval - RAGAS integration — replace custom eval with the RAGAS framework for standardized RAG evaluation metrics
- Table extraction — current pipeline skips tabular data in PDFs; add structured table parsing via Camelot
- HIPAA-aware deployment — add PII detection before indexing for production clinical use cases
These are publicly available and safe to use:
- WHO CAP Guideline
- NIH Clinical Guidelines
- Any publicly available radiology report template