3GPP Standards RAG

A retrieval-augmented question-answering system over 3GPP 5G-core telecom standards, built around one non-negotiable requirement: every claim in an answer is either backed by a checkable citation, or the system says it doesn't know.

"How does the AMF handle a registration reject?"

The AMF sends a REGISTRATION REJECT message carrying a 5GMM cause value
that indicates the reason for rejection. [S1] Upon receiving it, the UE
enters the 5GMM-DEREGISTERED state and, depending on the cause value, may
be barred from re-attempting registration on that PLMN. [S2]

  Sources: [S1] TS 24.501 §5.5.1.2.5   [S2] TS 24.501 §5.5.1.2.6
  Grounded · confidence 0.94 · 2 sources

Why this exists

Most "RAG chatbot" submissions demonstrate that retrieval-augmented generation works. This one demonstrates something narrower and harder: that it can be made to fail safely on a corpus where a wrong answer is actually costly -- 3GPP specs are the normative source of truth for how telecom equipment from different vendors interoperates, and a hallucinated "the AMF shall X" is indistinguishable from a real one to someone who doesn't already know the spec by heart.

The measurable claim isn't "the chatbot works" -- it's the pair of numbers in EVALUATION.md: on questions the corpus can answer, how often does it answer correctly and cite correctly; on questions it cannot, how often does it correctly refuse instead of confabulating.

Architecture

Ingest    3GPP archive (.docx) --> clause-aware parser --> clause-bounded chunks
Index     BGE-base dense embeddings + hand-rolled BM25 sparse vectors, both
          in Qdrant as named vectors on the same point
Retrieve  query analysis (spec/release/acronym extraction) --> hybrid search
          --> Reciprocal Rank Fusion --> cross-encoder rerank --> parent-
          clause expansion --> sufficiency gate (abstain before generating
          if evidence is weak)
Generate  Claude, evidence pinned to numbered blocks, one citation marker
          required per factual sentence
Verify    L1 mechanical citation check -> L2 local NLI entailment -> L3
          optional LLM-judge escalation for borderline claims -> strip
          unsupported claims or abstain outright
Serve     FastAPI (SSE stage progress, not raw token streaming -- see
          DESIGN_DECISIONS.md) + a React chat UI with a retrieval-trace panel

Full design rationale, including the decisions that don't have an obviously correct answer, is in docs/ARCHITECTURE.md and docs/DESIGN_DECISIONS.md.

Quickstart

Requires Python 3.12+, Node 20+, and Docker (for Qdrant).

cp .env.example .env        # fill in ANTHROPIC_API_KEY (or switch provider -- see below)
docker compose up -d qdrant

uv venv && uv pip install -e ".[dev]"
uv run tgpp ingest          # fetches the curated 5G-core spec set and indexes it (~10-15 min)
uv run tgpp serve           # http://localhost:8000/docs

cd apps/web && npm install && npm run dev   # http://localhost:5173

No Anthropic key on hand? Set TGPP_LLM__PROVIDER=ollama and TGPP_LLM__MODEL=llama3.1 with a local ollama serve running -- the whole pipeline, including verification, runs with zero API cost. See .env.example for every provider option.

Ask a question without the UI

uv run tgpp ask "What identifies a network slice in the 5G System?"

Run the eval suite

uv run tgpp eval

Runs the golden set (questions the corpus should answer) and the adversarial set (questions it shouldn't) and writes a metrics report to evals/results/latest.json. See EVALUATION.md for the metric definitions and the last recorded run.

Project layout

src/tgpp_rag/
  domain/       shared pydantic types passed between every stage
  ingest/       fetch, parse, normalise, chunk
  index/        embeddings, BM25, Qdrant store
  retrieval/    query analysis, RRF fusion, rerank, sufficiency gate
  generation/   provider abstraction (Anthropic/OpenAI/Ollama), prompts
  verification/ the three-layer anti-hallucination gate
  service/      FastAPI app, routes, wire schemas
  eval/         golden + adversarial harness and metrics
apps/web/       React (Vite + TypeScript) chat UI
evals/          the golden/adversarial datasets and recorded results
docs/           architecture, design decisions, runbook

Tests

uv run pytest                    # unit tests, no network/model downloads required
uv run pytest -m integration     # needs a running Qdrant + downloaded models

Documentation

• docs/ARCHITECTURE.md -- how data flows through the system and why it's shaped this way
• docs/DESIGN_DECISIONS.md -- the decisions with real trade-offs, recorded as ADRs
• EVALUATION.md -- metrics, methodology, and the last recorded eval run
• docs/RUNBOOK.md -- operating the service: config, health checks, common failures

License

MIT