Skip to content

PYPI_README.md

Latest commit

 

History

History
304 lines (216 loc) · 12.7 KB

PYPI_README.md

File metadata and controls

304 lines (216 loc) · 12.7 KB
Raw

Code-Graph-RAG

Code-Graph-RAG parses a multi-language codebase with Tree-sitter, sharpened by compiler-grade frontends and runtime traces where available, builds a knowledge graph of its structure in an embedded graph database that needs no server, and lets you query, edit, and optimise that code in plain English. It works across a monorepo of mixed languages under one unified graph schema.

What It Does

Point it at a repository and it reads every source file, extracts functions, classes, methods, and modules along with the relationships between them, and stores the result as an interconnected graph. Once the graph exists you can:

  • Ask questions about the codebase in natural language and get answers grounded in the real structure.
  • Retrieve the actual source of any function, class, or method by name or by intent.
  • Edit code through the agent with AST-based surgical patching and a diff preview before anything changes.
  • Search and rewrite code structurally by AST pattern with ast-grep, instead of text or regex.
  • Trace data flow through assignments, calls, and I/O sinks via FLOWS_TO taint edges.
  • Optimise code against language best practices or your own coding standards.
  • Find dead code by walking call and reference edges from entry points.
  • Overlay runtime behaviour: trace a test run (or pull production eBPF profiles) and merge the calls that actually happened into the graph, exposing dispatch that static analysis cannot see.
  • Group several repositories into a named workspace and query them as one graph.
  • Trace calls between microservices: route decorators become endpoint templates, and HTTP client URLs resolve to the handlers that serve them, linking services across project boundaries.

Supported Languages

Python, TypeScript, TSX, JavaScript, Rust, Go, Java, C, C++, C#, PHP, Lua, and Dart are fully supported. Scala is in development, and Ruby, Kotlin, Swift, Elixir, Haskell, Solidity, Bash, and Nix have structural support (modules, functions, classes where the language has them, and imports) through the pluggable ast-grep tier, which requires the ast-grep extra (pip install 'code-graph-rag[ast-grep]').

Install

pip install code-graph-rag

With all Tree-sitter grammars (Python, JS, TS, Rust, Go, Java, Scala, C, C++, C#, PHP, Lua, Dart) and the ast-grep tier (Ruby, Kotlin, Swift, Elixir, Haskell, Solidity, Bash, Nix):

pip install 'code-graph-rag[treesitter-full,ast-grep]'

Without ast-grep, files in the ast-grep tier's languages are indexed as bare File nodes, and a sync that meets them says so in a warning.

With semantic code search (UniXcoder embeddings):

pip install 'code-graph-rag[semantic]'

The vectors go to a SQLite file beside the embedded graph (~/.cgr/vectors.sqlite), so semantic search needs no server and no Docker either. To use Qdrant instead, set CGR_VECTOR_STORE_BACKEND=qdrant. To use Milvus Lite, install code-graph-rag[semantic,milvus], then set CGR_VECTOR_STORE_BACKEND=milvus and MILVUS_URI=./.milvus_code_embeddings.db before indexing.

To compute embeddings on an OpenAI-compatible endpoint (OpenAI, Ollama, vLLM) instead of locally, set CGR_EMBEDDING_PROVIDER=openai with OPENAI_EMBEDDING_BASE_URL and OPENAI_EMBEDDING_MODEL; torch and transformers are then not required locally.

Which version am I getting?

pip install code-graph-rag gives you the newest PyPI release, which is deliberately not the newest commit. Git tags exist for every version (one per merge), while PyPI uploads and GitHub Release binaries land on an every-50 cadence, so the tag on main usually runs ahead of the published version. A security fix, a breaking change or a merge a maintainer labels release now is the exception: each ships immediately rather than waiting for the cadence.

To run code newer than the latest release, install from git with uv:

uv tool install "code-graph-rag[treesitter-full,ast-grep] @ git+https://github.com/vitali87/code-graph-rag@main"

Use uv for this: it applies the project's [tool.uv.sources], which build the C and C++ grammars from forks that keep the declarations upstream drops after a #define whose value contains a block comment. pip ignores those sources and installs the upstream grammars.

To upgrade, rerun the install with --upgrade and the extras you chose, for example pip install --upgrade 'code-graph-rag[treesitter-full,ast-grep]'. With uv or pipx, run uv tool upgrade code-graph-rag or pipx upgrade code-graph-rag instead; see the installation guide for git installs and the other methods.

Prerequisites

  • Python 3.12+
  • ripgrep (rg) (for shell command text searching)
  • Only for the optional Memgraph backend: Docker, and cmake where no pymgclient wheel fits your platform

The knowledge graph is an embedded SQLite file (~/.cgr/graph.sqlite), so there is no database server to run. Memgraph and Neo4j remain available as opt-in backends; see Graph Backends.

The wheel is pure Python (py3-none-any), so it installs on any platform with Python 3.12 or newer. On Linux (x86_64, aarch64), macOS and Windows x64 it also pulls in cgr-graph-rs, a native extension that speeds up the embedded graph; elsewhere the same engine runs in pure Python.

Some distributions ship an older interpreter (Debian Bookworm, and so Raspberry Pi OS Bookworm, ships 3.11), which also explains the failed piwheels Bookworm build. On those systems, see the installation guide for the available interpreter-pinning alternatives.

CLI Quick Start

The package installs a cgr command.

Parse a repo, then query it:

cgr start --repo-path ./my-project --update-graph   # parse into the graph
cgr start --repo-path ./my-project                  # launch the interactive chat

The graph is written to ~/.cgr/graph.sqlite. To use Memgraph instead, install code-graph-rag[memgraph], set GRAPH_BACKEND=memgraph, and start the packaged Memgraph + Qdrant stack with cgr daemon up.

Index to protobuf for offline use:

cgr index -o ./index-output --repo-path ./my-project

Export knowledge graph to JSON:

cgr export -o graph.json

AI-guided optimisation:

cgr optimize python --repo-path ./my-project

Find dead code (functions unreachable from any entry point):

cgr dead-code                                   # scan the indexed project
cgr dead-code -e main --exclude '*.gen.*'       # add roots, skip generated code
cgr dead-code --format json --fail-on-found     # CI-friendly report

Results are candidates for review, not a guaranteed delete list. See the Dead Code Detection guide.

Group repositories into a workspace and query them together:

cgr workspace create my-platform
cgr workspace add-repo my-platform ./service-a
cgr workspace add-repo my-platform ./service-b
cgr start --workspace my-platform

Inspect the graph and the stack:

cgr stats                                  # node and relationship counts
cgr status                                 # stack state and last sync per project
cgr doctor                                 # check dependencies and configuration

Runtime Call Tracing

Static analysis cannot see calls through interfaces, reflection, registries, or framework routing. cgr trace records which functions actually called which while your code runs (typically the test suite) and merges the observations into the graph as CALLS edges with dynamic provenance: dynamic: true, observed call counts, the workloads (tests) that exercised each edge, and static_missed: true where no static edge existed.

Index the repository first (cgr start --repo-path ./my-project --update-graph), then:

Python (a pytest plugin ships with the package, inert unless enabled):

cd ./my-project
pytest --cgr-trace                              # writes cgr-trace.jsonl
cgr trace ingest cgr-trace.jsonl --repo-path .  # merge into the graph

Node.js / TypeScript (V8's built-in profiler, no agent needed; source maps are followed back to the original TypeScript):

node --cpu-prof --cpu-prof-name=run.cpuprofile app.js
cgr trace convert run.cpuprofile --repo-path ./my-project --workload smoke
cgr trace ingest cgr-trace.jsonl --repo-path ./my-project

Production overlay (eBPF continuous profilers: Parca, Pyroscope, OpenTelemetry). Fetch a pprof over HTTP and convert it in one step, re-anchoring build paths to your checkout:

cgr trace pull "https://parca.example/query?...&format=pprof" \
    --repo-path ./my-project --language go \
    --path-map /build/src/=./my-project/src/ \
    --label endpoint --header "Authorization=Bearer $TOKEN"
cgr trace ingest cgr-trace.jsonl --repo-path ./my-project

The JVM (Java, Scala), .NET, PHP, Lua, Dart, Go, Rust, and C/C++ each have a recording recipe in the Dynamic Call Tracing guide. Ingest is idempotent, so a cron'd pull plus ingest keeps a continuously refreshing production overlay. The absence of a dynamic edge never means dead code; it only means the traced workload did not exercise that path.

MCP Server

Run cgr mcp-server to serve the tools over stdio or HTTP for Claude Code and other MCP clients. The MCP surface registers:

  • Ask and retrieve: ask_agent, query_code_graph, get_code_snippet, and semantic_search (needs the semantic extra)
  • Structural editing: surgical_replace_code, plus structural_search and structural_replace (need the ast-grep extra)
  • Files and projects: read_file, write_file, list_directory, list_projects, index_repository, update_repository, delete_project, wipe_database

Python SDK

The cgr package provides short imports for programmatic use.

Load and query an exported graph

from cgr import load_graph

graph = load_graph("graph.json")
print(graph.summary())

functions = graph.find_nodes_by_label("Function")
for fn in functions[:5]:
    rels = graph.get_relationships_for_node(fn.node_id)
    print(f"{fn.properties['name']}: {len(rels)} relationships")

Query the graph with Cypher

MemgraphIngestor keeps its original name but works with whichever backend GRAPH_BACKEND selects. On the default embedded graph, host and port are not used.

from cgr import MemgraphIngestor

with MemgraphIngestor(host="localhost", port=7687) as db:
    rows = db.fetch_all("MATCH (f:Function) RETURN f.name LIMIT 10")
    for row in rows:
        print(row)

Generate Cypher from natural language

import asyncio
from cgr import CypherGenerator


async def main():
    gen = CypherGenerator()
    cypher = await gen.generate("Find all classes that inherit from BaseModel")
    print(cypher)


asyncio.run(main())

Semantic code search

Requires the semantic extra.

from cgr import embed_code

embedding = embed_code("def authenticate(user, password): ...")
print(f"Embedding dimension: {len(embedding)}")

Configuration

from cgr import settings

settings.set_orchestrator("openai", "gpt-5.6-terra", api_key="sk-...")
settings.set_cypher("google", "gemini-3.5-flash-lite", api_key="your-key")

Environment Variables

Configure via .env or environment variables:

Variable Default Description
GRAPH_BACKEND sqlite Graph engine: sqlite (embedded), memgraph, or neo4j
SQLITE_GRAPH_PATH ~/.cgr/graph.sqlite Embedded graph file
MEMGRAPH_HOST localhost Memgraph hostname (with GRAPH_BACKEND=memgraph)
MEMGRAPH_PORT 7687 Memgraph port (with GRAPH_BACKEND=memgraph)
ORCHESTRATOR_PROVIDER Provider: google, openai, anthropic, azure, ollama, minimax, litellm_proxy
ORCHESTRATOR_MODEL Model ID (e.g. gpt-5.6-terra, gemini-3.6-flash, claude-sonnet-5, qwen2.5-coder)
ORCHESTRATOR_API_KEY API key for the provider (not needed for ollama)
CYPHER_PROVIDER Provider for Cypher generation
CYPHER_MODEL Model ID for Cypher generation (e.g. qwen2.5-coder, gpt-5.6-luna, gemini-3.5-flash-lite)
CYPHER_API_KEY API key for Cypher provider (not needed for ollama)
TARGET_REPO_PATH . Default repository path

Documentation

Full documentation, architecture details, and contribution guide: docs.code-graph-rag.com

License

MIT