High-Performance Polyglot Code Clone Detection, DRY Health Analysis, AST Subtree Hasher & Embedded Studio WebUI.
Note
Primary Upstream SSoT: All development, issues, pull requests, and automated testing are managed on our self-hosted Gitea portal at git.gt-web-dev.com/gt-dev/cddm. The GitHub repository is an automated read-only downstream replica mirror.
- Overview
- The 4 Interface Pillars
- Key Features
- Supported Languages
- Installation
- Quick Start
- Interface Documentation
- DRY Health Score Formula
- Architecture & Documentation
- Performance Benchmarks
- Contributing
- License
CDDM (Code De-Duplication Meister) is an open-source, ultra-fast, multi-threaded polyglot code clone detection, structural AST refactoring, and architectural governance engine built natively in Rust. Designed to scale seamlessly across enterprise monorepos, CDDM detects exact (Type 1), renamed (Type 2), structural near-miss (Type 3), and semantic graph (Type 4) clones in milliseconds.
CDDM strictly enforces 100% Cross-Interface Feature Parity across all four primary interaction surfaces:
+----------------------------------------------------------------------------------------------------+
| CDDM Unified Core Engine |
+----------------------------------------------------------------------------------------------------+
| 1. CLI Engine | 2. WebUI Studio | 3. MCP Server | 4. TUI Studio |
| - 22 Subcommands | - React 19 Studio | - 30 Agent Tools | - 12 Ratatui Tabs |
| - Scriptable stdout | - Monaco Split Diffs | - 17 MCP Resources | - Split Diff Viewer |
| - Turnkey CI/CD | - SSE Live Stream | - 3 Agent Prompts | - Keyboard-Driven |
| [docs/CLI.md] | [docs/WEBUI.md] | [docs/MCP.md] | [docs/TUI.md] |
+----------------------------------------------------------------------------------------------------+
| Feature | Description |
|---|---|
| M61 Rolling Hash Winnowing | Sub-linear O(N) token fingerprinting using Mersenne Prime |
| SIMD Vectorized Reduction | Hardware-accelerated AVX2 (x86_64) and ARM NEON (AArch64) vector lanes computing parallel Mersenne-61 modular reduction. |
| Zero-Copy Memory Mapping | High-throughput memmap2 zero-copy memory mapping for large files (> 64KB) bypassing heap allocation overhead. |
| Tree-sitter AST Hasher | Parses source trees into ASTs and hashes subtrees with blake3 Merkle hashing to identify structural near-misses. |
| Cross-Language Semantic Matching | Detects algorithmic equivalents across different languages using Weisfeiler-Lehman graph kernels and neural subwords. |
| Polyglot Dead Code Detection | Detects unreferenced functions, unreachable blocks, and 0-hit duplicate clones correlated with coverage. |
| Ecosystem Library Overlap | Detects reimplemented standard and third-party library algorithms (e.g. lodash, itertools) and suggests native imports. |
| Organization Federation Hub | Multi-repository duplication scanner and cross-repo shared package extractor (.cddmhub.toml). |
| Runtime Coverage Correlation | Ingests LCOV, Cobertura, and Istanbul coverage tracefiles to calculate clone execution hit counts and risk scores. |
| DRY Health Scoring | Mathematical 0.0 - 100.0 codebase health index factoring in duplication percentage and cross-module boundaries. |
| In-Process Git Blame | Powered by gix (gitoxide) to annotate duplicate fragments with author names and commit timestamps in-process. |
| N-Way Graph Clustering | Disjoint-Set Union-Find algorithm partitioning pairwise clone graphs into transitive |
| Multi-Site Patch Synthesizer | Computes consensus invariant lines and synthesizes unified multi-file .patch diffs across all clone sites. |
| Autonomous AI Code Surgeon | Closed-loop iterative AI refactoring with automated test suite verification and rollback guarantees (cddm heal). |
| Distributed Cache Packs | Exports and imports portable .cddmpack binary fingerprint archives for lightning-fast CI/CD runs. |
| Turnkey CI/CD Generators | Turnkey workflow generators for Gitea Actions, GitHub Actions, GitLab CI, and Azure Pipelines. |
CDDM includes built-in tokenizers and Tree-sitter AST parsers for 23 programming languages:
Rust (.rs) TypeScript (.ts, .tsx) JavaScript (.js, .jsx, .mjs)
Python (.py) Go (.go) Java (.java)
C (.c, .h) C++ (.cpp, .hpp, .cc) C# (.cs)
Ruby (.rb) PHP (.php) Kotlin (.kt, .kts)
Swift (.swift) SQL (.sql) Bash / Shell (.sh, .bash)
Lua (.lua) CSS / SCSS / LESS HTML (.html, .htm)
JSON (.json) Zig (.zig, .zon) Scala (.scala, .sc)
Elixir (.ex, .exs) Dockerfile / Containerfile
brew install GrigorTonikyan/cddm/cddmscoop bucket add cddm https://git.gt-web-dev.com/gt-dev/cddm.git
scoop install cddmwinget install GrigorTonikyan.cddmcargo install cddmnpm install -g cddmInstall the official CDDM Studio Extension (editors/vscode) directly from the extension marketplace for in-editor squiggles, quick-fix refactorings, and the embedded WebUI sidebar.
# 1. Fast duplicate scan on current workspace
cddm scan ./src
# 2. Differential scan comparing working tree against main branch in CI
cddm diff main --fail-threshold 3.0
# 3. Launch interactive 12-tab Terminal UI (TUI)
cddm tui
# 4. Launch visual React 19 Studio WebUI in browser
cddm serve --port 3000 --open
# 5. Autonomous AI Code Surgeon refactoring with test loop
cddm heal --cluster 1 --provider gemini --model gemini-2.5-pro --verify --test-cmd "cargo test"
# 6. Generate turnkey Gitea Actions CI/CD workflow
cddm init gitea --writeFor comprehensive technical manuals on each interaction surface, explore our dedicated guides:
- CLI Command Reference — Exhaustive manual for all 22 CLI subcommands, flags, and recipes.
- MCP Server Specification — Stdio JSON-RPC 2.0 reference for all 30 AI agent tools, 17 resources, and prompts.
- Embedded Studio WebUI — React 19 Studio guide, 19 interactive modals, REST & SSE API catalog.
- Terminal UI (TUI) Studio — Keyboard shortcuts and 12-tab navigation guide.
- Language Server & IDE Setup — Real-time in-editor setup for VS Code, Cursor, Neovim, Zed, Helix, Sublime.
- 4-Pillar Feature Parity Matrix — Authoritative governance matrix across all 21 core capabilities.
CDDM evaluates codebase modularity health using a continuous mathematical scoring function:
Where:
-
Duplication Percentage: Ratio of duplicate clone tokens to total workspace tokens (
$\frac{\text{Clone Tokens}}{\text{Total Tokens}} \times 100$ ). - Cross Module Ratio: Proportion of clone pairs spanning distinct top-level directories.
- docs/ARCHITECTURE.md — Pipeline phases, SIMD reduction, and crate breakdown.
- docs/API.md — Axum REST endpoints and JSON schema specifications.
- docs/FEATURE_MATRIX.md — Detailed capability and test coverage matrix.
- docs/REQUIREMENTS.md — Performance SLAs, throughput bounds, and constraints.
- docs/ROADMAP.md — Enhancement proposals (EP-01 to EP-34) and release milestones.
- docs/TODO.md — Active engineering tasks and technical debt registry.
Scanning benchmark on a codebase of 500,000 LOC across 1,200 files:
| Tool | Engine Language | Scan Time | Memory Usage | Type 3 AST Clones |
|---|---|---|---|---|
| CDDM | Rust 2024 | 180 ms | 45 MB | Yes (Tree-sitter) |
jscpd |
Node.js | 4,200 ms | 320 MB | No |
PMD-CPD |
Java | 6,800 ms | 510 MB | No |
We welcome community contributions! Please check out CONTRIBUTING.md for our Git issue-driven development protocol, branch conventions, and testing guidelines.
Please read our Code of Conduct and Security Policy.
Dual-licensed under either of:
- Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.