@kongyo2/similarity-ts

TypeScript-only similarity analyzer

Private edition of mizchi/similarity.

Quick Start

npx @kongyo2/similarity-ts .

Refactor with AI

Use the following prompt with your AI coding assistant:

Run `npx @kongyo2/similarity-ts .` to detect semantic code similarities.
Execute this command, analyze the duplicate code patterns, and create a refactoring plan.

What it detects

The analyzer compares functions, types, and classes structurally. Every function is alpha-renamed before comparison — parameters, locals, inner functions, and the declaration name all get positional canonical names — so consistent renames never hide a duplicate, no matter how many identifiers changed. Free identifiers (imported helpers, globals, property names) keep their names, because which function you call is behavior.

On top of that, the comparison canonicalizes common style alternatives — two snippets a reviewer would call "the same code written differently" score as equal:

• arrow functions ⇔ function declarations ⇔ class methods ⇔ arrow-valued class fields, expression bodies ⇔ block bodies
• promise.then((v) => …) ⇔ const v = await promise; …
• items.forEach((item) => …) ⇔ for (const item of items) … ⇔ for (let i = 0; i < items.length; i++) { const item = items[i]; … }
• push-accumulator loops ⇔ items.map(...) / items.filter(...)
• `Hello ${name}` ⇔ "Hello " + name
• total += n ⇔ total = total + n ⇔ i++ (in statement position)
• return c ? a : b ⇔ if (c) { return a } else { return b } ⇔ if (c) { return a } return b (guard style)
• if (!c) { A } else { B } ⇔ if (c) { B } else { A }, !== guards ⇔ flipped === guards, De Morgan (!(a && b) ⇔ !a || !b)
• x ? x : y ⇔ x || y · x == null ? d : x ⇔ x ?? d · x === null || x === undefined ⇔ x == null · void 0 ⇔ undefined · Boolean(x) ⇔ !!x
• const { a, b } = obj ⇔ const a = obj.a; const b = obj.b, including single-use temporaries inlined either way
• const x = c ? a : b ⇔ let x; if (c) { x = a } else { x = b }
• jump-terminated switch ⇔ if/else if chains ⇔ guard ladders
• Object.assign({}, a, b) ⇔ { ...a, ...b }
• constructor(private readonly repo: Repo) {} ⇔ constructor(repo: Repo) { this.repo = repo; } (parameter properties are desugared the way the compiler emits them, after any super() call)
• for (; cond; ) ⇔ while (cond), braced ⇔ brace-less bodies, Yoda comparisons (0 === n) ⇔ natural order, arr[arr.length - 1] ⇔ arr.at(-1), reordered independent const declarations
• interfaces ⇔ structurally identical type aliases, reordered members, renamed-but-identically-typed property sets, Array<T> ⇔ T[], x?: T ⇔ x: T | undefined, reordered unions, renamed generic params
• classes: reordered members, renamed private fields, constructor parameter properties ⇔ explicit field + assignment, and fully renamed method names when the canonical method bodies match

Every declaration form takes part: function declarations, arrow and function expressions bound to a variable (a named function expression keeps recursing through its own name), export default classes and functions, class methods and arrow-function class fields, constructors, and declarations inside namespace/module blocks, which are reported under their qualified name (Legacy.Options). Bodiless declarations — overload signatures, declare function, abstract methods — are never reported: there is no code behind them.

Rewrites that change behavior keep their distinct shapes on purpose: swapped builtins (.map vs .filter, Math.max vs Math.min), a different free callee (sendEmail vs sendSms), flipped operators (* vs /, && vs ||, < vs <=), ?? vs ||, x == null vs x === null, optional chaining vs plain access, for-of vs for-in, an added break, reordered statements that share data, async vs sync contracts, then(onFulfilled, onRejected), fall-through switch cases, Object.assign with a mutated target, string prepend vs append folds (trail = trail + seg vs trail = seg + trail), boundary indexes (.at(-1) vs .at(0), .slice(0, n) vs .slice(n)), templates that stringify adjacent values (`${a}${b}` vs numeric a + b), and on the type side Promise<ShopUser> vs Promise<ShopOrder> payloads, Map<K, V> vs Map<V, K> swaps, index signatures vs concrete members, discriminated-union variants that differ in their kind literal or enum member (signed numbers and bigints included), differing extends heritage (type arguments included: Box<string> is not Box<number>), brand markers (_fooBrand: any), typeof/type-predicate members naming different things, and one- or two-member shapes whose only shared trait is a primitive. On the class side a constructor that does real work counts as a member, a different, qualified-vs-absent, or one-sided base class is a contract difference, an implements clause the other side lacks is one more difference, and a single shared field is not a duplicate class. Twins that differ only in data literals (a table name, a status code, a locale string) are reported — parameterizing them is the refactor.

Accuracy

Accuracy is tracked by a labeled benchmark (bench/cases.ts plus the extended corpora in bench/cases/) that mirrors the refactoring flow above: 300 ground-truth pairs across functions, types, and classes — semantic duplicates a refactoring plan must see, and similarly-shaped lookalikes it must not flag — evaluated at the default threshold. The corpus covers whole-function renames, guard/negation/ternary spellings, loop-form rewrites, destructuring, nullish sugar, literal-vs-behavior twins, realistic cross-file copy-paste, and — since 0.7.0 — the lookalike families found by auditing the engine's reports on a snapshot of the TypeScript compiler repository: bodiless overload signatures, discriminated-union variants (enum members, string, signed-number and bigint literals), heritage (one-sided, differing, differently instantiated or qualified bases), brand markers, tiny renamed shapes, typeof/predicate members, constructor work, and declaration scopes — namespaces, default exports, named function expressions.

Engine	Corpus	Wrong labels	Error rate	Accuracy
v0.3.0	71 pairs	15 / 71	21.13%	78.87%
v0.4.1	71 pairs	0 / 71	0.00%	100.00%
v0.4.1	261 pairs	89 / 261	34.10%	65.90%
v0.5.0	261 pairs	7 / 261	2.68%	97.32%
v0.6.0	261 pairs	0 / 261	0.00%	100.00%
v0.6.0	300 pairs	29 / 300	9.67%	90.33%
v0.7.0	300 pairs	0 / 300	0.00%	100.00%

v0.7.0 was audited against real code: run over the TypeScript compiler's API package, its VS Code extension, and 1,482 conformance test files, the v0.6.0 engine reported every AST-node interface pair that differs only in its kind discriminant, every overload signature as a 1.000 duplicate of the next, and thousands of single-field conformance classes as twins, while never extracting function expressions, arrow-function class fields, or anything declared inside a namespace. v0.7.0 fixes each family (type pairs on the 108-file API package drop from 685 to 419, conformance class pairs from 15,062 to 5,793, and the newly extracted declaration forms surface their twins), adds the families to the corpus, and keeps the original 261 pairs at 100% with unchanged margins. Reports are also deterministic now — type mode used to compare declarations in hash-map order, which moved borderline scores between runs.

Run it yourself with npm run bench:accuracy; the suite in tests/accuracy-benchmark.test.ts fails CI if any labeled pair is mislabeled.

CLI options

Option	Default	Description
--modes <list>	functions,types,classes,overlap	Comma-separated analysis modes
-t, --threshold <number>	0.8	Similarity threshold (0–1)
--min-lines <number>	3	Minimum function line count
--min-tokens <number>	—	Minimum function size in AST nodes (replaces the line gate; ~50 recommended for noisy code)
--no-size-penalty	off	Disable the short-function score penalty
--same-file-only / --cross-file-only	off	Restrict pair scope
--extensions <list>	ts,tsx,mts,cts	File extensions to scan
--exclude <pattern>	—	Exclude glob (repeatable)
--types-only <kind>	all	Restrict type mode to interface or type
--no-allow-cross-kind	off	Disable interface ⇔ type alias matching
--type-literals	off	Include anonymous type literals in type mode
--format <pretty|json>	pretty	Output format
--output <path>	—	Write the report to a file (parent directories are created)
--fail-on-warnings	off	Non-zero exit on analyzer warnings
--fail-on-duplicates	off	Non-zero exit when any pair is reported (CI gate)

Annotate a declaration with a // similarity-ignore comment on the preceding line to exclude it from the report.

References

The function comparison implements TSED (Tree Similarity of Edit Distance): both fragments are parsed to ASTs, an APTED-style tree edit distance δ with configurable per-operation weights (rename 0.3, delete 1.0, insert 1.0 — tuned against the labeled corpus) is computed, and the score is normalized as TSED = max(1 − δ / MaxNodes(G1, G2), 0). The alpha-renaming, refactor canonicalization, behavioral-atom guard, and size-penalty layers documented above are this project's additions on top of that metric; the operation-weight sensitivity they exploit is the paper's RQ3 finding that TSED's penalty weights are influential and language-dependent. Following that finding, the weights are re-validated against the labeled corpus on every accuracy release: for 0.7.0 every setting in rename 0.2–0.5 × delete/insert 0.8–1.0 classifies all 300 pairs correctly and moves the tightest margins by less than 0.03, so the calibrated defaults stand.

• Yewei Song, Cedric Lothritz, Daniel Tang, Tegawendé F. Bissyandé, and Jacques Klein. 2024. Revisiting Code Similarity Evaluation with Abstract Syntax Tree Edit Distance. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). ACL Anthology 2024.acl-short.3 · arXiv:2404.08817
• Yewei Song, Saad Ezzini, Xunzhu Tang, Cedric Lothritz, Jacques Klein, Tegawendé Bissyandé, Andrey Boytsov, Ulrick Ble, and Anne Goujon. 2023. Enhancing Text-to-SQL Translation for Financial System Design. The paper that introduced the original TSED metric. arXiv:2312.14725
• Mateusz Pawlik and Nikolaus Augsten. 2015. Efficient Computation of the Tree Edit Distance. ACM Transactions on Database Systems 40(1) — the APTED algorithm family used for δ. See also Pawlik and Augsten 2016, Tree edit distance: Robust and memory-efficient, Information Systems 56.