TypeScript-only similarity analyzer
Private edition of mizchi/similarity.
npx @kongyo2/similarity-ts .Use the following prompt with your AI coding assistant:
Run `npx @kongyo2/similarity-ts .` to detect semantic code similarities.
Execute this command, analyze the duplicate code patterns, and create a refactoring plan.
The analyzer compares functions, types, and classes structurally. Every function is alpha-renamed before comparison — parameters, locals, inner functions, and the declaration name all get positional canonical names — so consistent renames never hide a duplicate, no matter how many identifiers changed. Free identifiers (imported helpers, globals, property names) keep their names, because which function you call is behavior.
On top of that, the comparison canonicalizes common style alternatives — two snippets a reviewer would call "the same code written differently" score as equal:
- arrow functions ⇔ function declarations ⇔ class methods ⇔ arrow-valued class fields, expression bodies ⇔ block bodies
promise.then((v) => …)⇔const v = await promise; …items.forEach((item) => …)⇔for (const item of items) …⇔for (let i = 0; i < items.length; i++) { const item = items[i]; … }- push-accumulator loops ⇔
items.map(...)/items.filter(...) `Hello ${name}`⇔"Hello " + nametotal += n⇔total = total + n⇔i++(in statement position)return c ? a : b⇔if (c) { return a } else { return b }⇔if (c) { return a } return b(guard style)if (!c) { A } else { B }⇔if (c) { B } else { A },!==guards ⇔ flipped===guards, De Morgan (!(a && b)⇔!a || !b)x ? x : y⇔x || y·x == null ? d : x⇔x ?? d·x === null || x === undefined⇔x == null·void 0⇔undefined·Boolean(x)⇔!!xconst { a, b } = obj⇔const a = obj.a; const b = obj.b, including single-use temporaries inlined either wayconst x = c ? a : b⇔let x; if (c) { x = a } else { x = b }- jump-terminated
switch⇔if/else ifchains ⇔ guard ladders Object.assign({}, a, b)⇔{ ...a, ...b }constructor(private readonly repo: Repo) {}⇔constructor(repo: Repo) { this.repo = repo; }(parameter properties are desugared the way the compiler emits them, after anysuper()call)for (; cond; )⇔while (cond), braced ⇔ brace-less bodies, Yoda comparisons (0 === n) ⇔ natural order,arr[arr.length - 1]⇔arr.at(-1), reordered independentconstdeclarations- interfaces ⇔ structurally identical type aliases, reordered members,
renamed-but-identically-typed property sets,
Array<T>⇔T[],x?: T⇔x: T | undefined, reordered unions, renamed generic params - classes: reordered members, renamed private fields, constructor parameter properties ⇔ explicit field + assignment, and fully renamed method names when the canonical method bodies match
Every declaration form takes part: function declarations, arrow and
function expressions bound to a variable (a named function expression keeps
recursing through its own name), export default classes and functions,
class methods and arrow-function class fields, constructors, and
declarations inside namespace/module blocks, which are reported under
their qualified name (Legacy.Options). Bodiless declarations — overload
signatures, declare function, abstract methods — are never reported:
there is no code behind them.
Rewrites that change behavior keep their distinct shapes on purpose:
swapped builtins (.map vs .filter, Math.max vs Math.min), a
different free callee (sendEmail vs sendSms), flipped operators
(* vs /, && vs ||, < vs <=), ?? vs ||, x == null vs
x === null, optional chaining vs plain access, for-of vs for-in, an
added break, reordered statements that share data, async vs sync
contracts, then(onFulfilled, onRejected), fall-through switch cases,
Object.assign with a mutated target, string prepend vs append folds
(trail = trail + seg vs trail = seg + trail), boundary indexes
(.at(-1) vs .at(0), .slice(0, n) vs .slice(n)), templates that
stringify adjacent values (`${a}${b}` vs numeric a + b), and on
the type side Promise<ShopUser> vs Promise<ShopOrder> payloads,
Map<K, V> vs Map<V, K> swaps, index signatures vs concrete members,
discriminated-union variants that differ in their kind literal or enum
member (signed numbers and bigints included), differing extends heritage
(type arguments included: Box<string> is not Box<number>), brand
markers (_fooBrand: any), typeof/type-predicate members naming
different things, and one- or two-member shapes whose only shared trait is
a primitive. On the class side a constructor that does real work counts as
a member, a different, qualified-vs-absent, or one-sided base class is a
contract difference, an implements clause the other side lacks is one
more difference, and a single shared field is not a duplicate class. Twins
that differ
only in data literals (a table name, a status code, a locale string)
are reported — parameterizing them is the refactor.
Accuracy is tracked by a labeled benchmark (bench/cases.ts plus the
extended corpora in bench/cases/) that mirrors the refactoring flow
above: 300 ground-truth pairs across functions, types, and classes —
semantic duplicates a refactoring plan must see, and similarly-shaped
lookalikes it must not flag — evaluated at the default threshold. The
corpus covers whole-function renames, guard/negation/ternary spellings,
loop-form rewrites, destructuring, nullish sugar, literal-vs-behavior
twins, realistic cross-file copy-paste, and — since 0.7.0 — the lookalike
families found by auditing the engine's reports on a snapshot of the
TypeScript compiler repository: bodiless overload signatures,
discriminated-union variants (enum members, string, signed-number and
bigint literals), heritage (one-sided, differing, differently instantiated
or qualified bases), brand markers, tiny renamed shapes, typeof/predicate
members, constructor work, and declaration scopes — namespaces, default
exports, named function expressions.
| Engine | Corpus | Wrong labels | Error rate | Accuracy |
|---|---|---|---|---|
| v0.3.0 | 71 pairs | 15 / 71 | 21.13% | 78.87% |
| v0.4.1 | 71 pairs | 0 / 71 | 0.00% | 100.00% |
| v0.4.1 | 261 pairs | 89 / 261 | 34.10% | 65.90% |
| v0.5.0 | 261 pairs | 7 / 261 | 2.68% | 97.32% |
| v0.6.0 | 261 pairs | 0 / 261 | 0.00% | 100.00% |
| v0.6.0 | 300 pairs | 29 / 300 | 9.67% | 90.33% |
| v0.7.0 | 300 pairs | 0 / 300 | 0.00% | 100.00% |
v0.7.0 was audited against real code: run over the TypeScript compiler's
API package, its VS Code extension, and 1,482 conformance test files, the
v0.6.0 engine reported every AST-node interface pair that differs only in
its kind discriminant, every overload signature as a 1.000 duplicate of
the next, and thousands of single-field conformance classes as twins,
while never extracting function expressions, arrow-function class fields,
or anything declared inside a namespace. v0.7.0 fixes each family (type
pairs on the 108-file API package drop from 685 to 419, conformance class
pairs from 15,062 to 5,793, and the newly extracted declaration forms
surface their twins), adds the families to the corpus, and keeps the
original 261 pairs at 100% with unchanged margins. Reports are also
deterministic now — type mode used to compare declarations in hash-map
order, which moved borderline scores between runs.
Run it yourself with npm run bench:accuracy; the suite in
tests/accuracy-benchmark.test.ts fails CI if any labeled pair is
mislabeled.
| Option | Default | Description |
|---|---|---|
--modes <list> |
functions,types,classes,overlap |
Comma-separated analysis modes |
-t, --threshold <number> |
0.8 |
Similarity threshold (0–1) |
--min-lines <number> |
3 |
Minimum function line count |
--min-tokens <number> |
— | Minimum function size in AST nodes (replaces the line gate; ~50 recommended for noisy code) |
--no-size-penalty |
off | Disable the short-function score penalty |
--same-file-only / --cross-file-only |
off | Restrict pair scope |
--extensions <list> |
ts,tsx,mts,cts |
File extensions to scan |
--exclude <pattern> |
— | Exclude glob (repeatable) |
--types-only <kind> |
all |
Restrict type mode to interface or type |
--no-allow-cross-kind |
off | Disable interface ⇔ type alias matching |
--type-literals |
off | Include anonymous type literals in type mode |
--format <pretty|json> |
pretty |
Output format |
--output <path> |
— | Write the report to a file (parent directories are created) |
--fail-on-warnings |
off | Non-zero exit on analyzer warnings |
--fail-on-duplicates |
off | Non-zero exit when any pair is reported (CI gate) |
Annotate a declaration with a // similarity-ignore comment on the
preceding line to exclude it from the report.
The function comparison implements TSED (Tree Similarity of Edit
Distance): both fragments are parsed to ASTs, an APTED-style tree edit
distance δ with configurable per-operation weights (rename 0.3, delete
1.0, insert 1.0 — tuned against the labeled corpus) is computed, and the
score is normalized as TSED = max(1 − δ / MaxNodes(G1, G2), 0). The
alpha-renaming, refactor canonicalization, behavioral-atom guard, and
size-penalty layers documented above are this project's additions on top
of that metric; the operation-weight sensitivity they exploit is the
paper's RQ3 finding that TSED's penalty weights are influential and
language-dependent. Following that finding, the weights are re-validated
against the labeled corpus on every accuracy release: for 0.7.0 every
setting in rename 0.2–0.5 × delete/insert 0.8–1.0 classifies all 300 pairs
correctly and moves the tightest margins by less than 0.03, so the
calibrated defaults stand.
- Yewei Song, Cedric Lothritz, Daniel Tang, Tegawendé F. Bissyandé, and Jacques Klein. 2024. Revisiting Code Similarity Evaluation with Abstract Syntax Tree Edit Distance. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). ACL Anthology 2024.acl-short.3 · arXiv:2404.08817
- Yewei Song, Saad Ezzini, Xunzhu Tang, Cedric Lothritz, Jacques Klein, Tegawendé Bissyandé, Andrey Boytsov, Ulrick Ble, and Anne Goujon. 2023. Enhancing Text-to-SQL Translation for Financial System Design. The paper that introduced the original TSED metric. arXiv:2312.14725
- Mateusz Pawlik and Nikolaus Augsten. 2015. Efficient Computation of the Tree Edit Distance. ACM Transactions on Database Systems 40(1) — the APTED algorithm family used for δ. See also Pawlik and Augsten 2016, Tree edit distance: Robust and memory-efficient, Information Systems 56.