|
9 | 9 | [](https://github.com/jorge-martinez-gil/small-code-models/network/members) |
10 | 10 | [](https://colab.research.google.com/github/jorge-martinez-gil/small-code-models/blob/main/notebooks/quick_start.ipynb) |
11 | 11 | [](https://doi.org/10.48550/arXiv.2506.10995) |
12 | | -[](https://github.com/jorge-martinez-gil/small-code-models/actions/workflows/ci.yml) |
13 | 12 |
|
14 | 13 | **TL;DR:** This repository is the official, reproducible benchmark suite for the paper *Evaluating Small-Scale Code Models for Code Clone Detection*, with unified scripts to train and evaluate six compact code models across five clone-detection datasets. |
15 | 14 |
|
@@ -38,66 +37,6 @@ This work rigorously evaluates **small-scale transformer-based code models (<220 |
38 | 37 | | UniXCoder | ~200 M | Unified Enc-Dec | `microsoft/unixcoder-base` | |
39 | 38 | | Salesforce CodeT5 | 220 M | Encoder-Decoder | `Salesforce/codet5-base` | |
40 | 39 |
|
41 | | -## Benchmark Results |
42 | | - |
43 | | -The following values are **representative values — see paper for exact figures**. |
44 | | - |
45 | | -### BigCloneBench (BCB) |
46 | | - |
47 | | -| Model | Precision | Recall | F1 | |
48 | | -|---|---|---|---| |
49 | | -| CodeBERT | 0.91 | 0.89 | 0.90 | |
50 | | -| GraphCodeBERT | 0.92 | 0.90 | 0.91 | |
51 | | -| PLBART | 0.87 | 0.85 | 0.86 | |
52 | | -| PolyCoder | 0.82 | 0.80 | 0.81 | |
53 | | -| UniXCoder | 0.90 | 0.88 | 0.89 | |
54 | | -| CodeT5 | 0.88 | 0.86 | 0.87 | |
55 | | - |
56 | | -### POJ104 |
57 | | - |
58 | | -| Model | Precision | Recall | F1 | |
59 | | -|---|---|---|---| |
60 | | -| CodeBERT | 0.89 | 0.87 | 0.88 | |
61 | | -| GraphCodeBERT | 0.90 | 0.88 | 0.89 | |
62 | | -| PLBART | 0.84 | 0.83 | 0.84 | |
63 | | -| PolyCoder | 0.79 | 0.77 | 0.78 | |
64 | | -| UniXCoder | 0.88 | 0.86 | 0.87 | |
65 | | -| CodeT5 | 0.85 | 0.84 | 0.85 | |
66 | | - |
67 | | -### GCJ |
68 | | - |
69 | | -| Model | Precision | Recall | F1 | |
70 | | -|---|---|---|---| |
71 | | -| CodeBERT | 0.87 | 0.85 | 0.86 | |
72 | | -| GraphCodeBERT | 0.88 | 0.86 | 0.87 | |
73 | | -| PLBART | 0.82 | 0.80 | 0.81 | |
74 | | -| PolyCoder | 0.76 | 0.74 | 0.75 | |
75 | | -| UniXCoder | 0.86 | 0.84 | 0.85 | |
76 | | -| CodeT5 | 0.83 | 0.81 | 0.82 | |
77 | | - |
78 | | -### Karnalim |
79 | | - |
80 | | -| Model | Precision | Recall | F1 | |
81 | | -|---|---|---|---| |
82 | | -| CodeBERT | 0.85 | 0.83 | 0.84 | |
83 | | -| GraphCodeBERT | 0.86 | 0.84 | 0.85 | |
84 | | -| PLBART | 0.80 | 0.78 | 0.79 | |
85 | | -| PolyCoder | 0.73 | 0.71 | 0.72 | |
86 | | -| UniXCoder | 0.84 | 0.82 | 0.83 | |
87 | | -| CodeT5 | 0.81 | 0.79 | 0.80 | |
88 | | - |
89 | | -### PoolC |
90 | | - |
91 | | -| Model | Precision | Recall | F1 | |
92 | | -|---|---|---|---| |
93 | | -| CodeBERT | 0.88 | 0.86 | 0.87 | |
94 | | -| GraphCodeBERT | 0.89 | 0.87 | 0.88 | |
95 | | -| PLBART | 0.83 | 0.81 | 0.82 | |
96 | | -| PolyCoder | 0.77 | 0.75 | 0.76 | |
97 | | -| UniXCoder | 0.87 | 0.85 | 0.86 | |
98 | | -| CodeT5 | 0.84 | 0.82 | 0.83 | |
99 | | - |
100 | | -Exact figures are reported in Table 2 of the <a href="https://doi.org/10.48550/arXiv.2506.10995">paper</a>. Run the scripts to reproduce. |
101 | 40 |
|
102 | 41 | ## Quick Start |
103 | 42 |
|
|
0 commit comments