Elevate repository to reproducible research artifact with shared library, unified scripts, CI, tests, and publication-grade documentation

[Copilot]

This PR restructures the repository into a citation-ready, reproducible benchmark artifact aligned with the companion paper. It centralizes duplicated training/evaluation logic, formalizes packaging and CI, and upgrades documentation for faster adoption by researchers.

• Documentation and research framing

  • Replaced README.md with a paper-centric structure: expanded abstract, contributions, model/HF Hub matrix, per-dataset benchmark summary tables, quick start, repository tree, citation formats (BibTeX + APA), related work, and contact details.
  • Added docs/RESULTS.md with dataset-by-dataset result tables, key takeaways, and reproducibility notes.
  • Updated CHANGELOG.md with 1.1.0 entry reflecting the new architecture and reproducibility assets.

• Shared Python package (small_code_models/)

  • Added small_code_models/__init__.py, data.py, metrics.py, and trainer.py.
  • Introduced reusable dataset loading/building APIs, centralized metric computation/printing, and a thin Trainer wrapper with consistent defaults and run flow.

• Script refactor and CLI standardization

  • Refactored benchmark scripts to consume shared package utilities instead of per-file boilerplate.
  • Added consistent CLI surface (--data_dir, --output_dir, --sample_pct, --epochs) and removed hardcoded Google Drive paths.
  • Standardized script naming (including POJ104 PLBART path consistency).

• Reproducibility and developer ergonomics

  • Added pyproject.toml for editable installation and dependency metadata.
  • Added scripts/run_all_benchmarks.sh to execute all model × dataset runs and emit a compact summary.
  • Added notebooks/quick_start.ipynb (Colab-ready, self-contained synthetic demo).

• Quality gates

  • Added GitHub Actions workflow .github/workflows/ci.yml (Python 3.10, package install, lint, conditional tests).
  • Added tests/test_metrics.py and tests/test_data.py with focused unit coverage for shared utilities.
  • Added root .gitignore for Python artifact hygiene.

from small_code_models.data import build_datasets
from small_code_models.trainer import CloneDetectionTrainer, get_training_args
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("microsoft/codebert-base")
model = AutoModelForSequenceClassification.from_pretrained("microsoft/codebert-base", num_labels=2)

train_ds, val_ds, test_ds = build_datasets("/path/to/bcb", tokenizer)
trainer = CloneDetectionTrainer(model=model, args=get_training_args("results/codebert_bcb"))
metrics = trainer.run(train_ds, val_ds, test_ds)

Original prompt

Goal

Transform the small-code-models repository into a highly professional, academic-quality research artifact that maximises citations, reproducibility, and community adoption. The companion paper is "Evaluating Small-Scale Code Models for Code Clone Detection" (arXiv:2506.10995, DOI: https://doi.org/10.48550/arXiv.2506.10995), authored by Jorge Martinez-Gil (ORCID: 0000-0002-1632-1580).

---

1. Overhaul the README.md

Replace the existing README.md with a much richer version that includes all the following sections, in order:

Header section

• Title: # Evaluating Small-Scale Code Models for Code Clone Detection
• Badge row (keep existing badges, add new ones):
  • [![GitHub stars](https://img.shields.io/github/stars/jorge-martinez-gil/small-code-models?style=social)](…)
  • [![GitHub forks](https://img.shields.io/github/forks/jorge-martinez-gil/small-code-models?style=social)](…)
  • [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/jorge-martinez-gil/small-code-models/blob/main/notebooks/quick_start.ipynb)
  • [![DOI](https://img.shields.io/badge/DOI-10.48550%2FarXiv.2506.10995-blue)](https://doi.org/10.48550/arXiv.2506.10995)
  • [![CI](https://github.com/jorge-martinez-gil/small-code-models/actions/workflows/ci.yml/badge.svg)](…)
• A concise one-paragraph TL;DR immediately after the badges.

Abstract

Keep the existing paragraph but add a final sentence: "All evaluation scripts, pre-processed dataset loaders, and results are publicly available in this repository to facilitate reproducibility and further research."

Key Contributions (bullet list)

Clearly enumerate the main research contributions:

1. First systematic comparison of six <220 M parameter code models on five established clone-detection benchmarks.
2. Unified, reproducible evaluation harness (train → evaluate → report F1/Precision/Recall) runnable in a single command.
3. Publicly released, pre-configured training scripts for BigCloneBench, POJ104, GCJ, Karnalim, and PoolC.
4. Evidence that encoder-only models (CodeBERT, GraphCodeBERT) consistently outperform decoder-only counterparts on clone-detection tasks.
5. Lightweight shared library (small_code_models/) enabling researchers to plug in new models with ≤30 lines of code.

Models Evaluated (keep existing table, add a "HuggingFace Hub ID" column)

Model	Parameters	Architecture	HuggingFace Hub ID
CodeBERT	125 M	Encoder-only	microsoft/codebert-base
GraphCodeBERT	125 M	Encoder-only (Data-Flow)	microsoft/graphcodebert-base
PLBART	140 M	Encoder-Decoder	uclanlp/plbart-base
PolyCoder	160 M	Decoder-only	NinedayWang/PolyCoder-0.4B
UniXCoder	~200 M	Unified Enc-Dec	microsoft/unixcoder-base
Salesforce CodeT5	220 M	Encoder-Decoder	Salesforce/codet5-base

Benchmark Results

Add a Results Summary table for each dataset (use placeholder values clearly marked as "representative values — see paper for exact figures"):

BigCloneBench (BCB)

Model	Precision	Recall	F1
CodeBERT	0.91	0.89	0.90
GraphCodeBERT	0.92	0.90	0.91
PLBART	0.87	0.85	0.86
PolyCoder	0.82	0.80	0.81
UniXCoder	0.90	0.88	0.89
CodeT5	0.88	0.86	0.87

(Add equivalent tables for POJ104, GCJ, Karnalim, PoolC.)

Add a note: "Exact figures are reported in Table 2 of the paper. Run the scripts to reproduce."

Quick Start (new section)

# 1. Clone
git clone https://github.com/jorge-martinez-gil/small-code-models.git
cd small-code-models

# 2. Install
pip install -e ".[dev]"

# 3. Run CodeBERT on BigCloneBench
python bcb_detection_models/codebert-bcb-01.py \
    --data_dir /path/to/bcb \
    --output_dir results/codebert_bcb

# 4. Run ALL models on ALL datasets (bash)
bash scripts/run_all_benchmarks.sh /path/to/datasets

Also include a Google Colab badge pointing to notebooks/quick_start.ipynb.

Repository Structure (tree diagram)

small-code-models/
├── small_code_models/          # Shared Python library
│   ├── __init__.py
│   ├── data.py                 # Dataset loading utilities
│   ├── metrics.py              # Evaluation metrics
│   └── trainer.py              # Generic fine-tuning trainer
├── bcb_detection_models/       # BigCloneBench scripts (×6 models)
├── gcj_clone_detection_models/ # Google Code Jam scripts
├── karnalim_clone_detection_models/
├── poj104_clone_detection_models/
├── poolc_clone_detection_models/
├── notebooks/
│   └── quick_start.ipynb       # Interactive demo
├── scripts/
│   └── run_all_benchmarks.sh  # Full reproduction script
├── docs/
│   └── RESULTS.md              # Detailed results & analysis
├── pyproject.toml
└── requirements.txt

Citing this work (keep and expand)

Add both BibTeX and APA format, plus a note:...

This pull request was created from Copilot chat.