Skip to content

Commit f6ef59c

Browse files
SaveenaSolankiSaveenaSolanki
committed
Initial release: scaffold splitting and leakage audit toolkit
0 parents  commit f6ef59c

16 files changed

Lines changed: 1003 additions & 0 deletions

File tree

‎.github/workflows/tests.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
name: tests
2+
3+
on:
4+
push:
5+
branches: [main, master]
6+
pull_request:
7+
branches: [main, master]
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
strategy:
13+
matrix:
14+
python-version: ["3.10", "3.11", "3.12"]
15+
include:
16+
- python-version: "3.10"
17+
extras: ""
18+
- python-version: "3.11"
19+
extras: "[rdkit]"
20+
- python-version: "3.12"
21+
extras: "[rdkit]"
22+
23+
steps:
24+
- uses: actions/checkout@v4
25+
26+
- name: Set up Python ${{ matrix.python-version }}
27+
uses: actions/setup-python@v5
28+
with:
29+
python-version: ${{ matrix.python-version }}
30+
31+
- name: Install package and dev dependencies
32+
run: |
33+
python -m pip install --upgrade pip
34+
pip install -e ".[dev]${{ matrix.extras }}"
35+
36+
- name: Lint with ruff
37+
run: ruff check src/ tests/
38+
39+
- name: Run tests
40+
run: pytest --cov=scaffoldsplitlab --cov-report=term-missing

‎.gitignore‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
__pycache__/
2+
*.py[cod]
3+
*.egg-info/
4+
dist/
5+
build/
6+
.pytest_cache/
7+
.ruff_cache/
8+
*.so
9+
.env
10+
.venv/
11+
venv/
12+
.coverage
13+
htmlcov/
14+
output/

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2025
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# scaffoldsplitlab
2+
3+
**Scaffold-based train/val/test splitting for molecular machine learning.**
4+
5+
[![tests](https://github.com/your-org/scaffoldsplitlab/actions/workflows/tests.yml/badge.svg)](https://github.com/your-org/scaffoldsplitlab/actions/workflows/tests.yml)
6+
[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/)
7+
8+
## Overview
9+
10+
`scaffoldsplitlab` splits molecular datasets into training, validation, and test sets using either:
11+
12+
- **Scaffold-based splitting** — groups molecules by their Murcko scaffold and assigns whole scaffolds to the same split. This provides a more realistic estimate of generalization to novel chemical scaffolds. _(Requires optional RDKit.)_
13+
- **Random splitting** — standard stratified random split, optionally using a label column for stratification.
14+
- **Hash-based fallback** — when RDKit is not available, the scaffold method falls back to a deterministic hash-based split using molecule SMILES.
15+
16+
## Installation
17+
18+
```bash
19+
# Basic install (random splitting only)
20+
pip install scaffoldsplitlab
21+
22+
# With RDKit support (scaffold splitting)
23+
pip install scaffoldsplitlab[rdkit]
24+
25+
# Development install
26+
pip install -e ".[dev,rdkit]"
27+
```
28+
29+
## Quick Start
30+
31+
```bash
32+
# Random split
33+
scaffold-split --input molecules.csv --method random
34+
35+
# Scaffold split (requires RDKit)
36+
scaffold-split --input molecules.csv --method scaffold
37+
38+
# Stratified random split
39+
scaffold-split --input molecules.csv --method random --label-col label
40+
41+
# Custom fractions
42+
scaffold-split --input molecules.csv --train 0.7 --val 0.15 --test 0.15 --seed 123
43+
```
44+
45+
Run the demo:
46+
47+
```bash
48+
bash examples/run_demo.sh
49+
```
50+
51+
## CLI Reference
52+
53+
```
54+
scaffold-split [OPTIONS]
55+
56+
Options:
57+
-i, --input PATH Input CSV file (required)
58+
-s, --smiles-col TEXT SMILES column name [default: smiles]
59+
-l, --label-col TEXT Label column (for stratification)
60+
-m, --method TEXT Split method: 'random' or 'scaffold' [default: scaffold]
61+
--train FLOAT Training fraction [default: 0.8]
62+
--val FLOAT Validation fraction [default: 0.1]
63+
--test FLOAT Test fraction [default: 0.1]
64+
--seed INTEGER Random seed [default: 42]
65+
-o, --out PATH Output CSV path [default: split_output.csv]
66+
```
67+
68+
## Python API
69+
70+
```python
71+
from scaffoldsplitlab.core import scaffold_split, random_split, check_split_leakage
72+
from scaffoldsplitlab.io import read_csv, write_csv
73+
74+
df = read_csv("molecules.csv")
75+
result = scaffold_split(df, smiles_col="smiles", train_frac=0.8, val_frac=0.1, test_frac=0.1)
76+
write_csv(result, "split_output.csv")
77+
78+
# Check for duplicate SMILES across splits
79+
leaks = check_split_leakage(result, smiles_col="smiles")
80+
print(leaks) # {} if no leakage
81+
```
82+
83+
## Non-Sensitive Data Statement
84+
85+
This tool processes molecular structure data (SMILES strings) provided by the user. **It does not collect, transmit, or store any user data externally.** All computation is performed locally on the user's machine. No telemetry, analytics, or network calls are made. The tool does not ship with any pre-loaded chemical datasets. The example dataset (`examples/demo_input.csv`) contains 15 small molecules commonly used in cheminformatics tutorials and does not include any personally identifiable, proprietary, or sensitive information.
86+
87+
## Non-Sensitive Data Statement
88+
89+
This repository contains generic utility code and toy/public-data examples only. It does not include unpublished datasets, internal lab data, trained models, manuscript-specific results, or confidential project assets.
90+
91+
## License
92+
93+
MIT
94+
95+
## Development
96+
97+
```bash
98+
git clone https://github.com/your-org/scaffoldsplitlab.git
99+
cd scaffoldsplitlab
100+
pip install -e ".[dev,rdkit]"
101+
pytest
102+
ruff check src/ tests/
103+
```

‎environment.yml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
name: scaffoldsplitlab
2+
channels:
3+
- conda-forge
4+
- defaults
5+
dependencies:
6+
- python>=3.10
7+
- pip
8+
- typer>=0.9.0
9+
- rich>=13.0.0
10+
- pandas>=1.5.0
11+
- pytest>=7.0.0
12+
- ruff>=0.1.0
13+
- scikit-learn>=1.0.0
14+
- numpy>=1.21.0
15+
- pip:
16+
- rdkit # optional

‎examples/demo_input.csv‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
smiles,label
2+
CCO,0
3+
CC(=O)O,1
4+
c1ccccc1,0
5+
CCN(CC)CC,1
6+
C1=CC=C(C=C1)C(=O)O,0
7+
CC(C)CC(C)C(=O)O,1
8+
c1ccccc1O,0
9+
CC(C)Cc1ccc(C(C)C(=O)O)cc1,1
10+
CN1C=NC2=C1C(=O)N(C(=O)N2C)C,0
11+
CC(C)CC(C(=O)O)N,1
12+
O=C(O)C(N)Cc1ccccc1,0
13+
CCCCCC,1
14+
C1CCCCC1,0
15+
CC(C)O,1
16+
COc1ccc2[nH]cc(CCN)c2c1,0

‎examples/run_demo.sh‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
#!/usr/bin/env bash
2+
# run_demo.sh — Demo script for scaffoldsplitlab
3+
#
4+
# Runs both random and scaffold splits on the demo dataset.
5+
# Requires: scaffoldsplitlab installed (pip install -e .)
6+
# RDKit recommended but not required.
7+
set -euo pipefail
8+
9+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
10+
PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
11+
12+
INPUT="$SCRIPT_DIR/demo_input.csv"
13+
14+
echo "=============================================="
15+
echo " scaffoldsplitlab — Demo"
16+
echo "=============================================="
17+
echo ""
18+
echo "Input file: $INPUT"
19+
echo ""
20+
21+
# --------------------------------------------------
22+
# 1. Random split (no stratification)
23+
# --------------------------------------------------
24+
echo "--- Random split (80/10/10) ---"
25+
scaffold-split \
26+
--input "$INPUT" \
27+
--smiles-col smiles \
28+
--method random \
29+
--train 0.8 \
30+
--val 0.1 \
31+
--test 0.1 \
32+
--seed 42 \
33+
--out "$SCRIPT_DIR/demo_random_output.csv"
34+
35+
echo ""
36+
37+
# --------------------------------------------------
38+
# 2. Random split (stratified by label)
39+
# --------------------------------------------------
40+
echo "--- Random stratified split (80/10/10) by label ---"
41+
scaffold-split \
42+
--input "$INPUT" \
43+
--smiles-col smiles \
44+
--label-col label \
45+
--method random \
46+
--train 0.8 \
47+
--val 0.1 \
48+
--test 0.1 \
49+
--seed 42 \
50+
--out "$SCRIPT_DIR/demo_random_stratified_output.csv" \
51+
|| echo "(Stratified split skipped — dataset may be too small for stratification)"
52+
53+
echo ""
54+
55+
# --------------------------------------------------
56+
# 3. Scaffold split
57+
# --------------------------------------------------
58+
echo "--- Scaffold split (80/10/10) ---"
59+
scaffold-split \
60+
--input "$INPUT" \
61+
--smiles-col smiles \
62+
--method scaffold \
63+
--train 0.8 \
64+
--val 0.1 \
65+
--test 0.1 \
66+
--seed 42 \
67+
--out "$SCRIPT_DIR/demo_scaffold_output.csv"
68+
69+
echo ""
70+
echo "=============================================="
71+
echo " Demo complete. Output files in: $SCRIPT_DIR"
72+
echo "=============================================="
73+
ls -la "$SCRIPT_DIR"/demo_*_output.csv 2>/dev/null || echo "(No output CSV files found — scaffoldsplitlab may not be installed)"

‎pyproject.toml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
[build-system]
2+
requires = ["hatchling"]
3+
build-backend = "hatchling.build"
4+
5+
[project]
6+
name = "scaffoldsplitlab"
7+
version = "0.1.0"
8+
description = "Scaffold-based train/val/test splitting for molecular ML"
9+
requires-python = ">=3.10"
10+
dependencies = [
11+
"typer>=0.9.0",
12+
"rich>=13.0.0",
13+
"pandas>=1.5.0",
14+
"scikit-learn>=1.0.0",
15+
"numpy>=1.21.0",
16+
]
17+
18+
[project.optional-dependencies]
19+
rdkit = ["rdkit>=2022.9.0"]
20+
dev = ["pytest", "pytest-cov", "ruff"]
21+
22+
[project.scripts]
23+
scaffold-split = "scaffoldsplitlab.cli:app"
24+
25+
[tool.ruff]
26+
line-length = 100
27+
target-version = "py310"
28+
29+
[tool.ruff.lint]
30+
select = ["E", "F", "W", "I", "N", "UP"]
31+
32+
[tool.pytest.ini_options]
33+
testpaths = ["tests"]
34+
addopts = "-v --tb=short"

‎src/scaffoldsplitlab/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
"""scaffoldsplitlab - Scaffold-based train/val/test splitting for molecular ML."""
2+
3+
__version__ = "0.1.0"

0 commit comments

Comments
 (0)