Mech-BBB-lite

Does auxiliary supervision on P-glycoprotein substrate status actually help predict blood-brain barrier permeability, once you control for the descriptor features that usually get bundled in alongside it? A controlled multi-task learning study.

Table of Contents

• Overview
• Key Finding
• Results
• Data
• Quick Start
• Repository Structure
• Limitations & Future Work
• Contributing
• Citation
• License
• Acknowledgments

Overview

The blood-brain barrier (BBB) is the central bottleneck in CNS drug development; most promising drug candidates never make it into the brain in sufficient concentration. BBB permeability is really two distinct biological mechanisms: passive diffusion across the membrane, and active efflux by transporters like P-glycoprotein (P-gp). This project tests whether giving a model explicit auxiliary supervision on the active transport mechanism, via multi-task learning on BBB and P-gp labels jointly, improves BBB permeability prediction, and whether that effect survives a controlled comparison against simply adding standard physicochemical descriptor features.

Key Finding

An initial comparison suggested multi-task training with P-gp labels improved BBB AUROC by +0.009 to +0.019. A controlled follow-up experiment showed this was confounded: the multi-task run also included RDKit descriptor features that the original single-task baseline lacked. Holding descriptor features constant across both models, the P-gp auxiliary task provided no statistically significant additional benefit on either split, and trended slightly negative on the harder scaffold split. The descriptor features alone, independent of any auxiliary task, produced a real and statistically solid improvement (+0.034 AUROC on the scaffold split, 95% CI [0.005, 0.062]). In this setting, standard physicochemical descriptors appear to already carry most of the mechanistically-relevant signal that the P-gp task was intended to supply.

Results

Point estimates (test AUROC):

Model	Scaffold split	Random split
Chemprop single-task, no extra features	0.747	0.954
Chemprop single-task + RDKit descriptors	0.782	0.961
Chemprop multi-task (BBB + P-gp) + RDKit descriptors	0.766	0.962
RF + MACCS (validation-tuned threshold)	0.724	0.965

Paired bootstrap comparisons (2,000 resamples):

Comparison	Split	Delta AUROC	95% CI	Fraction favoring the second model
descriptors vs. no descriptors (single-task)	scaffold	+0.034	(0.005, 0.062)	0.990
multi-task vs. single-task+descriptors (P-gp task's own effect)	scaffold	-0.016	(-0.032, 0.000)	0.027
descriptors vs. no descriptors (single-task)	random	+0.007	(-0.001, 0.015)	0.956
multi-task vs. single-task+descriptors (P-gp task's own effect)	random	+0.002	(-0.003, 0.006)	0.754

The descriptor effect is real and robust on both splits (first and third rows). The P-gp auxiliary task's own marginal contribution, once descriptors are held equal, is not statistically significant on either split (second and fourth rows).

Data

• B3DB (Meng et al., 2021): BBB classification and logBB regression labels
• TDC BBB_Martins (Martins et al., 2012): BBB classification labels
• TDC Pgp_Broccatelli (Broccatelli et al., 2011): P-gp substrate labels

Merged and deduplicated by canonical InChIKey into 9,098 unique compounds. Evaluated on two splits:

• Scaffold split (Bemis-Murcko): harder, more realistic test of generalization to novel chemotypes
• Stratified random split: avoids the degenerate one-class test folds that naive scaffold splitting can produce on imbalanced BBB data, at the cost of overestimating real-world generalization due to chemical-similarity leakage

Both splits are confirmed leak-free (baselines/leak_check.py, zero exact-SMILES overlap across train/val/test on both).

Quick Start

git clone https://github.com/VoidEchoes42/mech-bbb-lite.git
cd mech-bbb-lite
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
pip install chemprop==1.6.1 descriptastorus   # version-pinned, v2 CLI differs

python data/download_data.py       # needs internet
python data/build_dataset.py
python baselines/leak_check.py
python baselines/train_rf_maccs.py
bash experiments/run_chemprop.sh
python experiments/bootstrap_compare.py
python experiments/bootstrap_compare_confound.py   # see script docstring for the two extra control runs it needs

Repository Structure

data/
  download_data.py     # pulls B3DB + TDC sources
  build_dataset.py       # dedup, canonicalize, generate both splits
baselines/
  train_rf_maccs.py       # RF + fingerprint baseline, validation-tuned threshold
  leak_check.py             # confirms zero train/val/test overlap
experiments/
  run_chemprop.sh                   # single-task and multi-task Chemprop commands
  bootstrap_compare.py                # original multi-task vs single-task comparison
  bootstrap_compare_confound.py         # controlled comparison isolating the P-gp task's
                                           # effect from the descriptor-feature effect
tests/                                    # unit tests for split/canonicalization logic

Limitations & Future Work

• The improvement first attributed to multi-task learning was substantially or entirely attributable to RDKit descriptor features rather than the P-gp auxiliary task itself; this was only caught via a dedicated controlled follow-up experiment. Future auxiliary-task studies on this problem should hold input features fixed across all conditions from the start.
• P-gp is only one active-transport mechanism; BCRP and influx transporters are not modeled.
• Deduplication across sources uses last-write-wins on conflicting labels.
• Undefined stereochemistry in source SMILES collapses some stereoisomer pairs during InChIKey-based deduplication.
• No 3D conformer representation, contrastive pretraining, or conformal calibration. A natural extension is a dual-branch architecture that explicitly separates passive-permeability and active-transport representations at the architecture level, rather than relying on a shared-encoder multi-task loss, which this result suggests may not be sufficient on its own.
• Chemprop was run with num_folds=1 per configuration; the per-run standard deviation it reports is not a measure of result stability. All significance claims here come from the bootstrap analyses, not from Chemprop's own reported variance.

Contributing

Issues and pull requests are welcome, particularly around testing whether an architecturally-separated (rather than shared-encoder) mechanism-decomposition approach recovers a benefit from P-gp supervision that this simpler multi-task setup did not find.

Citation

A formal citation will be added if/when this work is published. In the meantime, feel free to cite the repository directly:

@misc{mechbbblite2026,
  title  = {Mech-BBB-lite: A Controlled Test of Multi-Task P-glycoprotein Supervision for Blood-Brain Barrier Permeability Prediction},
  author = {Aditya, Aman},
  year   = {2026},
  url    = {https://github.com/VoidEchoes42/mech-bbb-lite}
}

License

MIT. See LICENSE.

Acknowledgments

Built on top of publicly released data from the B3DB and Therapeutics Data Commons projects, and the Chemprop library.