Skip to content

Commit c96eb44

Browse files
davek44davek44
committed
Merge remote-tracking branch 'origin/main' into rename-tf
# Conflicts: # docs/transfer/transfer.md # pyproject.toml
2 parents aa6a056 + 2e72644 commit c96eb44

19 files changed

Lines changed: 1359 additions & 213 deletions

‎.github/workflows/lint.yml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,7 @@ jobs:
2020
python-version: "3.11"
2121

2222
- name: Install dependencies
23-
run: python3 -m pip install black~=23.12.1 ruff~=0.1.11
24-
25-
- name: Check Black formatting
26-
run: black --check .
23+
run: python3 -m pip install ruff~=0.1.11
2724

2825
- name: Ruff
2926
run: ruff check src tests

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,8 @@ baskerville-tf provides researchers with tools to:
1616

1717
Documentation page: https://calico.github.io/baskerville-tf/index.html
1818

19-
[Document page for transfer learning](docs/transfer/transfer.md)
19+
- [Document page for transfer learning to hg38 tracks](docs/transfer_human/transfer.md)
20+
- [Document page for transfer learning to mm10 tracks](docs/transfer_mouse/transfer_mouse.md)
2021

2122
---
2223

‎docs/hound_scripts_doc.md‎

Lines changed: 221 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,221 @@
1+
# Baskerville Hound Scripts Documentation
2+
3+
This document describes the main functionality and differences between key scripts in the Baskerville suite.
4+
5+
## Overview
6+
7+
The Baskerville package includes several "hound" scripts for making genomic predictions using trained deep learning models. Each script serves a specific purpose in the genomic prediction pipeline, from basic sequence prediction to variant effect analysis and mutagenesis studies.
8+
9+
| Script | Input Type | Analysis Focus | Output Type | Parallelization |
10+
|--------|------------|----------------|-------------|-----------------|
11+
| `hound_predbed.py` | BED regions | Sequence prediction | Per-region predictions | Single process |
12+
| `hound_predvcf.py` | VCF variants | Full window prediction | Ref/alt predictions | Single process |
13+
| `hound_snp_slurm.py` | VCF variants | Variant effects | Effect statistics | SLURM cluster |
14+
| `hound_snp.py` | VCF variants | Variant effects | Effect statistics | Single/multi-worker |
15+
| `hound_snpgene.py` | VCF + GTF | Gene-context effects | Gene-level statistics | Single/multi-worker |
16+
| `hound_ism_bed.py` | BED regions | Saturation mutagenesis | Position × nucleotide effects | Single process |
17+
| `hound_ism_snp.py` | VCF variants | Saturation mutagenesis | Position × nucleotide effects | Single process |
18+
| `hound_isd_bed.py` | BED regions + GTF | Deletion mutagenesis | Gene-context deletion effects | Single process |
19+
20+
21+
## Script Descriptions
22+
23+
### 1. `hound_predbed.py` - BED File Sequence Prediction
24+
25+
**Purpose**: Predicts sequences from genomic regions defined in a BED file.
26+
27+
**Key Features**:
28+
- Takes genomic coordinates from a BED file as input
29+
- Generates model predictions for specified genomic regions
30+
- Supports both full predictions and summed predictions across sequence length
31+
- Can generate BigWig files for genome browser visualization
32+
- Supports embedding layer extraction for sequence representations
33+
34+
**Input**:
35+
- BED file with genomic coordinates
36+
- Model parameters and weights
37+
- Genome FASTA file
38+
- Optional targets file for specific prediction tracks
39+
40+
**Output**:
41+
- HDF5 file with predictions for each BED region
42+
- Optional BigWig files for visualization
43+
- Predictions can be per-position or summed across regions
44+
45+
---
46+
47+
### 2. `hound_predvcf.py` - VCF Variant Window Prediction
48+
49+
**Purpose**: Predicts full genomic windows for variants from a VCF file, generating predictions for both reference and alternate alleles.
50+
51+
**Key Features**:
52+
- Processes SNPs from VCF files (INDELs use reference sequence for alternate allele)
53+
- Generates full-window predictions at 32bp-bin resolution
54+
- Provides separate predictions for reference and alternate alleles
55+
- Focuses on complete genomic context around variants
56+
57+
**Input**:
58+
- VCF file with genetic variants
59+
- Model parameters and weights
60+
- Genome FASTA file
61+
- Targets file (required for stranded datasets)
62+
63+
**Output**:
64+
- HDF5 file with separate reference and alternate predictions
65+
- Full prediction windows for each variant
66+
- Variant metadata (chromosome, position, ref/alt alleles)
67+
68+
---
69+
70+
### 3. `hound_snp.py` - SNP Variant Effect Prediction
71+
72+
**Purpose**: Computes variant effect predictions for SNPs in a VCF file using various statistical measures.
73+
74+
**Key Features**:
75+
- Calculates multiple variant effect statistics (logSUM, SAD, D1, D2, JS, etc.)
76+
- metric summarized along the sequence length axis
77+
- Supports SNP clustering to reduce redundant predictions
78+
- Handles both single-worker and multi-worker execution modes
79+
- Comprehensive suite of distance and information-theory based metrics
80+
81+
**Available Statistics**:
82+
- **Basic Sum-based**: SUM, logSUM, logSED, sqrtSUM
83+
- **Maximum Effect**: SAX (finds position of maximum difference)
84+
- **Distance-based**: D1/D2 (L1/L2 distances), logD1/logD2, sqrtD1/sqrtD2
85+
- **Information Theory**: JS (Jensen-Shannon divergence), logJS
86+
87+
**Input**:
88+
- VCF file with SNPs
89+
- Model parameters and weights
90+
- Genome FASTA file
91+
- Targets file
92+
93+
**Output**:
94+
- HDF5 file with variant effect scores
95+
- Statistical summaries for each SNP across all prediction targets
96+
- Configurable output statistics
97+
98+
---
99+
100+
### 4. `hound_snp_slurm.py` - Cluster-Parallelized SNP Analysis
101+
102+
**Purpose**: Orchestrates parallel computation of variant effect predictions across a SLURM cluster.
103+
104+
**Key Features**:
105+
- Distributes SNP analysis across multiple compute nodes
106+
- Manages job submission and monitoring via SLURM
107+
- Handles result aggregation from parallel workers
108+
- Optimized for large-scale variant effect studies
109+
110+
**Input**:
111+
- Same inputs as `hound_snp.py`
112+
- Additional SLURM configuration parameters
113+
- Number of parallel processes to spawn
114+
115+
**Output**:
116+
- Aggregated results from all parallel workers
117+
- Job management and monitoring files
118+
- Same final output format as `hound_snp.py`
119+
120+
---
121+
122+
### 5. `hound_snpgene.py` - Gene-Context SNP Analysis
123+
124+
**Purpose**: Computes variant effect predictions for SNPs with respect to gene exons defined in a GTF file.
125+
126+
**Key Features**:
127+
- Focuses variant analysis on gene regions (exons)
128+
- metric computed for each variant-gene pair
129+
- Integrates GTF gene annotations
130+
- Supports gene span aggregation
131+
- Gene clustering to optimize predictions
132+
- Same statistical framework as `hound_snp.py`
133+
134+
**Input**:
135+
- VCF file with SNPs
136+
- GTF file with gene annotations
137+
- Model parameters and weights
138+
- Genome FASTA file
139+
- Targets file
140+
141+
---
142+
143+
### 6. `hound_ism_bed.py` - In Silico Saturation Mutagenesis (BED)
144+
145+
**Purpose**: Performs systematic in silico saturation mutagenesis of sequences from BED file regions.
146+
147+
**Key Features**:
148+
- Mutates every position in a specified region to all 4 possible nucleotides (A, C, G, T)
149+
- Configurable mutation window around sequence center
150+
- Measures regulatory effects of all possible single nucleotide changes
151+
- Supports ensemble predictions with multiple shifts
152+
153+
**Input**:
154+
- BED file with genomic regions to mutate
155+
- Model parameters and weights
156+
- Genome FASTA file
157+
- Targets file for prediction tracks
158+
159+
**Output**:
160+
- HDF5 file with mutation effect scores
161+
- 4D array: [sequences × positions × nucleotides × targets]
162+
- Identifies critical regulatory positions and nucleotides
163+
164+
---
165+
166+
### 7. `hound_ism_snp.py` - In Silico Saturation Mutagenesis (VCF)
167+
168+
**Purpose**: Performs systematic in silico saturation mutagenesis around variants from VCF files.
169+
170+
**Key Features**:
171+
- Centers mutagenesis around known variants from VCF
172+
- Systematically tests all nucleotide changes in variant neighborhoods
173+
- Default 200bp mutation window around variants
174+
- Same mutagenesis approach as `hound_ism_bed.py` but variant-focused
175+
176+
**Input**:
177+
- VCF file with variants of interest
178+
- Model parameters and weights
179+
- Genome FASTA file
180+
- Targets file for prediction tracks
181+
182+
**Output**:
183+
- HDF5 file with mutation effect scores
184+
- 4D array: [variants × positions × nucleotides × targets]
185+
- Variant metadata and sequence labels
186+
187+
---
188+
189+
### 8. `hound_isd_bed.py` - In Silico Deletion Mutagenesis (Gene-Context)
190+
191+
**Purpose**: Performs systematic in silico deletion mutagenesis of sequences with gene annotation context.
192+
193+
**Key Features**:
194+
- Systematically deletes nucleotides (default 1bp) across BED regions
195+
- Integrates gene annotations to focus on gene-relevant regions
196+
- Supports both exon-only and gene span analysis
197+
- Uses stitching to handle deletion compensation shifts
198+
- Gene-aware analysis with strand-specific effects
199+
200+
**Input**:
201+
- BED file with genomic regions
202+
- GTF file with gene annotations
203+
- Model parameters and weights
204+
- Genome FASTA file
205+
- Targets file and optional target genes list
206+
207+
**Output**:
208+
- Separate HDF5 files for each BED entry
209+
- Deletion effect scores focused on gene regions
210+
- SED/logSED statistics for measuring regulatory impact
211+
212+
## Choosing the Right Script
213+
214+
- **Use `hound_predbed.py`** when you want predictions for specific genomic regions
215+
- **Use `hound_predvcf.py`** when you need full prediction windows around variants
216+
- **Use `hound_snp.py`** for variant effect analysis with statistical summaries
217+
- **Use `hound_snp_slurm.py`** for large-scale variant studies requiring cluster computing
218+
- **Use `hound_snpgene.py`** when focusing on variant effects within gene contexts
219+
- **Use `hound_ism_bed.py`** for comprehensive mutagenesis analysis of regulatory regions
220+
- **Use `hound_ism_snp.py`** for detailed mutagenesis analysis around specific variants
221+
- **Use `hound_isd_bed.py`** for deletion-based mutagenesis with gene context

0 commit comments

Comments
 (0)