Skip to content

gnomAD Joint Freq/annotations into MT - #1166

Open
MattWellie wants to merge 20 commits into
mainfrom
gnomAD_4_into_MT
Open

MattWellie wants to merge 20 commits into
mainfrom
gnomAD_4_into_MT

Conversation

@MattWellie

Copy link
Copy Markdown
Contributor

Adds the gnomAD joint data in during the AnnotateCohort Stage (of the combiner pipeline). Keen to grab some input on the exact fields we want to pull over here.

I've left a lengthy docstring to explain the process used to find and display these data. I couldn't find a README on how to use the gnomAD HT, but I was remembering a conversation with Ben B in the Seqr team about how they stored keys in the globals dict and used that as an index to unpack the values in each row as a compression technique.

Here I'm pulling out the adj (quality-adjusted calls across all popuations) as the per-allele AC/AN/AF/Hom values, then using the max-population FAF and AF into separate fields to match the ones we're already extracting from exac/gnomad2/3

I'm also trying to pull out 'Hemi', which should be missing except for X/Y chrom variants. I might need to put some flexibility here for 'Hemi if defined, else missing'.

@MattWellie
MattWellie requested review from EddieLF and cassimons March 24, 2025 06:41
@codecov-commenter

codecov-commenter commented Mar 24, 2025

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 26.46%. Comparing base (c4c29fb) to head (ac9d41f).
⚠️ Report is 128 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1166   +/-   ##
=======================================
  Coverage   26.46%   26.46%           
=======================================
  Files           9        9           
  Lines        1757     1757           
=======================================
  Hits          465      465           
  Misses       1292     1292           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MattWellie
MattWellie marked this pull request as ready for review March 25, 2025 05:27
@MattWellie

Copy link
Copy Markdown
Contributor Author

Cool, I think this is now good for a review - the gnomad4 frequencies are accurate (spot-checked against the gnomad website), and the latest change replaces the existing mt.gnomad_genomes with the V4 data. We still have mt.gnomad_exomes (V2, from the seqr reference data), and mt.gnomad_genome_coverage (also from the reference data... the gnomad V4 table doesn't have this attribute, and the V4 release only seems to have a coverage table for Exomes?)

@MattWellie

Copy link
Copy Markdown
Contributor Author

This also deletes a test which was submitting 100+ python jobs in a chain to try and hit a niche recursion error. That's probably not something we need a test case for, and it was the only breaking test

@jmarshall

Copy link
Copy Markdown
Contributor

The weird test problem is already addressed by #1240.

# Conflicts:
#	test/test_size.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants