Linked sequences

[colinvwood]

Description

Allow denoise-paired to rescue and return unmerged sequences

AI Disclosure

• NO AI USED.
• AI USED.

AI Usage Details

Co-written with Codex

[colinvwood]

needs qiime2/q2-types#388

[colinvwood]

@ebolyen

[colinvwood]

disclaimer: the run_dada.R diff is 99% codex and the test_denoise.py diff is about 50/50 codex/de moi

[colinvwood]

One outstanding decision is how we want to restructure the stats table to accommodate these changes.

[Oddant1]

@colinvwood's comments came from the review. Needed to look it over with him to make sure I knew what the R was doing.

[colinvwood]

related to #129, #93, #189, qiime2/q2-feature-classifier#219, qiime2/q2-feature-classifier#224

[colinvwood]

Needs rachis-org/rachis-cli#401

[ebolyen]

Some grouped suggestions.

[nbokulich]

hey drive-by comment: I am testing this with some real data locally to see how it compares to single and paired, looking great! I like how the output stats report the number of concatenated seqs that are rescued, this is really useful. Could we also have a sum total of merged non-chimeric + concatenated, for simplicity we could just call this column "output" (and an additional column of output / input). Very cool to finally have this functionality in q2-dada2, thanks @colinvwood !

[ebolyen]

I think this looks good, but I did get lost in one spot.

[ebolyen]

I am 99% sure this will still work even with the new LinkedDNA class.

[colinvwood]

You mean the __init__ signature?

[ebolyen]

I am quite confused how this would be possible. Can you explain a little more in a comment?

[colinvwood]

Say we have two linked sequences like so:

>seq1
ACN TG
>seq2
AC NTG

The the unmerged.id.map becomes (imagine we're using a single N separator instead of ten):

ACNNTG -> ACN TG
ACNNTG -> AC NTG

Now it's ambiguous what ACNNTG maps to.

[colinvwood]

Needs qiime2/q2-types#397. (Need to merge and use those changes).

[colinvwood]

hey drive-by comment: I am testing this with some real data locally to see how it compares to single and paired, looking great! I like how the output stats report the number of concatenated seqs that are rescued, this is really useful. Could we also have a sum total of merged non-chimeric + concatenated, for simplicity we could just call this column "output" (and an additional column of output / input). Very cool to finally have this functionality in q2-dada2, thanks @colinvwood !

Thank you for the feedback, I agree the stats table needs polishing. Currently, your "output" column is actually just the "non-chimeric" column. (This is confusing and needs to change.)

Do you think it's useful to see chimera filtering stats stratified by merged/concatenated? If not we could just drop the last two columns (non-chimeric concatenated & its %). That would leave us with only two new columns compared to the pre-concatenation changes.

If we do want to see the stratified output we need to add a merged-only & its % columns for chimera filtering. This would leave us with six new columns compared to the pre-concatenation version, which feels like it's starting to make the stats table pretty dense. But I can also see the extra stats being useful.

What do you all think @ebolyen, @cherman2?

[nbokulich]

Do you think it's useful to see chimera filtering stats stratified by merged/concatenated? If not we could just drop the last two columns (non-chimeric concatenated & its %).

I don't think we need chimera filtering stats on the concatenated and merged reads separately... could just lump these together into one. I agree, otherwise this table becomes very dense.