Input data:
There are 5 independent paired-end fq.gz simulated with ngsngs:
ngsngs -i "$ref" -r 200 -ld Norm,350,50 -seq PE -f fq.gz -q1 R1.txt -q2 R2.txt -t 4 -o 200_${name}.
After simulation, simply cat *fq.gz > merge.fq.gz
I use this merge.fq.gz as fastp input, which should contain 1000 reads for one pair end (200 reads x 5 files).
Problem
v1.3.6 only reads 200 reads, but v1.1.0 can read 1000 reads.
suggested fix
I noticed that NGSNGS output is BGZF, and there is an EOF block at the end of each simulated file.
When these files are directly concatenated, the merged file contains multiple EOF blocks (one after each original file, plus the final one).
It seems the BGZF reader (introduced around v1.3.0?) stops reading as soon as it hits the first EOF block, so only the first 200 reads in the input data get processed.
Input data:
There are 5 independent paired-end fq.gz simulated with
ngsngs:ngsngs -i "$ref" -r 200 -ld Norm,350,50 -seq PE -f fq.gz -q1 R1.txt -q2 R2.txt -t 4 -o 200_${name}.After simulation, simply
cat *fq.gz > merge.fq.gzI use this merge.fq.gz as fastp input, which should contain 1000 reads for one pair end (200 reads x 5 files).
Problem
v1.3.6 only reads 200 reads, but v1.1.0 can read 1000 reads.
suggested fix
I noticed that NGSNGS output is BGZF, and there is an EOF block at the end of each simulated file.
When these files are directly concatenated, the merged file contains multiple EOF blocks (one after each original file, plus the final one).
It seems the BGZF reader (introduced around v1.3.0?) stops reading as soon as it hits the first EOF block, so only the first 200 reads in the input data get processed.