Due to the large files and vast amount of computational data required for this kind of data processing, nearly everything is run on a local supercomputer cluster through portable batch scripts (pbs). To make things easier I consolidated the necessary .pbs scripts to be run with through one shell script. When setting up the user simply has to specify the location of the directory that holds the folders for each fastq, and the location of where ever they're downloading the rest of the pbs.
Running the script will prompt the user to define a population (no file extensions), and then which script the would like to run on said population (again with no file extensions). For instance if they are mapping the fastq files for population H-2001 they would specify
H-2001
and then
map
Running the map.pbs script will also prompt which read (1 or 2) the user would like to map.
Program necessary for generating the transposon hierarchy file as PoPoolationTE2 requires this file format but does not generate it. The process is done in several phases and requires two .csv files generated by the user. First an annotation of transposable elements should be saved in .csv files (annotation.csv).
A smaller reference file (reference.csv) should be generated by taking the annotation file and trimming it down to unique names for each transposon. Then in a second column the user must manually input the family name for each transposon. Finally the third column must manually be filled in again with the order of the transposon. For example an annotation file may look like this
| Annotation |
|---|
| REX1_1_GA::chrI:536673-536882() |
| Helitron_1_GA::chrI:534938-534978() |
| L2_1_GA::chrI:534773-534936() |
| L2_1_GA::chrI:534633-534768() |
Removing extraneous information such as the chromosome, loci, and extra numbers it looks like
| Annotation |
|---|
| REX1 |
| Helitron |
| L2 |
| L2 |
We can get rid of duplicates and manually add in family and order information through simple literature search
| id | family | order |
|---|---|---|
| REX1 | REX | LINE |
| Helitron | helitron | DNA |
| L2 | L2 | LINE |
This then provides a template for the program. It will look for the id in the reference.csv within the name of each annotation in annotation.csv. Then, once found, it will add the respective family and order with tab delineation like PoPoolationTE2 requires for its hierarchy file.
Output: te.hier.txt