Run time problem?

[stevechen123]

Hi. I'm a novice to using Enrich2. I'm trying to count variants (using "count-only") directly by using a large number of reads (close to 4M forward reads and another 4M reverse reads). The WT sequence is also quite long.

In the terminal, it shows that I have been counting variants overnight. Is there an approximate amount of time that is required for each run? I'm just curious as to whether I'm doing something wrong or its simply taking a long time because I'm trying to run something quite complicated.

{
"fastq": {
"filters": {
"min quality": 20,
"remove unresolvable": true
},
"forward reads": "/media/Storage5/PHong/NGS/F300006022_L01_71_1_paired.fastq",
"reverse reads": "/media/Storage5/PHong/NGS/F300006022_L01_71_2_paired.fastq"
},
"name": "2021-09-27 71",
"output directory": "/media/Storage5/JingL/NGS/Enrich2_output",
"overlap": {
"forward start": 15,
"length": 120,
"max mismatches": 3,
"reverse start": 15,
"trim": true
},
"report filtered reads": false,
"timepoint": 0,
"variants": {
"use aligner": false,
"wild type": {
"coding": true,
"reference offset": 0,
"sequence": "ATGAATGGCACAGAAGGCCCTAACTTCTACGTGCCCTTCTCCAATGCGACGGGTGTGGTACGCAGCCCCTTCGAGTACCCACAGTACTACCTGGCTGAGCCATGGCAGTTCTCCATGCTGGCCGCCTACATGTTTCTGCTGATCGTGCTGGGCTTCCCCATCAACTTCCTCACGCTCTACGTCACCGTCCAGCACAAGAAGCTGCGCACGCCTCTCAACTACATCCTGCTCAACCTAGCCGTGGCTGACCTCTTCATGGTCCTAGGTGGCTTCACCAGCACCCTCTACACCTCTCTGCATGGATACTTCGTCTTCGGGCCCACAGGATGCAATTTGGAGGGCTTCTTTGCCACCCTGGGCGGTGAAATTGCCCTGTGGTCCTTGGTGGTCCTGGCCATCGAGCGGTACGTGGTGGTGTGTAAGCCCATGAGCAACTTCCGCTTCGGGGAGAACCATGCCATCATGGGCGTTGCCTTCACCTGGGTCATGGCGCTGGCCTGCGCCGCACCCCCACTCGCCGGCTGGTCCAGGTACATCCCCGAGGGCCTGCAGTGCTCGTGTGGAATCGACTACTACACGCTCAAGCCGGAGGTCAACAACGAGTCTTTTGTCATCTACATGTTCGTGGTCCACTTCACCATCCCCATGATTATCATCTTTTTCTGCTATGGGCAGCTCGTCTTCACCGTCAAGGAGGCCGCTGCCCAGCAGCAGGAGTCAGCCACCACACAGAAGGCAGAGAAGGAGGTCACCCGCATGGTCATCATCATGGTCATCGCTTTCCTGATCTGCTGGGTGCCCTACGCCAGCGTGGCATTCTACATCTTCACCCACCAGGGCTCCAACTTCGGTCCCATCTTCATGACCATCCCAGCGTTCTTTGCCAAGAGCGCCGCCATCTACAACCCTGTCATCTATATCATGATGAACAAGCAGTTCCGGAACTGCATGCTCACCACCATCTGCTGCGGCAAGAACCCACTGGGTGACGATGAGGCCTCTGCTACCGTGTCCAAGACGGAGACGAGCCAGGTGGCCCCGGCCGGTGGGGGTGGTTCCGGGGGAGGCGGAGTTTCTAAAGGTGAAGCAGTTATTAAGGAATTCATGAGATTCAAGGTACACATGGAAGGTTCTATGAATGGTCACGAATTTGAAATTGAAGGTGAAGGTGAAGGTAGACCATATGAAGGTACTCAAACTGCTAAGTTGAAGGTTACTAAAGGTGGTCCATTGCCATTTTCTTGGGATATTTTGTCTCCACAATTCATGTACGGTTCTAGAGCTTTTACAAAACATCCAGCAGATATTCCAGATTACTACAAGCAATCATTCCCAGAAGGTTTTAAATGGGAAAGAGTTATGAACTTCGAAGATGGTGGTGCAGTTACTGTTACACAAGATACTTCTTTGGAAGATGGTACATTGATCTATAAGGTTAAGTTGAGAGGTACTAATTTTCCACCAGATGGTCCAGTTATGCAAAAGAAAACTATGGGTTGGGAAGCTTCAACAGAAAGATTGTACCCAGAAGATGGTGTTTTGAAGGGTGACATTAAGATGGCATTGAGATTGAAGGATGGTGGTAGATATTTGGCTGATTTCAAGACTACATACAAGGCTAAGAAACCAGTTCAAATGCCAGGTGCTTACAACGTTGATAGAAAGTTGGATATTACTTCTCATAATGAAGATTACACAGTTGTTGAACAATATGAAAGAAGTGAAGGTAGACACAGTACAGGTGGTATGGATGAATTATACAAATGA"
}
}
}

[afrubin]

Thanks for using Enrich2!

The read counting step can be quite time consuming, and in my experience is limited by how fast your computer can read the files. One counter-intuitive consequence of this is that a laptop with an SSD can sometimes complete an analysis faster than a shared HPC environment.

To make sure that everything is configured correctly, I recommend doing a trial run with just the first 100 read pairs or so. On a Unix-based system you can accomplish this with head -n400 or similar.

You might also see a speedup if you compress the FASTQ files. Enrich2 supports gzip and bzip2, and will automatically decompress these files on the fly (if they have the right suffixes).