fastqc -java.lang.OutOfMemoryError: Java heap space - Solved!

$fastqc filename.fastq
Started analysis of filename.fastq
Exception in thread "Thread-1" java.lang.OutOfMemoryError: Java heap space
    at uk.ac.babraham.FastQC.Modules.GCModel.GCModel.<init>(GCModel.java:69)
    at uk.ac.babraham.FastQC.Modules.PerSequenceGCContent.processSequence(PerSequenceGCContent.java:198)
    at uk.ac.babraham.FastQC.Analysis.AnalysisRunner.run(AnalysisRunner.java:88)
    at java.base/java.lang.Thread.run(Thread.java:844)


$ grep -n 'Xm' /home/prakki/sw/FastQC/fastqc
167:    unshift @java_args,"-Xmx${memory}m";
170:    unshift @java_args,'-Xmx250m';


$ vi /home/prakki/sw/FastQC/fastqc


Changed the 250m to 10g (depends on the RAM of your computer) in the line 170

$ fastqc filename.fastq
Started analysis of filename.fastq
Approx 5% complete for filename.fastq
Approx 10% complete for filename.fastq
Approx 15% complete for filename.fastq
Approx 20% complete for filename.fastq
Approx 25% complete for filename.fastq
Approx 30% complete for filename.fastq
Approx 35% complete for filename.fastq
Approx 40% complete for filename.fastq
Approx 45% complete for filename.fastq
Approx 50% complete for filename.fastq
Approx 55% complete for filename.fastq
Approx 60% complete for filename.fastq
Approx 65% complete for filename.fastq
Approx 70% complete for filename.fastq
Approx 75% complete for filename.fastq
Approx 80% complete for filename.fastq
Approx 85% complete for filename.fastq
Approx 90% complete for filename.fastq
Approx 95% complete for filename.fastq
Analysis complete for filename.fastq

Reverse complement fastq to fastq/a

When dealing with long insert mate pairs, some tools like Reapr might need the reads to be reverse complemented fastq in order to run the tool. So, I started searching for some softwares which can actually do what I wanted that.

One of the tool, I most of the times used was fastx_toolkit. It has a utility called fastx_reverse_complement by which I could convert all fastq to rc fastq files.

FASTQ to REV-COMP FASTQ:

$ fastx_reverse_complement -i sample.fastq -o rc_sample_fastx_tk.fastq

FASTQ to REV-COMP FASTA:

Sometimes, we might have to reverse complement fastq and need to convert to fasta file. This could be achieved by running revseq tool in EMBOSS with the following command:

$ revseq -sequence sample.fastq -outseq rc_sample.fasta -notag

* If there is a following error

fastx_reverse_complement: Invalid quality score value (char '0' ord 48 quality value -16) on line 4

Try pasting "-Q33" (without quotes) after fastx_reverse_complement.

fastest way to count reads in fastq file - use parallel

Unzipped fastq files with 16 cores, took very minute time to count number of reads.

$ time parallel "echo {} && cat {} | wc -l | awk '{d=\$1; print d/4;}'" ::: 500bp_insert*.fastq

500bp_insert_lane1_2_read2PE_single_appended.Genome.completely.cleaned.fastq_ORPHAN.fastq
408600
500bp_insert_lane1_2_read1PE_single_appended.Genome.completely.cleaned.fastq_ORPHAN.fastq
2668085
500bp_insert_lane1_2_read2PE_single_appended.Genome.completely.cleaned.fastq_matched.fastq
148713855
500bp_insert_lane1_2_read1PE_single_appended.Genome.completely.cleaned.fastq_matched.fastq
148713855

real    0m19.850s
user    0m13.144s
sys     0m56.604s

But the zipped files of size around ~ 23 Gb took a bit of around ~5 min.

$time parallel "echo {} && gunzip -c {} | wc -l | awk '{d=\$1; print d/4;}'" ::: 500bp_insert_lane*_read1.fastq.gz

500bp_insert_lane1_read1.fastq.gz
75640549

500bp_insert_lane2_read1.fastq.gz
76849862

real    2m40.358s
user    5m6.051s
sys     0m24.710s

$ time parallel "echo {} && gunzip -c {} | wc -l | awk '{d=\$1; print d/4;}'" ::: 500bp_insert_lane*_read2.fastq.gz

500bp_insert_lane2_read2.fastq.gz
76849862
500bp_insert_lane1_read2.fastq.gz
75640549

real    2m49.668s
user    5m17.894s
sys     0m28.807s