GitLab

As part of the effort to enable conda-based installation, it occured that samtools-1.9 was installed, despite that 0.1.19 should be installed (iris had newer version for an as-of-yet unknown reason, i.e., all.yaml specified samtools==0.1.19 and was identical between iris and litcrit).

It is speculated that this lead to the host filtering step failing on iris:

rule mt_filtering:
    input: Preprocessing/mt.r1.trimmed.rna_filtered.fq, Preprocessing/mt.r2.trimmed.rna_filtered.fq, Preprocessing/mt.se.trimmed.rna_filtered.fq,
/home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa, /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa, /home/
users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.amb, /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.ann, /hom
e/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.bwt, /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.pac, /h
ome/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.sa
    output: Preprocessing/mt.r1.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt.r2.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt
.se.trimmed.rna_filtered.hg38_filtered.fq
    jobid: 13


        TMP_FILE=$(mktemp --tmpdir=/tmp -t "alignment_XXXXXX.bam")
        BUFFER=$(mktemp --tmpdir=/tmp -t "alignment_XXXXXX.bam")
        bwa mem -v 1 -t 26 /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa Preprocessing/mt.r1.trimmed.rna_filtered.fq Prepro
cessing/mt.r2.trimmed.rna_filtered.fq | samtools view -@ 26 -bS - > $TMP_FILE
        samtools merge -@ 26 -u -         <(samtools view -@ 26 -u  -f 4 -F 264 $TMP_FILE)         <(samtools view -@ 26 -u -f 8 -F 260 $TMP_FILE)
         <(samtools view -@ 26 -u -f 12 -F 256 $TMP_FILE) |         samtools view -@ 26 -bF 0x800 -  | samtools sort -o -@ 26 -m 2G -n - $BUFFER -
 |         bamToFastq -i stdin -fq Preprocessing/mt.r1.trimmed.rna_filtered.hg38_filtered.fq -fq2 Preprocessing/mt.r2.trimmed.rna_filtered.hg38_fi
ltered.fq
        if [[ -s Preprocessing/mt.se.trimmed.rna_filtered.fq ]]
        then
        bwa mem -v 1 -t 26 /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa Preprocessing/mt.se.trimmed.rna_filtered.fq | samt
ools view -@ 26 -bS - |         samtools view -@ 26 -uf 4 - | bamToFastq -i stdin -fq Preprocessing/mt.se.trimmed.rna_filtered.hg38_filtered.fq
        else
        echo "Preprocessing/mt.se.trimmed.rna_filtered.fq is empty, skipping single end human sequence filtering, but creating it anyway..."
        touch Preprocessing/mt.se.trimmed.rna_filtered.hg38_filtered.fq
        fi
        rm -rf $BUFFER* $TMP_FILE
[...]
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[main] Version: 0.7.9a-r786
[main] CMD: bwa mem -v 1 -t 26 /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa Preprocessing/mt.r1.trimmed.rna_filtered.fq Preprocessing/mt.r2.trimmed.rna_filtered.fq
[main] Real time: 255.606 sec; CPU: 4680.023 sec
Usage: samtools sort [options...] [in.bam]
Options:
  -l INT     Set compression level, from 0 (uncompressed) to 9 (best)
  -m INT     Set maximum memory per thread; suffix K/M/G recognized [768M]
  -n         Sort by read name
  -t TAG     Sort by value of TAG. Uses position as secondary index (or read name if -n is set)
  -o FILE    Write final output to FILE rather than standard output
  -T PREFIX  Write temporary files to PREFIX.nnnn.bam
      --input-fmt-option OPT[=VAL]
               Specify a single input file format option in the form
               of OPTION or OPTION=VALUE
  -O, --output-fmt FORMAT[,OPT[=VAL]]...
               Specify output format (SAM, BAM, CRAM)
      --output-fmt-option OPT[=VAL]
               Specify a single output file format option in the form
               of OPTION or OPTION=VALUE
      --reference FILE
               Reference sequence FASTA FILE [null]
  -@, --threads INT
               Number of additional threads to use [0]
[W::bam_merge_core2] No @HD tag found.
Error in rule mt_filtering:
    jobid: 13
    output: Preprocessing/mt.r1.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt.r2.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt.se.trimmed.rna_filtered.hg38_filtered.fq

This needs further inspection.