IMP issueshttps://git-r3lab.uni.lu/IMP/IMP/-/issues2018-10-08T16:28:24+02:00https://git-r3lab.uni.lu/IMP/IMP/-/issues/137samtools-1.9 might break host filtering step2018-10-08T16:28:24+02:00Cedric Lacznysamtools-1.9 might break host filtering stepAs part of the effort to enable conda-based installation, it occured that `samtools-1.9` was installed, despite that `0.1.19` **should** be installed (iris had newer version for an as-of-yet unknown reason, i.e., `all.yaml` specified `sa...As part of the effort to enable conda-based installation, it occured that `samtools-1.9` was installed, despite that `0.1.19` **should** be installed (iris had newer version for an as-of-yet unknown reason, i.e., `all.yaml` specified `samtools==0.1.19` and was **identical** between iris and litcrit).
It is speculated that this lead to the host filtering step failing on iris:
```
rule mt_filtering:
input: Preprocessing/mt.r1.trimmed.rna_filtered.fq, Preprocessing/mt.r2.trimmed.rna_filtered.fq, Preprocessing/mt.se.trimmed.rna_filtered.fq,
/home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa, /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa, /home/
users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.amb, /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.ann, /hom
e/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.bwt, /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.pac, /h
ome/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.sa
output: Preprocessing/mt.r1.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt.r2.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt
.se.trimmed.rna_filtered.hg38_filtered.fq
jobid: 13
TMP_FILE=$(mktemp --tmpdir=/tmp -t "alignment_XXXXXX.bam")
BUFFER=$(mktemp --tmpdir=/tmp -t "alignment_XXXXXX.bam")
bwa mem -v 1 -t 26 /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa Preprocessing/mt.r1.trimmed.rna_filtered.fq Prepro
cessing/mt.r2.trimmed.rna_filtered.fq | samtools view -@ 26 -bS - > $TMP_FILE
samtools merge -@ 26 -u - <(samtools view -@ 26 -u -f 4 -F 264 $TMP_FILE) <(samtools view -@ 26 -u -f 8 -F 260 $TMP_FILE)
<(samtools view -@ 26 -u -f 12 -F 256 $TMP_FILE) | samtools view -@ 26 -bF 0x800 - | samtools sort -o -@ 26 -m 2G -n - $BUFFER -
| bamToFastq -i stdin -fq Preprocessing/mt.r1.trimmed.rna_filtered.hg38_filtered.fq -fq2 Preprocessing/mt.r2.trimmed.rna_filtered.hg38_fi
ltered.fq
if [[ -s Preprocessing/mt.se.trimmed.rna_filtered.fq ]]
then
bwa mem -v 1 -t 26 /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa Preprocessing/mt.se.trimmed.rna_filtered.fq | samt
ools view -@ 26 -bS - | samtools view -@ 26 -uf 4 - | bamToFastq -i stdin -fq Preprocessing/mt.se.trimmed.rna_filtered.hg38_filtered.fq
else
echo "Preprocessing/mt.se.trimmed.rna_filtered.fq is empty, skipping single end human sequence filtering, but creating it anyway..."
touch Preprocessing/mt.se.trimmed.rna_filtered.hg38_filtered.fq
fi
rm -rf $BUFFER* $TMP_FILE
[...]
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[main] Version: 0.7.9a-r786
[main] CMD: bwa mem -v 1 -t 26 /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa Preprocessing/mt.r1.trimmed.rna_filtered.fq Preprocessing/mt.r2.trimmed.rna_filtered.fq
[main] Real time: 255.606 sec; CPU: 4680.023 sec
Usage: samtools sort [options...] [in.bam]
Options:
-l INT Set compression level, from 0 (uncompressed) to 9 (best)
-m INT Set maximum memory per thread; suffix K/M/G recognized [768M]
-n Sort by read name
-t TAG Sort by value of TAG. Uses position as secondary index (or read name if -n is set)
-o FILE Write final output to FILE rather than standard output
-T PREFIX Write temporary files to PREFIX.nnnn.bam
--input-fmt-option OPT[=VAL]
Specify a single input file format option in the form
of OPTION or OPTION=VALUE
-O, --output-fmt FORMAT[,OPT[=VAL]]...
Specify output format (SAM, BAM, CRAM)
--output-fmt-option OPT[=VAL]
Specify a single output file format option in the form
of OPTION or OPTION=VALUE
--reference FILE
Reference sequence FASTA FILE [null]
-@, --threads INT
Number of additional threads to use [0]
[W::bam_merge_core2] No @HD tag found.
Error in rule mt_filtering:
jobid: 13
output: Preprocessing/mt.r1.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt.r2.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt.se.trimmed.rna_filtered.hg38_filtered.fq
```
This needs further inspection.https://git-r3lab.uni.lu/IMP/IMP/-/issues/126`metaquast` rule: ERROR! Failed downloading SILVA rRNA gene database2018-10-19T15:42:47+02:00Fredrik Boulund`metaquast` rule: ERROR! Failed downloading SILVA rRNA gene databaseHi, just encountered this message in my output from the `metaquast` rule. It seems everything continues anyway, but I couldn't find any other mention of this warning message in any other issues so far, so figured I'd report it anyway.
`...Hi, just encountered this message in my output from the `metaquast` rule. It seems everything continues anyway, but I couldn't find any other mention of this warning message in any other issues so far, so figured I'd report it anyway.
```
Downloading SILVA ribosomal RNA gene database...
ERROR! Failed downloading SILVA rRNA gene database (http://www.arb-silva.de/fileadmin/silva_databases/release_119/Exports/SILVA_119_SSURef_Nr9_tax_silva.fasta.gz)! The search for reference genomes cannot be performed. Try to download it manually in /home/imp/lib/quast-release_3.1/lbs/blast/16S_RNA_blastdb and restart MetaQUAST.
NOTICE: No references are provided, starting regular QUAST with MetaGeneMark gene finder
```Deepti MittalDeepti Mittalhttps://git-r3lab.uni.lu/IMP/IMP/-/issues/105megahit iterations: 99-25=74 modulo 4 is 2 and not 02017-10-04T12:29:15+02:00Patrick Maymegahit iterations: 99-25=74 modulo 4 is 2 and not 0you are producing with megahit kmer assemblies from 25 to 97 with increment 4 and additional the 99 kmer assembly, which is an artefact by the megahit tool.
At some point you should change this, also in the documentation, it is misleadin...you are producing with megahit kmer assemblies from 25 to 97 with increment 4 and additional the 99 kmer assembly, which is an artefact by the megahit tool.
At some point you should change this, also in the documentation, it is misleading because 99-25=74 modulo 4 is 2 and not 0https://git-r3lab.uni.lu/IMP/IMP/-/issues/13Handling multiple fastq files2017-12-19T09:20:54+01:00Shaman NarayanasamyHandling multiple fastq filesSometimes `fastq` files may not exist as a single pair of R1 and R2 files. They may exist as multiple pairs of fastq pairs. This is a potential request when we eventually release IMP to the public.
This may require a major refactor.
Sometimes `fastq` files may not exist as a single pair of R1 and R2 files. They may exist as multiple pairs of fastq pairs. This is a potential request when we eventually release IMP to the public.
This may require a major refactor.