IMP issueshttps://git-r3lab.uni.lu/IMP/IMP/-/issues2017-05-31T09:51:20+02:00https://git-r3lab.uni.lu/IMP/IMP/-/issues/81Initialization problems2017-05-31T09:51:20+02:00Laurent Heirendtlaurent.heirendt@uni.luInitialization problems@yjarosz and @shaman.narayanasamy ,
I have the following issues when running `impy init` on `Ubuntu 16.04`:
- Download of `hg38.tgz` is constantly stuck. Would be nice to have the code recognize that there is already the file `hg38.tg...@yjarosz and @shaman.narayanasamy ,
I have the following issues when running `impy init` on `Ubuntu 16.04`:
- Download of `hg38.tgz` is constantly stuck. Would be nice to have the code recognize that there is already the file `hg38.tgz` present when downloaded separately.
- Once the download passed, following traceback error appeared:
```sh
(envImpy) ➜ imp-db impy init [15:32:23]
[x] Found IMP docker-r3lab.uni.lu/imp/imp 1.4
[x] Downloading IMP databases 'https://webdav-r3lab.uni.lu/public/R3lab/IMP/db/hg38.tgz' to '/home/laurent/imp-db/db.tgz'
100.0% 6333071005 / 6333000000
[x] Extracting to /home/laurent/imp-db/imp-db
[x] Removing tmp file: /home/laurent/imp-db/db.tgz
Traceback (most recent call last):
File "/home/laurent/.linuxbrew/bin/impy", line 11, in <module>
sys.exit(cli())
File "/home/laurent/.local/lib/python3.5/site-packages/click/core.py", line 716, in __call__
return self.main(*args, **kwargs)
File "/home/laurent/.local/lib/python3.5/site-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/home/laurent/.local/lib/python3.5/site-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/laurent/.local/lib/python3.5/site-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/laurent/.local/lib/python3.5/site-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/home/laurent/.local/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/laurent/.linuxbrew/opt/python3/lib/python3.5/site-packages/impy.py", line 384, in init
shutil.remove(tmp)
```Yohan Jaroszyohan.jarosz@uni.luYohan Jaroszyohan.jarosz@uni.luhttps://git-r3lab.uni.lu/IMP/IMP/-/issues/105megahit iterations: 99-25=74 modulo 4 is 2 and not 02017-10-04T12:29:15+02:00Patrick Maymegahit iterations: 99-25=74 modulo 4 is 2 and not 0you are producing with megahit kmer assemblies from 25 to 97 with increment 4 and additional the 99 kmer assembly, which is an artefact by the megahit tool.
At some point you should change this, also in the documentation, it is misleadin...you are producing with megahit kmer assemblies from 25 to 97 with increment 4 and additional the 99 kmer assembly, which is an artefact by the megahit tool.
At some point you should change this, also in the documentation, it is misleading because 99-25=74 modulo 4 is 2 and not 0https://git-r3lab.uni.lu/IMP/IMP/-/issues/13Handling multiple fastq files2017-12-19T09:20:54+01:00Shaman NarayanasamyHandling multiple fastq filesSometimes `fastq` files may not exist as a single pair of R1 and R2 files. They may exist as multiple pairs of fastq pairs. This is a potential request when we eventually release IMP to the public.
This may require a major refactor.
Sometimes `fastq` files may not exist as a single pair of R1 and R2 files. They may exist as multiple pairs of fastq pairs. This is a potential request when we eventually release IMP to the public.
This may require a major refactor.
https://git-r3lab.uni.lu/IMP/IMP/-/issues/24Prokka 1.112018-02-09T09:10:34+01:00Shaman NarayanasamyProkka 1.11https://git-r3lab.uni.lu/IMP/IMP/-/issues/60IMP execution outside of code directory2018-02-09T09:10:34+01:00Shaman NarayanasamyIMP execution outside of code directoryIMP is currently not executable outside of the directory containing the code. Or I am doing it the wrong way. This is an issue because it is not efficient. At present, everyone running IMP has their own copy or the repository, the databa...IMP is currently not executable outside of the directory containing the code. Or I am doing it the wrong way. This is an issue because it is not efficient. At present, everyone running IMP has their own copy or the repository, the databases etc. Is it possible to have a centralized repository which everyone can run from?
Below is the error I get when trying to execute outide the IMP directory.
```
fatal: Not a git repository (or any parent up to mount point /mnt/md1200)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Traceback (most recent call last):
File "/home/snarayanasamy/Work/tools/IMP/IMP", line 252, in <module>
args = docopt(__doc__, version=get_git_version(), options_first=True)
File "/home/snarayanasamy/Work/tools/IMP/IMP", line 122, in get_git_version
['git', '--no-pager', 'log', '-n', '1', '--pretty=format:%H']
File "/usr/local/lib/python3.4/subprocess.py", line 620, in check_output
raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['git', '--no-pager', 'log', '-n', '1', '--pretty=format:%H']' returned non-zero exit status 128
```
Is there any way we can fix this? My IMP code directory is getting so messy with all the log files etc :six:
Yohan Jaroszyohan.jarosz@uni.luYohan Jaroszyohan.jarosz@uni.luhttps://git-r3lab.uni.lu/IMP/IMP/-/issues/23Incorporate VizBin Java implementation2018-02-09T09:10:34+01:00Shaman NarayanasamyIncorporate VizBin Java implementationFor consistency reasons.For consistency reasons.Yohan Jaroszyohan.jarosz@uni.luYohan Jaroszyohan.jarosz@uni.luhttps://git-r3lab.uni.lu/IMP/IMP/-/issues/43Metagenomic and metatranscriptomic only options for Docker container2018-02-09T09:10:34+01:00Shaman NarayanasamyMetagenomic and metatranscriptomic only options for Docker containerYohan Jaroszyohan.jarosz@uni.luYohan Jaroszyohan.jarosz@uni.luhttps://git-r3lab.uni.lu/IMP/IMP/-/issues/56Update MEGAHIT2018-02-09T09:10:34+01:00Shaman NarayanasamyUpdate MEGAHITUpdate to latest version that accommodates paired end information.
Update corresponding command for MEGAHIT within snakemake rules.Update to latest version that accommodates paired end information.
Update corresponding command for MEGAHIT within snakemake rules.Yohan Jaroszyohan.jarosz@uni.luYohan Jaroszyohan.jarosz@uni.luhttps://git-r3lab.uni.lu/IMP/IMP/-/issues/27Deprecate DATADIR parameter2018-02-09T09:10:34+01:00Shaman NarayanasamyDeprecate DATADIR parameterI think this simply adds complexity to the running of the pipeline. The entire paths can be specified in the MG and MT parameters.I think this simply adds complexity to the running of the pipeline. The entire paths can be specified in the MG and MT parameters.Yohan Jaroszyohan.jarosz@uni.luYohan Jaroszyohan.jarosz@uni.luhttps://git-r3lab.uni.lu/IMP/IMP/-/issues/126`metaquast` rule: ERROR! Failed downloading SILVA rRNA gene database2018-10-19T15:42:47+02:00Fredrik Boulund`metaquast` rule: ERROR! Failed downloading SILVA rRNA gene databaseHi, just encountered this message in my output from the `metaquast` rule. It seems everything continues anyway, but I couldn't find any other mention of this warning message in any other issues so far, so figured I'd report it anyway.
`...Hi, just encountered this message in my output from the `metaquast` rule. It seems everything continues anyway, but I couldn't find any other mention of this warning message in any other issues so far, so figured I'd report it anyway.
```
Downloading SILVA ribosomal RNA gene database...
ERROR! Failed downloading SILVA rRNA gene database (http://www.arb-silva.de/fileadmin/silva_databases/release_119/Exports/SILVA_119_SSURef_Nr9_tax_silva.fasta.gz)! The search for reference genomes cannot be performed. Try to download it manually in /home/imp/lib/quast-release_3.1/lbs/blast/16S_RNA_blastdb and restart MetaQUAST.
NOTICE: No references are provided, starting regular QUAST with MetaGeneMark gene finder
```Deepti MittalDeepti Mittalhttps://git-r3lab.uni.lu/IMP/IMP/-/issues/137samtools-1.9 might break host filtering step2018-10-08T16:28:24+02:00Cedric Lacznysamtools-1.9 might break host filtering stepAs part of the effort to enable conda-based installation, it occured that `samtools-1.9` was installed, despite that `0.1.19` **should** be installed (iris had newer version for an as-of-yet unknown reason, i.e., `all.yaml` specified `sa...As part of the effort to enable conda-based installation, it occured that `samtools-1.9` was installed, despite that `0.1.19` **should** be installed (iris had newer version for an as-of-yet unknown reason, i.e., `all.yaml` specified `samtools==0.1.19` and was **identical** between iris and litcrit).
It is speculated that this lead to the host filtering step failing on iris:
```
rule mt_filtering:
input: Preprocessing/mt.r1.trimmed.rna_filtered.fq, Preprocessing/mt.r2.trimmed.rna_filtered.fq, Preprocessing/mt.se.trimmed.rna_filtered.fq,
/home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa, /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa, /home/
users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.amb, /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.ann, /hom
e/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.bwt, /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.pac, /h
ome/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa.sa
output: Preprocessing/mt.r1.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt.r2.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt
.se.trimmed.rna_filtered.hg38_filtered.fq
jobid: 13
TMP_FILE=$(mktemp --tmpdir=/tmp -t "alignment_XXXXXX.bam")
BUFFER=$(mktemp --tmpdir=/tmp -t "alignment_XXXXXX.bam")
bwa mem -v 1 -t 26 /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa Preprocessing/mt.r1.trimmed.rna_filtered.fq Prepro
cessing/mt.r2.trimmed.rna_filtered.fq | samtools view -@ 26 -bS - > $TMP_FILE
samtools merge -@ 26 -u - <(samtools view -@ 26 -u -f 4 -F 264 $TMP_FILE) <(samtools view -@ 26 -u -f 8 -F 260 $TMP_FILE)
<(samtools view -@ 26 -u -f 12 -F 256 $TMP_FILE) | samtools view -@ 26 -bF 0x800 - | samtools sort -o -@ 26 -m 2G -n - $BUFFER -
| bamToFastq -i stdin -fq Preprocessing/mt.r1.trimmed.rna_filtered.hg38_filtered.fq -fq2 Preprocessing/mt.r2.trimmed.rna_filtered.hg38_fi
ltered.fq
if [[ -s Preprocessing/mt.se.trimmed.rna_filtered.fq ]]
then
bwa mem -v 1 -t 26 /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa Preprocessing/mt.se.trimmed.rna_filtered.fq | samt
ools view -@ 26 -bS - | samtools view -@ 26 -uf 4 - | bamToFastq -i stdin -fq Preprocessing/mt.se.trimmed.rna_filtered.hg38_filtered.fq
else
echo "Preprocessing/mt.se.trimmed.rna_filtered.fq is empty, skipping single end human sequence filtering, but creating it anyway..."
touch Preprocessing/mt.se.trimmed.rna_filtered.hg38_filtered.fq
fi
rm -rf $BUFFER* $TMP_FILE
[...]
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[main] Version: 0.7.9a-r786
[main] CMD: bwa mem -v 1 -t 26 /home/users/claczny/projects/imp-devel/dbs/hg38-db/filtering/hg38.fa Preprocessing/mt.r1.trimmed.rna_filtered.fq Preprocessing/mt.r2.trimmed.rna_filtered.fq
[main] Real time: 255.606 sec; CPU: 4680.023 sec
Usage: samtools sort [options...] [in.bam]
Options:
-l INT Set compression level, from 0 (uncompressed) to 9 (best)
-m INT Set maximum memory per thread; suffix K/M/G recognized [768M]
-n Sort by read name
-t TAG Sort by value of TAG. Uses position as secondary index (or read name if -n is set)
-o FILE Write final output to FILE rather than standard output
-T PREFIX Write temporary files to PREFIX.nnnn.bam
--input-fmt-option OPT[=VAL]
Specify a single input file format option in the form
of OPTION or OPTION=VALUE
-O, --output-fmt FORMAT[,OPT[=VAL]]...
Specify output format (SAM, BAM, CRAM)
--output-fmt-option OPT[=VAL]
Specify a single output file format option in the form
of OPTION or OPTION=VALUE
--reference FILE
Reference sequence FASTA FILE [null]
-@, --threads INT
Number of additional threads to use [0]
[W::bam_merge_core2] No @HD tag found.
Error in rule mt_filtering:
jobid: 13
output: Preprocessing/mt.r1.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt.r2.trimmed.rna_filtered.hg38_filtered.fq, Preprocessing/mt.se.trimmed.rna_filtered.hg38_filtered.fq
```
This needs further inspection.