Custom screening does not work well for launching in batches
When launching several instances of IMP, in parallel, the screen parameter seems to be indexing the fasta file each time IMP is launched. This is not an issue in a normal scenario, but in parallel, all the IMP instances are clashing and trying to index the same file, which makes corrupts the process, which is then terminated. Not sure why Snakemake doesn't recognize the existence of the bwa index files that were indexed previously. In addition, if a given screen file is very big, we would also ideally want to index it only once. For example, the human genome takes two hours to index...
So, looks like we figured out why this is happening. The
--screen parameter first copies the relevant fasta file into the
db/filtering) folder. This causes the indexed files (from the previous run) to have a later time stamp compared to the copied fasta file. Therefore
Snakemake invokes the indexing parameter again. @yjarosz, any idea what is the best way to solve this. I was thinking that we could give some conditions (
bash) within the rules to deal with it. Let me know what you think.