GitLab

* preseq and picard collectalignmentsummarymetrics added

* changed PICARD COLLECTALIGNMENTSUMMARYMETRICS to PICARD COLLECTMULTIPLEMETRICS and integrated the new wrapper as a temporary script

* integration of next steps with temporary use of future wrappers as scripts

* wrapper integration for collectmultiplemetrics and genomecov, rule to create an igv-file from bigWig files

* deeptools and phantompeakqualtools integration

* phantompeakqualtools, headers and multiqc integration

* draft for integration of additional plots from phantompeakqualtools data into multiqc

* Changes according view #4

* Cross-correlation plots are grouped now. Changes in the description of the plots.

* change to newer wrapper versions n all rules and code cleanup

* Changes according to view #4, temporary matplotlib dependency to multiqc added, github actions added

* github actions

* test config and test data

* changes according to PR #4

* update config

* more logs added

* lint: Mixed rules and functions in same snakefile -> moved a part of the rule_all input to common.smk, input functions for rule_all added

* undo the changes of the last commit

* moved all function from Snakefile to common.smk

* --cache flag added for github actions

* --cache flag added for github actions

* snakemake_output_cache location added

* test snakemake_output_cache location

* another test snakemake_output_cache location

* another test snakemake_output_cache location

* set cache in github actions

* fix: dependencies in pysam resulted in ContextualVersionConflict in multiqc

* test: set cache location in github actions

* removed config files in .test from gitignore

* pysam depenencies and changes for github actions

* directory for ngs-test-data added

* gitmodules

* config

* test submodules

* test submodules

* config added

* directory for snakemake output cache changed

* cache location removed

* creating directory for snakemake output cache in github actions

* test cache directory with mkdir and chmod

* code cleanup github actions

* code cleanup github actions

* conda-forge channel added to pysam env

* conda-forge channel added to pysam env

* rule phantompeak added in a script instead of a shell command via Rscript

* testing on saccharomyces cerevisiae data set with deactivated preseq_lc_extrap rule

* r-base environment added to rule phantompeak_correlation

* changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)

* rule preseq_lc_extrap activated again, changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)

* adopt changes from bam_post_analysis branch, control grouping for samples and input rule to get sample-control combinations

* minimal cleanup

* adjustment of the plot_fingerprint: input function, matching of each treatment sample to its control sample for common output, integration of the JSD calculation, new wildcard for the control samples

* changes on wildcard handling for controls

* rule for macs2 callpeak added

* rule for bedtools intersect added, drafts for multiqc peaks count added

* broad and narrow option handling via config, additional rule for narrow peaks output, peaks count and frip score for multiqc, peaks for igv

* adaptation and integration of plot scripts for results from homer and macs2 analysis, script for plot_peaks_count and it's integration in snakemake-report, integraion of older plots in snakemake-report

* changes for linter

* changes for linter

* changes for linter

* changes for linter

* changes for linter

* changes on input functions and on params parsing for rules plot_macs_qc and plot_homrer_annotatepeaks, peaks wildcard added to all outputs

* test for the behavior of the linter

* test for the behavior of the linter

* changes for the linter

* test for the linter

* refactoring the config variable, restoring the input functions

* plot for FRiP score and some reports added, plot for annotatepeaks summary as draft added

* plot for homer annotatepeaks summary and report description, changes on frip score and peak count plots, changes according to PR #5

* some code cleanup

* changes for PR #5 added

* code cleanup

* preseq and picard collectalignmentsummarymetrics added

* changed PICARD COLLECTALIGNMENTSUMMARYMETRICS to PICARD COLLECTMULTIPLEMETRICS and integrated the new wrapper as a temporary script

* integration of next steps with temporary use of future wrappers as scripts

* wrapper integration for collectmultiplemetrics and genomecov, rule to create an igv-file from bigWig files

* deeptools and phantompeakqualtools integration

* phantompeakqualtools, headers and multiqc integration

* draft for integration of additional plots from phantompeakqualtools data into multiqc

* Changes according view #4

* Cross-correlation plots are grouped now. Changes in the description of the plots.

* change to newer wrapper versions n all rules and code cleanup

* Changes according to view #4, temporary matplotlib dependency to multiqc added, github actions added

* github actions

* test config and test data

* changes according to PR #4

* update config

* more logs added

* lint: Mixed rules and functions in same snakefile -> moved a part of the rule_all input to common.smk, input functions for rule_all added

* undo the changes of the last commit

* moved all function from Snakefile to common.smk

* --cache flag added for github actions

* --cache flag added for github actions

* snakemake_output_cache location added

* test snakemake_output_cache location

* another test snakemake_output_cache location

* another test snakemake_output_cache location

* set cache in github actions

* fix: dependencies in pysam resulted in ContextualVersionConflict in multiqc

* test: set cache location in github actions

* removed config files in .test from gitignore

* pysam depenencies and changes for github actions

* directory for ngs-test-data added

* gitmodules

* config

* test submodules

* test submodules

* config added

* directory for snakemake output cache changed

* cache location removed

* creating directory for snakemake output cache in github actions

* test cache directory with mkdir and chmod

* code cleanup github actions

* code cleanup github actions

* conda-forge channel added to pysam env

* conda-forge channel added to pysam env

* rule phantompeak added in a script instead of a shell command via Rscript

* testing on saccharomyces cerevisiae data set with deactivated preseq_lc_extrap rule

* r-base environment added to rule phantompeak_correlation

* changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)

* rule preseq_lc_extrap activated again, changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)

* Add some TODOs for later

* reflect that compute_matrix requests two threads in the extra params

* also cache bwa_index, as this can take quite some time
Co-authored-by: David Laehnemann <david.laehnemann@hhu.de>

* trimming with cutadapt added

* additions to cutadapt according to View #1, BWA added as draft

* adding cutadapt comments back, bwa mem will be moved to a new branch

* mapping, duplicate marking and merging

* changes according View #2 added, some minor additional changes with handling cutadapt-log-files for multiqc input in common.smk were necessary

* code formatting with black

* changes according view #2 added and replaced directory name mapped with merged in bwa_mem output, because its bam files are only temporary and otherwise the empty mapped directory would remain after the workflow-run

* rule for samtools view added

* rule samtools_view added, rules for samtools_index, flagstat, idxstats and samtools_stats were created as drafts and are therefore still commented out

* adding filtering rules, rule for removing orphan reads, adding rule for samtools sorting and statistics for multiqc

* Higher parallelization of the stats rules by outsourcing general rules for this purpose in stats.smk and utils.smk. Statistics for multiqc input of BAM files after 'bwa_mem' and 'merge_duplicates' added.

* Changes according to View #3.

* refactoring _dedup

* Log-path refactoring according to view #3. In multiqc-report the hovering of data points for samtools stats, flagstat and idxstats works for Chromium browser, but not for Firefox. Additional plots can be generated from the multiqc_data statistics, as described here: https://github.com/ewels/MultiQC/issues/512.

* trimming with cutadapt added

* additions to cutadapt according to View #1, BWA added as draft

* adding cutadapt comments back, bwa mem will be moved to a new branch

* mapping, duplicate marking and merging

* changes according View #2 added, some minor additional changes with handling cutadapt-log-files for multiqc input in common.smk were necessary

* code formatting with black

* changes according view #2 added and replaced directory name mapped with merged in bwa_mem output, because its bam files are only temporary and otherwise the empty mapped directory would remain after the workflow-run

* view #2: output path for rule bwa_mem and input path for rule merge_bams changed

* trimming with cutadapt added

* additions to cutadapt according to View #1, BWA added as draft

* adding cutadapt comments back, bwa mem will be moved to a new branch