1. 24 Jun, 2021 1 commit
  2. 23 Jun, 2021 1 commit
  3. 16 Jun, 2021 2 commits
  4. 09 Jun, 2021 1 commit
  5. 04 Jun, 2021 1 commit
    • Antonie Vietor's avatar
      Consensus peak analysis (#6) · 1345df8a
      Antonie Vietor authored
      * preseq and picard collectalignmentsummarymetrics added
      
      * changed PICARD COLLECTALIGNMENTSUMMARYMETRICS to PICARD COLLECTMULTIPLEMETRICS and integrated the new wrapper as a temporary script
      
      * integration of next steps with temporary use of future wrappers as scripts
      
      * wrapper integration for collectmultiplemetrics and genomecov, rule to create an igv-file from bigWig files
      
      * deeptools and phantompeakqualtools integration
      
      * phantompeakqualtools, headers and multiqc integration
      
      * draft for integration of additional plots from phantompeakqualtools data into multiqc
      
      * Changes according view #4
      
      * Cross-correlation plots are grouped now. Changes in the description of the plots.
      
      * change to newer wrapper versions n all rules and code cleanup
      
      * Changes according to view #4, temporary matplotlib dependency to multiqc added, github actions added
      
      * github actions
      
      * test config and test data
      
      * changes according to PR #4
      
      * update config
      
      * more logs added
      
      * lint: Mixed rules and functions in same snakefile -> moved a part of the rule_all input to common.smk, input functions for rule_all added
      
      * undo the changes of the last commit
      
      * moved all function from Snakefile to common.smk
      
      * --cache flag added for github actions
      
      * --cache flag added for github actions
      
      * snakemake_output_cache location added
      
      * test snakemake_output_cache location
      
      * another test snakemake_output_cache location
      
      * another test snakemake_output_cache location
      
      * set cache in github actions
      
      * fix: dependencies in pysam resulted in ContextualVersionConflict in multiqc
      
      * test: set cache location in github actions
      
      * removed config files in .test from gitignore
      
      * pysam depenencies and changes for github actions
      
      * directory for ngs-test-data added
      
      * gitmodules
      
      * config
      
      * test submodules
      
      * test submodules
      
      * config added
      
      * directory for snakemake output cache changed
      
      * cache location removed
      
      * creating directory for snakemake output cache in github actions
      
      * test cache directory with mkdir and chmod
      
      * code cleanup github actions
      
      * code cleanup github actions
      
      * conda-forge channel added to pysam env
      
      * conda-forge channel added to pysam env
      
      * rule phantompeak added in a script instead of a shell command via Rscript
      
      * testing on saccharomyces cerevisiae data set with deactivated preseq_lc_extrap rule
      
      * r-base environment added to rule phantompeak_correlation
      
      * changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)
      
      * rule preseq_lc_extrap activated again, changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)
      
      * adopt changes from bam_post_analysis branch, control grouping for samples and input rule to get sample-control combinations
      
      * minimal cleanup
      
      * adjustment of the plot_fingerprint: input function, matching of each treatment sample to its control sample for common output, integration of the JSD calculation, new wildcard for the control samples
      
      * changes on wildcard handling for controls
      
      * rule for macs2 callpeak added
      
      * rule for bedtools intersect added, drafts for multiqc peaks count added
      
      * broad and narrow option handling via config, additional rule for narrow peaks output, peaks count and frip score for multiqc, peaks for igv
      
      * adaptation and integration of plot scripts for results from homer and macs2 analysis, script for plot_peaks_count and it's integration in snakemake-report, integraion of older plots in snakemake-report
      
      * changes for linter
      
      * changes for linter
      
      * changes for linter
      
      * changes for linter
      
      * changes for linter
      
      * changes on input functions and on params parsing for rules plot_macs_qc and plot_homrer_annotatepeaks, peaks wildcard added to all outputs
      
      * test for the behavior of the linter
      
      * test for the behavior of the linter
      
      * changes for the linter
      
      * test for the linter
      
      * refactoring the config variable, restoring the input functions
      
      * plot for FRiP score and some reports added, plot for annotatepeaks summary as draft added
      
      * plot for homer annotatepeaks summary and report description, changes on frip score and peak count plots, changes according to PR #5
      
      * some code cleanup
      
      * changes for PR #5 added
      
      * rules for merging peaks added
      
      * output logic for peak analysis and consensus peak analysis integrated, rule macs2_merged_expand added, config for output logic adapted, samples.tsv adapted to groups
      
      * functions for checking exists_multiple_groups and exists_replicates adapted to check this for each antibody, integration of igenomes.yaml for macs_gsize param, blacklist .bed files added to which igenomes.yaml references, added seperate activation param for optional outputs in config file, rule for consensus peak .saf and .bed filed, plot for consens peak intersected, rule to create igv for consensus peak analysis
      
      * a few minor corrections
      
      * rules for creating genome-filter - integration of blacklist to the workflow
      
      * homer annotation for consensus peaks, grouping of samples to antibodies, rule and script for featureCounts
      
      * bedtools sort bug fixes
      
      * draft for deseq2-analysis step, does not work yet
      
      * removed submodule ngs-test-data
      
      * removed submodule test-datasets
      
      * removes submodule atacseq
      
      * new test data sets added, integration of new data in samples and units tables, rule for modification of the featurecounts outputs, integration and modification of the featurecounts_deseq2.R script, integration of released wrappers
      
      * changes for linting
      
      * changes for linting
      
      * wildcards handling for rule feature_counts, update on snakemake-github-action
      
      * the new test data for PE now work correctly, ToDo: featurecounts_deseq2.R script still needs to be fixed, parallelization not yet possible
      
      * integration of sra download for SE test data sets, integration of accession numbers to cutadapt and fastqc, fixes on blacklists integration
      
      * workflow adjustment up to step 5, samples.tsv and units.tsv added for large real dataset
      
      * automation of sra download without the need for an entry in config, additional column for sra accession number in units.tsv and handling, changes according to suggestions, bug fixes
      
      * adaptation of the workflow for se: bypassing of orphan_rm and the stats from it, adaptations in post-analysis step
      
      * spliting and merging workflow for se and pe data on orphan remove
      
      * changes
      
      * changes
      
      * changes
      
      * split se and pe workflow at orphan remove step
      
      * adjustments for se data in CollectMultipleMetrics and genomecov, extraction of fragment size for genomcov from samtools stats
      
      * workflow has been adapted for se data
      
      * changes on design file and configuration for debugging deseq2 analysis, changes on github actions configuration
      
      * changes on design file and configuration for debugging deseq2 analysis, changes on github actions configuration
      
      * testing changes on design file and config
      
      * testing changes on design file and config
      
      * testing changes on design file: ambiguous samples
      
      * deseq2 analysis established
      
      * changes for linting
      
      * changes for linting
      
      * changes for github actions
      
      * changes for github actions
      
      * changes for github actions
      
      * changes for github actions
      
      * changes for github actions
      
      * changes for github actions
      
      * changes for github actions
      
      * changes for github actions
      
      * changes for github actions
      
      * changes for github actions
      
      * changes for github actions
      
      * changes on common.smk for configfile
      
      * changes on common.smk for configfile
      
      * changes on common.smk for configfile
      
      * changes on common.smk for configfile
      
      * changes on common.smk for configfile
      
      * changes on common.smk for configfile
      
      * changes on directory structure for config
      
      * changes on directory structure for config
      
      * Integration of controls for DESeq2 analysis, debugging of DESeq2 FDR calculation, separate output of each plot, reduced dataset created, debugging of compute matrix output for large data
      
      * config and data of reduced dataset
      
      * typo removed on main.yaml
      
      * new minimal single end test dataset from chromosome 14
      
      * some minor changes for github actions
      
      * some minor changes on units.tsv and debugging mode removed
      
      * new minimal paired end test dataset from chromosome VII saccharomyces cerevisiae
      
      * some minor changes on github actions
      
      * igenomes download, parser for igenomes, blacklist download and and option to customize the blacklist for a specific chromosome added, changes according to PR
      
      * changes for linting
      
      * changes for linting
      
      * changes for linting
      
      * some changes to satisfy the linter
      
      * separate rules for igenomes and blacklist download and blacklist handling
      
      * draft for creating checkpoint and rules to handle igenomes, blacklists and macs_gsize value
      
      * testing blacklist handling on new snakemake release
      
      * blacklists and genome size handling with snakemake 6.4.0 release
      
      * test data for single end changed to chromosome
      21 for data reduction
      
      * test refactoring for github actions
      
      * some minor changes on blacklist formatting
      
      * some additional minor changes on blacklist formatting
      
      * minimal changes on blacklist formatting
      
      * test data for single end reads changed back to chromosome 14
      
      * single end test data on chromosome 21
      
      * additional reduction of single end test data
      
      * some minor changes for testing
      
      * some minor changes for testing
      
      * some minor changes for testing
      
      * some minor changes for testing
      
      * some minor changes in github actions
      
      * some minor changes in github actions
      
      * changes according to PR
      1345df8a
  6. 21 Oct, 2020 1 commit
    • Antonie Vietor's avatar
      Peak analysis v2 (#5) · bc1ebc47
      Antonie Vietor authored
      * preseq and picard collectalignmentsummarymetrics added
      
      * changed PICARD COLLECTALIGNMENTSUMMARYMETRICS to PICARD COLLECTMULTIPLEMETRICS and integrated the new wrapper as a temporary script
      
      * integration of next steps with temporary use of future wrappers as scripts
      
      * wrapper integration for collectmultiplemetrics and genomecov, rule to create an igv-file from bigWig files
      
      * deeptools and phantompeakqualtools integration
      
      * phantompeakqualtools, headers and multiqc integration
      
      * draft for integration of additional plots from phantompeakqualtools data into multiqc
      
      * Changes according view #4
      
      * Cross-correlation plots are grouped now. Changes in the description of the plots.
      
      * change to newer wrapper versions n all rules and code cleanup
      
      * Changes according to view #4, temporary matplotlib dependency to multiqc added, github actions added
      
      * github actions
      
      * test config and test data
      
      * changes according to PR #4
      
      * update config
      
      * more logs added
      
      * lint: Mixed rules and functions in same snakefile -> moved a part of the rule_all input to common.smk, input functions for rule_all added
      
      * undo the changes of the last commit
      
      * moved all function from Snakefile to common.smk
      
      * --cache flag added for github actions
      
      * --cache flag added for github actions
      
      * snakemake_output_cache location added
      
      * test snakemake_output_cache location
      
      * another test snakemake_output_cache location
      
      * another test snakemake_output_cache location
      
      * set cache in github actions
      
      * fix: dependencies in pysam resulted in ContextualVersionConflict in multiqc
      
      * test: set cache location in github actions
      
      * removed config files in .test from gitignore
      
      * pysam depenencies and changes for github actions
      
      * directory for ngs-test-data added
      
      * gitmodules
      
      * config
      
      * test submodules
      
      * test submodules
      
      * config added
      
      * directory for snakemake output cache changed
      
      * cache location removed
      
      * creating directory for snakemake output cache in github actions
      
      * test cache directory with mkdir and chmod
      
      * code cleanup github actions
      
      * code cleanup github actions
      
      * conda-forge channel added to pysam env
      
      * conda-forge channel added to pysam env
      
      * rule phantompeak added in a script instead of a shell command via Rscript
      
      * testing on saccharomyces cerevisiae data set with deactivated preseq_lc_extrap rule
      
      * r-base environment added to rule phantompeak_correlation
      
      * changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)
      
      * rule preseq_lc_extrap activated again, changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)
      
      * adopt changes from bam_post_analysis branch, control grouping for samples and input rule to get sample-control combinations
      
      * minimal cleanup
      
      * adjustment of the plot_fingerprint: input function, matching of each treatment sample to its control sample for common output, integration of the JSD calculation, new wildcard for the control samples
      
      * changes on wildcard handling for controls
      
      * rule for macs2 callpeak added
      
      * rule for bedtools intersect added, drafts for multiqc peaks count added
      
      * broad and narrow option handling via config, additional rule for narrow peaks output, peaks count and frip score for multiqc, peaks for igv
      
      * adaptation and integration of plot scripts for results from homer and macs2 analysis, script for plot_peaks_count and it's integration in snakemake-report, integraion of older plots in snakemake-report
      
      * changes for linter
      
      * changes for linter
      
      * changes for linter
      
      * changes for linter
      
      * changes for linter
      
      * changes on input functions and on params parsing for rules plot_macs_qc and plot_homrer_annotatepeaks, peaks wildcard added to all outputs
      
      * test for the behavior of the linter
      
      * test for the behavior of the linter
      
      * changes for the linter
      
      * test for the linter
      
      * refactoring the config variable, restoring the input functions
      
      * plot for FRiP score and some reports added, plot for annotatepeaks summary as draft added
      
      * plot for homer annotatepeaks summary and report description, changes on frip score and peak count plots, changes according to PR #5
      
      * some code cleanup
      
      * changes for PR #5 added
      
      * code cleanup
      bc1ebc47
  7. 09 Sep, 2020 1 commit
    • Antonie Vietor's avatar
      Bam post analysis (#4) · 268eb612
      Antonie Vietor authored
      
      
      * preseq and picard collectalignmentsummarymetrics added
      
      * changed PICARD COLLECTALIGNMENTSUMMARYMETRICS to PICARD COLLECTMULTIPLEMETRICS and integrated the new wrapper as a temporary script
      
      * integration of next steps with temporary use of future wrappers as scripts
      
      * wrapper integration for collectmultiplemetrics and genomecov, rule to create an igv-file from bigWig files
      
      * deeptools and phantompeakqualtools integration
      
      * phantompeakqualtools, headers and multiqc integration
      
      * draft for integration of additional plots from phantompeakqualtools data into multiqc
      
      * Changes according view #4
      
      * Cross-correlation plots are grouped now. Changes in the description of the plots.
      
      * change to newer wrapper versions n all rules and code cleanup
      
      * Changes according to view #4, temporary matplotlib dependency to multiqc added, github actions added
      
      * github actions
      
      * test config and test data
      
      * changes according to PR #4
      
      * update config
      
      * more logs added
      
      * lint: Mixed rules and functions in same snakefile -> moved a part of the rule_all input to common.smk, input functions for rule_all added
      
      * undo the changes of the last commit
      
      * moved all function from Snakefile to common.smk
      
      * --cache flag added for github actions
      
      * --cache flag added for github actions
      
      * snakemake_output_cache location added
      
      * test snakemake_output_cache location
      
      * another test snakemake_output_cache location
      
      * another test snakemake_output_cache location
      
      * set cache in github actions
      
      * fix: dependencies in pysam resulted in ContextualVersionConflict in multiqc
      
      * test: set cache location in github actions
      
      * removed config files in .test from gitignore
      
      * pysam depenencies and changes for github actions
      
      * directory for ngs-test-data added
      
      * gitmodules
      
      * config
      
      * test submodules
      
      * test submodules
      
      * config added
      
      * directory for snakemake output cache changed
      
      * cache location removed
      
      * creating directory for snakemake output cache in github actions
      
      * test cache directory with mkdir and chmod
      
      * code cleanup github actions
      
      * code cleanup github actions
      
      * conda-forge channel added to pysam env
      
      * conda-forge channel added to pysam env
      
      * rule phantompeak added in a script instead of a shell command via Rscript
      
      * testing on saccharomyces cerevisiae data set with deactivated preseq_lc_extrap rule
      
      * r-base environment added to rule phantompeak_correlation
      
      * changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)
      
      * rule preseq_lc_extrap activated again, changed genome data back to human, added rule for downloading single chromosomes from the reference genome (to generate smaller test data)
      
      * Add some TODOs for later
      
      * reflect that compute_matrix requests two threads in the extra params
      
      * also cache bwa_index, as this can take quite some time
      Co-authored-by: default avatarDavid Laehnemann <david.laehnemann@hhu.de>
      268eb612
  8. 15 Jun, 2020 1 commit
    • Antonie Vietor's avatar
      Filtering (#3) · 4607b5fc
      Antonie Vietor authored
      * trimming with cutadapt added
      
      * additions to cutadapt according to View #1, BWA added as draft
      
      * adding cutadapt comments back, bwa mem will be moved to a new branch
      
      * mapping, duplicate marking and merging
      
      * changes according View #2 added, some minor additional changes with handling cutadapt-log-files for multiqc input in common.smk were necessary
      
      * code formatting with black
      
      * changes according view #2 added and replaced directory name mapped with merged in bwa_mem output, because its bam files are only temporary and otherwise the empty mapped directory would remain after the workflow-run
      
      * rule for samtools view added
      
      * rule samtools_view added, rules for samtools_index, flagstat, idxstats and samtools_stats were created as drafts and are therefore still commented out
      
      * adding filtering rules, rule for removing orphan reads, adding rule for samtools sorting and statistics for multiqc
      
      * Higher parallelization of the stats rules by outsourcing general rules for this purpose in stats.smk and utils.smk. Statistics for multiqc input of BAM files after 'bwa_mem' and 'merge_duplicates' added.
      
      * Changes according to View #3.
      
      * refactoring _dedup
      
      * Log-path refactoring according to view #3. In multiqc-report the hovering of data points for samtools stats, flagstat and idxstats works for Chromium browser, but not for Firefox. Additional plots can be generated from the multiqc_data statistics, as described here: https://github.com/ewels/MultiQC/issues/512.
      4607b5fc
  9. 18 May, 2020 1 commit
    • Antonie Vietor's avatar
      Mapping and duplicate marking (#2) · bb5a88a1
      Antonie Vietor authored
      * trimming with cutadapt added
      
      * additions to cutadapt according to View #1, BWA added as draft
      
      * adding cutadapt comments back, bwa mem will be moved to a new branch
      
      * mapping, duplicate marking and merging
      
      * changes according View #2 added, some minor additional changes with handling cutadapt-log-files for multiqc input in common.smk were necessary
      
      * code formatting with black
      
      * changes according view #2 added and replaced directory name mapped with merged in bwa_mem output, because its bam files are only temporary and otherwise the empty mapped directory would remain after the workflow-run
      
      * view #2: output path for rule bwa_mem and input path for rule merge_bams changed
      bb5a88a1
  10. 07 May, 2020 1 commit
    • AntonieV's avatar
      trimming with cutadapt added (#1) · 3f3ed4f5
      AntonieV authored
      * trimming with cutadapt added
      
      * additions to cutadapt according to View #1, BWA added as draft
      
      * adding cutadapt comments back, bwa mem will be moved to a new branch
      3f3ed4f5
  11. 30 Apr, 2020 1 commit
  12. 29 Apr, 2020 1 commit
  13. 28 Apr, 2020 1 commit
  14. 22 Apr, 2020 1 commit
  15. 15 Apr, 2020 2 commits