Commit ec745749 authored by Neil Kindlon's avatar Neil Kindlon
Browse files

Merge remote-tracking branch 'upstream/master'

parents cd012c6c b3504cee
......@@ -28,7 +28,7 @@ extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest',
'sphinx.ext.intersphinx', 'sphinx.ext.todo',
'sphinx.ext.coverage', 'sphinx.ext.pngmath',
'sphinx.ext.ifconfig', 'sphinx.ext.viewcode',
'matplotlib.sphinxext.plot_directive']
'matplotlib.sphinxext.plot_directive']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['templates']
......@@ -44,16 +44,16 @@ master_doc = 'index'
# General information about the project.
project = u'bedtools'
copyright = u'2009 - 2014, Aaron R. Quinlan & Neil Kindlon'
copyright = u'2009 - 2015, Aaron R. Quinlan & Neil Kindlon'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '2.23.0'
version = '2.24.0'
# The full version, including alpha/beta/rc tags.
release = '2.23.0'
release = '2.24.0'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
......@@ -187,7 +187,7 @@ htmlhelp_basename = 'bedtools-docs'
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'bedtools.tex', u'Bedtools Documentation',
u'Quinlan lab @ UVa', 'manual'),
u'Quinlan lab @ Univ. of Utah', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
......@@ -219,7 +219,7 @@ latex_documents = [
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'bedtools', u'Bedtools Documentation', [u'UVa'], 1)
('index', 'bedtools', u'Bedtools Documentation', [u'UU'], 1)
]
# Example configuration for intersphinx: refer to the Python standard library.
......
......@@ -2,6 +2,51 @@
Release History
###############
Version 2.24.0 (27-May-2015)
============================
1. The `coverage` tool now takes advantage of pre-sorted intervals via the `-sorted` option. This allows the `coverage` tool to be much faster, use far less memory, and report coverage for intervals in their original order in the input file.
2. We have changed the behavior of the `coverage` tool such that it is consistent with the other tools. Specifically, coverage is now computed for the intervals in the A file based on the overlaps with the B file, rather than vice versa.
3. The ``subtract`` tool now supports pre-sorted data via the ``-sorted`` option and is therefore much faster and scalable.
4. The ``-nonamecheck`` option provides greater tolerance for chromosome labeling when using the ``-sorted`` option.
5. Support for multiple SVLEN tags in VCF format, and fixed a bug that failed to process SVLEN tags coming at the end of a VCF INFO field.
6. Support for reverse complementing IUPAC codes in the ``getfasta`` tool.
7. Provided greater flexibility for "BED+" files, where the first 3 columns are chrom, start, and end, and the remaining columns are free-form.
8. We now detect stale FAI files and recreate an index thanks to a fix from @gtamazian.
9. New feature from Pierre Lindenbaum allowing the ``sort`` tool to sort files based on the chromosome order in a ``faidx`` file.
10. Eliminated multiple compilation warnings thanks to John Marshall.
11. Fixed bug in handling INS variants in VCF files.
Version 2.23.0 (22-Feb-2015)
============================
1. Added ``-k`` option to the closest tool to report the k-closest features in one or more -b files.
2. Added ``-fd`` option to the closest tool to for the reporting of downstream features in one or more -b files. Requires -D to dictate how "downstream" should be defined.
3. Added ``-fu`` option to the closest tool to for the reporting of downstream features in one or more -b files. Requires -D to dictate how "downstream" should be defined.
4. Pierre Lindenbaum added a new split tool that will split an input file into multiple sub files. Unlike UNIX split, it can balance the chunking of the sub files not just by number of lines, but also by total number of base pairs in each sub file.
5. Added a new spacing tool that reports the distances between features in a file.
6. Jay Hesselberth added a ``-reverse`` option to the makewindows tool that reverses the order of the assigned window numbers.
7. Fixed a bug that caused incorrect reporting of overlap for zero-length BED records. Thanks to @roryk.
8. Fixed a bug that caused the map tool to not allow ``-b`` to be specified before ``-a``. Thanks to @semenko.
9. Fixed a bug in ``makewindows`` that mistakenly required ``-s`` with ``-n``.
Version 2.22.1 (01-Jan-2015)
============================
1. When using -sorted with intersect, map, and closest, bedtools can now detect and warn you when your input datasets employ different chromosome sorting orders.
2. Fixed multiple bugs in the new, faster closest tool. Specifically, the -iu, -id, and -D options were not behaving properly with the new "sweeping" algorithm that was implemented for the 2.22.0 release. Many thanks to Sol Katzman for reporting these issues and for providing a detailed analysis and example files.
3. We FINALLY wrote proper documentation for the closest tool (http://bedtools.readthedocs.org/en/latest/content/tools/closest.html)
4. Fixed bug in the tag tool when using -intervals, -names, or -scores. Thanks to Yarden Katz for reporting this.
5. Fixed issues with chromosome boundaries in the slop tool when using negative distances. Thanks to @acdaugherty!
6. Multiple improvements to the fisher tool. Added a -m option to the fisher tool to merge overlapping intervals prior to comparing overlaps between two input files. Thanks to@brentp
7. Fixed a bug in makewindows tool requiring the use of -b with -s.
8. Fixed a bug in intersect that prevented -split from detecting complete overlaps with -f 1. Thanks to @tleonardi .
9. Restored the default decimal precision to the groupby tool.
10. Added the -prec option to the merge and map tools to specific the decimal precision of the output.
Version 2.22.0 (12-Nov-2014)
============================
1. The "closest" tool now requires sorted files, but this requirement now enables it to simultaneously find the closest intervals from many (not just one) files.
2. We now have proper support for "imprecise" SVs in VCF format. This addresses a long standing (sorry) limitation in the way bedtools handles VCF files.
Version 2.21.0 (18-Sep-2014)
============================
1. Added ability to intersect against multiple `-b` files in the `intersect` tool.
......
......@@ -3,67 +3,101 @@
###############
*coverage*
###############
**coverageBed** computes both the *depth* and *breadth* of coverage of features in file A across the features
in file B. For example, **coverageBed** can compute the coverage of sequence alignments (file A) across 1
kilobase (arbitrary) windows (file B) tiling a genome of interest. One advantage that **coverageBed**
offers is that it not only *counts* the number of features that overlap an interval in file B, it also
computes the fraction of bases in B interval that were overlapped by one or more features. Thus,
**coverageBed** also computes the *breadth* of coverage for each interval in B.
The ``bedtools coverage`` tool computes both the *depth* and *breadth* of coverage of features in file B on the features
in file A. For example, ``bedtools coverage`` can compute the coverage of sequence alignments (file B) across 1
kilobase (arbitrary) windows (file A) tiling a genome of interest. One advantage that ``bedtools coverage``
offers is that it not only *counts* the number of features that overlap an interval in file A, it also
computes the fraction of bases in the interval in A that were overlapped by one or more features. Thus,
``bedtools coverage`` also computes the *breadth* of coverage observed for each interval in A.
==========================================================================
Usage and option summary
==========================================================================
Usage:
.. note::
::
If you are trying to compute coverage for very large files and are having trouble
with excessive memory usage, please presort your data by chromosome and
then by start position (e.g., ``sort -k1,1 -k2,2n in.bed > in.sorted.bed``
for BED files) and then use the ``-sorted`` option. This invokes a
memory-efficient algorithm designed for large files.
coverageBed [OPTIONS] -a <BED/GFF/VCF> -b <BED/GFF/VCF>
=========================== ===============================================================================================================================================================================================================
Option Description
=========================== ===============================================================================================================================================================================================================
**-abam** BAM file A. Each BAM alignment in A is compared to B in search of overlaps. Use "stdin" if passing A with a UNIX pipe: For example:
.. important::
As of version 2.24.0, the `coverage` tool has changed such that the coverage is
computed for the A file, not the B file. This changes the command line interface
to be consistent with the other tools. Also, the `coverage` tool
can accept multiple files for the `-b` option. This allows one to measure
coverage between a single query (`-a`) file and multiple database files (`-b`) at once!
| samtools view -b <BAM> | intersectBed -abam stdin -b genes.bed
**-s** Force strandedness. That is, only features in A are only counted towards coverage in B if they are the same strand. *By default, this is disabled and coverage is counted without respect to strand*.
**-hist** Report a histogram of coverage for each feature in B as well as a summary histogram for _all_ features in B.
| Output (tab delimited) after each feature in B:
| 1) depth
| 2) # bases at depth
| 3) size of B
| 4) % of B at depth
**-d** Report the depth at each position in each B feature. Positions reported are one based. Each position and depth follow the complete B feature.
**-split** Treat "split" BAM or BED12 entries as distinct BED intervals when computing coverage. For BAM files, this uses the CIGAR "N" and "D" operations to infer the blocks for computing coverage. For BED12 files, this uses the BlockCount, BlockStarts, and BlockEnds fields (i.e., columns 10,11,12).
=========================== ===============================================================================================================================================================================================================
.. seealso::
:doc:`../tools/intersect`
:doc:`../tools/genomecov`
===============================
Usage and option summary
===============================
**Usage**:
::
bedtools coverage [OPTIONS] -a <FILE> \
-b <FILE1, FILE2, ..., FILEN>
**(or)**:
::
coverageBed [OPTIONS] -a <FILE> \
-b <FILE1, FILE2, ..., FILEN>
=========================== =========================================================================================================================================================
Option Description
=========================== =========================================================================================================================================================
**-a** BAM/BED/GFF/VCF file "A". Each feature in A is compared to B in search of overlaps. Use "stdin" if passing A with a UNIX pipe.
**-b** One or more BAM/BED/GFF/VCF file(s) "B". Use "stdin" if passing B with a UNIX pipe.
**NEW!!!**: -b may be followed with multiple databases and/or wildcard (*) character(s).
**-abam** BAM file A. Each BAM alignment in A is compared to B in search of overlaps. Use "stdin" if passing A with a UNIX pipe: For example: samtools view -b <BAM> | bedtools intersect -abam stdin -b genes.bed. **Note**: no longer necessary after version 2.19.0
**-hist** | Report a histogram of coverage for each feature in A as well as a summary histogram for _all_ features in A.
| Output (tab delimited) after each feature in A:
| 1) depth
| 2) # bases at depth
| 3) size of A
| 4) % of A at depth
**-d** Report the depth at each position in each A feature. Positions reported are one based. Each position and depth follow the complete A feature.
**-counts** Only report the count of overlaps, don't compute fraction, etc. Restricted by -f and -r.
**-f** Minimum overlap required as a fraction of A. Default is 1E-9 (i.e. 1bp).
**-r** Require that the fraction of overlap be reciprocal for A and B. In other words, if -f is 0.90 and -r is used, this requires that B overlap at least 90% of A and that A also overlaps at least 90% of B.
**-s** Force "strandedness". That is, only report hits in B that overlap A on the same strand. By default, overlaps are reported without respect to strand.
**-S** Require different strandedness. That is, only report hits in B that overlap A on the _opposite_ strand. By default, overlaps are reported without respect to strand.
**-split** Treat "split" BAM (i.e., having an "N" CIGAR operation) or BED12 entries as distinct BED intervals.
**-sorted** For very large B files, invoke a "sweeping" algorithm that requires position-sorted (e.g., ``sort -k1,1 -k2,2n`` for BED files) input. When using -sorted, memory usage remains low even for very large files.
**-g** Specify a genome file the defines the expected chromosome order in the input files for use with the ``-sorted`` option.
**-header** Print the header from the A file prior to results.
**-sortout** When using *multiple databases* (`-b`), sort the output DB hits for each record.
**-nobuf** Disable buffered output. Using this option will cause each line of output to be printed as it is generated, rather than saved in a buffer. This will make printing large output files noticeably slower, but can be useful in conjunction with other software tools and scripts that need to process one line of bedtools output at a time.
**-iobuf** Follow with desired integer size of read buffer. Optional suffixes `K/M/G` supported. **Note**: currently has no effect with compressed files.
=========================== =========================================================================================================================================================
==========================================================================
Default behavior
==========================================================================
After each interval in B, **coverageBed** will report:
After each interval in A, ``bedtools coverage`` will report:
1) The number of features in A that overlapped (by at least one base pair) the B interval.
2) The number of bases in B that had non-zero coverage from features in A.
3) The length of the entry in B.
4) The fraction of bases in B that had non-zero coverage from features in A.
1) The number of features in B that overlapped (by at least one base pair) the A interval.
2) The number of bases in A that had non-zero coverage from features in B.
3) The length of the entry in A.
4) The fraction of bases in A that had non-zero coverage from features in B.
Below are the number of features in A (N=...) overlapping B and fraction of bases in B with coverage.
Below are the number of features in B (N=...) overlapping A and fraction of bases in A with coverage.
::
Chromosome ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BED FILE B *************** *************** ****** **************
BED FILE A *************** *************** ****** **************
BED File A ^^^^ ^^^^ ^^ ^^^^^^^^^ ^^^ ^^ ^^^^
BED File B ^^^^ ^^^^ ^^ ^^^^^^^^^ ^^^ ^^ ^^^^
^^^^^^^^ ^^^^^ ^^^^^ ^^
Result [ N=3, 10/15 ] [ N=1, 2/15 ] [N=1,6/6] [N=6, 12/14 ]
......@@ -71,20 +105,20 @@ Below are the number of features in A (N=...) overlapping B and fraction of base
For example:
::
.. code-block:: bash
$ cat A.bed
chr1 0 100
chr1 100 200
chr2 0 100
cat A.bed
$ cat B.bed
chr1 10 20
chr1 20 30
chr1 30 40
chr1 100 200
cat B.bed
chr1 0 100
chr1 100 200
chr2 0 100
coverageBed -a A.bed -b B.bed
$ bedtools coverage -a A.bed -b B.bed
chr1 0 100 3 30 100 0.3000000
chr1 100 200 1 100 100 1.0000000
chr2 0 100 0 0 100 0.0000000
......@@ -95,57 +129,58 @@ For example:
``-s`` Calculating coverage by strand
==========================================================================
Use the "**-s**" option if one wants to only count coverage if features in A are on the same strand as the
feature / window in B. This is especially useful for RNA-seq experiments.
feature / window in A. This is especially useful for RNA-seq experiments.
For example (note the difference in coverage with and without **-s**:
::
.. code-block:: bash
$ cat A.bed
chr1 0 100 b1 1 +
chr1 100 200 b2 1 -
chr2 0 100 b3 1 +
cat A.bed
$ cat B.bed
chr1 10 20 a1 1 -
chr1 20 30 a2 1 -
chr1 30 40 a3 1 -
chr1 100 200 a4 1 +
cat B.bed
chr1 0 100 b1 1 +
chr1 100 200 b2 1 -
chr2 0 100 b3 1 +
coverageBed -a A.bed -b B.bed
$ bedtools coverage -a A.bed -b B.bed
chr1 0 100 b1 1 + 3 30 100 0.3000000
chr1 100 200 b2 1 - 1 100 100 1.0000000
chr2 0 100 b3 1 + 0 0 100 0.0000000
coverageBed -a A.bed -b B.bed -s
$ bedtools coverage -a A.bed -b B.bed -s
chr1 0 100 b1 1 + 0 0 100 0.0000000
chr1 100 200 b2 1 - 0 0 100 0.0000000
chr2 0 100 b3 1 + 0 0 100 0.0000000
==========================================================================
``-hist`` Creating a histogram of coverage for each feature in the B file
``-hist`` Creating a histogram of coverage for each feature in the A file
==========================================================================
One should use the "**-hist**" option to create, for each interval in B, a histogram of coverage of the
features in A across B.
One should use the "**-hist**" option to create, for each interval in A, a histogram of coverage of the
features in B across A.
In this case, each entire feature in B will be reported, followed by the depth of coverage, the number of
bases at that depth, the size of the feature, and the fraction covered. After all of the features in B have
been reported, a histogram summarizing the coverage among all features in B will be reported.
In this case, each entire feature in A will be reported, followed by the depth of coverage, the number of
bases at that depth, the size of the feature, and the fraction covered. After all of the features in A have
been reported, a histogram summarizing the coverage among all features in A will be reported.
::
.. code-block:: bash
cat A.bed
$ cat A.bed
chr1 0 100 b1 1 +
chr1 100 200 b2 1 -
chr2 0 100 b3 1 +
$ cat B.bed
chr1 10 20 a1 1 -
chr1 20 30 a2 1 -
chr1 30 40 a3 1 -
chr1 100 200 a4 1 +
cat B.bed
chr1 0 100 b1 1 +
chr1 100 200 b2 1 -
chr2 0 100 b3 1 +
coverageBed -a A.bed -b B.bed -hist
$ bedtools coverage -a A.bed -b B.bed -hist
chr1 0 100 b1 1 + 0 70 100 0.7000000
chr1 0 100 b1 1 + 1 30 100 0.3000000
chr1 100 200 b2 1 - 1 100 100 1.0000000
......@@ -154,28 +189,27 @@ been reported, a histogram summarizing the coverage among all features in B will
all 1 130 300 0.4333333
===========================================================================
``-d`` Reporting the per-base of coverage for each feature in the B file
``-d`` Reporting the per-base of coverage for each feature in the A file
===========================================================================
One should use the "**-d**" option to create, for each interval in B, a detailed list of coverage at each of the
positions across each B interval.
One should use the "**-d**" option to create, for each interval in A, a detailed list of coverage at each of the
positions across each A interval.
The output will consist of a line for each one-based position in each B feature, followed by the coverage
The output will consist of a line for each one-based position in each A feature, followed by the coverage
detected at that position.
::
.. code-block:: bash
$ cat A.bed
chr1 0 10
cat A.bed
$ cat B.bed
chr1 0 5
chr1 3 8
chr1 4 8
chr1 5 9
cat B.bed
chr1 0 10
coverageBed -a A.bed -b B.bed -d
$ bedtools coverage -a A.bed -b B.bed -d
chr1 0 10 B 1 1
chr1 0 10 B 2 1
chr1 0 10 B 3 1
......@@ -186,14 +220,3 @@ detected at that position.
chr1 0 10 B 8 3
chr1 0 10 B 9 1
chr1 0 10 B 10 0
=============================================================================
``-split`` Reporting coverage with spliced alignments or blocked BED features
=============================================================================
As described in section 1.3.19, coverageBed will, by default, screen for overlaps against the entire span
of a spliced/split BAM alignment or blocked BED12 feature. When dealing with RNA-seq reads, for
example, one typically wants to only tabulate coverage for the portions of the reads that come from
exons (and ignore the interstitial intron sequence). The **-split** command allows for such coverage to be
performed.
......@@ -23,14 +23,14 @@ void coverage_help(void) {
cerr << "Options: " << endl;
cerr << "\t-hist\t" << "Report a histogram of coverage for each feature in B" << endl;
cerr << "\t\tas well as a summary histogram for _all_ features in B." << endl << endl;
cerr << "\t\tOutput (tab delimited) after each feature in B:" << endl;
cerr << "\t\t 1) depth\n\t\t 2) # bases at depth\n\t\t 3) size of B\n\t\t 4) % of B at depth" << endl << endl;
cerr << "\t-hist\t" << "Report a histogram of coverage for each feature in A" << endl;
cerr << "\t\tas well as a summary histogram for _all_ features in A." << endl << endl;
cerr << "\t\tOutput (tab delimited) after each feature in A:" << endl;
cerr << "\t\t 1) depth\n\t\t 2) # bases at depth\n\t\t 3) size of A\n\t\t 4) % of A at depth" << endl << endl;
cerr << "\t-d\t" << "Report the depth at each position in each B feature." << endl;
cerr << "\t-d\t" << "Report the depth at each position in each A feature." << endl;
cerr << "\t\tPositions reported are one based. Each position" << endl;
cerr << "\t\tand depth follow the complete B feature." << endl << endl;
cerr << "\t\tand depth follow the complete A feature." << endl << endl;
cerr << "\t-counts\t" << "Only report the count of overlaps, don't compute fraction, etc." << endl << endl;
......
......@@ -86,7 +86,11 @@ void BedSlop::AddSlop(BED &bed) {
if ( ((int)bed.end + (int)_leftSlop) <= chromSize )
bed.end = bed.end + (int)_leftSlop;
else
bed.end = chromSize;
// if the _leftSlop is negative and pushes bed.end to be < 0, set to 1
if ( (((int)bed.end + (int)_leftSlop) <= 0) && _leftSlop < 0)
bed.end = 1;
else
bed.end = chromSize;
}
else {
if ( ((int)bed.start - (int)_leftSlop) >= 0 )
......@@ -97,7 +101,7 @@ void BedSlop::AddSlop(BED &bed) {
bed.end = bed.end + (int)_rightSlop;
else
// if the _rightSlop is negative and pushes bed.end to be < 0, set to 1
if ( ((int)bed.end + (int)_rightSlop) <= 0 )
if ( (((int)bed.end + (int)_rightSlop) <= 0) && _rightSlop < 0)
bed.end = 1;
else
bed.end = chromSize;
......
......@@ -244,7 +244,14 @@ void FastaReference::open(string reffilename, bool usemmap, bool useFullHeader)
struct stat stFileInfo;
string indexFileName = filename + index->indexFileExtension();
// if we can find an index file, use it
if(stat(indexFileName.c_str(), &stFileInfo) == 0) {
if(stat(indexFileName.c_str(), &stFileInfo) == 0) {
// check if the index file is older than the FASTA file
struct stat index_attrib, fasta_attrib;
stat(indexFileName.c_str(), &index_attrib);
stat(reffilename.c_str(), &fasta_attrib);
if (fasta_attrib.st_mtime > index_attrib.st_mtime) {
cerr << "Warning: the index file is older than the FASTA file." << endl;
}
index->readIndexFile(indexFileName);
} else { // otherwise, read the reference and generate the index file in the cwd
cerr << "index file " << indexFileName << " not found, generating..." << endl;
......
# This file was auto-generated by running "make setversion VERSION=v2.23.0"
# on Sun Feb 22 17:33:17 EST 2015 .
# This file was auto-generated by running "make setversion VERSION=v2.24.0"
# on Wed May 27 20:44:34 EDT 2015 .
# Please do not edit or commit this file manually.
#
v2.23.0
v2.24.0
......@@ -68,7 +68,7 @@ echo \
#_________________________________________
# p-values for fisher's exact test
left right two-tail ratio
1 1 1 -nan" > exp
1 1 1 nan" > exp
$BT fisher -a a_merge.bed -b b.bed -g t.60.genome > obs
check obs exp
rm obs exp
......
......@@ -88,7 +88,7 @@ AGCTYRWSKMDVHBXN
agctyrwskmdvhbxn" > exp
$BT getfasta -fi test.iupac.fa -bed test.iupac.bed -fo - > obs
check obs exp
rm obs exp
rm obs exp test.iupac.fa.fai
# test IUPAC revcomp
......@@ -100,4 +100,23 @@ NXVDBHKMSWYRAGCT
nxvdbhkmswyragct" > exp
$BT getfasta -fi test.iupac.fa -bed test.iupac.bed -s -fo - > obs
check obs exp
rm obs exp
rm obs exp test.iupac.fa.fai
# test the warning about an outdated FASTA index file
echo " getfasta.t10...\c"
echo \
">chr1
cggggggggg
>chr2
AAATTTTTTTTTT" > test.fa
# create an index file
echo -e "chr2\t2\t10" | $BT getfasta -fi test.fa -bed - -fo - > /dev/null
# modify the FASTA file in a second
sleep 1
touch test.fa
echo -e "chr2\t2\t10" | $BT getfasta -fi test.fa -bed - -fo - \
> /dev/null 2> obs
echo "Warning: the index file is older than the FASTA file." > exp
check obs exp
rm obs exp test.fa test.fa.fai
1 0 16 1 1000 -
2 0 16 1 1000 -
>1
AGCTYRWSKMDVHBXNACGT
>2
agctyrwskmdvhbxnacgt
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment