forked from samtools/samtools
-
Notifications
You must be signed in to change notification settings - Fork 0
/
NEWS
1318 lines (848 loc) · 46.3 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Release a.b
-----------
Release 1.9 (18th July 2018)
----------------------------
* Samtools mpileup VCF and BCF output is now deprecated. It is still
functional, but will warn. Please use bcftools mpileup instead. (#884)
* Samtools mpileup now handles the '-d' max_depth option differently. There
is no longer an enforced minimum, and '-d 0' is interpreted as limitless
(no maximum - warning this may be slow). The default per-file depth is
now 8000, which matches the value mpileup used to use when processing
a single sample. To get the previous default behaviour use the higher
of 8000 divided by the number of samples across all input files, or 250.
(#859)
* Samtools stats new features:
- The '--remove-overlaps' option discounts overlapping portions of
templates when computing coverage and mapped base counting. (#855)
- When a target file is in use, the number of bases inside the
target is printed and the percentage of target bases with coverage
above a given threshold specified by the '--cov-threshold' option. (#855)
- Split base composition and length statistics by first and last reads.
(#814, #816)
* Samtools faidx new features:
- Now takes long options. (#509, thanks to Pierre Lindenbaum)
- Now warns about zero-length and truncated sequences due to the
requested range being beyond the end of the sequence. (#834)
- Gets a new option (--continue) that allows it to carry on
when a requested sequence was not in the index. (#834)
- It is now possible to supply the list of regions to output in a text
file using the new '--region-file' option. (#840)
- New '-i' option to make faidx return the reverse complement of
the regions requested. (#878)
- faidx now works on FASTQ (returning FASTA) and added a new
fqidx command to index and return FASTQ. (#852)
* Samtools collate now has a fast option '-f' that only operates on
primary pairs, dropping secondary and supplementary. It tries to write
pairs to the final output file as soon as both reads have been found. (#818)
* Samtools bedcov gets a new '-j' option to make it ignore deletions (D) and
reference skips (N) when computing coverage. (#843)
* Small speed up to samtools coordinate sort, by converting it to use
radix sort. (#835, thanks to Zhuravleva Aleksandra)
* Samtools idxstats now works on SAM and CRAM files, however this
isn't fast due to some information lacking from indices. (#832)
* Compression levels may now be specified with the level=N
output-fmt-option. E.g. with -O bam,level=3.
* Various documentation improvements.
* Bug-fixes:
- Improved error reporting in several places. (#827, #834, #877, cd7197)
- Various test improvements.
- Fixed failures in the multi-region iterator (view -M) when regions
provided via BED files include overlaps (#819, reported by Dave Larson).
- Samtools stats now counts '=' and 'X' CIGAR operators when
counting mapped bases. (#855)
- Samtools stats has fixes for insert size filtering (-m, -i). (#845; #697
reported by Soumitra Pal)
- Samtools stats -F now longer negates an earlier -d option. (#830)
- Fix samtools stats crash when using a target region. (#875, reported by
John Marshall)
- Samtools sort now keeps to a single thread when the -@ option is absent.
Previously it would spawn a writer thread, which could cause the CPU
usage to go slightly over 100%. (#833, reported by Matthias Bernt)
- Fixed samtools phase '-A' option which was incorrectly defined to take
a parameter. (#850; #846 reported by Dianne Velasco)
- Fixed compilation problems when using C_INCLUDE_PATH. (#870; #817 reported
by Robert Boissy)
- Fixed --version when built from a Git repository. (#844, thanks to John
Marshall)
- Use noenhanced mode for title in plot-bamstats. Prevents unwanted
interpretation of characters like underscore in gnuplot version 5. (#829,
thanks to M. Zapukhlyak)
- blast2sam.pl now reports perfect match hits (no indels or mismatches).
(#873, thanks to Nils Homer)
- Fixed bug in fasta and fastq subcommands where stdout would not be flushed
correctly if the -0 option was used.
- Fixed invalid memory access in mpileup and depth on alignment records
where the sequence is absent.
Release 1.8 (3rd April 2018)
----------------------------
* samtools calmd now has a quiet mode. This can be enabled by passing `-Q` to
calmd. (Thanks to Colin Davenport)
* In samtools depth `-d 0` will effectively remove the depth limit. (#764)
* Improvements made to samtools collate's interface and documentation. It is
now possible to specify an output file name using `-o`, instead of deriving
it from the prefix used for temporary files. The prefix itself is now
optional if `-o` or `-O` (to stdout) is used. (#780)
* Bug-fixes:
- Make samtools addreplacerg choose output format by file extension. (#767;
reported by Argy Megalios)
- Merge tests now work on ungzipped data, allowing tests to be run against
different deflate libraries.
- samtools markdup error messages about missing tags have been updated with
the suggestion that samtools fixmate is run beforehand. (#765; reported by
Yudong Cai)
- Enables the `--reference` option for samtools fastq. Now works like other
programs when a reference sequence is needed for CRAM files. (#791,
reported by Milana Kaljevic)
Release 1.7 (26th January 2018)
--------------------
* HTSlib, and so samtools, now support BAMs which include CIGARs with more
than 65535 operations as per HTS-Specs 18th November (dab57f4 and 2f915a8).
* samtools quickcheck will now write a warning to stderr if it finds
any problems. These messages can be suppressed with a new `-q` option.
* samtools markdup can now mark supplementary alignments of reads where
the primary alignment is found to be a duplicate. Supplementary marking
can be turned on by passing the `-S` option to markdup. When this
option is enabled, all the alignment data will be written to a temporary
file so that supplementary alignments that occur before a duplicated
primary can be correctly marked in the final output. The location
of this temporary file can be influenced using the new `-T` option.
* samtools view now supports HTSlib's new multi-region iterator.
This can be enabled by passing the `-M` option to view. When using
this option:
- The BED filter (`-L` option) will use the index to skip through the file
- Reads from overlapping regions will only be output once
* samtools bedcov will now ignore BED comment and header lines (#571; thanks
to Daniel Baker).
* samtools collate now updates the @HD SO: and GO: tags, and sort will
remove a GO: tag if present. (#757; reported by Imran Haque).
* Bug-fixes:
- maq2sam now checks for input files that end early. (#751; patch supplied
by Alexandre Rebert of the Mayhem team, via Andreas Tille from Debian.)
- Fixed incorrect check when looking up header tags that could lead
to a crash in samtools stats. (#208; thanks to Dave Larson.)
- Fixed bug in samtools fastq `-O` option where it would fail if
the OQ tag in the input file had an unexpected type. (#758;
reported by Taejeong Bae)
- The MD5 calculations in samtools dict and md5fa did not handle
non-alphabetic characters in the same way as the CRAM MD5 function.
They have now been updated to match. (#704; reported by Chris Norman).
- Fix possible infinite loop in samtools targetcut.
- Building bam_tview_curses should no longer fail if a curses header file
cannot be found.
Release 1.6 (28th September 2017)
--------------------
* Added new markdup sub-command and '-m' option for fixmate. Used together,
they allow duplicates to be marked and optionally removed. This
fixes a number of problems with the old 'rmdup' sub-command, for
example samtools issue #497. 'rmdup' is kept for backwards compatibility
but 'markdup' should be used in preference.
* Sort is now much better at keeping within the requested memory limit. It
should also be slightly faster and need fewer temporary files when the file
to be sorted does not fit in memory. (#593; thanks to Nathan Weeks.)
* Sort no longer rewrites the header when merging from files. It can also
now merge from memory, so fewer temporary files need to be written and
it is better at sorting in parallel when everything fits in memory.
* Both sort and merge now resolve ties when merging based on the position
in the input file(s). This makes them fully stable for all ordering
options. (Previously position sort was stable, but name and by tag
sorts were not).
* New --output-qname option for mpileup.
* Support for building on Windows using msys2/mingw64 or cygwin has
been improved.
Release 1.5 [Solstice Release] (21st June 2017)
--------------------
* Samtools fastq now has a -i option to create a fastq file from an index
tag, and a -T option (similar to -t) to add user specified aux tags to
the fastq header line.
* Samtools fastq can now create compressed fastq files, by giving the
output filenames an extention of .gq, .bgz, or .bgzf
* Samtools sort has a -t TAG option, that allows records to be sorted by
the value of the specified aux tag, then by position or name. Merge
gets a similar option, allowing files sorted this way to be merged.
(#675; thanks to Patrick Marks of 10xgenomics).
Release 1.4.1 (8th May 2017)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Added options to fastq to create fastq files from BC (or other)
tags.
* Samtools view has gained a -G <flags> option to exclude on all bits
set. For example to discard reads where neither end has been
mapped use "-G 12".
* Samtools cat has a -b <fofn> option to ease concatenation of many
files.
* Added misc/samtools_tab_completion for bash auto-completion of
samtools sub-commands. (#560)
* Samtools tview now has J and K keys for verticale movement by 20
lines. (#257)
* Various compilation / portability improvements.
* Fixed issue with more than 65536 CIGAR operations and SAM/CRAM files.
(#667)
Release 1.4 (13 March 2017)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Noteworthy changes in samtools:
* Fixed Issue #345 - out-by-one error in insert-size in samtools stats
* bam_split now add a @PG header to the bam file
* Added mate cigar tag support to fixmate
* Multi-threading is now supported for decoding BAM and CRAM (as well
as the previously supported encoding). Most commands that read BAM
or CRAM have gained an -@ or --threads arguments, providing a
significant speed bonus. For commands that both read and write
files the threads are shared between decoding and encoding tasks.
* Added -a option to samtools mpileup to show all locations, including
sites with zero depth; repeating the option as -aa or -a -a additionally
shows reference sequences without any reads mapped to them (#496).
* The mpileup text output no longer contains empty columns at zero coverage
positions. Previously it would output "...0\t\t..." in some circumstances
(zero coverage due to being below a minumum base quality); this has been
fixed to output as "...0\t*\t*..." with placeholder '*' characters as in
other zero coverage circumstances (see PR #537).
* To stop it from creating too many temporary files, samtools sort
will now not run unless its per-thread memory limit (-m) is set to
at least 1 megabyte (#547).
* The misc/plot-bamstats script now has a -l / --log-y option to change
various graphs to display their Y axis log-scaled. Currently this
affects the Insert Size graph (PR #589; thanks to Anton Kratz).
* Fixmate will now also add and update MC (mate CIGAR) tags.
Beta Release 1.3.1 (22 April 2016)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Noteworthy changes in samtools:
* The sort command creates any needed temporary files alongside the final
output file (similarly to the pre-1.3 behaviour), and now aborts when
it detects a collision with another sort invocation's temporary files.
When the -T PREFIX option specified is a directory (or when sorting to
standard output), a random component is now added to temporary filenames
to try to avoid collisions (#432, #523, #529, #535, PR #530).
* All samtools commands now check for I/O errors more carefully, especially
when writing output files (#111, #253, #470, PR #467).
* Build fixes for 32-bit systems; be sure to run configure on such systems
to enable large file support and access to 2GiB+ files.
* The fasta/fastq/bam2fq command no longer ignores reads when the -s option
is used (#532).
* The fastq -O option no longer crashes on reads that do not have an OQ tag
field (#517).
* The merge and sort commands now handle (unusual) BAM files that have no
textual @SQ headers (#548, #550).
* Sorting files containing @CO headers no longer duplicates the comment
headers, which previously happened on large sorts for which temporary
files were needed (#563).
* The rmdup and view -l commands no longer crash on @RG headers that do not
have a LB field (#538).
* Fixed miscellaneous issues #128, #130, #131, #489, and #514.
Beta Release 1.3 (15 December 2015)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Noteworthy changes in samtools:
* The obsolete "samtools sort in.bam out.prefix" usage has been removed.
If you are still using -f, -o, or out.prefix, convert to use -T PREFIX
and/or -o FILE instead. (#295, #349, #356, #418, PR #441; see also
discussions in #171, #213.)
* When writing CRAM output, samtools now defaults to writing CRAM v3.0
rather than v2.1.
* The "bamshuf" command has been renamed to "collate" (hence the term
bamshuf no longer appears in the documentation, though it still works
on the command line for compatibility with existing scripts).
* The mpileup command now outputs the unseen allele in VCF/BCF as <*>
rather than X or <X> as previously, and now has AD, ADF, ADR, INFO/AD,
INFO/ADF, INFO/ADR --output-tags annotations that largely supersede
the existing DV, DP4, DPR annotations.
* The mpileup command now applies BAQ calculations at all base positions,
regardless of which -l or -r options are used (previously with -l it was
not applied to the first few tens of bases of each chromosome, leading
to different mpileup results with -l vs. -r; #79, #125, #286, #407).
* Samtools now has a configure script which checks your build environment
and facilitates choosing which HTSlib to build against. See INSTALL
for details.
* Samtools's Makefile now fully supports the standard convention of
allowing CC/CPPFLAGS/CFLAGS/LDFLAGS/LIBS to be overridden as needed.
Previously it listened to $(LDLIBS) instead; if you were overriding
that, you should now override LIBS rather than LDLIBS.
* A new addreplacerg command that adds or alters @RG headers and RG:Z
record tags has been added.
* The rmdup command no longer immediately aborts (previously it always
aborted with "bam_get_library() not yet implemented"), but remains
not recommended for most use (#159, #252, #291, #393).
* Merging files with millions of headers now completes in a reasonable
amount of time (#337, #373, #419, #453; thanks to Nathan Weeks,
Chris Smowton, Martin Pollard, Rob Davies).
* Samtools index's optional index output path argument works again (#199).
* Fixed calmd, targetcut, and potential mpileup segfaults when given broken
alignments with POS far beyond the end of their reference sequences.
* If you have source code using bam_md.c's bam_fillmd1_core(), bam_cap_mapQ(),
or bam_prob_realn_core() functions, note that these now take an additional
ref_len parameter. (The versions named without "_core" are unchanged.)
* The tview command's colour scheme has been altered to be more suitable
for users with colour blindness (#457).
* Samtools depad command now handles CIGAR N operators and accepts
CRAM files (#201, #404).
* Samtools stats now outputs separate "N" and "other" columns in the
ACGT content per cycle section (#376).
* Added -a option to samtools depth to show all locations, including
zero depth sites (#374).
* New samtools dict command, which creates a sequence dictionary
(as used by Picard) from a FASTA reference file.
* Samtools stats --target-regions option works again.
* Added legacy API sam.h functions sam_index_load() and samfetch() providing
bam_fetch()-style iteration over either BAM or CRAM files. (In general
we recommend recoding against the htslib API directly, but this addition
may help existing libbam-using programs to be CRAM-enabled easily.)
* Fixed legacy API's samopen() to write headers only with "wh" when writing
SAM files. Plain "w" suppresses headers for SAM file output, but this
was broken in 1.2.
* "samtools fixmate - -" works in pipelines again; with 1.0 to 1.2,
this failed with "[bam_mating] cannot determine output format".
* Restored previous "samtools calmd -u" behaviour of writing compression
level 0 BAM files. Samtools 1.0 to 1.2 incorrectly wrote raw non-BGZF
BAM files, which cannot be read by most other tools. (Samtools commands
other than calmd were unaffected by this bug.)
* Restored bam_nt16_nt4_table[] to legacy API header bam.h.
* Fixed bugs #269, #305, #320, #328, #346, #353, #365, #392, #410, #445,
#462, #475, and #495.
Beta Release 1.2 (2 February 2015)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Noteworthy changes in samtools:
* Flagstat now works on SAM, BAM, or CRAM files (rather than BAM only)
* Stats calculates mismatches per cycle for unclipped length
* Merge can now merge SAM input files
* CRAM reference files are now cached by default (see HTSlib release
notes and samtools(1) man page)
* Tested against Intel-optimised zlib (https://github.com/jtkukunas/zlib;
see README for details)
* Fixed bugs #302, #309, #318, and #327 and many other improvements
and bugs fixed in HTSlib -- see the HTSlib release notes
Beta Release 1.1 (19 September, 2014)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Notable changes in samtools:
* Sorting files with thousands of reference contigs now completes in
a reasonable amount of time (#260)
* Fixmate and flagstat now consider supplementary reads
* Fixmate now only adds a template cigar tag ("ct:Z") when requested
via a new -c option, and never adds it repeatedly (#276)
* Mpileup DPR annotation fixed (#274)
* Checksum added to stats command output
* Removed view -Q option
Beta Release 1.0 (15 August, 2014)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
First release of HTSlib-based samtools.
Numerous changes, notably support for the CRAM sequencing file format.
The faidx command now reads either uncompressed or BGZF-compressed FASTA files
compressed with bgzip. In previous samtools-0.1.x versions, faidx could read
either uncompressed or RAZF-compressed FASTA files, but RAZF and razip are
superseded by BGZF/bgzip and have been removed from samtools.
Beta Release 0.1.20 (15 August, 2014)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Final release of standalone samtools.
Beta Release 0.1.19 (15 March, 2013)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Notable changes in samtools and bcftools:
* The latest source code and development moved to github,
http://github.com/samtools/samtools
* Many important bugfixes and contributions by many people. Thanks to all!
* Performance improvements (multi-threading)
* Important changes in calling, see
- samtools mpileup -p
- bcftools view -m
* New annotations useful for filtering (RPB, HWE, QBD, MDV)
* New tools, bamcheck and plot-bamcheck
* New features in samtools tview
* And much more..
For a detailed list of commits, please see
http://github.com/samtools/samtools/commits/master
(0.1.19: 15 March 2013, commit 96b5f2294ac0054230e88913c4983d548069ea4e)
Beta Release 0.1.18 (2 September, 2011)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Notable changes in samtools:
* Support the new =/X CIGAR operators (by Peter Cock).
* Allow to subsample BAM while keeping the pairing intact (view -s).
* Implemented variant distance bias as a new filter (by Petr Danecek).
* Bugfix: huge memory usage during indexing
* Bugfix: use of uninitialized variable in mpileup (rare)
* Bugfix: wrong BAQ probability (rare)
Notable changes in bcftools:
* Support indel in the contrast caller.
* Bugfix: LRT2=nan in rare cases
(0.1.18: 2 September 2011, r982:295)
Beta Release 0.1.17 (6 July, 2011)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
With the maturity of `mpileup' and the lack of update in the `pileup' command,
the `pileup' command is now formally dropped. Most of the pileup functionality,
such as outputting mapping quality and read positions, have been added
`mpileup'.
Since this release, `bcftools view' is able to perform contrast SNP calling
(option -T) for discovering de novo and/or somatic mutations between a pair of
samples or in a family trio. Potential mutations are scored by a log likelihood
ratio, which is very simple in math, but should be comparable to more
sophisticated methods. Note that getting the score is only the very first step.
A lot more need to be done to reduce systematical errors due to mapping and
reference errors and structural variations.
Other notable changes in samtools:
* Improved sorting order checking during indexing.
* Improved region parsing. Colons in reference sequence names are parsed
properly.
* Fixed an issue where mpileup does not apply BAQ for the first few reads when
a region is specified.
* Fixed an issue where `faidx' does not work with FASTA files with long lines.
* Bugfix: wrong SP genotype information in the BCF output.
Other notable changes in bcftools:
* Output the ML esitmate of the allele count.
* Added the HWE plus F<0 filter to varFilter. For multiple samples, it
effectively filters false heterozygous calls around centromeres.
* For association mapping, perform both 1-degree and 2-degree test. The
2-degree test is conservative but more robust to HWE violation.
(0.1.17: 6 July 2011, r973:277)
Beta Release 0.1.16 (21 April, 2011)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Notable changes in samtools:
* Support the new SAM/BAM type `B' in the latest SAM spec v1.4.
* When the output file of `samtools merge' exists, do not overwrite it unless
a new command-line option `-f' is applied.
* Bugfix: BED support is not working when the input BED is not sorted.
* Bugfix: some reads without coordinates but given on the reverse strand are
lost in merging.
Notable changes in bcftools:
* Code cleanup: separated max-likelihood inference and Bayesian inference.
* Test Hardy-Weinberg equilibrium with a likelihood-ratio test.
* Provided another association test P-value by likelihood-ratio test.
* Use Brent's method to estimate the site allele frequency when EM converges
slowly. The resulting ML estimate of allele frequnecy is more accurate.
* Added the `ldpair' command, which computes r^2 between SNP pairs given in
an input file.
Also, the `pileup' command, which has been deprecated by `mpileup' since
version 0.1.10, will be dropped in the next release. The old `pileup' command
is substandard and causing a lot of confusion.
(0.1.16: 21 April 2011, r963:234)
Beta Release 0.1.15 (10 April, 2011)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Noteable changes:
* Allow to perform variant calling or to extract information in multiple
regions specified by a BED file (`samtools mpileup -l', `samtools view -L'
and `bcftools view -l').
* Added the `depth' command to samtools to compute the per-base depth with a
simpler interface. File `bam2depth.c', which implements this command, is the
recommended example on how to use the mpileup APIs.
* Estimate genotype frequencies with ML; perform chi^2 based Hardy-Weinberg
test using this estimate.
* For `samtools view', when `-R' is specified, drop read groups in the header
that are not contained in the specified file.
* For `samtools flagstat', separate QC-pass and QC-fail reads.
* Improved the command line help of `samtools mpileup' and `bcftools view'.
* Use a global variable to control the verbose level of samtools stderr
output. Nonetheless, it has not been full utilized.
* Fixed an issue in association test which may report false associations,
possibly due to floating point underflow.
(0.1.15: 10 April 2011, r949:203)
Beta release 0.1.14 (21 March, 2011)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This release implements a method for testing associations for case-control
data. The method does not call genotypes but instead sums over all genotype
configurations to compute a chi^2 based test statistics. It can be potentially
applied to comparing a pair of samples (e.g. a tumor-normal pair), but this
has not been evaluated on real data.
Another new feature is to make X chromosome variant calls when female and male
samples are both present. The user needs to provide a file indicating the
ploidy of each sample (see also manual bcftools/bcftools.1).
Other notable changes:
* Added `bcftools view -F' to parse BCF files generated by samtools r921 or
older which encodes PL in a different way.
* Changed the behavior of `bcftools view -s'. Now when a list of samples is
provided, the samples in the output will be reordered to match the ordering
in the sample list. This change is mainly designed for association test.
* Sped up `bcftools view -v' for target sequencing given thousands of samples.
Also added a new option `view -d' to skip loci where only a few samples are
covered by reads.
* Dropped HWE test. This feature has never been implemented properly. An EM
should be much better. To be implemented in future.
* Added the `cat' command to samtools. This command concatenate BAMs with
identical sequence dictionaries in an efficient way. Modified from bam_cat.c
written by Chris Saunders.
* Added `samtools view -1' to write BAMs at a low compression level but twice
faster to create. The `sort' command generates temporary files at a low
compression level as well.
* Added `samtools mpileup -6' to accept "BAM" with Illumina 1.3+ quality
strings (strictly speaking, such a file is not BAM).
* Added `samtools mpileup -L' to skip INDEL calling in regions with
excessively high coverage. Such regions dramatically slow down mpileup.
* Updated `misc/export2sam.pl', provided by Chris Saunders from Illumina Inc.
(0.1.14: 21 March 2011, r933:170)
Beta release 0.1.13 (1 March, 2011)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The most important though largely invisible modification is the change of the
order of genotypes in the PL VCF/BCF tag. This is to conform the upcoming VCF
spec v4.1. The change means that 0.1.13 is not backward compatible with VCF/BCF
generated by samtools older than r921 inclusive. VCF/BCF generated by the new
samtools will contain a line `##fileformat=VCFv4.1' as well as the samtools
version number.
Single Individual Haplotyping (SIH) is added as an experimental feature. It
originally aims to produce haploid consensus from fosmid pool sequencing, but
also works with short-read data. For short reads, phased blocks are usually too
short to be useful in many applications, but they can help to rule out part of
SNPs close to INDELs or between copies of CNVs.
Other notable changes in samtools:
* Construct per-sample consensus to reduce the effect of nearby SNPs in INDEL
calling. This reduces the power but improves specificity.
* Improved sorting order checking in indexing. Now indexing is the preferred way
to check if a BAM is sorted.
* Added a switch `-E' to mpileup and calmd. This option uses an alternative way
to apply BAQ, which increases sensistivity, especially to MNPs, at the cost of
a little loss in specificity.
* Added `mpileup -A' to allow to use reads in anomalous pairs in SNP calling.
* Added `mpileup -m' to allow fine control of the collection of INDEL candidates.
* Added `mpileup -S' to compute per-sample strand bias P-value.
* Added `mpileup -G' to exclude read groups in variant calling.
* Fixed segfault in indel calling related to unmapped and refskip reads.
* Fixed an integer overflow in INDEL calling. This bug produces wrong INDEL
genotypes for longer short INDELs, typically over 10bp.
* Fixed a bug in tview on big-endian machines.
* Fixed a very rare memory issue in bam_md.c
* Fixed an out-of-boundary bug in mpileup when the read base is `N'.
* Fixed a compiling error when the knetfile library is not used. Fixed a
library compiling error due to the lack of bam_nt16_nt4_table[] table.
Suppress a compiling warning related to the latest zlib.
Other notable changes in bcftools:
* Updated the BCF spec.
* Added the `FQ' VCF INFO field, which gives the phred-scaled probability
of all samples being the same (identical to the reference or all homozygous
variants). Option `view -f' has been dropped.
* Implementated of "vcfutils.pl vcf2fq" to generate a consensus sequence
similar to "samtools.pl pileup2fq".
* Make sure the GT FORMAT field is always the first FORMAT to conform the VCF
spec. Drop bcf-fix.pl.
* Output bcftools specific INFO and FORMAT in the VCF header.
* Added `view -s' to call variants from a subset of samples.
* Properly convert VCF to BCF with a user provided sequence dictionary. Nonetheless,
custom fields are still unparsed and will be stored as a missing value.
* Fixed a minor bug in Fisher's exact test; the results are rarely changed.
(0.1.13: 1 March 2011, r926:134)
Beta release 0.1.12a (2 December, 2010)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is another bug fix release:
* Fixed a memory violation in mpileup, which causes segfault. Release
0.1.9 and above are affected.
* Fixed a memory violation in the indel caller, which does not causes
segfault, but may potentially affect deletion calls in an unexpected
way. Release 0.1.10 and above are affected.
* Fixed a bug in computing r-square in bcftools. Few are using this
functionality and it only has minor effect.
* Fixed a memory leak in bam_fetch().
* Fixed a bug in writing meta information to the BAM index for the last
sequence. This bug is invisible to most users, but it is a bug anyway.
* Fixed a bug in bcftools which causes false "DP4=0,0,0,0" annotations.
(0.1.12: 2 December 2010, r862)
Beta release 0.1.11 (21 November, 2010)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is mainly a bug fix release:
* Fixed a bug in random retrieval (since 0.1.8). It occurs when reads
are retrieved from a small region containing no reads.
* Fixed a bug in pileup (since 0.1.9). The bug causes an assertion
failure when the first CIGAR operation is a deletion.
* Improved fault tolerence in remote access.
One minor feature has been implemented in bcftools:
* Added a reference-free variant calling mode. In this mode, a site is
regarded as a variat iff the sample(s) contains two or more alleles;
the meaning of the QUAL field in the VCF output is changed
accordingly. Effectively, the reference allele is irrelevant to the
result in the new mode, although the reference sequence has to be
used in realignment when SAMtools computes genotype likelihoods.
In addition, since 0.1.10, the `pileup' command has been deprecated by
`mpileup' which is more powerful and more accurate. The `pileup' command
will not be removed in the next few releases, but new features will not
be added.
(0.1.11: 21 November 2010, r851)
Beta Release 0.1.10 (16 November, 2010)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This release is featured as the first major improvement to the indel
caller. The method is similar to the old one implemented in the pileup
command, but the details are handled more carefully both in theory and
in practice. As a result, the new indel caller usually gives more
accurate indel calls, though at the cost of sensitivity. The caller is
implemented in the mpileup command and is invoked by default. It works
with multiple samples.
Other notable changes:
* With the -r option, the calmd command writes the difference between
the original base quality and the BAQ capped base quality at the BQ
tag but does not modify the base quality. Please use -Ar to overwrite
the original base quality (the 0.1.9 behavior).
* Allow to set a maximum per-sample read depth to reduce memory. In
0.1.9, most of memory is wasted for the ultra high read depth in some
regions (e.g. the chr1 centromere).
* Optionally write per-sample read depth and per-sample strand bias
P-value.
* Compute equal-tail (Bayesian) credible interval of site allele
frequency at the CI95 VCF annotation.
* Merged the vcfutils.pl varFilter and filter4vcf for better SNP/indel
filtering.
(0.1.10: 16 November 2010, r829)
Beta Release 0.1.9 (27 October, 2010)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This release is featured as the first major improvement to the samtools'
SNP caller. It comes with a revised MAQ error model, the support of
multi-sample SNP calling and the computation of base alignment quality
(BAQ).
The revised MAQ error model is based on the original model. It solves an
issue of miscalling SNPs in repetitive regions. Althought such SNPs can
usually be filtered at a later step, they mess up unfiltered calls. This
is a theoretical flaw in the original model. The revised MAQ model
deprecates the orginal MAQ model and the simplified SOAPsnp model.
Multi-sample SNP calling is separated in two steps. The first is done by
samtools mpileup and the second by a new program, bcftools, which is
included in the samtools source code tree. Multi-sample SNP calling also
works for single sample and has the advantage of enabling more powerful
filtration. It is likely to deprecate pileup in future once a proper
indel calling method is implemented.
BAQ is the Phred-scaled probability of a read base being wrongly
aligned. Capping base quality by BAQ has been shown to be very effective
in suppressing false SNPs caused by misalignments around indels or in
low-complexity regions with acceptable compromise on computation
time. This strategy is highly recommended and can be used with other SNP
callers as well.
In addition to the three major improvements, other notable changes are:
* Changes to the pileup format. A reference skip (the N CIGAR operator)
is shown as '<' or '>' depending on the strand. Tview is also changed
accordingly.
* Accelerated pileup. The plain pileup is about 50% faster.
* Regional merge. The merge command now accepts a new option to merge
files in a specified region.
* Fixed a bug in bgzip and razip which causes source files to be
deleted even if option -c is applied.
* In APIs, propogate errors to downstream callers and make samtools
return non-zero values once errors occur.
(0.1.9: 27 October 2010, r783)
Beta Release 0.1.8 (11 July, 2010)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Notable functional changes:
* Added the `reheader' command which replaces a BAM header with a new
header. This command is much faster than replacing header by
BAM->SAM->BAM conversions.
* Added the `mpileup' command which computes the pileup of multiple
alignments.
* The `index' command now stores the number of mapped and unmapped
reads in the index file. This information can be retrieved quickly by
the new `idxstats' command.
* By default, pileup used the SOAPsnp model for SNP calling. This
avoids the floating overflow in the MAQ model which leads to spurious
calls in repetitive regions, although these calls will be immediately
filtered by varFilter.
* The `tview' command now correctly handles CIGARs like 7I10M and
10M1P1I10M which cause assertion failure in earlier versions.
* Tview accepts a region like `=10,000' where `=' stands for the
current sequence name. This saves typing for long sequence names.
* Added the `-d' option to `pileup' which avoids slow indel calling
in ultradeep regions by subsampling reads locally.
* Added the `-R' option to `view' which retrieves alignments in read
groups listed in the specified file.
Performance improvements:
* The BAM->SAM conversion is up to twice faster, depending on the
characteristic of the input.
* Parsing SAM headers with a lot of reference sequences is now much
faster.
* The number of lseek() calls per query is reduced when the query
region contains no read alignments.
Bug fixes:
* Fixed an issue in the indel caller that leads to miscall of indels.
Note that this solution may not work well when the sequencing indel
error rate is higher than the rate of SNPs.
* Fixed another issue in the indel caller which may lead to incorrect
genotype.
* Fixed a bug in `sort' when option `-o' is applied.
* Fixed a bug in `view -r'.
APIs and other changes:
* Added iterator interfaces to random access and pileup. The callback
interfaces directly call the iterator interfaces.
* The BGZF blocks holding the BAM header are indepedent of alignment
BGZF blocks. Alignment records shorter than 64kB is guaranteed to be
fully contained in one BGZF block. This change is fully compatible
with the old version of samtools/picard.
Changes in other utilities:
* Updated export2sam.pl by Chris Saunders.
* Improved the sam2vcf.pl script.
* Added a Python version of varfilter.py by Aylwyn Scally.
(0.1.8: 11 July 2010, r613)