Formal changes: tests, documentation, copyright

Add zstd to the test suite - add zstd to history_002_pos.ksh - add random levels of zstd to history_002_pos.ksh - add zstd-fast to history_002_pos.ksh - add random levels of zstd-fast to history_002_pos.ksh Add documentation - add man page content - add README for contrib/zstd Fixup copyright headers of touched and new files - Adds copyright headers for Allan Jude / Klara Inc. - Added Ornias1993 Copyright - Cleans copyright header formatting Co-authored-by: Allan Jude <[email protected]> Co-authored-by: Sebastian Gottschall <[email protected]> Co-authored-by: Kjeld Schouten-Lebbing <[email protected]> Co-authored-by: Michael Niewöhner <[email protected]> Signed-off-by: Allan Jude <[email protected]> Signed-off-by: Sebastian Gottschall <[email protected]> Signed-off-by: Kjeld Schouten-Lebbing <[email protected]> Signed-off-by: Michael Niewöhner <[email protected]>
openzfs · May 2, 2020 · 8f71e1e · 8f71e1e
1 parent db154a3
commit 8f71e1e
Show file tree

Hide file tree

Showing 23 changed files with 256 additions and 117 deletions.
diff --git a/contrib/zstd/README.md b/contrib/zstd/README.md
@@ -0,0 +1,24 @@
+# ZSTD Contrib Library Manual
+
+## Introduction
+
+This `contrib` contains the ZSTD library used in ZFS. It is heavily cut-down by
+dropping any unneeded files but otherwise is intentionally unmodified. Please do
+not alter these files in any way, besides upgrading to a newer ZSTD release.
+
+## Updating ZSTD
+
+To update ZSTD the following steps need to be taken:
+
+1. Grab the latest release of [ZSTD](https://github.com/facebook/zstd/releases).
+2. Replace (not merge) `common`, `compress`, `decompress` and `zstd.h` with the
+   new versions from `lib/`.
+3. Make sure any newly required files and/or folders are also included.
+   1. Add an empty file `Makefile.am` inside any new folder.
+   2. Add them to `AC_CONFIG_FILES` in `configure.ac` accordingly.
+   3. Make sure new files/folders are listed in
+      - `contrib/zstd/Makefile.am`
+      - `lib/libzstd/Makefile.am`
+      - `module/zstd/Makefile.in`
+      - this README
+4. Update the version `ZFS_MODULE_VERSION("x.y.z")` in `module/zstd/zstd.c`.
diff --git a/include/sys/arc.h b/include/sys/arc.h
@@ -18,10 +18,14 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2012, 2016 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2012, 2016, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #ifndef	_SYS_ARC_H

diff --git a/include/sys/spa.h b/include/sys/spa.h
@@ -18,16 +18,20 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2011, 2019 by Delphix. All rights reserved.
- * Copyright 2011 Nexenta Systems, Inc.  All rights reserved.
- * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved.
- * Copyright 2013 Saso Kiselkov. All rights reserved.
- * Copyright (c) 2014 Integros [integros.com]
- * Copyright 2017 Joyent, Inc.
- * Copyright (c) 2017, 2019, Datto Inc. All rights reserved.
- * Copyright (c) 2017, Intel Corporation.
+ * Copyright (c) 2011, 2019, Delphix. All rights reserved.
+ * Copyright (c) 2011, Nexenta Systems Inc. All rights reserved.
+ * Copyright (c) 2014, Spectra Logic Corporation. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2014, Integros. All rights reserved.
+ * Copyright (c) 2017, Joyent, Inc. All rights reserved.
+ * Copyright (c) 2017, Datto Inc. All rights reserved.
+ * Copyright (c) 2017, Intel Corporation. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #ifndef _SYS_SPA_H

diff --git a/include/sys/zio.h b/include/sys/zio.h
@@ -21,11 +21,14 @@
 
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright 2011 Nexenta Systems, Inc. All rights reserved.
- * Copyright (c) 2012, 2018 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2011, Nexenta Systems, Inc. All rights reserved.
+ * Copyright (c) 2012, 2018, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
  * Copyright (c) 2013, Joyent, Inc. All rights reserved.
- * Copyright 2016 Toomas Soome <[email protected]>
+ * Copyright (c) 2016, Toomas Soome. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #ifndef _ZIO_H

diff --git a/include/sys/zio_compress.h b/include/sys/zio_compress.h
@@ -20,9 +20,11 @@
  */
 
 /*
- * Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
+ * Copyright (c) 2009, Sun Microsystems Inc. All rights reserved.
+ * Copyright (c) 2015, 2016, Delphix. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
  * Use is subject to license terms.
- * Copyright (c) 2015, 2016 by Delphix. All rights reserved.
  */
 
 #ifndef _SYS_ZIO_COMPRESS_H

diff --git a/man/man5/zpool-features.5 b/man/man5/zpool-features.5
@@ -1,7 +1,4 @@
 '\" te
-.\" Copyright (c) 2012, 2018 by Delphix. All rights reserved.
-.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
-.\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
 .\" The contents of this file are subject to the terms of the Common Development
 .\" and Distribution License (the "License").  You may not use this file except
 .\" in compliance with the License. You can obtain a copy of the license at
@@ -14,6 +11,11 @@
 .\" CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your
 .\" own identifying information:
 .\" Portions Copyright [yyyy] [name of copyright owner]
+.\" Copyright (c) 2012, 2018, Delphix. All rights reserved.
+.\" Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+.\" Copyright (c) 2014, Joyent Inc. All rights reserved.
+.\" Copyright (c) 2019, Klara Inc. All rights reserved.
+.\" Copyright (c) 2019, Allan Jude. All rights reserved.
 .TH ZPOOL-FEATURES 5 "Jun 8, 2018"
 .SH NAME
 zpool\-features \- ZFS pool feature descriptions
@@ -926,6 +928,42 @@ This feature becomes \fBactive\fR when the \fBzpool checkpoint\fR subcommand
 is used to checkpoint the pool.
 The feature will only return back to being \fBenabled\fR when the pool
 is rewound or the checkpoint has been discarded.
+.RE
+
+.sp
+.ne 2
+.na
+\fBzstd_compress\fR
+.ad
+.RS 4n
+.TS
+l l .
+GUID	org.freebsd:zstd_compress
+READ\-ONLY COMPATIBLE	no
+DEPENDENCIES	none
+.TE
+
+\fBzstd\fR is a high-performance compression algorithm that features a
+combination of high compression ratios and high speed. Compared to \fBgzip\fR,
+\fBzstd\fR offers slighty better compression at much higher speeds. Compared
+to \fBlz4\fR, \fBzstd\fR offers much better compression while being only
+modestly slower. Typically, \fBzstd\fR compression speed ranges from 250 to 500
+MB/s per thread and decompression speed is over 1 GB/s per thread.
+
+When the \fBzstd\fR feature is set to \fBenabled\fR, the administrator can turn
+on \fBzstd\fR compression of any dataset by running
+`zfs set compress=zstd <pool/fs>`.
+
+This feature becomes \fBactive\fR once a \fBcompress\fR property has been set to
+\fBzstd\fR, and will return to being \fBenabled\fR once all filesystems that
+have ever had their compress property set to \fBzstd\fR are destroyed.
+
+Booting off of \fBzstd\fR-compressed root pools is not yet supported.
+
+Sending a \fBzstd\fR dataset to a non-\fBzstd zpool\fR by \fBzfs send -e\fR is
+not possible and will result in an error. Either drop the \fB-e\fR option or
+enable \fBzstd\fR on the receiver \fBzpool\fR.
+.RE
 
 .SH "SEE ALSO"
 zpool(8)
diff --git a/man/man8/zfsprops.8 b/man/man8/zfsprops.8
@@ -19,23 +19,22 @@
 .\" CDDL HEADER END
 .\"
 .\"
-.\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved.
-.\" Copyright 2011 Joshua M. Clulow <[email protected]>
-.\" Copyright (c) 2011, 2019 by Delphix. All rights reserved.
-.\" Copyright (c) 2011, Pawel Jakub Dawidek <[email protected]>
-.\" Copyright (c) 2012, Glen Barber <[email protected]>
-.\" Copyright (c) 2012, Bryan Drewery <[email protected]>
-.\" Copyright (c) 2013, Steven Hartland <[email protected]>
-.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
-.\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
-.\" Copyright (c) 2014 by Adam Stevko. All rights reserved.
-.\" Copyright (c) 2014 Integros [integros.com]
-.\" Copyright (c) 2016 Nexenta Systems, Inc. All Rights Reserved.
-.\" Copyright (c) 2014, Xin LI <[email protected]>
-.\" Copyright (c) 2014-2015, The FreeBSD Foundation, All Rights Reserved.
-.\" Copyright 2019 Richard Laager. All rights reserved.
-.\" Copyright 2018 Nexenta Systems, Inc.
-.\" Copyright 2019 Joyent, Inc.
+.\" Copyright (c) 2009, Sun Microsystems, Inc. All Rights Reserved.
+.\" Copyright (c) 2011, Joshua M. Clulow. All Rights Reserved.
+.\" Copyright (c) 2011, 2019, Delphix. All rights reserved.
+.\" Copyright (c) 2011, Pawel Jakub Dawidek. All rights reserved.
+.\" Copyright (c) 2012, Glen Barber. All rights reserved.
+.\" Copyright (c) 2012, Bryan Drewery. All rights reserved.
+.\" Copyright (c) 2013, Steven Hartland. All rights reserved.
+.\" Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+.\" Copyright (c) 2014, 2019, Joyent, Inc. All rights reserved.
+.\" Copyright (c) 2014, Adam Stevko. All rights reserved.
+.\" Copyright (c) 2014, Integros. All rights reserved.
+.\" Copyright (c) 2014, Xin LI. All rights reserved.
+.\" Copyright (c) 2014-2015, The FreeBSD Foundation. All Rights Reserved.
+.\" Copyright (c) 2016, 2018, Nexenta Systems, Inc. All rights reserved.
+.\" Copyright (c) 2019, Richard Laager. All rights reserved.
+.\" Copyright (c) 2019, Kjeld Schouten-Lebbing. All rights reserved.
 .\"
 .Dd January 30, 2020
 .Dt ZFSPROPS 8
@@ -774,7 +773,8 @@ for more information on these algorithms.
 Changing this property affects only newly-written data.
 .It Xo
 .Sy compression Ns = Ns Sy on Ns | Ns Sy off Ns | Ns Sy gzip Ns | Ns
-.Sy gzip- Ns Em N Ns | Ns Sy lz4 Ns | Ns Sy lzjb Ns | Ns Sy zle
+.Sy gzip- Ns Em N Ns | Ns Sy lz4 Ns | Ns Sy lzjb Ns | Ns Sy zle Ns | Ns Sy zstd Ns | Ns
+.Sy zstd- Ns Em N Ns | Ns Sy zstd-fast- Ns Em N
 .Xc
 Controls the compression algorithm used for this dataset.
 .Pp
@@ -842,6 +842,35 @@ is equivalent to
 .Pc .
 .Pp
 The
+.Sy zstd
+compression using the zstd algorithm ,
+You can specify the
+.Sy zstd
+level by using the value
+.Sy zstd- Ns Em N ,
+where
+.Em N
+is an integer from 1
+.Pq fastest
+to 19
+.Pq best compression ratio .
+.Sy zstd
+You can also specify a negative
+.Sy zstd
+level by using the value
+.Sy zstd- Ns Em N ,
+where
+.Em N
+is an integer from 100
+.Pq fastest
+to 1
+.Pq best compression ratio .
+.Sy zstd
+is equivalent to
+.Sy zstd-3
+.Pc .
+.Pp
+The
 .Sy zle
 compression algorithm compresses runs of zeros.
 .Pp

diff --git a/module/zcommon/zfeature_common.c b/module/zcommon/zfeature_common.c
@@ -20,11 +20,14 @@
  */
 
 /*
- * Copyright (c) 2011, 2018 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
- * Copyright (c) 2013, Joyent, Inc. All rights reserved.
- * Copyright (c) 2014, Nexenta Systems, Inc. All rights reserved.
- * Copyright (c) 2017, Intel Corporation.
+ * Copyright (c) 2011, 2018, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2013, Joyent Inc. All rights reserved.
+ * Copyright (c) 2014, Nexenta Systems Inc. All rights reserved.
+ * Copyright (c) 2017, Intel Corporation. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #ifndef _KERNEL

diff --git a/module/zcommon/zfs_prop.c b/module/zcommon/zfs_prop.c
@@ -18,16 +18,17 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2011, 2018 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
- * Copyright 2016, Joyent, Inc.
- * Portions Copyright (c) 2019 by Klara Inc.
+ * Copyright (c) 2010, Robert Milkowski. All rights reserved.
+ * Copyright (c) 2011, 2018, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2016, Joyent, Inc. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Use is subject to license terms.
  */
 
-/* Portions Copyright 2010 Robert Milkowski */
-
 #include <sys/zio.h>
 #include <sys/spa.h>
 #include <sys/u8_textprep.h>

diff --git a/module/zfs/arc.c b/module/zfs/arc.c
@@ -18,14 +18,15 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2018, Joyent, Inc.
+ * Copyright (c) 2018, Joyent Inc. All rights reserved.
  * Copyright (c) 2011, 2019, Delphix. All rights reserved.
  * Copyright (c) 2014, Saso Kiselkov. All rights reserved.
- * Copyright (c) 2017, Nexenta Systems, Inc.  All rights reserved.
+ * Copyright (c) 2017, Nexenta Systems Inc. All rights reserved.
  * Copyright (c) 2019, loli10K <[email protected]>. All rights reserved.
- * Portions Copyright (c) 2019 by Klara Inc.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
  * Copyright (c) 2020, George Amanakis. All rights reserved.
  */
 

diff --git a/module/zfs/dbuf.c b/module/zfs/dbuf.c
@@ -18,12 +18,16 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright 2011 Nexenta Systems, Inc.  All rights reserved.
- * Copyright (c) 2012, 2019 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
- * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved.
+ * Copyright (c) 2011, Nexenta Systems Inc. All rights reserved.
+ * Copyright (c) 2012, 2019, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2014, Spectra Logic Corporation. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #include <sys/zfs_context.h>

diff --git a/module/zfs/dmu.c b/module/zfs/dmu.c
@@ -18,14 +18,18 @@
  *
  * CDDL HEADER END
  */
+
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2011, 2020 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2011, 2020, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
  * Copyright (c) 2013, Joyent, Inc. All rights reserved.
  * Copyright (c) 2016, Nexenta Systems, Inc. All rights reserved.
- * Copyright (c) 2015 by Chunwei Chen. All rights reserved.
- * Copyright (c) 2019 Datto Inc.
+ * Copyright (c) 2015, Chunwei Chen. All rights reserved.
+ * Copyright (c) 2019, Datto Inc. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Use is subject to license terms.
  */
 
 #include <sys/dmu.h>

diff --git a/module/zfs/dmu_objset.c b/module/zfs/dmu_objset.c
@@ -21,19 +21,21 @@
 
 /*
  * Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
- * Copyright (c) 2012, 2018 by Delphix. All rights reserved.
- * Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
- * Copyright (c) 2013, Joyent, Inc. All rights reserved.
- * Copyright (c) 2014 Spectra Logic Corporation, All rights reserved.
- * Copyright (c) 2015, STRATO AG, Inc. All rights reserved.
- * Copyright (c) 2016 Actifio, Inc. All rights reserved.
- * Copyright 2017 Nexenta Systems, Inc.
- * Copyright (c) 2017 Open-E, Inc. All Rights Reserved.
+ * Copyright (c) 2010, Robert Milkowski. All rights reserved.
+ * Copyright (c) 2012, 2018, Delphix. All rights reserved.
+ * Copyright (c) 2013, Saso Kiselkov. All rights reserved.
+ * Copyright (c) 2013, Joyent Inc. All rights reserved.
+ * Copyright (c) 2014, Spectra Logic Corporation. All rights reserved.
+ * Copyright (c) 2015, STRATO AG. All rights reserved.
+ * Copyright (c) 2016, Actifio Inc. All rights reserved.
+ * Copyright (c) 2017, Nexenta Systems Inc. All Rights Reserved.
+ * Copyright (c) 2017, Open-E Inc. All Rights Reserved.
  * Copyright (c) 2018, loli10K <[email protected]>. All rights reserved.
+ * Copyright (c) 2019, Klara Inc. All rights reserved.
+ * Copyright (c) 2019, Allan Jude. All rights reserved.
+ * Use is subject to license terms.
  */
 
-/* Portions Copyright 2010 Robert Milkowski */
-
 #include <sys/zfeature.h>
 #include <sys/cred.h>
 #include <sys/zfs_context.h>