Zabbix 3.0

.gitignore

@@ -0,0 +1 @@
+/.project

LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 Jens Berthold
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,99 @@
+# Backup script for Zabbix configuration data (MySQL/PostgreSQL)
+
+This is a MySQL/PostgreSQL database backup script for the [Zabbix](http://www.zabbix.com/) monitoring software from version 1.3.1 up to 4.4.
+
+## Download
+
+Download the latest (stable) release here:
+
+https://github.com/maxhq/zabbix-backup/releases/latest
+
+## More informations
+
+Please see the [Project Wiki](https://github.com/maxhq/zabbix-backup/wiki).
+
+## Version history
+
+**0.9.3 (2020-01-17)**
+
+- ENH: Check for unknown tables
+- ENH: Speed up MySQL backup by not calling mysqldump for every single table anymore
+- ENH: New option -S to specify PostgreSQL schema
+- FIX: Stabilize and enhance PostgreSQL dump
+- FIX: Skip IP reverse lookup for localhost, fix multiline dig answers
+
+**0.9.2 (2020-01-16)**
+
+- ENH: Support for Zabbix 4.4
+- ENH: Fix (non-critical) shellcheck issues (Mario Trangoni)
+- CHG: Fix and enhance helper script get-table-list.pl
+- FIX: Escape special characters while reading password (ironbishop)
+- FIX: Re-enable accidentally disabled cleanup of postgresql password file
+- FIX: Insert hostname into backup file also if database resides on localhost
+
+**0.9.1 (2019-03-21)**
+
+- FIX: Correctly process hostname option -H (Tiago Cruz)
+
+**0.9.0 (2019-03-14)**
+
+- NEW: Support for PostgreSQL databases (Sergey Galkin)
+- NEW: Option -P to specify database server port (Sergey Galkin)
+- NEW: Support for socket connections to MySQL and PostgreSQL server (Greg Cockburn)
+- NEW: Database connection parameters are read from Zabbix servern config by default (Greg Cockburn)
+- ENH: Support for Zabbix 4.0 (Wesley Schaft)
+- ENH: Options -h and --help to show help (hostname is now specified using -H)
+- ENH: Options -Z to skip reading the Zabbix server config file
+- CHG: Rename script to "zabbix-dump"
+- FIX: Support for whitespaces in database parameters (Greg Cockburn)
+- FIX: Support for backslashes in manually entered password (Greg Cockburn)
+
+**0.8.2 (2016-09-08)**
+
+- NEW: Option -x to use XZ instead of GZ for compression (Jonathan Wright)
+- NEW: Option -0 for "no compression"
+- FIX: Evil space was masking end of here-document (fixed in #8 by @msjmeyer)
+- FIX: Prevent "Warning: Using a password on the command line interface can be insecure."
+
+**0.8.1 (2016-07-11)**
+
+- ENH: Added Zabbix 3.0.x tables to list (added & tested by Ruslan Ohitin)
+
+**0.8.0 (2016-01-22)**
+
+- FIX: Only invoke `dig` if available
+- ENH: Option -c to use a MySQL config ("options") file (suggested by Daniel Schneller)
+- ENH: Option -r to rotate backup files (Daniel Schneller)
+- ENH: Add database version to filename if available
+- ENH: Add quiet mode. IP reverse lookup optional (Daniel Schneller)
+- ENH: Bash related fixes (Misu Moldovan)
+- CHG: Default output directory is now $PWD instead of script dir
+
+**0.7.1 (2015-01-27)**
+
+- NEW: Parsing of commandline arguments implemented
+- ENH: Try reverse lookup of IPs and include hostname/IP in filename
+- REV: Stop if database password is wrong
+
+**0.7.0 (2014-10-02)**
+
+- ENH: Complete overhaul to make script work with lots of Zabbix versions
+
+**0.6.0 (2014-09-15)**
+
+- REV: Updated the table list for use with zabbix v2.2.3
+
+**0.5.0 (2013-05-13)**
+
+- NEW: Added table list comparison between database and script
+
+**0.4.0 (2012-03-02)**
+
+- REV: Incorporated mysqldump options (suggested by Jonathan Bayer)
+
+**0.3.0 (2012-02-06)**
+
+- ENH: Backup of Zabbix 1.9.x / 2.0.0, removed unnecessary use of
+  variables (DATEBIN etc) for commands that use to be in $PATH
+
+**0.2.0 (2011-11-05)**

RELEASE.md

@@ -0,0 +1,49 @@
+## Releasing a new version
+
+### Before commit
+
+* Update README.md
+
+### After commit
+
+* Tag the release commit: `git tag v0.9.2 && git push --tags`
+* Add release at https://github.com/maxhq/zabbix-backup/releases
+
+  **Summary of main changes**
+
+  ```
+This release fixes ...
+
+### Direct download
+
+[zabbix-mysql-dump](https://raw.githubusercontent.com/maxhq/zabbix-backup/v0.9.2/zabbix-dump) (v0.9.2)
+
+### Changelog
+
+Please see the [version history in the README](https://github.com/maxhq/zabbix-backup/blob/v0.9.2/README.md#version-history).
+
+### Thanks!
+- @username
+  ```
+
+* Update https://github.com/maxhq/zabbix-backup/wiki
+* Update https://zabbix.org/wiki/Docs/howto/database_backup_script
+* Announce release at https://www.linkedin.com/groups/161448
+
+  > **New version x.x.x of zabbix-dump**
+  >
+  > zabbix-dump is a Linux bash script for backing up the Zabbix configuration by saving MySQL or PostgreSQL database tables into a compressed file.
+  > Tables holding configuration data will be fully backed up. For mass data tables (events, history, trends, ...) only the table schema is stored without any data (to keep the backup small).
+  >
+  > Overiew: https://github.com/maxhq/zabbix-backup/blob/master/README.md
+  > Latest release: https://github.com/maxhq/zabbix-backup/releases/latest
+
+* Announce release at https://www.xing.com/communities/forums/100845147
+
+  > **Neue Version x.x.x von zabbix-dump**
+  >
+  > zabbix-dump ist ein Linux-Bash-Skript zum Backup der Zabbix-Konfiguration durch Sicherung der MySQL- bzw. PostgreSQL-Datenbanktabellen in eine komprimierte Datei.
+  > Es sichert Konfigurationsdaten komplett, bei Tabellen mit Massendaten (Historie, Events, Trends etc.) jedoch nur das "leere" Datenbankschema (um das Backup zu minimieren).
+  >
+  > Übersicht: https://github.com/maxhq/zabbix-backup/blob/master/README.md
+  > Neueste Version: https://github.com/maxhq/zabbix-backup/releases/latest

get-table-list.pl

@@ -0,0 +1,167 @@
+#!/usr/bin/env perl
+# NAME
+#     get-table-list.pl - List current and historic Zabbix database tables
+#
+# SYNOPSIS
+#     This is mainly a helper script for developing the backup script.
+#
+#     It connects to svn://svn.zabbix.com (using Subversion client "svn") and
+#     fetches the schema definitions of all tagged Zabbix versions beginning
+#     from 1.3.1 (this takes a while).
+#
+#     It then prints out a list of all tables together with the first and last
+#     Zabbix version where they were used.
+#
+# HISTORY
+#     v0.1 - 2014-09-19 First version
+#
+# AUTHOR
+#     Jens Berthold (maxhq), 2020
+use strict;
+use warnings;
+
+use version;
+
+my $REPO = 'https://git.zabbix.com/scm/zbx/zabbix.git';
+my $REPO_WEB = 'https://git.zabbix.com/projects/ZBX/repos/zabbix/raw';
+
+sub stop {
+	my ($msg) = @_;
+	print "ERROR: $msg\n";
+	exit;
+}
+
+# sort version numbers correctly
+sub cmpver {
+	my ($a, $b) = @_;
+
+	# split version parts: 1.2.3rc1 --> 1  2  3rc1
+	my @a_parts = split /\./, $a;
+	my @b_parts = split /\./, $b;
+
+	for (my $i=0; $i<scalar(@a_parts); $i++) {
+		return 1 if $i >= scalar(@b_parts);
+		# split number parts: 3rc1 --> 3  rc  1
+		my ($a_num, $a_type, $a_idx) = $a_parts[$i] =~ m/^(\d+)(\D+)?(\d+)?$/;
+		my ($b_num, $b_type, $b_idx) = $b_parts[$i] =~ m/^(\d+)(\D+)?(\d+)?$/;
+		my $cmp;
+		# 3 before 4
+		$cmp = $a_num <=> $b_num;   return $cmp unless $cmp == 0;
+		# 3rc1 before 3
+		return -1 if     $a_type and not $b_type;
+		return  1 if not $a_type and     $b_type;
+		# a1 before b1
+		$cmp = ($a_type//"") cmp ($b_type//""); return $cmp unless $cmp == 0;
+		# rc1 before rc2
+		$cmp = ($a_idx//0) <=> ($b_idx//0);   return $cmp unless $cmp == 0;
+	}
+	# 1.2 before 1.2.1
+	return -1 if scalar(@a_parts) < scalar(@b_parts);
+	# equal
+	return 0;
+}
+
+# Get tag list from repo:
+sub get_taglist {
+	print "Querying existing tags from $REPO...\n";
+
+	# Returned format:
+	#	7f6b20903537b9bbf72fe2b75ab7fac557856aad	refs/tags/1.0
+	#	693709cc4a80777f7759856c853b38cbc920f068	refs/tags/1.1
+	my @tags_raw = `git ls-remote -t $REPO`;
+	# remove trailing newline
+	chomp @tags_raw;
+	# skip release candidates, betas and tags like "zabicom-xxx"
+	@tags_raw = grep { m{ refs/tags/ \d+ \. \d+ ( \. \d+ )? $}x } @tags_raw;
+
+	# Create HashRef:
+	#   1.0 => 7f6b20903537b9bbf72fe2b75ab7fac557856aad
+	#   1.1 => 693709cc4a80777f7759856c853b38cbc920f068
+	return {
+		map { m{^ (\w+) \s+ refs/tags/ (.*) $}x; $2 => $1 } @tags_raw
+	};
+}
+
+# Read old table informations from zabbix-dump
+sub get_old_tabinfo {
+	my $tabinfo_old = {};
+
+	open my $fh, '<', './zabbix-dump' or stop("Couldn't find 'zabbix-dump': $!");
+
+	my $within_data_section = 0;
+	while (<$fh>) {
+	    chomp;
+	    if (/^__DATA__/) { $within_data_section = 1; next }
+	    next unless $within_data_section;
+	    my ($table, $from, undef, $to, $mode) = split /\s+/;
+
+	    $tabinfo_old->{$table} = {
+		    from => $from,
+	        to => $to,
+	        schema_only => ($mode//"") eq "SCHEMAONLY" ? 1 : 0,
+		};
+	}
+
+	return $tabinfo_old;
+}
+
+
+
+`which git` or stop("No Git client found");
+
+my $tabinfo_old = get_old_tabinfo(); # old data from zabbix_dump
+
+my $tags = get_taglist();
+my $tabinfo = {}; # for each table, create a list of Zabbix versions that know it
+
+print "Reading table schemas...\n";
+
+# Loop over tags and read table schema
+for my $tag (sort { cmpver($a,$b) } keys %$tags) {
+	next if cmpver($tag, "1.3.1") < 0; # before Zabbix 1.3.1, schema was stored as pure SQL
+
+	my ($schema, $subdir);
+
+	printf " - %-8s %s", $tag, "Looking for schema...";
+	# search in subdir /schema (<= 1.9.8) and /src for schema.(sql|tmpl)
+	for my $sub (qw(schema src)) {
+		# file list:
+		#	100644 blob 1f0a05eb826dfcdb26f9429ad30c720454374ca1	data.tmpl
+		#	100644 blob b98b5eecc62731508c09d9e76d7aed9d4eb201f2	schema.tmpl
+		my @files_raw = `curl -s $REPO_WEB/create/$sub?at=refs%2Ftags%2F$tag`;
+		next unless @files_raw; # directory not found?
+		chomp @files_raw; # remove trailing newline
+		my @files = map { /^ \d+ \s+ \w+ \s+ \w+ \s+ (.*) $/x; $1 } @files_raw;
+
+		($schema) = grep /^schema\.(sql|tmpl)/, @files;
+		$subdir = $sub;
+		last;
+	}
+	if (!$schema) {
+		print "\nNo schema found in tag $tag\n";
+		next;
+	}
+	print " Processing ($schema)... ";
+	my @table = `curl -s $REPO_WEB/create/$subdir/$schema?at=refs%2Ftags%2F$tag`;
+	for (@table) {
+		chomp;
+		next unless m/^TABLE/;
+		my (undef, $table) = split /\|/;
+		$tabinfo->{$table} //= [];
+		push @{$tabinfo->{$table}}, $tag;
+	}
+	print " Done\n";
+}
+
+#
+# Print out results
+#
+print "\n\n";
+print "TABLE                      FIRST USE  LAST USE  MODE\n";
+print "----------------------------------------------------\n";
+for my $tab (sort keys %$tabinfo) {
+	my $mode = $tabinfo_old->{$tab}
+		? ($tabinfo_old->{$tab}->{schema_only} ? '  SCHEMAONLY' : '')
+		: '  <-- NEW TABLE! Only store schema?';
+	printf "%-26s %-8s - %-8s%s\n", $tab, $tabinfo->{$tab}->[0], $tabinfo->{$tab}->[-1], $mode;
+}