dnsdbq.man

.\" Copyright (c) 2014-2017 by Farsight Security, Inc.
.\"
.\" Licensed under the Apache License, Version 2.0 (the "License");
.\" you may not use this file except in compliance with the License.
.\" You may obtain a copy of the License at
.\"
.\"  http://www.apache.org/licenses/LICENSE-2.0
.\"
.\" Unless required by applicable law or agreed to in writing, software
.\" distributed under the License is distributed on an "AS IS" BASIS,
.\" WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
.\" See the License for the specific language governing permissions and
.\" limitations under the License.
.\"
.Dd 2018-01-30
.Dt dnsdbq 1 DNSDB
.Os " "
.Sh NAME
.Nm dnsdbq
.Nd DNSDB query tool
.Sh SYNOPSIS
.Nm dnsdbq
.Op Fl dfmhjsShcI
.Op Fl A Ar timestamp
.Op Fl B Ar timestamp
.Op Fl b Ar bailiwick
.Op Fl i Ar ip
.Op Fl l Ar limit
.Op Fl L Ar output_limit
.Op Fl u Ar server_sys
.Op Fl n Ar name
.Op Fl k Ar sort_keys
.Op Fl p Ar output_type
.Op Fl R Ar raw_rdata
.Op Fl r Ar rdata
.Op Fl t Ar rrtype
.Op Fl J Ar input_file
.Sh DESCRIPTION
.Nm dnsdbq
constructs and issues queries to the Farsight DNSDB and displays responses. It
is commonly used as a production command line interface to the DNSDB API
server.
.Pp
DNSDB is a database that stores and indexes both the passive DNS data
available via Farsight Security's Security Information Exchange as well as the
authoritative DNS data that various zone operators make available. DNSDB makes
it easy to search for individual DNS RRsets and provides additional metadata
for search results such as first seen and last seen timestamps as well as the
DNS bailiwick associated with an RRset. DNSDB also has the ability to perform
inverse or rdata searches.
.Pp
You'll need to get an API key from Farsight to use
.Ic dnsdbq
with DNSDB.
.Pp
Farsight's passive DNS infrastructure performs a complex process
of "bailiwick reconstruction" where an RRset's position within the DNS
hierarchy is approximated. This serves two purposes:
.Bl -enum -offset indent
.It
Provide context of the location of a given DNS record within the DNS hierarchy
.It
Prevent "untrustworthy" records that are a result of intentional or
unintentional cache poisoning attempts from being replicated by downstream
consumers.
.El
.Pp
For example, given the fully qualified domain name
.Ic www.dachshund.example.com ,
valid bailiwicks would be
.Ic dachshund.example.com ,
.Ic example.com ,
or
.Ic com .
.Sh OPTIONS
.Bl -tag -width 3n
.It Fl A Ar timestamp
Specify a "time_first_after" timestamp. Only results first seen by the
Farsight sensor network on or after this date will be emitted. See below for
timestamp formats and examples.
.It Fl B Ar timestamp
Specify a "time_first_before" timestamp. Only results first seen by the
Farsight sensor network on or before this date will be emitted. See below for
timestamp formats and examples.
.It Fl b Ar bailiwick
specify bailiwick (only valid with
.Fl r
queries).  If bailiwick is specified, but rrtype is not specified, then a default rrtype of ANY will be used.
.It Fl c
by default, -A and -B (separately or together) will select partial overlaps of
database tuples and time search criteria. To match only complete overlaps, add
the -c ("completeness") command line option (this is also known as "strict"
mode).
.It Fl d
enable debug mode.
.It Fl f
specify batch lookup mode allowing one or more queries to be performed.
Queries will be read from standard input and are expected to be be in
one of the following formats:
.Bl -enum -offset indent
.It
RRSet query:
.Ic rrset/name/NAME[/RRTYPE[/BAILIWICK]]
.It
Rdata (name) query:
.Ic rdata/name/NAME[/RRTYPE]
.It
Rdata (IP address) query:
.Ic rdata/ip/ADDR[/PFXLEN]
.It
Rdata (raw) query:
.Ic rdata/raw/HEX-PAIRS[/RRTYPE]
.El
.Pp
This option cannot be mixed with
.Fl n ,
.Fl r ,
.Fl R ,
or
.Fl i .
An example of how to use
.Fl f
is below.
.It Fl h
emit usage and quit.
.It Fl I
request information from the API server concerning the API key itself, which
may include rate limit, query quota, query allowance, or privilege levels; the
output format and content is dependent on the server_sys argument (see -u) and
upon the -p argument.  -I -p json prints the raw info.  -I -p text prints
the information in a more understandable textual form, including converting
any epoch integer times into UTC formatted times.
.It Fl i Ar ip
specify rdata ip ("right-hand side") query.
The value is one of an IPv4 address, an IPv6 address, an IPv4 network with prefix length, an IPv4 address range,
or an IPv6 network with prefix length. If a network lookup is being performed,
the delimiter between network address and prefix length is a single comma (",")
character rather than the usual slash ("/") character to avoid clashing with
the HTTP URI path name separator.  Examples are below.
.It Fl J Ar input_file
opens input_file and reads newline-separated JSON objects therefrom, in
preference to -f (batch mode) or -r, -n, or -i (query mode). This can be used
to reprocess the output from a prior invocation which used -j (-p json).  If
inut_file is "-" then standard input (stdin) willl be read.
.It Fl j
specify newline delimited json output mode.
.It Fl k Ar sort_keys
when sorting with -s or -S, selects one or more comma separated sort keys,
among "first", "last", "count", "name", and/or "data".
If sorting was specified, but this -k option is not specified, then will default
to sorting by first, last, count.
.It Fl l Ar limit
query for that limit's number of responses. If specified as 0 then the DNSDB
API server will return the maximum limit of results allowed.  If
.Fl l ,
is not specified, then the query will not specify a limit, and the DNSDB API
server may use its default limit.
.It Fl L Ar output_limit
clamps the number of responses output to
.Ic output_limit .
The
.Fl L
output limit defaults to the
.Fl l
limit's value.
.It Fl m
used only with
.Fl f ,
this causes all output to be merged rather than serialized, and causes up
to ten (10) API jobs to execute in parallel.
.It Fl n Ar name
specify
.Ic rdata
name ("right-hand side") query.  The value is a DNS domain name in
presentation format, or a left-hand (".example.com") or right-hand
("www.example.") wildcard domain name. Note that left-hand wildcard queries
are somewhat more expensive than right-hand wildcard queries.
.It Fl p Ar output_type
select output type. Specify
.Ic csv
for comma separated value output,
.Ic dns
for presentation output similar to that of dig(1), or
.Ic json
for newline delimited json output.
.It Fl R Ar raw-data
specify raw
.Ic rdata
data ("right-hand side") query.  The value is an even number of
hexadecimal digits specifying a raw octet string.
.It Fl r Ar rdata
specify rrset ("left-hand side") query.
.It Fl s
sort output in ascending key order.
.It Fl S
sort output in descending key order.
.It Fl t Ar rrtype
specify the resource record type desired.  Valid values include those
defined in DNS RFCs, including ANY.  A special-case supported in DNSDB
is ANY-DNSSEC, which matches on DS, RRSIG, NSEC, DNSKEY, NSEC3,
NSEC3PARAM, and DLV resource record types.
.It Fl u Ar server_sys
specifies the syntax of the RESTful URL, default is "dnsdb".
.El
.Sh "TIMESTAMP FORMATS"
Timestamps may be one of following forms.
.Bl -enum -offset indent
.It
positive unsigned integer : in Unix epoch format.
.It
negative unsigned integer : negative offset in seconds from now.
.It
YYYY-MM-DD [HH:MM:SS] : in absolute form, in UTC time, as DNSDB does its
fencing using UTC time.
.It
%dw%dd%dh%dm%ds : the relative form with explicit labels.  Calculates offsite
from UTC time, as DNSDB does its fencing using UTC time.
.Pp
.El
A few examples of how to use timefencing options.
.Bd -literal -offset 4n
# only responses after Aug 22, 2015 (midnight)
$ dnsdbq ... -A 2015-08-22
# only responses before Jan 22, 2013 (midnight)
$ dnsdbq ... -B 2013-01-22
# only responses from 2015 (midnight to midnight)
$ dnsdbq ... -B 2016-01-01 -A 2015-01-01
# only responses after 2015-08-22 14:36:10
$ dnsdbq ... -A "2015-08-22 14:36:10"
# only responses from the last 60 minutes
$ dnsdbq ... -A "-3600"
# only responses after "just now"
$ date +%s
1485284066
$ dnsdbq ... -A 1485284066
.Ed
.Sh EXAMPLES
.Pp
A few examples of how to specify IP address information.
.Bd -literal -offset 4n
# specify a single IPv4 address
$ dnsdbq ... -i 128.223.32.35
# specify an IPv4 CIDR
$ dnsdbq ... -i 128.223.32.0/24
# specify a range of IPv4 addresses
$ dnsdbq ... -i 128.223.32.0-128.223.32.32
.Ed
.Pp
Perform an rrset query for a single A record for
.Ic farsightsecurity.com .
The output is serialized as JSON and is piped to the
.Ic jq
program (a command-line JSON processor) for pretty printing.
.Bd -literal -offset 4n
$ dnsdbq -r farsightsecurity.com/A -l 1 -j | jq .
{
  "count": 6350,
  "time_first": 1380123423,
  "time_last": 1427869045,
  "rrname": "farsightsecurity.com.",
  "rrtype": "A",
  "bailiwick": "farsightsecurity.com.",
  "rdata": [
    "66.160.140.81"
  ]
}
.Ed
.Pp
Perform a batched operation for a several different
.Ic rrset
and
.Ic rdata
queries. Output is again serialized as JSON and redirected to a file.
.Bd -literal -offset 4n
$ cat batch.txt
rrset/name/\*.wikipedia.org
rrset/name/\*.dmoz.org
rdata/name/\*.pbs.org
rdata/name/\*.opb.org
rdata/ip/198.35.26.96
rdata/ip/23.21.237.247
rdata/raw/0b763d73706631202d616c6c
$ dnsdbq -j -f < batch.txt > batch-output.json
$ head -1 batch-output.json | jq .
{
  "count": 2411,
  "zone_time_first": 1275401003,
  "zone_time_last": 1484841664,
  "rrname": "wikipedia.org.",
  "rrtype": "NS",
  "bailiwick": "org.",
  "rdata": [
    "ns0.wikimedia.org.",
    "ns1.wikimedia.org.",
    "ns2.wikimedia.org."
  ]
}
.Ed
.Sh FILES
.Ic ~/.isc-dnsdb-query.conf ,
.Ic ~/.dnsdb-query.conf ,
.Ic /etc/isc-dnsdb-query.conf ,
or
.Ic /etc/dnsdb-query.conf :
configuration file which should contain the user's apikey and server URL.
.Bl -tag -width ".Ev DNSDB_SERVER"
.It Ev APIKEY
contains the user's apikey (no default).
.It Ev DNSDB_SERVER
contains the URL of the DNSDB API server (default is https://api.dnsdb.info),
and optionally the URI prefix for the database (default is "/lookup").
.El
.Sh ENVIRONMENT
The following environment variables affect the execution of
.Nm :
.Bl -tag -width ".Ev DNSDB_API_KEY , APIKEY"
.It Ev DNSDB_API_KEY , APIKEY
contains the user's apikey. If DNSDB_API_KEY is not present, then APIKEY will
be used. If neither variable is present, the configuration file is consulted.
.It Ev DNSDB_SERVER
contains the URL of the DNSDB API server, and optionally a URI prefix to be
used (default is "/lookup"). If not set, the configuration file is consulted.
.It Ev DNSDB_TIME_FORMAT
controls how human readable date times are displayed.  If "iso" then ISO8601
(RFC3339) format is used, for example; "2018-09-06T22:48:00Z".  If "csv" then
an Excel CSV compatible format is used; for example, "2018-09-06 22:48:00".
.El
.Sh "EXIT STATUS"
Success (exit status zero) occurs if a connection could be established
to the back end database server, even if no records matched the search
criteria. Failure (exit status nonzero) occurs if no connection could be
established, perhaps due to a network or service failure, or a configuration
error such as specifying the wrong server hostname.
.Sh "SEE ALSO"
.Xr dig 1 ,
.Xr jq 1 ,
.Xr libcurl 3