-
Notifications
You must be signed in to change notification settings - Fork 132
/
md5deep.html
374 lines (331 loc) · 14.4 KB
/
md5deep.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
<!-- manual page source format generated by PolyglotMan v3.2, -->
<!-- available at http://polyglotman.sourceforge.net/ -->
<html>
<head>
<title>md5deep man page</title>
</head>
<body bgcolor='white'>
<a href='#toc'>Table of Contents</a><p>
<p>
<h2><a name='sect0' href='#toc0'>Name</a></h2>
md5deep - Compute and compare MD5 message digests <br>
sha1deep - Compute and compare SHA-1 message digests <br>
sha256deep - Compute and compare SHA-256 message digests <br>
tigerdeep - Compute and compare Tiger message digests <br>
whirlpooldeep - Compute and compare Whirlpool message digests
<p>
<h2><a name='sect1' href='#toc1'>Synopsis</a></h2>
<b>md5deep
</b> -v | -V | -h <br>
<b>md5deep</b> [-m|-M|-x|-X <file>] [-a|-A <hash>] [-f <file>] [-p <size>] [-i <size>] [-tnwzresS0lbkqZud]
[-F <bum>] [-o <fbcplsde>] [-j <num>] [[<b>FILES</b>]
<p>
<h2><a name='sect2' href='#toc2'>Description</a></h2>
<p>
Computes the hashes,
or message digest, for any number of files while optionally recursively
digging through the directory structure. Can also take a list of known hashes
and display the filenames of input files whose hashes either do or do not
match any of the known hashes. Errors are reported to standard error. If
no FILES are specified, reads from standard input.
<p>
<dl>
<dt><b>-p <size> </b> </dt>
<dd>Piecewise mode.
Breaks files into chunks before hashing. Chunks may be specified using
IEC multipliers b, k, m, g, t, p, or e. (Never let it be said that the author
didn’t plan ahead!) This mode cannot be used with the -z mode.
<p> </dd>
<dt><b>-i|-I <size> </b>
</dt>
<dd>Size threshold mode. Only hash files smaller than the given the threshold.
In -i mode, simply omits those files larger than the threshold. In -I mode,
displays all files, but uses asterisks for the hashes of files larger than
the threshold. Sizes may be specified using IEC multipliers b, k, m, g,
t, p, or e.
<p> </dd>
<dt><b>-r</b> </dt>
<dd>Enables recursive mode. All subdirectories are traversed. Please
note that recursive mode cannot be used to examine all files of a given
file extension. For example, calling md5deep -r *.txt will examine all files
in <i>directories</i> that end in .txt.
<p> </dd>
<dt><b>-e</b> </dt>
<dd>Displays a progress indicator and estimate
of time remaining for each file being processed. Time estimates for files
larger than 4GB are not available on Windows. This mode may not be used
with th -p mode.
<p> </dd>
<dt><b>-m</b> <file> </dt>
<dd>Enables matching mode. The file given should be a
list of known hashes. The input files are examined one at a time, and only
those files that match the list of known hashes are output. This flag may
be used more than once to add multiple sets of known hashes. Acceptable
formats for lists of known hashes are plain (such as those generated by
md5deep or md5sum), Hashkeeper files, iLook, and the National Software
Reference Library (NSRL) as produced by the National Institute for Standards
in Technology. <br>
<b></b> If standard input is used with the -m flag, displays "stdin" if the input
matches one of the hashes in the list of known hashes. If the hash does
not match, the program displays no output. <br>
<b></b> This flag may not be used in conjunction with the -x, -X, or -A flags. See
the section "UNICODE SUPPORT" below.
<p> </dd>
<dt><b>-x</b> <file> </dt>
<dd>Same as the -m flag above, but
does negative matching. That is, only those files NOT in the list of known
hashes are displayed. <br>
<b></b> This flag may not be used in conjunction with the -m, -M, or -a flags. See
the section "UNICODE SUPPORT" below. </dd>
<dt><b>-M</b> and <b>-X</b> <file> </dt>
<dd>Same as -m and -x above,
but displays the hash for each file that does (or does not) match the
list of known hashes.
<p> </dd>
<dt><b>-a</b> <hash> </dt>
<dd>Adds a single hash to the list of known hashes
used for matching mode, and if not already enabled, enables matching mode.
Adding single hashes cannot, by itself, be used to print the hashes of
matching files like the -M flag does. When used in conjunction with the -w
flag, the filename displayed is just the hash submitted on the command
line. <br>
<b></b> This flag may not be used in conjunction with the -x, -X, or -A flags.
<p> </dd>
<dt><b>-A</b>
<hash> </dt>
<dd>Same as -a above, but does negative matching. This flag may not be used
in conjunction with the -m, -M, or -A flags.
<p> </dd>
<dt><b>-f</b> <file> </dt>
<dd>Takes a list of files
to be hashed from the specified file. Each line is assumed to be a filename.
This flag can only be used once per invocation. If it’s used a second time,
the second instance will clobber the first. <br>
Note that you can still use other flags, such as the -m or -x modes, and
submit additional FILES on the command line.
<p> </dd>
<dt><b>-w</b> </dt>
<dd>During any of the matching
modes (-m,-M,-x,or -X), displays the filename of the known hash that matched
the input file. See the section "UNICODE SUPPORT" below.
<p> </dd>
<dt><b>-t</b> </dt>
<dd>Display a timestamp
in GMT with each result. On Windows this timestamp will be the file’s creation
time. On all other systems it should be the file’s change time.
<p> </dd>
<dt><b>-n</b> </dt>
<dd>During
any of the matching modes (-m,-M,-x,or -X), displays only the filenames of
any known hashes that were not matched by any of the input files.
<p> </dd>
<dt><b>-s</b> </dt>
<dd>Enables
silent mode. All error messages are supressed.
<p> </dd>
<dt><b>-S</b> </dt>
<dd>Like silent mode, but still
displays warnings on improperly formatted hashes in the list of known hashes.
<p> </dd>
<dt><b>-z</b> </dt>
<dd>Enables file size mode. Prepends the hash with a ten digit representation
of the size of each file processed. If the file size is greater than 9999999999
bytes (about 9.3GB) the program displays 9999999999 for the size.
<p> </dd>
<dt><b>-q</b> </dt>
<dd>Quiet
mode. File names are omitted from the output. Each hash is still followed
by two spaces before the newline.
<p> </dd>
<dt><b>-Z</b> </dt>
<dd>Produces output in Triage format. Each
line contans the file’s size, a tab, a hash of the first 512 bytes, a tab,
the hash of the complete file, a tab, and the file name. These values are
intended in increasing order of specificity. That is, two files with different
sizes cannot possibly match. This is a fast comparison and should be done
first. Next, two files with different partial hashes cannot possibly match.
This is often faster than hashing the whole file. Finally, if those two
pieces align, then it’s worth reading and hashing the entire file.
<p> </dd>
<dt><b>-0</b> </dt>
<dd>Uses
a NULL character (/0) to terminate each line instead of a newline. Useful
for processing filenames with strange characters.
<p> </dd>
<dt><b>-l</b> </dt>
<dd>Enables relative file
paths. Instead of printing the absolute path for each file, displays the
relative file path as indicated on the command line. This flag may not
be used in conjunction with the -b flag.
<p> </dd>
<dt><b>-b</b> </dt>
<dd>Enables bare mode. Strips any
leading directory information from displayed filenames. This flag may not
be used in conjunction with the -l flag.
<p> </dd>
<dt><b>-k</b> </dt>
<dd>Enables asterisk mode. An asterisk
is inserted in lieu of a second space between the filename and the hash,
just like md5sum in its binary (-b) mode.
<p> </dd>
<dt><b>-c</b> </dt>
<dd>Enables comma separated values
output, or CSV mode. This mode has the side effect of removing the 10 digit
size limitation from -z mode. Also note that asterisks from -k mode are not
displayed when in CSV mode.
<p> </dd>
<dt><b>-o</b> <bcpflsd> </dt>
<dd>Enables expert mode. Allows the user
specify which (and only which) types of files are processed. Directory processing
is still controlled with the -r flag. The expert mode options allowed are:
<br>
f - Regular files <br>
b - Block Devices <br>
c - Character Devices <br>
p - Named Pipes <br>
l - Symbolic Links <br>
s - Sockets <br>
d - Solaris Doors <br>
e - Windows PE executables
<p> </dd>
<dt><b>-jnn</b> </dt>
<dd>Controls multi-threading. By default the program
will create one producer thread to scan the file system and one hashing
thread per CPU core. Multi-threading causes output filenames to be in non-deterministic
order, as files that take longer to hash will be delayed while they are
hashed. If a deterministic order is required, specify <b>-j0</b> to disable multi-threading
<p> </dd>
<dt><b>-d</b> </dt>
<dd>Output in Digital Forensics XML (DFXML) format.
<p> </dd>
<dt><b>-u</b> </dt>
<dd>Quote Unicode output.
For example, the snowman is shown as <b>U+C426</b>.
<p> </dd>
<dt><b>-F<bum></b> </dt>
<dd>Specifies the input mode
that is used to read files. The default is <b>-Fb</b> (buffered I/O) which reads
files with fopen(). Specifying <b>-Fu</b> will use unbuffered I/O and read the file
with open(). Specifying <b>-Fm</b> will use memory-mapped I/O which will be faster
on some platforms, but which (currently) will not work with files that
produce I/O errors.
<p> </dd>
<dt><b>-h</b> </dt>
<dd>Show a help screen and exit.
<p> </dd>
<dt><b>-v</b> </dt>
<dd>Show the version number
and exit.
<p> </dd>
<dt><b>-V</b> </dt>
<dd>Show copyright information and exit.
<p> </dd>
</dl>
<h2><a name='sect3' href='#toc3'>Unicode Support</a></h2>
As of version
3.0 the program supports Unicode characters in filenames on Microsoft Windows
systems for filenames specified on the command line with globbing (e.g. *),
for files specified with the <b>-f</b> of files to hash, and for files read from
directories using the <b>-r</b> option.
<p> By default all program input and output
should be in UTF-8. The program automatically converts this to UTF-16 for
opening files).
<p> On Unix/Linux/MacOS, you should use a terminal emulator
that supports UTF-8 and UTF-8 characters in filenames will be properly displayed.
<p> On Windows, please note that the onsole is not capiable of displaying
Unicode characters. You must either redirect output to a file and open the
file with Wordpad (which can display Unicode), or you must specify the
<b>-u</b> option to quote Unicode using standard <b>U+XXXX</b> notation.
<p> Currently the
file name of a file containing known hashes may not be specified as a unicode
filename, but you can specify the name using tab completition or an asterisk
(e.g. md5deep -m *.txt where there is only one file with a .txt extension).
<p>
<h2><a name='sect4' href='#toc4'>Return Value</a></h2>
Returns a bit-wise value based on the success of the operation
and the status of any matching operations. <p>
<ol>
<li>Success. Note that the program
considers itself successful even when it encounters read errors, permission
denied errors, or finds directories when not in recursive mode. </li><li>Unused hashes.
Under any of the matching modes, returns this value if one or more of
the known hashes was not matched by any of the input files. </li><li>Unmatched inputs.
Under any of the matching modes, returns this value if one or more of the
input values did not match any of the known hashes. </li><li>User error, such as
trying to do both positive and negative matching at the same time. </li><li>Internal
error, such as memory corruption or uncaught cycle. All internal errors
should be reported to the developer! See the section "Reporting Bugs" below.
<p>
<p> </li>
</ol>
<h2><a name='sect5' href='#toc5'>Author</a></h2>
md5deep was written by Jesse Kornblum, [email protected]
and Simson Garfinkel.
<p>
<h2><a name='sect6' href='#toc6'>Known Issues</a></h2>
Using the -r flag cannot be used to recursively
process all files of a given extension in a directory. This is a feature,
not a bug. If you need to do this, use the <a href='find.1'><b>find</b>(1)</a>
command.
<p>
<h2><a name='sect7' href='#toc7'>Reporting Bugs</a></h2>
We
take all bug reports <i>very</i> seriously. Any bug that jeopardizes the forensic
integrity of this program could have serious consequences on people’s lives.
When submitting a bug report, please include a description of the problem,
how you found it, and your contact information. <p>
Send bug reports to the
author at the address above.
<p> <p>
<h2><a name='sect8' href='#toc8'>Copyright</a></h2>
This program is a work of the US
Government. In accordance with 17 USC 105, copyright protection is not available
for any work of the US Government. This program is PUBLIC DOMAIN. Portions
of this program contain code that is licensed under the terms of the General
Public License (GPL). Those portions retain their original copyright and
license. See the file COPYING for more details. <p>
There is NO warranty for
this program; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
<p>
<h2><a name='sect9' href='#toc9'>See Also</a></h2>
More information and installation instructions can be
found in the README file. Current versions of both documents can be found
on the project homepage: <a href='http://md5deep.sourceforge.net/'>http://md5deep.sourceforge.net/</a>
<p>
The MD5 specification,
RFC 1321, is available at <br>
<a href='http://www.ietf.org/rfc/rfc1321.txt'>http://www.ietf.org/rfc/rfc1321.txt</a>
<p>
The SHA-1 specification, RFC 3174, is available
at <br>
<a href='http://www.faqs.org/rfcs/rfc3174.html'>http://www.faqs.org/rfcs/rfc3174.html</a>
<p>
The SHA-256 specification, FIPS 180-2,
is available at <br>
<a href='http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf'>http://csrc.nist.gov/publications/fips/fips180-2/fips180-2.pdf</a>
<p>
The Tiger specification
is available at <br>
<a href='http://www.cs.technion.ac.il/~biham/Reports/Tiger/'>http://www.cs.technion.ac.il/~biham/Reports/Tiger/</a>
<p>
The Whirlpool specification
is available at <br>
<a href='http://planeta.terra.com.br/informatica/paulobarreto/WhirlpoolPage.html'>http://planeta.terra.com.br/informatica/paulobarreto/WhirlpoolPage.html</a>
<p>
<hr><p>
<a name='toc'><b>Table of Contents</b></a><p>
<ul>
<li><a name='toc0' href='#sect0'>Name</a></li>
<li><a name='toc1' href='#sect1'>Synopsis</a></li>
<li><a name='toc2' href='#sect2'>Description</a></li>
<li><a name='toc3' href='#sect3'>Unicode Support</a></li>
<li><a name='toc4' href='#sect4'>Return Value</a></li>
<li><a name='toc5' href='#sect5'>Author</a></li>
<li><a name='toc6' href='#sect6'>Known Issues</a></li>
<li><a name='toc7' href='#sect7'>Reporting Bugs</a></li>
<li><a name='toc8' href='#sect8'>Copyright</a></li>
<li><a name='toc9' href='#sect9'>See Also</a></li>
</ul>
</body>
</html>