forked from jnweiger/perl-File-Unpack
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Changes
354 lines (286 loc) · 14.1 KB
/
Changes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
0.69 jw Mon Jul 21 17:21:13 CEST 2014
- fixed https://github.com/jnweiger/perl-File-Unpack/issues/2
0.67 jw Mon Jul 21 16:14:49 CEST 2014
- Semantics change of file_unpack.pl:
Default to one_shot unless $0 contains 'deep'. New --deep option to also switch this one.
This is a move towards least surprise. No other unpacker has this deep feature.
- Fixed builtin helper for application=x-lzip to conform to IPC::Run::run() semantics.
- Added an example how to write a mime helper script to the documentation.
- Fixed diagnostics in case a helper script fails.
- Several smallish doku improvements.
- Updated test suite to accept encoding 'unknown' as a synonym to 'binary'.
0.66 jw Mon Jan 27 18:57:16 CET 2014
- harmful filename test removed, fixing bnc#860561.
The name may not match, when symlinking back. Readlink is the only thing that counts.
- _run_mime_helper() now returns diagnostics even when 'successful' so that we
can see reasoning for 'passed'.
- added t/08-survive-corrupt.t to define behavoiur for good10.pdf and bad34.pdf
- registered the two new test files with t/02-mime.t and t/07-cmdline.t
0.65 jw Tue Aug 6 11:23:43 CEST 2013
- CPAN no longer fails with 'Using $< in a non-suffix rule context is
a GNUmake idiom'
0.64 jw Mon Jul 29 20:54:55 CEST 2013
- assert writable dirs in _run_mime_helper while disassembling jail.
needed for ksh/ast-base.2012-08-01.tar.bz2/src/cmd/pax/data/ro.dat
- all rename()s in _run_mime_helper() equipped with or die.
- DESTROY() now handles more cases: Adding the epilog when missing is paramount.
0.63 jw Fri Jun 14 12:08:25 CEST 2013
- added check for lzip, please test with
/space/legaldb_nfs/sha256/pu/fF/6EtJ9m+nlIZ_aDn2nL2QiBahLsFUhJxKcLAv3zA
0.62 jw Tue May 7 20:04:15 CEST 2013
- support mac pax files. They are gzipped cpio:
'ASCII cpio archive (pre-SVR4 or odc)' and are reported by flm
as application/octet-stream.
(0.61 had a defunct testsuite)
0.60 jw, Mon Apr 29 16:22:33 CEST 2013
- report text/x-application-xml instead of application/xml
0.59 jw, Tue Mar 26 22:53:51 CET 2013
- helping upx to survive symlinks. "%{lsrc}s" format added.
- POD fixes from from anirvan/patch-1
- application/msword helper added. Using antiword.
- helper diagnostics improved: first line of stderr is captured in
the log.
- {verbose} control improved.
0.58 jw, Fri Feb 8 11:15:33 CET 2013
- bugfix: {inside_archives} was not decremented after leaving an
archive.
0.57 jw, Thu Feb 7 19:01:36 CET 2013
- fixed -f to really allow crawling symlinks.
* the logfile will contain the link name, not the link target.
* it thus preserves the illusion that all is inside the destination
tree.
- The following command can now be used to recreate the logfile (and
retry more unpacking) in $DIR:
file_unpack -D $DIR -f -L $DIR.unpack.log $DIR
- _run_mime_helper(): cleanup $jail_base added when cmd fails to run.
Helper failure may still may result in missing files.
- t/07-cmdline.t added.
0.56 jw, Tue Feb 5 18:58:06 CET 2013
- guard _my_shell_quote against undef.
- no 'running as root warning' when !{verbose}
- Relax testsuite on test.mht
- relax DESTROY to not start a logfile, when none is requested.
- testsuite really silent now.
0.55 jw, Fri Jan 25 22:09:04 CET 2013
- option --follow-file-symlinks or new(follow_file_symlinks => $yes) added;
where $yes is 0, 1, or 2.
We may want to use '1', if we are e.g. inside a legaldb_unpacked
tree, where everything is a symlink. This usage is experimental.
A value of '2' is hideous and requires studying perldoc Unpack.pm
first.
- tracking $self->{inside_archives} added. Also experimental.
0.54 jw, Fri Mar 9 17:13:48 CET 2012
- no longer add special files like fifos, sockets, dev nodes to the logfile.
- loggable_pathname adds {srcdir => 'input'} if path is in
$self->{input}
0.53 jw, Thu Feb 16 22:26:06 UTC 2012
- new parameter -A and new(archive_name_as_dir=>1) added.
- new(log_type => 'PLAIN') added, used unless -L
0.52 jw, Wed Feb 1 00:35:47 UTC 2012
- logfile: input menitoned on 'passed' items.
- Makefile.PL now actually installs file_unpack (rt#73670).
Man page is a hack. Sorry.
- file_unpack can now unpack multiple archives on the command line.
- helper/handler wording now consistently speaks of helpers.
0.51 jw, Wed Jan 25 20:23:22 CET 2012
- input archives that do not exist in destdir (because they were
unpacked)
now show up in the log prefixed with './input/./' unless fullpath.
- added loggable_pathname()
0.50 jw, Tue Jan 24 18:43:01 CET 2012
- made testsuite survive on SLE11_SP1, put IO::Handle::tell() inside
eval to survive SLES11_SP1
0.49 jw, Mon Jan 23 19:12:44 CET 2012
- made survive ntop, in{file} must not modify, so that reopen inside
mime() works.
0.48 jw, Tue Nov 29 11:23:30 UTC 2011
- fixed Use of qw(...) as parentheses. This is deprecated with newer
perl.
0.47 jw, Wed Oct 5 00:58:11 CEST 2011
- fixed rt#70885, new(logfile => \$log); now works as advertised.
- somewhat experimental code: we now assert there is no utf8 in the log.
0.46 jw, Wed Sep 7 16:08:45 CEST 2011
- No more uninitialized value in printf line 780.
- Incompatible change: unpacked filenames are now relative to destdir
unless -F or ->new(log_fullname => 1)
- New parameter -P or ->new(log_params => {}) to feed params into the
logfile.
0.45 jw, Fri May 27 13:10:16 CEST 2011
Added option world_readable=>1, to assert 0755/0444 modes for
dirs/files.
0.44 jw, Fri May 20 12:47:27 CEST 2011
converted log to use syswrite() attempt to catch write errors...
0.43 jw, Thu May 12 13:22:02 CEST 2011
Implemented no_op option in _run_mime_helper().
0.42 jw, Thu Apr 21 15:56:27 CEST 2011
Unexpected DESTROY handler closes json logfile properly with an {error}
message, if any.
0.41 jw, Mon Apr 18 14:55:45 CEST 2011
Statistics in logfile about skipped (exclude and symlink) files.
0.40 jw, Fri Apr 15 15:31:54 CEST 2011
less verbose with differing destnames...
Survive crashes in run().
logfile json syntax preserved, even when terminating error message. No
more trailing dummy "/":{}
0.39 jw, Wed Apr 6 18:25:26 CEST 2011
Try /._\d?/ name suffixes on collisions to avoid _fu_* directories if
possible. Added _chmod_add() to help creating readable files and
executable directories. Raised default maxfilesize from 100M to 2.5G,
honors FILE_UNPACK_MAXFILESIZE too.
0.38 Both File::LibMagic and File::MimeInfo::Magic fail on .info files.
We now ask perl -T and return application/x-text-mixed in that case.
Fixed comparison with RLIM_INFINITY, silencing false warnings.
0.37 Fixed setrlimit() calls, to not explode, when the limit is already lower.
Minor cleanup.
0.36 removed dependency on lsof, in favour of /proc and builtin
_children_fuser(). tick-tick now shows the percentage of the fastest
moving fd.
0.35 tick-tick now shows file offsets via crude lsof.
tested with gcc-ejc, parsing lsof output with $u->run() during
$u->run(). Yeah!
0.34 jw, Thu Mar 24 09:24:42 CET 2011
Libmagic perl module explodes with malformed utf8, assert LC_ALL=C.
0.33 jw, Fri Mar 11 18:04:09 CET 2011
run() untaint argv0. needed for running helpers under -T
Empty file is text/x-empty now. not application/x-empty.
0.32 jw, Thu Mar 10 01:52:53 CET 2011
Fixed setlimit code, to be more robust.
Fixed logfile JSON syntax, with a dummy Element in the epilog.
Added 06-logfile.t to test the logfile syntax.
0.31 jw, Wed Mar 9 00:01:50 CET 2011
Added list() method.
Spellchecked pod.
0.30 jw, Fri Nov 19 19:13:53 CET 2010
Run through perlcritic.
Switch back to gz to help smoke testers
0.29 jw, Tue Dec 14 23:19:02 CET 2010
Mention /usr/bin/file_unpack in perldoc
0.28 jw, Fri Oct 22 14:29:09 CEST 2010
Look only at files and dirs, ignore special files.
Reading a pipe could block us endlessly.
0.27 jw, Sat Sep 11 22:01:49 CEST 2010
new feature in mime(): attempt full file access
not only if flm reports currupt, but also when it reports
a/octet-stream. this helps to recognize a/x-iso9660-image
Added .iso by including my old unpack_iso.pl as a helper.
Added .lha
0.26 jw, Thu Sep 9 19:26:21 CEST 2010
Added .upx; we are testing all application/*-executable for upx to get
the mime type. Fixed test results on SLE-11, 02-mime.t now can match
regexp too.
0.25 jw, Wed Sep 8 21:44:37 CEST 2010
Added .cpio, .deb, .cab, .7z
Removed stupid debug die in helper.
Removed done_testing() for the sake of old Test::More on 11.1
0.23 jw, Wed Sep 1 23:17:52 CEST 2010
Added option one_shot, to new() and file_unpack.pl
Added unpacking of: application/pdf, %tar+lzma, %rar, %uuencode
Added '# Requires: ....' comments with the builtin handlers to document
the non-perlish dependencies.
Fixed recognition of .xls -- File::LibMagic said corrupt, with only a
short buffer. Swapped 2nd and 3rd parameter of _run_mime_helper():
mime helpers have now their suggested destname as second parameter.
mime(): added suffix rules for 'text/plain' and
'application/octet-stream', to harvest even the least trusted source of
information, in the absence of anything else.
Implemented first draft of helper/text/=x-shellscript: tested embedded
gzip.
Added -L option to file_unpack.pl;
_run_mime_helper() now untaints its argv. run() errors are now
propagated.
t/04-subdir.t now does a real-life unzip test, if unzip is available.
Lowered $RECURSION_LIMIT to 200. 1000 takes ages with big files.
Call bunzip2 with -f, but don't call xz with -f.
Fixed unpack() to not hit 256 char limit per filename so easily.
Tested mime_handler_dir(), now it actually works. Introduced a default
directory /usr/share/File-Unpack/helper.
Preparing env variable PERL5LIB for helpers.
mime(): Fixed usage of LZMA alone decoder: count as positive, only of
output appeared.
text/x-shellscript: uudecode and shar support tested; works now.
0.22 jw, Sun Aug 29 23:27:59 CEST 2010
Fixed descending of destdir into subdirs of subdirs
Added t/04-subdir.t,
Added first support for running unpack() under -T
Replaced MANIFEST.SKIP with a filter in t/manifest.t
Fixed t/02-mime.t to survive missing shared-mime-info.
(Makefile.PL cannot require non-perl packages, can it?)
0.21 jw, Wed Aug 25 22:15:48 CEST 2010
Added 'nothing-to-do' semantic for mime-helpers: Point back by symlink.
Added helper/text=x-shellscript to demonstrate this.
Made external mime-helpers work: registering, calling.
Remember absolut path, call mime_type() correctly.
Started to propagate errors from _run_mime_helper() to unpack().
mime(): In SLES11 we get 'text/plain charset=utf-8' without semicolon.
0.20 jw, Tue Aug 24 21:15:49 CEST 2010
POD polishing. Synopsis now has some illustrative code.
0.19 jw, Tue Aug 24 15:42:49 CEST 2010
Added _finalize_argvv() to call _locate_*() functions only if
find_mime_handler() needs it. Added _locate_tar() and _locate_cpio_i;
Converted sub _builtin_mime_handlers back into @builtin_mime_handlers;
Debugged run(); piping was broken. Made new(jail_chmod0 => 1) optional.
0.18 jw, Tue Aug 24 02:13:47 CEST 2010
Added handlers for x-tar+bzip2, x-tar+gzip, x-rpm;
Converted @builtin_mime_handlers into sub _builtin_mime_handlers;
Converted system() tests of tar options to run() tests. Added tests for
cpio too. Secured pump inside eval {}.
Added fmt_run_shellcmd() to the API; thus fixed file_unpac.pl -m ...
0.17 jw, Mon Aug 23 11:36:04 CEST 2010
Made the test suite run again with -t
0.16 jw, Mon Aug 23 01:28:09 CEST 2010
added tar handler. debugged dir unpacking, copy-in,
destdir=jail, and def_mime_handler_fmt.
0.15 jw, Mon Aug 16 01:08:32 CEST 2010
destroyed unpack(): recreated subdir code. To be completed.
0.14 jw, Thu Aug 12 00:14:00 CEST 2010
Using Text::Sprintf::Named for parsing builtin_mime_handlers.
Added _fmt_run_shellcmd(). Tested _run_mime_helper. Finished.
0.13 jw, Wed Aug 11 10:30:49 CEST 2010
regression fixed: application/oasis.*.text no longer had +zip suffix.
0.12 jw, Fri Aug 6 01:38:50 CEST 2010
Possibly finalized _run_mime_helper(), needs testing.
0.11 jw, Tue Aug 3 20:15:04 CEST 2010
Reporting @missing_unpacker in log.
0.10 jw, Tue Aug 3 00:30:25 CEST 2010
File::Unppack::run() finished. Oops, IPC::Run opens all redirects
before init. Jail construction done in _run_mime_helper(). Jail
destruction TBD.
@builtin_mime_handlers now reflects the harness structure, knows which
suffixes to remove, and uses words in curly braces as templates, to
make it less ugly. Delayed template substitution into
_run_mime_helper(), as extra array-refs are no good idea in a harness.
0.09 jw, Sun Aug 1 23:42:19 CEST 2010
Added self->{done}, to protect against re-unpacking; enabled recursion
in unpack. More consistent use of C<...> in POD. _run_mime_helper()
prepared for IPC::Run. Changed use_mime_handler() to parse an array,
rather than a format string.
0.08 jw, Mon Jul 12 00:54:09 CEST 2010
Added mime handler methods: use_mime_handler_dir(), use_mime_handler(),
find_mime_handler() using $self->{mime_orcish};
Added builtin handlers to new(), using "%{destdir} ..." format string
syntax Added _prep_configdir(), shell and json support.
Continued unpack() to dispatch into mime handlers.
Started _run_mime_helper()
0.07 jw, Fri Jul 9 01:19:19 CEST 2010
Added support for stacked compressions recognition without looking
at file suffixes at all:
mime(file => '.tar.gz.lzma') says: "application/x-tar+gzip+lzma"
Hack to make tar.bz2 appear as "application/x-tar+bzip2"
tar.bz2.lzma does not work, that is why I call bz2 support a hack.
Now we can work on efficient unpack helpers.
0.06 jw, Thu Jul 8 00:31:24 CEST 2010
using Compress::Raw::Lzma::AloneDecoder instead of lzcat.
0.05 jw, Wed Jun 30 01:35:17 CEST 2010
started unpack(), added _not_excluded(),
added file_unpack.pl for testing.
0.04 jw, Mon Jun 28 21:38:31 CEST 2010
added exclude(), with tests & docs.
added _fs_check() and _fs_warn() logic.
added and used _bytes_unit().
0.03 jw, Sun Jun 27 03:09:21 CEST 2010
new() completed, private log(), logf() added.
0.02 jw, Sat Jun 26 21:55:12 CEST 2010
mime() documented, implemented, tested.
0.01 jw, Sat Jun 26 12:45:63 CEST 2010
First version, skeleton drafted.