Redirected output, row-value shift, and other features
The principal feature of Miller 4.4.0 is redirected output. Inspired by awk
, Miller lets you tap/tee your data as it's processed, run output through subordinate processes such as gzip
and jq
, split a single file into multiple files per an account-ID column, and so on.
Details: http://johnkerl.org/miller/doc/reference.html#Redirected-output_statements_for_put
Other features:
mlr step -a shift
allows you to place the previous record's values alongside the current record's values: http://johnkerl.org/miller/doc/reference.html#stepmlr head
, when used without the group-by flag (-g
), stops after the specified number of records has been output. For example, even with a multi-gigabyte data file,mlr head -n 10 hugefile.dat
will complete quickly after producing the first ten records from the file.- The
sec2gmtdate
verb, andsec2gmtdate
function forfilter
/put
, is new: please see http://johnkerl.org/miller/doc/reference.html#sec2gmtdate and http://johnkerl.org/miller/doc/reference.html#Functions_for_filter_and_put. sec2gmt
andsec2gmtdate
both leave non-numbers as-is, rather than formatting them as(error)
. This is particularly relevant for formatting nullable epoch-seconds columns in SQL-table output: if a column value isNULL
then aftersec2gmt
orsec2gmtdate
it will still beNULL
.- The dot operator has been universalized to work with any data type and produce a string. For example, if the field
n
has integers, then instead of typingmlr put '$name = "value:".string($n)'
you can now simply domlr put '$name = "value:".$n'
. This is particularly timely for creating filenames for redirectedprint
/dump
/tee
/emit
output. - The online documents now have a copy of the Miller manpage: http://johnkerl.org/miller/doc/manpage.html
- Bugfix: inside
filter
/put
,$x==""
was distinct fromisempty($x)
. This was nonsensical; now both are the same.
Brew update: Homebrew/homebrew-core#3820