Interpolated percentiles, markdown-tabular output format, CSV-quote preservation
Major features:
- Interpolated percentiles are now available using
mlr stats1 -i
ormlr merge-fields -i
. Non-interpolated percentiles are the default. The former resemble R'stype=7
quantiles and the latter resemble R'stype=1
quantiles. See also http://johnkerl.org/miller/doc/reference.html#stats1 and http://johnkerl.org/miller/doc/reference.html#merge-fields. - Markdown-tabular output format is now available using
--omd
: please see http://johnkerl.org/miller/doc/file-formats.html#Markdown_tabular and #106. - For files using CSV input as well as CSV output, there is now a --quote-original option which outputs fields with quotes if they had them on input. The was-quoted flag isn't tracked on derived fields, e.g. if fields
a
andb
were quoted on input, then inmlr put '$c = $a . $b
thec
field won't be quoted on output. As such, this option is most useful withmlr cut
,mlr filter
, etc. The use-case from the original feature request #77 (comment) is in trimming down a huge CSV file in order to facilitate subsequent in-memory processing using spreadsheet software. - The cookbook at http://johnkerl.org/miller/doc/cookbook.html has been extended significantly.
Minor features:
- You can now set a
MLR_CSV_DEFAULT_RS=lf
environment variable if you're tired of always putting--rs lf
arguments for your CSV files: http://johnkerl.org/miller/doc/file-formats.html#CSV/TSV/etc. - The
printn
andeprintn
commands formlr put
are identical toprint
andeprint
except they don't print final newlines. - It is now an error if boundvars in the same for-loop expression have duplicate names, e.g.
for (a,a in $*) {...}
results in the error messagemlr: duplicate for-loop boundvars "a" and "a"
. - The
strptime
function would announce an internal coding error on malformed format strings; now, it correctly points out the user-level error.
Bug fixes:
- Percentiles in
merge-fields
were not working. This was fixed; also, the lacking unit-test cases which would have caught this sooner have been filled in. - Miller's CSV output-quoting was non-RFC-compliant: double-quotes within field names were not being duplicated. This has been fixed (#104).
Brew update: Homebrew/homebrew-core#2698