File-stat DSL function, new stats accumulator, misc. bugfixes

New features

• Add a stat DSL function by @johnkerl in #1560
• Add mad accumulator for stats1 DSL function by @johnkerl in #1561
• Support $NO_COLOR by @johnkerl in #1580

Bug fixes

• Fraction bugfix by @oandrew in #1579
• Fix local time when TZ is not set by @balki in #1649
• Bash process substitution not working with put -f by @johnkerl in #1583
• Be smarter about auto-unflatten by @johnkerl in #1584
• RS aliases for ASCII top-of-table control characters are misnamed by @johnkerl in #1620
• Fix binary data in JSON output by @johnkerl in #1626
• Fix prepipe handling when filenames have whitespace by @johnkerl in #1627
• Error in splita/splitax when field contains a single non-string value by @johnkerl in #1629

Documentation updates

• Update reference-verbs.md by @aborruso in #1665
• Characters to be removed by @aborruso in #1668
• Fix minor typo by @austinletson in #1673
• Enable admonition extension by @aborruso in #1636
• To realize which chapter and section are active by @aborruso in #1631
• To have edit and copy code in each page by @aborruso in #1632
• Update extra.css by @aborruso in #1633
• A note about positional field names by @aborruso in #1634
• Fix typo in online help for --no-jlistwrap by @johnkerl in #1541
• Try to build readthedocs .epub and .pdf by @johnkerl in #1548
• On-line help for mlr summary --transpose by @johnkerl in #1581
• Note IANA TSV support by @johnkerl in #1582
• Source-file update for PR 1634 by @johnkerl in #1635
• Update source material for #1665 by @johnkerl in #1666
• Fix 1668 error-source by @johnkerl in #1672

Minor changes

• The package version must match the major tag version by @lespea in #1654
• Use string version of regexp methods to reduce allocs by @Juneezee in #1614
• Chore: fix function name in comment by @camcui in #1543
• Fix mismatched method names in comments by @forcedebug in #1549
• Compiling on newer go versions doesn't work by @lespea in #1655
• Misc. codespell findings by @johnkerl in #1628

New Contributors

• @camcui made their first contribution in #1543
• @forcedebug made their first contribution in #1549
• @oandrew made their first contribution in #1579
• @balki made their first contribution in #1649
• @lespea made their first contribution in #1654
• @austinletson made their first contribution in #1673

Dependency updates

• Bump actions/cache from 4.0.1 to 4.0.2 by @dependabot in #1532
• Bump golang.org/x/term from 0.18.0 to 0.19.0 by @dependabot in #1536
• Bump github.com/klauspost/compress from 1.17.7 to 1.17.8 by @dependabot in #1538
• Bump actions/upload-artifact from 4.3.1 to 4.3.2 by @dependabot in #1547
• Bump actions/checkout from 4.1.2 to 4.1.3 by @dependabot in #1550
• Bump actions/upload-artifact from 4.3.2 to 4.3.3 by @dependabot in #1551
• Bump actions/checkout from 4.1.3 to 4.1.4 by @dependabot in #1552
• Bump actions/setup-go from 5.0.0 to 5.0.1 by @dependabot in #1553
• Bump golang.org/x/sys from 0.19.0 to 0.20.0 by @dependabot in #1554
• Bump golang.org/x/text from 0.14.0 to 0.15.0 by @dependabot in #1556
• Bump golang.org/x/term from 0.19.0 to 0.20.0 by @dependabot in #1555
• Bump actions/checkout from 4.1.4 to 4.1.5 by @dependabot in #1557
• Bump goreleaser/goreleaser-action from 5.0.0 to 5.1.0 by @dependabot in #1563
• Bump actions/checkout from 4.1.5 to 4.1.6 by @dependabot in #1566
• Bump github/codeql-action from 2.13.4 to 3.25.5 by @dependabot in #1567
• Bump github/codeql-action from 3.25.5 to 3.25.6 by @dependabot in #1568
• Bump github/codeql-action from 3.25.6 to 3.25.7 by @dependabot in #1570
• Bump goreleaser/goreleaser-action from 5.1.0 to 6.0.0 by @dependabot in #1574
• Bump github/codeql-action from 3.25.7 to 3.25.8 by @dependabot in #1575
• Bump golang.org/x/text from 0.15.0 to 0.16.0 by @dependabot in #1576
• Bump golang.org/x/sys from 0.20.0 to 0.21.0 by @dependabot in #1578
• Bump golang.org/x/term from 0.20.0 to 0.21.0 by @dependabot in #1577
• Bump github.com/klauspost/compress from 1.17.8 to 1.17.9 by @dependabot in #1585
• Bump actions/checkout from 4.1.6 to 4.1.7 by @dependabot in #1586
• Bump github/codeql-action from 3.25.8 to 3.25.9 by @dependabot in #1587
• Bump github/codeql-action from 3.25.9 to 3.25.10 by @dependabot in #1588
• Bump github/codeql-action from 3.25.10 to 3.25.11 by @dependabot in #1593
• Bump golang.org/x/sys from 0.21.0 to 0.22.0 by @dependabot in #1595
• Bump golang.org/x/term from 0.21.0 to 0.22.0 by @dependabot in #1594
• Bump actions/upload-artifact from 4.3.3 to 4.3.4 by @dependabot in #1596
• Bump actions/setup-go from 5.0.1 to 5.0.2 by @dependabot in #1597
• Bump github/codeql-action from 3.25.11 to 3.25.12 by @dependabot in #1598
• Bump github/codeql-action from 3.25.12 to 3.25.13 by @dependabot in #1602
• Bump github/codeql-action from 3.25.13 to 3.25.14 by @dependabot in #1603
• Bump github/codeql-action from 3.25.14 to 3.25.15 by @dependabot in #1604
• Bump golang.org/x/sys from 0.22.0 to 0.23.0 by @dependabot in #1605
• Bump actions/upload-artifact from 4.3.4 to 4.3.5 by @dependabot in #1606
• Bump golang.org/x/term from 0.22.0 to 0.23.0 by @dependabot in #1612
• Bump actions/upload-artifact from 4.3.5 to 4.3.6 by @dependabot in #1609
• Bump github/codeql-action from 3.25.15 to 3.26.0 by @dependabot in #1610
• Bump golang.org/x/text from 0.16.0 to 0.17.0 by @dependabot in #1611
• Bump golang.org/x/sys from 0.23.0 to 0.24.0 by @dependabot in #1613
• Bump github/codeql-action from 3.26.0 to 3.26.1 by @dependabot in #1615
• Bump github/codeql-action from 3.26.1 to 3.26.2 by @dependabot in #1617
• Bump codespell-project/actions-codespell from 2.0 to 2.1 by @dependabot in #1622
• Bump github/codeql-action from 3.26.2 to 3.26.3 by @dependabot in #1623
• Bump github/codeql-action from 3.26.3 to 3.26.4 by @dependabot in #1624
• Bump github/codeql-action from 3.26.4 to 3.26.5 by @dependabot in #1630
• Bump github.com/lestrrat-go/strftime from 1.0.6 to 1.1.0 by @dependabot in #1637
• Bump github/codeql-action from 3.26.5 to 3.26.6 by @dependabot in #1638
• Bump actions/upload-artifact from 4.3.6 to 4.4.0 by @dependabot in #1640
• Bump golang.org/x/text from 0.17.0 to 0.18.0 by @dependabot in #1641
• Bump golang.org/x/term from 0.23.0 to 0.24.0 by @dependabot in #1642
• Bump github/codeql-action from 3.26.6 to 3.26.7 by @dependabot in #1648
• Bump github/codeql-action from 3.26.7 to 3.26.8 by @dependabot in #1652
• Bump github.com/klauspost/compress from 1.17.9 to 1.17.10 by @dependabot in #1659
• Bump github/codeql-action from 3.26.8 to 3.26.9 by @dependabot in #1660
• Bump actions/checkout from 4.1.7 to 4.2.0 by @dependabot in #1662
• Bump github/codeql-action from 3.26.9 to 3.26.10 by @dependabot in #1664
• Bump github/codeql-action from 3.26.10 to 3.26.11 by @dependabot in #1669

Full Changelog: v6.12.0...v6.13.0

Miller 6.12.0: New sparsify verb, wide-table performance improvement, thousands separator for fmtnum function

Features

• New mlr sparsify verb by @johnkerl in #1498
• Support thousands separator in fmtnum by @johnkerl in #1499
• Add descriptions for put and filter verbs by @johnkerl in #1529

Bugfixes

• Miller produces no output on TSV with > 64K characters per line by @johnkerl in #1505
• Enable record-hashing by default by @johnkerl in #1507
• Improved file-not-found handling by @johnkerl in #1508
• Avoid spurious [] on JSON output in some cases by @johnkerl in #1528

Internal

• 6.11.0-dev by @johnkerl in #1484
• Separate out ILineReader abstraction by @johnkerl in #1504

Dependency updates

• Bump actions/upload-artifact from 4.2.0 to 4.3.0 by @dependabot in #1483
• Bump github.com/klauspost/compress from 1.16.7 to 1.17.5 by @dependabot in #1486
• Bump actions/upload-artifact from 4.3.0 to 4.3.1 by @dependabot in #1491
• Bump github.com/klauspost/compress from 1.17.5 to 1.17.6 by @dependabot in #1492
• Bump golang.org/x/term from 0.16.0 to 0.17.0 by @dependabot in #1494
• Bump github.com/klauspost/compress from 1.17.6 to 1.17.7 by @dependabot in #1502
• Bump actions/cache from 4.0.0 to 4.0.1 by @dependabot in #1511
• Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 by @dependabot in #1516
• Bump golang.org/x/sys from 0.17.0 to 0.18.0 by @dependabot in #1521
• Bump golang.org/x/term from 0.17.0 to 0.18.0 by @dependabot in #1522
• Bump actions/checkout from 4.1.1 to 4.1.2 by @dependabot in #1526

Full Changelog: v6.11.0...v6.12.0

Miller 6.11.0: CSV/TSV auto-unsparsify, regex-fieldname support for reorder/sub/ssub/gsub, strmatch DSL function, and more

Features

• Auto-unsparsify CSV and TSV on output by @johnkerl in #1479
  • Issue #1418
  • https://miller.readthedocs.io/en/latest/record-heterogeneity/#rectangular-file-formats-csv-and-pretty-print
  • https://miller.readthedocs.io/en/latest/file-formats/#csvtsvasvusvetc
• mlr reorder with regex support by @johnkerl in #1473
  • https://miller.readthedocs.io/en/latest/reference-verbs/index.html#reorder
• Implement all/by-regex field selection (-a/-r) for mlr sub, gsub, and ssub by @johnkerl in #1480
  • https://miller.readthedocs.io/en/latest/reference-dsl-builtin-functions/index.html#sub
  • https://miller.readthedocs.io/en/latest/reference-dsl-builtin-functions/index.html#gsub
  • https://miller.readthedocs.io/en/latest/reference-dsl-builtin-functions/index.html#ssub
• Preserve regex captures across stack frames by @johnkerl in #1447
• Document and unit-test regex-capture reset logic by @johnkerl in #1451
• New strmatch/strmatchx DSL functions by @johnkerl in #1448
  • https://miller.readthedocs.io/en/latest/reference-dsl-builtin-functions/index.html#strmatch
  • https://miller.readthedocs.io/en/latest/reference-dsl-builtin-functions/index.html#strmatchx
• Implement mlr uniq -x by @johnkerl in #1457
  • https://miller.readthedocs.io/en/latest/reference-verbs/index.html#uniq
• On-line help info for mlr join --lk "" by @johnkerl in #1458
  • https://miller.readthedocs.io/en/latest/reference-verbs/index.html#join
• Fix PR 1462: remove limit of 1000 on dedupe field names by @johnkerl in #1463
• Support PPRINT barred input by @johnkerl in #1472
  • https://miller.readthedocs.io/en/latest/file-formats/
• Support markdown format on input by @johnkerl in #1478
  • https://miller.readthedocs.io/en/latest/file-formats/

Bugfixes

• mlr --norc was erroring by @johnkerl in #1450
• Have clean_whitespace re-run type inference by @johnkerl in #1464

Internals

• Rename internal regex functions by @johnkerl in #1446
• Replace deprecated io/ioutil functions by @Juneezee in #1452
• Internal name-neatens by @johnkerl in #1475
• Fix typos in tests for PPRINT barred input by @johnkerl in #1476
• Don't run regression tests twice in GitHub CI by @johnkerl in #1477
• Miller 6.11.0 by @johnkerl in #1481

Dependencies

• Bump actions/upload-artifact from 3.1.3 to 4.0.0 by @dependabot in #1445
• Bump golang.org/x/term from 0.15.0 to 0.16.0 by @dependabot in #1466
• Bump actions/cache from 3.3.2 to 3.3.3 by @dependabot in #1468
• Bump actions/upload-artifact from 4.0.0 to 4.1.0 by @dependabot in #1469
• Bump actions/cache from 3.3.3 to 4.0.0 by @dependabot in #1470
• Bump actions/upload-artifact from 4.1.0 to 4.2.0 by @dependabot in #1471

Full Changelog: v6.10.0...v6.11.0

Miller 6.10.0: Add --files option; bugfixes; use Go 1.19

Features

• Add a --files option by @johnkerl in #1426

Bugfixes

• Fix ragged-CSV auto-pad by @johnkerl in #1428
• Absent variable on left side of boolean OR (||) expression makes it absent by @johnkerl in #1434
• Include null in any typemask by @johnkerl in #1395
• transformers/grep: avoid allocations with (*regexp.Regexp).MatchString by @Juneezee in #1416
• JSONL output does not properly handle keys with quotes by @johnkerl in #1425

Minor changes

• Update to Go 1.19 by @johnkerl in #1441
• miller 6.10.0 by @johnkerl in #1442
• Add winget to README.md by @rursprung in #1414
• Name-neaten for #1392 by @johnkerl in #1393

Miller as API

• Export library code in pkg/ by @johnkerl in #1391
• Better API example by @johnkerl in #1392

Dependencies

• Bump golang.org/x/text from 0.12.0 to 0.13.0 by @dependabot in #1382
• Bump golang.org/x/sys from 0.11.0 to 0.12.0 by @dependabot in #1381
• Bump golang.org/x/term from 0.11.0 to 0.12.0 by @dependabot in #1380
• Bump actions/checkout from 3.6.0 to 4.0.0 by @dependabot in #1383
• Bump goreleaser/goreleaser-action from 4.4.0 to 4.6.0 by @dependabot in #1385
• Bump actions/upload-artifact from 3.1.2 to 3.1.3 by @dependabot in #1387
• Bump actions/cache from 3.3.1 to 3.3.2 by @dependabot in #1390
• Bump goreleaser/goreleaser-action from 4.6.0 to 5.0.0 by @dependabot in #1396
• Bump actions/checkout from 4.0.0 to 4.1.0 by @dependabot in #1400
• Bump golang.org/x/term from 0.12.0 to 0.13.0 by @dependabot in #1404
• Bump github.com/mattn/go-isatty from 0.0.19 to 0.0.20 by @dependabot in #1411
• Bump actions/checkout from 4.1.0 to 4.1.1 by @dependabot in #1412
• Bump golang.org/x/text from 0.13.0 to 0.14.0 by @dependabot in #1419
• Bump golang.org/x/sys from 0.13.0 to 0.14.0 by @dependabot in #1420
• Bump golang.org/x/term from 0.13.0 to 0.14.0 by @dependabot in #1423
• Bump golang.org/x/term from 0.14.0 to 0.15.0 by @dependabot in #1432
• Bump actions/setup-go from 4.1.0 to 5.0.0 by @dependabot in #1436

New Contributors

• @rursprung made their first contribution in #1414

Full Changelog: v6.9.0...v6.10.0

Miller 6.9.0: Nanosecond timestamps, ZSTD compression, improved data-error handling, and more

New features

Support for nanosecond-resolution timestamps:

• Add DSL functions for integer nanoseconds since the epoch by @johnkerl in #1326
• Add %N and %O for strfntime by @johnkerl in #1334
• Add %s format specifier for strftime by @johnkerl in #1335
• Requested on issue #1152 by @derekmahar
• See also https://miller.readthedocs.io/en/6.9.0/reference-dsl-builtin-functions/index.html#time-functions

Statistical functions from the stats verb may now be computed over arbitrary arrays and maps:

• New DSL functions for summary stats over arrays / maps by @johnkerl in #1364
• Requested on issue #1345 by @janxkoci
• See also https://miller.readthedocs.io/en/6.9.0/reference-dsl-builtin-functions/index.html#stats-functions

Additional control over filenames for the split verb:

• Filename options for split by @sloanlance in #1366
• Requested on issue #1365 by @sloanlance

Support for details of data-computation errors beyond the current (error):

• Fatal-on-data-error mlr -x option by @johnkerl in #1373
• See also https://miller.readthedocs.io/en/6.9.0/reference-dsl-errors/#handling-for-data-errors
• Requested on issue #1106 by @honzajde

New verbs and DSL functions:

• New sub, gsub, and ssub verbs by @johnkerl in #1361, requested by @janxkoci on #1356. See also:
  • https://miller.readthedocs.io/en/6.9.0/reference-verbs/#sub
  • https://miller.readthedocs.io/en/6.9.0/reference-verbs/#gsub
  • https://miller.readthedocs.io/en/6.9.0/reference-verbs/#ssub
• New contains DSL function by @johnkerl in #1374
  • https://miller.readthedocs.io/en/6.9.0/reference-dsl-builtin-functions/index.html#contains

Other updates:

• Support ZSTD compression in-process by @johnkerl in #1360, from issue #1342 by @aborruso
  • See also https://miller.readthedocs.io/en/6.9.0/reference-main-compressed-data/
• Support comments in mlr -s files by @johnkerl in #1359, from issue #1343 by @janxkoci
• Add empty-key check to mlr check by @johnkerl in #1330, from issue #1050 by @aborruso

Bug fixes

• Do wildcard globbing on Windows by @johnkerl in #1362, from issue #1341 by @coolbq
• Treat empty like absent in + - * by @johnkerl in #1371, from issue #1001 by @Poshi
• Can't use ${field_name} if it contains UTF-8 characters also encodeable as Latin-1 by @johnkerl in #1363, from issue #1358 by @clemente
• Typofix in uif/uof percentiles by @johnkerl in #1375

Documentation updates

• Update readthedocs notes in the how-to-release page by @johnkerl in #1308
• Fix mlr grep docs re OFS/OPS by @johnkerl in #1309
• Update Fedora link by @bkmgit in #1339
• Small typos in documentation of mlr nest by @johnkerl in #1352, from issue #1350 by @kusalananda

Internal

• Update 2015-era Python sketch to Python 3 by @johnkerl in #1372
• Remove redundant nil check by @Juneezee in #1367
• Bump actions/checkout from 3.5.2 to 3.5.3 by @dependabot in #1319
• Bump github/codeql-action from 2.3.6 to 2.13.4 by @dependabot in #1318
• Bump golang.org/x/term from 0.8.0 to 0.9.0 by @dependabot in #1321
• Bump goreleaser/goreleaser-action from 4.2.0 to 4.3.0 by @dependabot in #1320
• Bump golang.org/x/text from 0.9.0 to 0.10.0 by @dependabot in #1322
• Bump golang.org/x/text from 0.10.0 to 0.11.0 by @dependabot in #1337
• Bump golang.org/x/sys from 0.9.0 to 0.10.0 by @dependabot in #1336
• Bump golang.org/x/term from 0.9.0 to 0.10.0 by @dependabot in #1338
• Bump golang.org/x/sys from 0.10.0 to 0.11.0 by @dependabot in #1347
• Bump golang.org/x/text from 0.11.0 to 0.12.0 by @dependabot in #1349
• Bump actions/setup-go from 4.0.1 to 4.1.0 by @dependabot in #1351
• Bump goreleaser/goreleaser-action from 4.3.0 to 4.4.0 by @dependabot in #1354
• Bump golang.org/x/term from 0.10.0 to 0.11.0 by @dependabot in #1348
• Bump actions/checkout from 3.5.3 to 3.6.0 by @dependabot in #1369

New Contributors

• @bkmgit made their first contribution in #1339
• @Juneezee made their first contribution in #1367
• @sloanlance made their first contribution in #1366

Full Changelog: v6.8.0...v6.9.0

Miller 6.8.0: New case verb, index DSL function, and more

New features

New case verb:

• Unify the case verb, and add options by @johnkerl in #1306
• Add new upcase and downcase verbs by @johnkerl in #1217

New index DSL function:

• index DSL function by @johnkerl in #1247

Enhancements:

• Add mlr step -a rprod for running products by @johnkerl in #1228
• Add optional second base argument to int DSL function by @johnkerl in #1244
• Implement --csv-trim-leading-space flag by @johnkerl in #1272
• New mlr json-parse -k flag by @johnkerl in #1291
• Let mlr help take pre-flags, such as --always-color by @johnkerl in #1292
• Values-only -a option for mlr grep by @johnkerl in #1305

Bugfixes

• Fix bug on DSL comment with no final newline by @johnkerl in #1216
• Better error message on unparseable TZ environment variable by @johnkerl in #1249
• Fix typo by @dnicolson in #1252
• Treat data-file numbers with leading + as numeric by @johnkerl in #1269
• Fix precedence of coalesce operators ?? and ??? by @johnkerl in #1270

Documentation

• Docs re tail -f and --records-per-batch 1 by @johnkerl in #1218
• Fix issue links in README-dev.md by @kcwu in #1248

Miscellaneous

• Include tools in the release tarball by @skitt in #1221
• Run go mod tidy by @skitt in #1220

Dependencies

Go:

• Bump minimum compiler version from Go 1.15 to 1.18 by @johnkerl in #1246

Others:

• Bump github/codeql-action from 2.2.5 to 2.2.6 by @dependabot in #1230
• Bump actions/cache from 3.2.6 to 3.3.1 by @dependabot in #1229
• Bump github/codeql-action from 2.2.6 to 2.2.7 by @dependabot in #1232
• Bump actions/setup-go from 3.5.0 to 4.0.0 by @dependabot in #1233
• Bump actions/checkout from 3.3.0 to 3.4.0 by @dependabot in #1234
• Bump github/codeql-action from 2.2.7 to 2.2.8 by @dependabot in #1242
• Bump actions/checkout from 3.4.0 to 3.5.0 by @dependabot in #1245
• Bump golang.org/x/term from 0.0.0-20210927222741-03fcf44c2211 to 0.6.0 by @dependabot in #1222
• Bump github.com/mattn/go-isatty from 0.0.17 to 0.0.18 by @dependabot in #1243
• Bump github/codeql-action from 2.2.8 to 2.2.9 by @dependabot in #1250
• Bump codespell-project/actions-codespell from 9c63fddd79f483308bfaea379a505dcd361b5d1d to 57beb9f38f49d773d641ac555d1565c3b6a59938 by @dependabot in #1253
• Bump golang.org/x/term from 0.6.0 to 0.7.0 by @dependabot in #1256
• Bump github/codeql-action from 2.2.9 to 2.2.10 by @dependabot in #1259
• Bump github/codeql-action from 2.2.10 to 2.2.11 by @dependabot in #1261
• Bump actions/checkout from 3.5.0 to 3.5.1 by @dependabot in #1263
• Bump actions/checkout from 3.5.1 to 3.5.2 by @dependabot in #1264
• Bump github/codeql-action from 2.2.11 to 2.2.12 by @dependabot in #1265
• Bump github/codeql-action from 2.2.12 to 2.3.0 by @dependabot in #1274
• Bump github/codeql-action from 2.3.0 to 2.3.1 by @dependabot in #1277
• Bump github/codeql-action from 2.3.1 to 2.3.2 by @dependabot in #1279
• Bump codespell-project/actions-codespell from 57beb9f38f49d773d641ac555d1565c3b6a59938 to 94259cd8be02ad2903ba34a22d9c13de21a74461 by @dependabot in #1282
• Bump github/codeql-action from 2.3.2 to 2.3.3 by @dependabot in #1284
• Bump golang.org/x/term from 0.7.0 to 0.8.0 by @dependabot in #1285
• Bump actions/setup-go from 4.0.0 to 4.0.1 by @dependabot in #1294
• Bump github.com/stretchr/testify from 1.8.2 to 1.8.3 by @dependabot in #1295
• Bump github.com/mattn/go-isatty from 0.0.18 to 0.0.19 by @dependabot in #1296
• Bump github/codeql-action from 2.3.3 to 2.3.4 by @dependabot in #1299
• Bump github/codeql-action from 2.3.4 to 2.3.5 by @dependabot in #1300
• Bump github.com/stretchr/testify from 1.8.3 to 1.8.4 by @dependabot in #1301
• Bump github/codeql-action from 2.3.5 to 2.3.6 by @dependabot in #1303

New Contributors

• @kcwu made their first contribution in #1248
• @dnicolson made their first contribution in #1252

Full Changelog: v6.7.0...v6.8.0

Miller 6.7.0: New leftpad/rightpad DSL functions, unspace verb, and more

Features

• New leftpad and rightpad DSL functions by @johnkerl in #1205
• mlr unspace verb by @johnkerl in #1167
• Support more backslashed special characters in DSL strings by @johnkerl in #1212
• Add --ofmte, --ofmtf, --ofmtg command-line flags by @johnkerl in #1206

Documentation updates

• Fixed missing double quote in documentation sample by @Clindbergh in #1181
• Complete #1181 by @johnkerl in #1184
• Add doc info on DSL code-comment syntax by @johnkerl in #1165
• Fix typos by @jwilk in #1135
• Fix typo in mlr put documentation by @johnkerl in #1140

Bug fixes

• Fix #1164: regression on CSV blank-line handling by @johnkerl in #1168
• Fix #1146: bug in lo/hi limits for non-auto histogram by @johnkerl in #1157
• Fix #1102: empty-string field in single-column TSV should not be a schema-restart by @johnkerl in #1163

Minor changes

• Add Go LICENSE file by @skitt in #1171

Dependency updates

• Bump github/codeql-action from 2.1.33 to 2.1.35 by @dependabot in #1137
• Bump actions/setup-go from 3.3.1 to 3.4.0 by @dependabot in #1136
• Bump github/codeql-action from 2.1.35 to 2.1.36 by @dependabot in #1143
• Bump actions/checkout from 3.1.0 to 3.2.0 by @dependabot in #1145
• Bump goreleaser/goreleaser-action from 3.2.0 to 4.1.0 by @dependabot in #1147
• Bump actions/setup-go from 3.4.0 to 3.5.0 by @dependabot in #1148
• Bump github/codeql-action from 2.1.36 to 2.1.37 by @dependabot in #1151
• Bump actions/cache from 3.0.11 to 3.2.0 by @dependabot in #1155
• Bump actions/cache from 3.2.0 to 3.2.1 by @dependabot in #1156
• Bump actions/cache from 3.2.1 to 3.2.2 by @dependabot in #1160
• Bump github.com/mattn/go-isatty from 0.0.16 to 0.0.17 by @dependabot in #1162
• Bump codespell-project/actions-codespell from bcf481f4d5cce7b92b65f05aebe8f552d4f1442c to 9c63fddd79f483308bfaea379a505dcd361b5d1d by @dependabot in #1172
• Bump actions/checkout from 3.2.0 to 3.3.0 by @dependabot in #1173
• Bump actions/cache from 3.2.2 to 3.2.3 by @dependabot in #1174
• Bump actions/upload-artifact from 3.1.1 to 3.1.2 by @dependabot in #1175
• Bump github/codeql-action from 2.1.37 to 2.1.38 by @dependabot in #1176
• Bump github/codeql-action from 2.1.38 to 2.1.39 by @dependabot in #1179
• Bump github/codeql-action from 2.1.39 to 2.2.1 by @dependabot in #1183
• Bump goreleaser/goreleaser-action from 4.1.0 to 4.1.1 by @dependabot in #1185
• Bump goreleaser/goreleaser-action from 4.1.1 to 4.2.0 by @dependabot in #1187
• Bump actions/cache from 3.2.3 to 3.2.4 by @dependabot in #1186
• Bump actions/cache from 3.2.4 to 3.2.5 by @dependabot in #1192
• Bump github/codeql-action from 2.2.1 to 2.2.3 by @dependabot in #1191
• Bump github/codeql-action from 2.2.3 to 2.2.4 by @dependabot in #1193
• Bump actions/cache from 3.2.5 to 3.2.6 by @dependabot in #1196
• Bump golang.org/x/sys from 0.0.0-20210326220804-49726bf1d181 to 0.1.0 in /cmd/experiments/cli_parser by @dependabot in #1203
• Bump github.com/stretchr/testify from 1.8.1 to 1.8.2 by @dependabot in #1208
• Bump github/codeql-action from 2.2.4 to 2.2.5 by @dependabot in #1207

New Contributors

• @jwilk made their first contribution in #1135
• @Clindbergh made their first contribution in #1181

Full Changelog: v6.5.0...v6.7.0

Miller 6.6.0: Bugfixes and unspace verb

Features

• mlr unspace verb by @johnkerl in #1167

Bugfixes

• Add doc info on DSL code-comment syntax by @johnkerl in #1165
• Fix typos by @jwilk in #1135
• Fix typo in mlr put documentation by @johnkerl in #1140
• Fix #1146: bug in lo/hi limits for non-auto histogram by @johnkerl in #1157
• Fix #1102: empty-string field in single-column TSV should not be a schema-restart by @johnkerl in #1163
• Fix #1164: regression on CSV blank-line handling by @johnkerl in #1168

Internal

• Bump github/codeql-action from 2.1.33 to 2.1.35 by @dependabot in #1137
• Bump actions/setup-go from 3.3.1 to 3.4.0 by @dependabot in #1136
• Bump github/codeql-action from 2.1.35 to 2.1.36 by @dependabot in #1143
• Bump actions/checkout from 3.1.0 to 3.2.0 by @dependabot in #1145
• Bump goreleaser/goreleaser-action from 3.2.0 to 4.1.0 by @dependabot in #1147
• Bump actions/setup-go from 3.4.0 to 3.5.0 by @dependabot in #1148
• Bump github/codeql-action from 2.1.36 to 2.1.37 by @dependabot in #1151
• Bump actions/cache from 3.0.11 to 3.2.0 by @dependabot in #1155
• Bump actions/cache from 3.2.0 to 3.2.1 by @dependabot in #1156
• Bump actions/cache from 3.2.1 to 3.2.2 by @dependabot in #1160
• Bump github.com/mattn/go-isatty from 0.0.16 to 0.0.17 by @dependabot in #1162

New Contributors

• @jwilk made their first contribution in #1135

Full Changelog: v6.5.0...v6.6.0

Miller 6.5.0: Bugfixes and memory-reduction optimizations

What's Changed

Features:

• Restore the --jvquoteall flag by @johnkerl in #1083
• Restore --quote-all for CSV output by @johnkerl in #1084

Bugfixes:

• Fix labels for mlr histogram --auto by @johnkerl in #1089
• Correctly support multiple regexes in mlr reshape -r by @johnkerl in #1091
• Check -- terminator on --mfrom by @johnkerl in #1098
• Type-safety in exec by @johnkerl in #1099
• Don't double-quote a CSV field only for having a leading space by @johnkerl in #1101

Performance/memory-reduction:

• Use int8 for mvtype (memory reduction) by @johnkerl in #1130
• Exclude median from summary default by @johnkerl in #1131
• More mlrval size-reduction by @johnkerl in #1132
• Convert mlrval polymorphism from struct to unionish interface by @johnkerl in #1133

Minor/internal:

• Account for varying mlr locations by @skitt in #1086
• Account for varying mlr locations, continued by @johnkerl in #1087
• [StepSecurity] ci: Harden GitHub Actions by @step-security-bot in #1107
• Bump github.com/pkg/profile from 1.6.0 to 1.7.0 by @dependabot in #1110
• Bump github/codeql-action from 2.1.28 to 2.1.33 by @dependabot in #1126
• Bump actions/cache from 3 to 3.0.11 by @dependabot in #1109
• Bump actions/upload-artifact from 3.1.0 to 3.1.1 by @dependabot in #1112
• Bump github.com/stretchr/testify from 1.8.0 to 1.8.1 by @dependabot in #1113
• Miller 6.5.0 by @johnkerl in #1134

New Contributors

• @step-security-bot made their first contribution in #1107

Full Changelog: v6.4.0...v6.5.0

5.10 bugfix for issue #1108

Miller 5 is long-gone; 6.0.0 was released almost a year ago. Yet issue #1108 reports a critical memory-corruption bug on 5.10.3; this fixes that.