Text

Text

echo

To create a file and supply text

echo "foo" > file.txt

To append text at the end of a file

echo "bar" >> file.txt

tr

Replace some character by another

tr <FROM_SET> <TO_SET>

To replace all as by bs

echo aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa | tr a b
# bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb

To replace [a-c] (a,b,c) by d

echo abcd | tr a-c d
# ddddd

grep

Returns 0 if a match happens (read until match, stop afterwards)

echo Foobar | grep --max-count=1 --ignore-case foo | echo $?

cat (concatenate)

https://github.com/coreutils/coreutils/blob/master/src/cat.c

Original use

To produce a file containing both file content

cat file1.txt file2.txt > file3.txt

Hacks

To display a file content (showing special characters and line number)

cat --show-all --number README.md

To edit a file interactively (hit Ctrl-D after keying in, you can key in several lines) Does this works ?

cat file.txt - > file.txt

To create a file and supply text interactively (hit Ctrl-D after keying in, you can key in several lines)

cat > file.txt

head

tail

split

Split one files into several files (per lines)

Take this half-million lines and split them into 500k chunks named chunk-<ID_DECIMAL>.csv

head -n 5000000 $RAW_DATA_FILE_PATH | split -d -l 100000 --additional-suffix=.csv - chunk

cut

wc

awk

sed

uuid

uuidgen

It can be time-consuming:

• 10s for 10k
• 1 min 30 for 100k
• 30 minutes for 3 millions

time (1>/dev/null for i in {1..10000}; do uuidgen; done)
( for i in {1..10000}; do; uuidgen; done > /dev/null; )  5,69s user 2,95s system 99% cpu 8,670 total

Random or time-based does not matter much

> time (1>/dev/null for i in {1..10000}; do uuidgen --random; done)
( for i in {1..10000}; do; uuidgen --random; done > /dev/null; )  5,71s user 2,86s system 99% cpu 8,620 total
> time (1>/dev/null for i in {1..10000}; do uuidgen --time; done)
( for i in {1..10000}; do; uuidgen --time; done > /dev/null; )  5,84s user 3,03s system 92% cpu 9,621 total

pandoc

Convert from one format to another

pandoc --from <SOURCE_TYPE> --to <TARGET_TYPE> $TARGET_FILE>

From mediawiki to markdown, removing the original file.

FILE_NAME=<FILE>; pandoc --from mediawiki --to markdown $FILE_NAME.mediawiki > $FILE_NAME.md; rm $FILE_NAME.mediawiki

Formats are supported (from and to):

• Microsoft Word: docx
• Markdown: markdown

Specific

JSON

jq

Arrays

https://jqlang.github.io/jq/manual/#basic-filters

> echo '[{"foo":"bar"}]' | jq '.[0].foo'
"bar

CSV

Miller

https://miller.readthedocs.io

mlr --csv cat $FILENAME