A helpful reference of common bash script features and functions. If the answer isn't here, Stack Overflow is your friend.
if [ true == true ]; then
echo "Yes, it is true"
fi
Alternative formatting
if [ true == true ]
then # "then" is placed on its own line
echo "Yes, it is true"
fi
if [ false == true ]; then
echo "Yes, it is true"
else
echo "No, it is false"
fi
Variables are easy to declare, use, and concatenate
a="hello"
b="world"
echo "$a"
echo "$a $b" # concatenation
Boolean variables are not recognized. In the example below, false returns 0, which allows the if statement to run.
if [ false ]; then
echo "Yes, it is true"
fi
Variables are always referenced using a $ symbol, but are not declared using this symbol. In general, it is best to reference the variable inside of double quotes to make sure it is interpreted as a variable and not a string. However, it is not always necessary to use double quotes. This example shows both approaches to referencing a variable.
counter="4"
cat /etc/passwd | head -n "$counter"
echo $counter
Bash has special variables which will not be covered in detail here. Some useful special variables are:
$0: current shell environment$$: PID of current shell process$_: The name of the file from which the code is being run$1,$2,$n, etc.: the value of the nth argument provided to the bash command or function. Useful in functions that accept inputs.
See man test for additional logic tests and explanations.
Technically there's some distinction between a variable being unset vs. null,
(foo= vs. foo=""), but this distinction usually doesn't matter in practice.
z_test_variable="Not an empty variable"
if [ -z "$z_test_variable" ]; then
echo "the length of z_test_variable is zero"
else
echo "the length of z_test_variable is not zero"
fi
n_test_variable="Not an empty variable"
if [ -n "$n_test_variable" ]; then
echo "the length of n_test_variable is not zero"
else
echo "the length of n_test_variable is zero"
fi
list="a b c d e f"
my_array=($list)
echo $list # print out entire string
echo ${my_array[1]} # print out single character from array
echo ${my_array[3]}
Running a command inside of $() lets you store the command output as a
variable. This is helpful when a command's output is later used as input.
passwordfile=$(cat /etc/passwd)
echo "$passwordfile"
Integer arithmetic should occur inside of double parentheses. Otherwise, integers are treated as strings and concatenated.
c=12
d=15
echo "$c+$d" # integers are concatenated, not added
echo "$(($c+$d))" # integers are added
for i in $(seq 1 3 30); do
echo "Number: $i"
done
Observe how the following example stumbles over whitespace. It is usually easier to iterate over a variable using a while loop.
password_file="/etc/passwd"
password_file_lines=$(cat $password_file)
for single_line in $password_file_lines; do
echo "$single_line"
done
list="a b c d e f"
my_array=($list) # convert string to array
for i in "${my_array[@]}"; do
echo "$i"
done
while read myline; do
echo "$myline"
done < /etc/passwd
while IFS= read -r thisfile; do
cat "$thisfile"
done <<< $(ls -1)
print_hello_world () {
echo "Hello world!"
}
print_hello_world
Use the special variable names $1, $2...$n to use input arguments
add_two_numbers () {
number1=$1
number2=$2
echo "$(($number1+$number2))"
}
add_two_numbers 20 9
small_string="hello world!"
echo "${small_string:6:6}"
Combine ls -1 (lists files on separate lines), ls -p (appends / to the
end of directory names), and grep -v / (removes lines with / character) to
get an iterable list of only files, not directories. Combine with
iterating over multiline variable
files_list=$(ls -p -1 /var/log/ | grep -v /) # Each filename on a separate line
while IFS= read -r thisfile; do
cat "/var/log/$thisfile"
done <<< $files_list
If a bash command prints a row (or rows) at the start or end that you want to remove, you can use the head and tail commands to remove these rows.
ls -l /var/log/ | tail -n+2 # remove the first row of text
cat /etc/passwd | head -n 5 # print only the first 5 rows of text
The awk utility is very useful as extracting individual columns of text, even
when there is variable whitespace in different rows of text. The following
example first removes the first row of text, then prints out the owner of all
files in /var/log/ (data found in column 3).
ls -l /var/log/ | tail -n+2 | awk '{ print $3 }'
The sort utility allows duplicate rows of data to be deleted. For example, the
previous example can be reused but with duplicate usernames remove using the
-u (unique) flag of the sort utility. Another useful flag is -b, which
ignores whitespace.
ls -l /var/log/ | tail -n+2 | awk '{ print $3 }' | sort -ub
The cut utility allows data to be parsed using a delimiter of a single
character.
# Using delimiter " ", cut the first word of each line
ls -l /var/log/ | cut -d " " -f 1
# A range of data to cut using this delimiter can be specified
ls -l /var/log/ | cut -d " " -f 1-3
# To cut all text after the 4th word of a line, use
# an open-ended hyphen to indicate a range
ls -l /var/log/ | cut -d " " -f 4-
Most Linux tools act on data from left to right, not right to left. If you need
to parse data in the other direction, the rev utility is the best approach.
It reverses rows of data, and it can be used a second time to return the data
to the proper orientation. The following example shows how
ls -1 -p /etc/ | grep -v / # prints out file names
# This first attempt (below) at extracting file extensions fails
# because not every line contains a delimiter.
ls -1 -p /etc/ | grep -v / | cut -d "." -f 2
# Solve this by grepping for delimiter
ls -1 -p /etc/ | grep -v / | grep "\." | cut -d "." -f 2
# Now use sort to remove duplicate rows
ls -1 -p /etc/ | grep -v / | grep "\." | cut -d "." -f 2 | sort -u
# Another solution would be to use `rev` to parse from right to left
# This is useful if any file has two delimited (i.e. .tar.gz)
# Observe how `rev` is used twice
ls -1 -p /etc/ | grep -v / | grep "\." | rev | cut -d "." -f 1 | rev | sort -u
Use the tr utility to remove or replace a single character in text.
# Remove all hyphens from ls -l output
ls -l /etc/ | tr -d "-"
# Replace all hyphens from ls -l output with the + char
ls -l /etc/ | tr "-" "+"
If you want to replace specific text substrings with other substrings,
the sed utility is an extremely versatile tool for doing so. Review the sed
man file for extended details.
# Output /etc/passwd and replace the string :x: with ++++++
cat /etc/passwd | sed 's/:x:/++++++/'
# Remove the 3rd line of text (more granular control than head or tail)
cat /etc/passwd | sed '3d'
# Remove all whitespace of any kind
sed -r 's/\s+//g'
If you want to search data inside of files, the grep utility is an
extremely versatile tool for doing so. Review the grep man file for
extended details.
# Search for the phrase "password" in log files.
# Ignore case and search recursively.
grep -inr "password" /var/log
# Search for multiple phrases in log files, and ignore lines with "username"
# Print out the 3 lines after the found phrase
grep -inrE -A 3 "(password|passwd)" /var/log | grep -vE "(username|user)"
# Sometimes it is nice to have the word you search for highlighted
# Piping output to grep -v removes the highlighting
# The 1st solution to this is to use "--color=always"
grep -inrE -A 3 --color=always "(password|passwd)" /var/log | grep -vE "(username|user)"
# The 2nd alternative solution is to pipe to another grep to recreate the color
grep -inrE -A 3 "(password|passwd)" /var/log | grep -vE "(username|user)" | grep -iE -A 3 "(password|passwd)"
If you want to search data in the name of a file (either a file extension,
substring of the file name, or more), the find utility is an
extremely versatile tool for doing so. Review the find man file for
extended details.
# Find files under /var/log/ with an extension of .0 or .1
find /var/log/ -name "*.0" -o -name "*.1"
# Find files under /etc/ with name matching *.conf and ignore case
# For each file found, run the ls command on the file and the file command
find /etc/ -iname "*.conf" -ls -exec file {} \;
If you want to create a bash one-liner and pipe input into a utility that
doesn't support piped input (for example, it requires the input to be specified
after a certain flag), the xargs utility is a versatile tool for this case.
Review the xargs man file for extended details, since this tool has
many features.
# First, print the full path of all files in /etc/
ls -ldp /etc/* | grep -vE "/$" | awk '{ print $9 }'
# Now use xargs to pipe into the wc utility
ls -ldp /etc/* | grep -vE "/$" | awk '{ print $9 }' | xargs wc
When you're building a script to be used across many different environments, it's helpful to use the shellcheck tool to verify whether any of your code may encounter issues in other shell environments. Making a script compatible between bash, sh, and other shells for certain edge cases can be tough!
shellcheck myscript.sh
Bash has a built-in getopts utility which is useful to working with input flags
when you are writing a bash script.
#!/bin/bash
print_help() {
echo "Here is the help text!"
echo "Use: $0 [options]"
echo
echo " OPTIONS"
echo " -e num1 num2 Evaluate the mathematical expression provided"
echo " -s num1 Print the square of the number"
echo " -c num1 Print the cube of the number"
echo " -h Help information"
}
square_number() {
echo "$(($1*$1))"
}
cube_number() {
echo "$(($1*$1*$1))"
}
eval_expr() {
echo "Received expression $1. Result is:"
echo "$(($1))"
}
while getopts "hs:c:e:" option; do
case "${option}" in
s) square_number;;
c) cube_number;;
e) eval_expr "${OPTARG}";;
h) print_help; exit 0;;
*) print_help; exit 1;;
esac
done