The System Administrator's Guide To Bash Scripting: Terry Cox Aug 1, 2019
The System Administrator's Guide To Bash Scripting: Terry Cox Aug 1, 2019
Terry Cox
[email protected]
Aug 1, 2019
Study Guide | The System Administrator’s Guide to Bash Scripting
Contents
Terminal Shortcuts 10
Special Files 11
Wildcards 12
History 13
Screen 14
Executing Scripts 15
I/O 16
Redirection 17
2
Study Guide | The System Administrator’s Guide to Bash Scripting
Piping 18
Grouping Commands 21
Command Substitution 22
Jobs 23
Text Processing 24
Scripts 25
Basic Syntax 25
3
Study Guide | The System Administrator’s Guide to Bash Scripting
Shell 26
Shell Variables 26
Environment Variables 27
Aliases 30
If Statements 31
Basic Syntax 31
Else If Syntax 31
4
Study Guide | The System Administrator’s Guide to Bash Scripting
Case Statements 33
Basic Syntax 33
Operators 34
File Tests 34
String Tests 35
Arithmetic Tests 35
Misc Syntax 35
While Loop 37
Basic Syntax 37
5
Study Guide | The System Administrator’s Guide to Bash Scripting
For Loop 38
Basic Syntax 38
Variables 40
Basic Syntax 40
Booleans 41
Arrays 42
Basic Syntax 42
Declaration 42
Assignment 42
6
Study Guide | The System Administrator’s Guide to Bash Scripting
Positional Parameters 44
Basic Syntax 44
Basic Syntax 46
Exit Statuses 47
Global Variable 47
In Conditional Statements 47
Create a Function 50
Basic Syntax 50
7
Study Guide | The System Administrator’s Guide to Bash Scripting
Call a Function 50
Positional Parameters 50
Return Codes 51
Checklist 52
Syslog Standard 54
Basic Syntax 55
Debugging 56
8
Study Guide | The System Administrator’s Guide to Bash Scripting
Exit on Error 56
Verbose Debugging 56
Manual Debugging 57
9
Study Guide | The System Administrator’s Guide to Bash Scripting
Terminal Shortcuts
Up arrow: shows the previous command; you can cycle through all previous commands one by one by pressing it
repeatedly.
Tab: Tab completion, used to automatically complete long strings, such as file names or locations
CTRL + A: Positions the cursor at the front of line
CTRL + E: Positions the cursor at the end of the line
CTRL + C: Cancels the current operation
CTRL + K: Deletes everything after the cursor location
CTRL + U: Deletes all text before the cursor location
CTRL + L: Clears the terminal
CTRL + R: Searches command history
CTRL + Z: Sends the current operation to the background
10
Study Guide | The System Administrator’s Guide to Bash Scripting
Special Files
. Current directory
.. Parent directory
../ Parent directory, including slash; used to navigate from the parent
../../ The parent of the parent directory
~/ The current user’s home directory
.hiddenfile Files that start with a dot are hidden files. They are generally configuration files
11
Study Guide | The System Administrator’s Guide to Bash Scripting
Wildcards
12
Study Guide | The System Administrator’s Guide to Bash Scripting
History
13
Study Guide | The System Administrator’s Guide to Bash Scripting
Screen
14
Study Guide | The System Administrator’s Guide to Bash Scripting
Executing Scripts
15
Study Guide | The System Administrator’s Guide to Bash Scripting
I/O
16
Study Guide | The System Administrator’s Guide to Bash Scripting
Redirection
These redirect the output or input of a command into files, devices, and the input of other commands.
> Redirects the standard output of a command into a file; replaces the contents of a file
>> Appends into the end of a file
< Imports the contents of a file into the command
<< Appends the contents of a file into the command
2> Redirects standard error of a command into a file
2>> Appends standard error of a command into the end of a file
&> Redirects standard error and standard output when redirecting text
&>> Appends standard error and standard output when redirecting text
Example: cat < test.txt >> existingfile.txt
Uses the contents of test.txt on the cat command, then appends the results to existingfile.txt
17
Study Guide | The System Administrator’s Guide to Bash Scripting
Piping
18
Study Guide | The System Administrator’s Guide to Bash Scripting
xargs : Reads items from the standard input and allows commands to be run on the items:
<commands> | <xargs> <command>
Example: ls | grep test | xargs rm –fv
Lists all items in the current directory, then filters the results for the string test, then performs a file removal
with verbose output. This basically removes all files that have the string test in them.
19
Study Guide | The System Administrator’s Guide to Bash Scripting
In Bash, you can run multiple commands based on the following format: <Command> <option> <Command>
Options:
; Run the following command even if the previous command fails or succeeds
&& Run the following command only if the previous succeeds or has no errors
|| Run the following command only if the previous fails or results in error
& Run the previous command in the background
20
Study Guide | The System Administrator’s Guide to Bash Scripting
Grouping Commands
Bash provides two ways to group a list of commands meant to be executed as a unit
21
Study Guide | The System Administrator’s Guide to Bash Scripting
Command Substitution
22
Study Guide | The System Administrator’s Guide to Bash Scripting
Jobs
Commands run from the terminal, whether in the foreground or in the background
In the terminal, while running a command, you can use CTRL+Z to stop, but not kill, a command/job. You can start
it up again later, either in the foreground or background.
jobs Shows jobs and commands running in the background
fg <job number> Short for Foreground, and sends the specified job to the foreground of the terminal
bg <job number> Short for Background, and sends the specified job to the background of the terminal
<command> & Runs the command in the background, allowing you to run other commands while it processes
nohup Runs a command immune to hang-ups and allows a command to run even after a terminal is closed or the
user who ran the command is logged out
23
Study Guide | The System Administrator’s Guide to Bash Scripting
Text Processing
"Double Quotation marks" Meta-characters enclosed within the quotes are treated literally with the exception of
variables which have already been set.
Example: name=Cameron ; echo "My name is $name"
'single quotation marks' All meta-characters processed literally, with no variable processing
24
Study Guide | The System Administrator’s Guide to Bash Scripting
Scripts
Basic Syntax
#! /bin/bash
# Commands
Shebang / HashBang: #! /bin/bash
Informs Linux which command line interpreter to use for the script. In this example, it's the Bourne Again Shell
25
Study Guide | The System Administrator’s Guide to Bash Scripting
Shell
/etc/profile
/etc/profile.d
/etc/bashrc
/etc/bash.bashrc
/etc/skel
Contents of this directory are copied to new users directories when a new user is created
~/.bash_login Executes whatever commands are within the file ( ~/.bash_login ) when a user logs in
~/.profile User-specific Bash configuration file
~/.bash_profile User-specific Bash configuration file
~/.bashrc User-specific Bash configuration file that executes whatever commands are within the file
( ~/.bash_login ) when a user logs in
~/.bash_logout Executes whatever commands are within the file ( ~.bash_logout ) when a user logs out
Shell Variables
set Shows shell variables for the current instance of the running shell
Set your own shell variables: EXAMPLE=VAR ; echo $EXAMPLE
Creates the shell variable EXAMPLE and sets the value to VAR , then prints the variable's value
Remove shell variables: unset EXAMPLE ; echo $EXAMPLE
Removes the shell variable EXAMPLE ; echo will show no display since $EXAMPLE is no longer set to any
value
26
Study Guide | The System Administrator’s Guide to Bash Scripting
Environment Variables
27
Study Guide | The System Administrator’s Guide to Bash Scripting
28
Study Guide | The System Administrator’s Guide to Bash Scripting
BRed='\e[1;31m' # Red
BGreen='\e[1;32m' # Green
BYellow='\e[1;33m' # Yellow
BBlue='\e[1;34m' # Blue
BPurple='\e[1;35m' # Purple
BCyan='\e[1;36m' # Cyan
BWhite='\e[1;37m' # White
Underline:
UBlack='\e[4;30m' # Black
URed='\e[4;31m' # Red
UGreen='\e[4;32m' # Green
UYellow='\e[4;33m' # Yellow
UBlue='\e[4;34m' # Blue
UPurple='\e[4;35m' # Purple
UCyan='\e[4;36m' # Cyan
UWhite='\e[4;37m' # White
Background:
On_Black='\e[40m' # Black
On_Red='\e[41m' # Red
On_Green='\e[42m' # Green
On_Yellow='\e[43m' # Yellow
On_Blue='\e[44m' # Blue
On_Purple='\e[45m' # Purple
On_Cyan='\e[46m' # Cyan
On_White='\e[47m' # White
High Intensity
IBlack='\e[0;90m' # Black
IRed='\e[0;91m' # Red
IGreen='\e[0;92m' # Green
IYellow='\e[0;93m' # Yellow
IBlue='\e[0;94m' # Blue
29
Study Guide | The System Administrator’s Guide to Bash Scripting
IPurple='\e[0;95m' # Purple
ICyan='\e[0;96m' # Cyan
IWhite='\e[0;97m' # White
Bold High Intensity
BIBlack='\e[1;90m' # Black
BIRed='\e[1;91m' # Red
BIGreen='\e[1;92m' # Green
BIYellow='\e[1;93m' # Yellow
BIBlue='\e[1;94m' # Blue
BIPurple='\e[1;95m' # Purple
BICyan='\e[1;96m' # Cyan
BIWhite='\e[1;97m' # White
High Intensity backgrounds
On_IBlack='\e[0;100m' # Black
On_IRed='\e[0;101m' # Red
On_IGreen='\e[0;102m' # Green
On_IYellow='\e[0;103m' # Yellow
On_IBlue='\e[0;104m' # Blue
On_IPurple='\e[0;105m' # Purple
On_ICyan='\e[0;106m' # Cyan
On_IWhite='\e[0;107m' # White
Aliases
30
Study Guide | The System Administrator’s Guide to Bash Scripting
If Statements
Basic Syntax
if [ condition ];
then
#commands to be run if true
else
#commands to be run if false
fi
Else If Syntax
if [ condition ];
then
#commands to be run if true
elif [ condition ];
then
#commands to be run if true
else
#commands to be run if false
fi
31
Study Guide | The System Administrator’s Guide to Bash Scripting
32
Study Guide | The System Administrator’s Guide to Bash Scripting
Case Statements
Case statements are used to check the value of a parameter and execute code depending on the value.
This is similar to the switch statement in other languages with some slight differences:
Basic Syntax
case "$VAR" in
pattern_1 )
# Commands to be executed
;;
pattern_2 )
# Commands to be executed
;;
* )
# Default
;;
esac
33
Study Guide | The System Administrator’s Guide to Bash Scripting
Operators
File Tests
34
Study Guide | The System Administrator’s Guide to Bash Scripting
String Tests
Arithmetic Tests
Misc Syntax
35
Study Guide | The System Administrator’s Guide to Bash Scripting
36
Study Guide | The System Administrator’s Guide to Bash Scripting
While Loop
Basic Syntax
while [ condition ] do
#command(s)
#increment
done
Example:
x=1
while [ $x -le 5 ]
do
echo "Welcome $x times"
x=$(( $x + 1 ))
done
The above loop will run a command while x is less than or equal to 5
The last line adds 1 to x on each iteration
37
Study Guide | The System Administrator’s Guide to Bash Scripting
For Loop
Basic Syntax
Example:
Output:
# Color: red
# Color: green
# Color: blue
C-Like Syntax
38
Study Guide | The System Administrator’s Guide to Bash Scripting
# Command 3
done
Expression1 : The first expression in the list is only checked the first time the for loop is ran. This is useful for
setting the starting criteria of the loop.
Expression2 : The second expression is the condition that will be evaluated at the start of each loop to see if it is
true or false.
Expression3 : The last expression is executed at the end of each loop. This comes in handy when we need to add a
counter.
Example:
39
Study Guide | The System Administrator’s Guide to Bash Scripting
Variables
Because everything in bash is case sensitive, it is best practice to make variables in ALL CAPS
Basic Syntax
40
Study Guide | The System Administrator’s Guide to Bash Scripting
Booleans
Booleans are simple in Bash. Just declare a variable and assign it a true or false value
VAR_NAME=true
VAR_NAME=false
Boolean exit statuses: 0 = true 1 = false
41
Study Guide | The System Administrator’s Guide to Bash Scripting
Arrays
Basic Syntax
Declaration
Assignment
42
Study Guide | The System Administrator’s Guide to Bash Scripting
This method does not use explicit indexes and an associative array cannot be set like this
Clearing an associative array using ARRAY=() works
ARRAY=([X]=E1 [Y]=E2 ...) : Compound assignment for indexed arrays with index-value pairs declared
individually (here, X and Y)
X and Y are arithmetic expressions
This syntax can be combined with the above
Elements declared without an explicitly-specified index are assigned sequentially starting at either the last
element with an explicit index, or zero
ARRAY=([S1]=E1 [S2]=E2 ...) : Individual mass-setting for associative arrays
The named indexes (here, S1 and S2) are strings.
ARRAY+=(E1 E2 ...) : Appends to ARRAY
${ARRAY[N]} : Expands to the value of the index N in the indexed array ARRAY
If N is a negative number, it's treated as the offset from the maximum assigned index (can't be used for
assignment), 1
${ARRAY[S]} : Expands to the value of the index S in the associative array ARRAY
"${ARRAY[@]}" , ${ARRAY[@]} , "${ARRAY[*]}" , ${ARRAY[*]} : Similar to mass-expanding positional parameters,
this expands to all elements
If unquoted, both subscripts __*__ and __@__ expand to the same result
If quoted, @__ expands to all elements individually quoted, *__ expands to all elements quoted as a whole
"${ARRAY[@]:N:M}" , ${ARRAY[@]:N:M} , "${ARRAY[*]:N:M}" , ${ARRAY[*]:N:M} : Similar to what this syntax
does for the characters of a single string, when doing substring expansion, this expands to M elements starting
with element N. This way you can mass-expand individual indexes
The rules for quoting and the subscripts
__*__ and __@__ are the same as above for the other mass expansions
43
Study Guide | The System Administrator’s Guide to Bash Scripting
Positional Parameters
Basic Syntax
Example:
#! /bin/bash
echo $1
#This echos the first argument after the script name
echo -e "\n" #New Line
echo $2
#This echos the second argument after the script name
echo -e "\n" #New Line
44
Study Guide | The System Administrator’s Guide to Bash Scripting
echo $3
#This echos the third argument after the script name
echo -e "\n" #New Line
Tom
Dick
Harry
Script:
#! /bin/bash
echo -e "Logging into host $2 with user \"${1}\" \n"
ssh -p 22 ${1}@${2}
Output:
Logging into host 192.168.1.4 with user "root"
45
Study Guide | The System Administrator’s Guide to Bash Scripting
Sometimes you need to allow users running scripts to input custom data. This can be accomplished with the read
command.
Basic Syntax
Example:
#! /bin/bash
read -p "Type Your Username" USERNAME
echo -e "\n"
read -p "Type The IP Address" IPADDR
echo -e "Logging into host $IPADDR with user \"${USERNAME}\" \n"\
ssh -p 22 ${IPADDR}@${USERNAME}
To have formatted text at the command line, you need to know the escape sequences for echo
Escape sequences:
echo -e " text <escape sequence> text
\a : Alert (bell)
\b : Backspace
\c : Supress trailing newline
\e : Escape
\f : Form feed
\n : Newline
\r : Carriage return
\v : Vertical tab
\\ : Backslash
46
Study Guide | The System Administrator’s Guide to Bash Scripting
Exit Statuses
This is the error status of a command. All commands return an exit status, allowing for granular control of your scripts,
based on those statuses
In Bash, there are up to 255 exit statuses with 0 being the first
Exit status meanings: - 0 : Success - 1 : General Errors - 2 : Misuse of Shell Built-ins; syntax errors, missing keyword or
command permission errors, etc - Other : Error
Global Variable
Example:
#! /bin/bash
ls /path/does/not/exist
echo "$?"
## Output of (echo "$?") = 2
In Conditional Statements
In most cases, you use exit statuses within a conditional statement to perform an action based on whether your
program is having errors or not.
Example:
47
Study Guide | The System Administrator’s Guide to Bash Scripting
#! bin/bash
HOST="google.com" ping c 1
$HOST
if [ "$?" eq "0"] then
echo "$HOST is reachable" else
echo "$HOST is unreachable"
fi
Because we're able to successfully ping google, our exit status would be 0
We ask if our exit status is equal to 0 because if it is our output would be google.com is reachable
It may not be necessary to write out conditional statements with exit statuses. In Bash, there are two logical operators
that can take the place of some conditional statements:
command && command - The second command will only run if the previous command succeeds
command || command - The second command will only run if the previous command fails
There are conditions in which you may need to tell your program to halt its execution and return an exit status, whether
Bash determines there is an error or not.
To tell bash to halt execution of a script and return an exit status, you would use the exit command.
Basic Syntax
Example:
48
Study Guide | The System Administrator’s Guide to Bash Scripting
This pings google.com with one packet, then it asks if the exit status is not equal to 0
If exit status is not equal to 0 , then we exit with a status of 1
If the exit status is 0, then we simply exit with a status of 0
49
Study Guide | The System Administrator’s Guide to Bash Scripting
Create a Function
Functions are blocks of reusable code; used when you need to do the same tasks multiple times.
Basic Syntax
myFunction() {
# Code Goes Here
}
Call a Function
Unlike other languages, calling a function in Bash does not entail using parentheses: - myfunction parameter1
parameter2 parameter3
Positional Parameters
In functions, it’s possible to use positional parameters as arguments. To use positional parameters, you must first
reference them within your function. Once defined, you can use your function with arguments that take on the place of
the parameters:
Example:
function myfunction () {
# echo -e "$1 \n"
# echo -e "$2 \n"
# echo -e "$3 \n"
50
Study Guide | The System Administrator’s Guide to Bash Scripting
Output:
John
Mary
Fred
Susan
Return Codes
Each function has an exit status, and functions have their own method of dealing with exit statuses. Return codes are
simply exit statuses for functions. By default, the return code of a function is simply the exit status of the last command
executed within the function:
functionName() {
# Code Goes Here
return <Return Code>
}
51
Study Guide | The System Administrator’s Guide to Bash Scripting
Checklist
if [ ! d "$HTML_DIR" ]; then
echo "$HTML_DIR does not exist. Exiting."
exit 1
fi
52
Study Guide | The System Administrator’s Guide to Bash Scripting
#!/bin/bash
#
# Replace with the description and/or purpose of this shell script.
GLOBAL_VAR1="one"
GLOBAL_VAR2="two"
function function_one() {
local LOCAL_VAR1="one"
# Replace with function code.
}
# Main body of the shell script starts here.
#
# Replace with the main commands of your shell script.
# Exit with an explicit exit status.
exit 0
53
Study Guide | The System Administrator’s Guide to Bash Scripting
Syslog Standard
54
Study Guide | The System Administrator’s Guide to Bash Scripting
Basic Syntax
55
Study Guide | The System Administrator’s Guide to Bash Scripting
Debugging
For detailed information regarding debugging tools for Bash, use the help set command.
X-tracing or print debugging is an option built into Bash that lets you display commands and their arguments as
they are executed
Additionally, the values of variables and regex expansions will be shown.
To enable print debugging, place a -x after the hashbang: - #!/bin/bash -x
Or call it with set : - set -x # Start debugging set +x # Stop debugging
Exit on Error
Exit on error immediately halts the execution of code if any command within the script has a non-zero exit status
To enable exit on error, place a -e after the hashbang: - #!/bin/bash -e
Or call it with set : - set -e # Start exit on error set +e # Stop exit on error
Both the -x and -e options can be combined: -xe
Verbose Debugging
56
Study Guide | The System Administrator’s Guide to Bash Scripting
Manual Debugging
With manual debugging, we create our own debugging code. Normally, we create a special variable known as DEBUG to
inform our script whether debugging is on or off:
#!/bin/bash DEBUG=true
if $DEBUG
then
echo "Debug Mode On." else
echo "Debug Mode Off."
fi
57