Bash Scripting Basic For Devops Beginners
Bash Scripting Basic For Devops Beginners
Introduction: Bash, which stands for "Bourne Again Shell," is a widely used
command-line tool for Unix and Unix-like systems, such as Linux and macOS. It allows
users to interact with their operating system by typing text commands. Bash is
popular in DevOps for automating tasks, improving productivity, and maintaining
consistency in managing infrastructure.
Key Features of Bash Scripting
1. Easy to Learn: Bash scripting has a simple and straightforward syntax, making
it easy for both beginners and experienced developers to write automation
scripts.
2. Command-Line Power: Bash scripts can use the many command-line tools
available in Unix-like systems, making it easy to integrate with system
commands and utilities.
3. Text Processing: Bash is great at handling text, which is useful for tasks like
analyzing logs, extracting information, and manipulating text data.
4. Control Structures: Bash supports if-else statements, loops, and other control
structures, allowing you to create dynamic and flexible scripts.
Practical Uses of Bash in DevOps
1. Automating Deployments: Bash scripts can automate the process of
deploying applications, setting up environments, configuring files, and starting
services.
2. System Maintenance: DevOps engineers use Bash to configure system
settings, install software packages, and apply updates, ensuring consistency
across multiple servers.
3. Log Analysis: Bash is useful for reading and analyzing log files, helping with
monitoring and troubleshooting.
4. Scheduled Tasks: Bash scripts can be scheduled to run at specific times using
cron jobs, automating routine tasks like backups and log rotations.
5. Infrastructure as Code (IaaC): Bash scripts help manage infrastructure, often
working with tools like Ansible, Terraform, or Docker to configure, deploy, and
maintain systems.
6. Command-Line Proficiency: Being skilled in Bash allows you to efficiently
navigate and interact with Unix-based systems, which is essential for tasks like
debugging and system administration.
7. Task Automation: Bash can automate various tasks, such as parsing logs,
manipulating files, and processing data.
8. Troubleshooting: Bash is useful for writing quick scripts to diagnose and fix
problems, analyze logs, and check system configurations.
9. Customization: You can write custom Bash scripts to tailor your processes and
extend the functionality of existing tools.
10. Integration with DevOps Tools: Bash scripts can interact with various
DevOps tools and services, such as version control systems, CI/CD pipelines,
and monitoring tools.
11. Cross-Platform Compatibility: Bash scripts work on different Unix-
based systems, providing compatibility across platforms.
Best Practices for Bash Scripting in DevOps
1. Error Handling: Ensure your scripts handle errors gracefully and provide useful
feedback.
2. Documentation: Comment and document your scripts thoroughly to improve
maintainability and collaboration.
3. Security: Follow security best practices to avoid vulnerabilities like code
injection and unauthorized access.
4. Version Control: Use version control systems like Git to manage changes to
your scripts, facilitating collaboration and tracking.
5. Modularity: Modular programming involves breaking down a program into
smaller, independent modules (functions or classes). Each module focuses on a
specific task, making the code easier to understand, debug, and maintain.
By understanding and using Bash, you can automate tasks, streamline
operations, and improve efficiency in a DevOps environment.
As a DevOps engineer, you need to know how to give instructions to the
operating system and this method is what is called “scripting”. Here you write small
programs, called ‘scripts’, that automate repetitive tasks, manage system operations,
or perform specific functions within a software environment. These scripts are typically
written in scripting languages, which are designed to be easy to write and execute.
Here are the key points about scripting:
1. Automation:
Scripts automate repetitive tasks, reducing manual effort and minimizing
errors.
Commonly used in system administration, software development, and
data processing.
2. Scripting Languages:
Scripting languages are typically high-level, easy-to-write languages.
Examples include Bash, Python, Perl, Ruby, and JavaScript.
They often require an interpreter to run, rather than being compiled into
machine code.
3. Command Execution:
Scripts execute a series of commands to accomplish a task.
Commands can include file operations, text processing, or running other
programs.
4. Ease of Use:
Scripting languages are generally easier to learn and use compared to
full-fledged programming languages.
They provide a straightforward way to write and run code quickly.
5. Flexibility and Integration:
Scripts can interact with other software, tools, and systems.
They are flexible and can be modified easily to adapt to different
requirements.
Practical Uses of Scripting
1. System Administration:
Automate tasks like backups, updates, and system monitoring.
Example: A script to update and upgrade all packages in a Linux system.
2. Web Development:
Automate web-related tasks such as deploying websites, running servers,
or processing web forms.
Example: JavaScript to validate user input in a web form.
3. Data Processing:
Automate data analysis, manipulation, and reporting tasks.
Example: A Python script to read data from a CSV file, process it, and
generate a report.
4. Software Development:
Automate build processes, testing, and deployment.
Example: A script to compile code, run tests, and deploy the application
to a server.
Why Use Scripting?
Efficiency: Automates repetitive tasks, saving time and reducing errors.
Productivity: Simplifies complex tasks and workflows, making them faster to
execute.
Consistency: Ensures tasks are performed the same way every time, reducing
the risk of mistakes.
Adaptability: Easy to modify and adapt scripts to changing requirements.
In a Linux/Unix system, there are several different types of shells that users can utilize.
Here are some of the most commonly used shells:
1. Bourne Shell (sh)
The original Unix shell developed by Stephen Bourne.
Standard shell on Unix systems.
Output: /bin/sh
2. Bourne Again Shell (bash)
An enhanced version of the Bourne Shell.
Default shell on many Linux distributions.
Includes features from the Korn shell (ksh) and C shell (csh).
Output: /bin/bash
3. Korn Shell (ksh)
Developed by David Korn.
Compatible with the Bourne shell, but includes many enhancements.
Identification of shell: /usr/bin/ksh
4. C Shell (csh)
Developed by Bill Joy.
Syntax resembles the C programming language.
Includes features like history and job control.
Identification of shell: /usr/bin/csh
5. TENEX C Shell (tcsh)
An enhanced version of the C shell.
Adds filename completion and command-line editing.
Identification of shell: /usr/bin/tcsh
6. Z Shell (zsh)
Includes features from bash, ksh, and tcsh.
Highly customizable and supports many advanced features.
Identification of shell: /bin/zsh
7. Almquist Shell (ash)
A lightweight shell, often used in embedded systems.
Compatible with the Bourne shell.
Identification of shell: /bin/ash
8. Dash (Debian Almquist Shell)
A modern replacement for ash.
Lightweight and fast, often used as the default /bin/sh on Debian systems.
Identification of shell: /bin/dash
9. Fish (Friendly Interactive Shell)
Focuses on user-friendliness and interactive use.
Includes features like syntax highlighting, autosuggestions, and a web-
based configuration interface.
Indentification of shell: /usr/bin/fish
10. /bin/rbash
Shell Name: Restricted Bash
Description: A restricted version of Bash that limits certain capabilities
for security purposes.
11./usr/bin/tmux
Tool Name: tmux
Description: A terminal multiplexer, not a shell. It allows users to
manage multiple terminal sessions within a single window.
12./usr/bin/screen
Tool Name: GNU Screen
Description: Another terminal multiplexer, similar to tmux, used to
manage multiple terminal sessions.
Now let’s delve into the full class which is going to be hands-on
1. Understanding Concepts
Before diving into the hands-on exercises, let's briefly understand some key concepts.
Variables: Placeholders for storing data that can be used and manipulated
within scripts.
Inputs and Data Streams: Methods to accept input from the user or other
data sources and handle output.
Conditional Statements: Statements that allow decision-making based on
certain conditions (if-else statements).
Loops: Code structures that repeat a sequence of instructions until a specific
condition is met (for, while loops).
Bash Functions: Reusable blocks of code that can be defined and called within
scripts.
Working with Variables
Every programming language should have what is called ‘variables’.
Variables helps in automating repitition of commands. A variable is a pointer to data in
memory. So anytime you want to store data in memory you use ‘variable’. So you say
a variables is a placeholder for data in memory- meaning that variables represent data
in memory.
In the computer, there are different storages. Variables are replacing you craming the
memory address. Variables arer easier to remember compared to memory address. So
for example if I want to store James inside the computer all I have to do is :
MyName=James. MyName is the variable.
So that means means that you set a variable by typing whatever you like and = to
store the data you want.
To output or call the variable or get the data stored in the variable you type: echo
$MyName.
In Bash, using quotes around strings is important for managing spaces and special
characters correctly. Here are the key points:
Single Quotes (')
Purpose: Preserve the literal value of each character within the quotes.
Usage: Use single quotes when the string contains special characters or spaces
that you do not want to be interpreted.
Example
echo 'Hello World'
echo 'Special characters like $ and * are preserved as literals'
Double Quotes (")
Purpose: Preserve the literal value of most characters within the quotes, but
allow for variable expansion and command substitution.
Usage: Use double quotes when you need to include spaces or special
characters, but also want to allow variable and command substitution.
Example
name="World"
echo "Hello $name"
echo "The date is $(date)"
No Quotes
Purpose: Allow Bash to interpret special characters and split words based on
spaces.
Usage: Use no quotes when you are sure the string contains no spaces or
special characters that need to be preserved.
Example
echo Hello
echo $USER
Practical Rules
1. No Spaces or Special Characters: You can omit quotes if the string contains
only alphanumeric characters without spaces or special characters.
echo HelloWorld
echo myfile.txt
2. With Spaces or Special Characters: Use quotes to prevent issues.
echo "Hello World" # Correct
echo Hello World # Incorrect, will print "Hello" and "World" on separate lines
3. Using Variables: Use double quotes to expand variables properly.
greeting="Hello World"
echo "$greeting" # Correct
echo $greeting # Correct, but not safe if greeting contains spaces or special
characters
Examples in Practice
Correct Usage
file="my file.txt"
echo "$file" # Correct, handles spaces
# Declare variables
name="David"
age=25
city="Lagos"
Are methods to accept input from the user or other data sources and handle output.
Basic Concepts
1. Standard Input (stdin): The default source of input for a program, typically
the keyboard.
2. Standard Output (stdout): The default destination for output from a program,
typically the terminal screen.
3. Standard Error (stderr): The default destination for error messages, typically
the terminal screen.
Redirection Operators
>: Redirects stdout to a file (overwrites the file).
>>: Redirects stdout to a file (appends to the file).
2>: Redirects stderr to a file.
<: Redirects stdin from a file.
Hands-On Examples
Redirecting Output to a File
1. Writing to a File
#!/bin/bash
This script checks if the user's age is 18 or older and prints a message accordingly.
Basic ‘else’ statement:
#!/bin/bash
age=12
if [ $age == “James” ]
then
echo “Positive”
else
echo “This is the wrong user”
fi
file="example.txt"
if [ -f "$file" ]; then
echo "File $file exists and is a regular file."
else
echo "File $file does not exist or is not a regular file."
fi
Checking if a Directory Exists:
#!/bin/bash
directory="example_directory"
if [ -d "$directory" ]; then
echo "Directory $directory exists."
else
echo "Directory $directory does not exist."
fi
Other examples:
For files
#!/bin/bash
if [ -f "images.png" ]; then # Check if "images.png" exists and is a regular
file
echo "The images folder exists already moving on..." # Print message if it
exists
sleep 0.5 # Sleep for 0.5 seconds
else # If "images.png" doesn't exist
touch "images.png" # Create an empty file named "images.png"
fi
For Directory
#!/bin/bash
if [ -d "images" ]; then # Check if "images" directory exists
echo "The images directory exists already, moving on..." # Print message
if it exists
sleep 0.5 # Sleep for 0.5 seconds
else # If "images" directory doesn't exist
mkdir "images" # Create the "images" directory
fi
LOOPS IN BASH SCRIPTING
Loops in bash scripting allows you to execute a set of commands repeatedly until the
command is carried out completely. Loops in Bash allow you to execute a block of code
repeatedly based on a condition. There are mainly three types of loops in Bash:
1. for loop: Executes a sequence of commands for each item in a list or range. It
goes through a set of items, elements or range of value.
2. while loop: Executes a sequence of commands as long as a condition is true. It
continues to run a particular condition until has changed.
3. until loop: Executes a sequence of commands until a condition becomes true.
You can use various commands within loops such as ‘read’ and ‘test’ commands in
bash scripting.
The ‘test’ command use [], while the read command uses ‘read’ within the loop in
bash scripting.
Examples:
1. for Loop:
#!/bin/bash
for fruit in apple banana cherry; do
echo "I like $fruit"
done
In this example, the echo command is used within the for loop to print
each fruit.
2. while Loop:
#!/bin/bash
counter=1
while [ $counter -le 5 ]; do
echo "Count: $counter"
counter=$((counter + 1))
done
Here, the echo command is used within the while loop to print the count.
3. until Loop:
#!/bin/bash
counter=1
until [ $counter -gt 5 ]; do
echo "Count: $counter"
counter=$((counter + 1))
COMPARISON OPERATORS IN LOOPS
String Comparison:
==: Equal to
!=: Not equal to
Numeric Comparison:
-eq: Equal to
-ne: Not equal to
-lt: Less than
-le: Less than or equal to
-gt: Greater than
-ge: Greater than or equal to
Logical Operators
&&: Logical AND
||: Logical OR
!: Logical NOT
Examples:
‘for’ Loop with String Comparison:
#!/bin/bash
# List of fruits
fruits=("apple" "banana" "cherry")
# Loop through each fruit
for fruit in "${fruits[@]}"; do
if [ "$fruit" == "banana" ]; then
echo "Found a banana!"
else
echo "This is a $fruit."
fi
done
‘while’ Loop with Numeric Comparison
#!/bin/bash
# Initialize a counter
counter=1
# Iterate through a list of filenames and echo a message if the file exists or
is readable
for file in file1.txt file2.txt file3.txt; do
[ -e "$file" ] || echo "$file does not exist"
done
This for loop iterates through a list of filenames and checks if each file exists. If
a file does not exist (-e), it echoes a message.
3. Logical NOT (!) in a for Loop:
#!/bin/bash
Example:
list_inventory(){
echo " Welcome to Directory Creator
-------------------------------
by Yusuf Clust"
echo $inventory_name
if [[ ! -f $inventory_name ]]
then
echo "File does not exist"
exit 1
else
while read -r line
do
echo $line
done < $inventory_name
fi
list_inventory
echo $username
greet "Alice" "Monday" ## This is to call greet function with two arguments
In Bash Scripting the ‘exit’ command is used to exit a script or a function. The
argument to exit is a numeric value called an ‘exit status’. Conventionally, a script or
function returning ‘0’ indicates success and any other value indicates an error or
some kind of failure.
‘exit 1’ is often used to indicate error or failure condition. So if ‘exit 1’ is
returned it means something went wrong within the function and the script should exit
with a non-zero status to signal an error to the calling environment.
‘$?’ checks the exit status for the last executed command or function.
‘$#’ is used to get the number of argument that the function receives.