II.

Bash Scripting
1.Introduction
This tutorial will give you a solid platform in how to create bash scripts and automate day to day
system admin tasks. Certainly, everything cannot be covered in this chapter but you will be
equipped with right amount of knowledge to make your own scripts and excel in it if you put in
your own efforts.

Bash scripting is used by many of the system admins and DevOps geeks to get things done quickly
and efficiently. There are so many automation tools in the market like Ansible, Puppet, Chef etc.
Which are way more sophisticated but sometimes to get things done quickly in Linux systems we
use Bash scripts. Also, scripting will make you understand what automation means and then you
can quickly grasp the features that is used in Configuration Management tools like Ansible or
puppet.

What are scripts?

A Bash script is a plain text file which contains a series of commands. These commands are a
mixture of commands we would normally type ourselves on the command line (such as ls or cp for
example) and commands we could type on the command line but generally wouldn't (you'll
discover these over the next few pages). A crucial point to remember though is:

Anything you can run normally on the command line can be put into a script and it will do
exactly the same thing. Similarly, anything you can put into a script can also be run normally
on the command line and it will do exactly the same thing.

First Script
As we discussed earlier that script is a normal text file with commands in it.
We will open a file vi editor and add some commands in it.
It is convention to give files that are Bash scripts an extension of .sh (print.sh for example)

$ vi print.sh

•#!/bin/bash

•# A sample Bash script

•echo Hello World!

Explanation:

Line 1 - #! is called as the SHEBANG character, it tells the script to interpret the rest of the lines with an
Interpreter /bin/bash. So, if we change that to /usr/bin/python then it tells the script to use python interpreter.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Line 2- This is a comment. Anything after # is not executed. It is for our reference only. Comments is for us so or
anybody who reads this script will have some reference.

Line 3- Is the command echo which will print a message to the screen. You can type this command yourself on
the command line and it will behave exactly the same.

Running or Executing a script?

Running a Bash script is fairly easy. Sometimes you will hear people saying execute the script, both means same
thing. Before we can execute a script, it must have the execute permission set. If you forget to grant this
permission before running the script you'll just get an error message “Permission denied”. Whenever you create a
file in Linux system by default it will not have an execute permission, this is for security reasons. You make your
script executable and then you can run it.

imran@DevOps:.../bash$ ./print.sh

bash: ./print.sh: Permission denied

imran@DevOps:.../bash$ ls -l

total 4

-rw-rw-r-- 1 imran imran 53 Oct 21 17:33 print.sh

imran@DevOps:.../bash$ chmod 755 print.sh

imran@DevOps:.../bash$ ls -l

total 4

-rwxr-xr-x 1 imran imran 53 Oct 21 17:33 print.sh

imran@DevOps:.../bash$ ./print.sh

Hello World!

Without giving execute permission also we can run the script but then we provide a shell and ask it
to run all the command in the script on that shell.

imran@DevOps:.../bash$ bash print.sh

Hello World!

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 2.Variables

Temporary stores of information in memory.

How do they Work?

A variable is a temporary store for a piece of information. There are two actions we may perform for variables:

Setting a value for a variable.

Reading or using the value for a variable.

To assign a variable we use = sign, VariableName=Value

To read/access the value of variable we use $VariableName

imran@DevOps:.../bash$ VAR1=123

imran@DevOps:.../bash$ echo $VAR1

123

Command line arguments

When we run a program on the command line you would be familiar with supplying arguments after it to control
its behaviour.

For instance we could run the command ls -l /tmp. -l and /tmp are both command line arguments to the command
ls.

We can do similar with our bash scripts. To do this we use the variables $1 to represent the first command line
argument, $2 to represent the second command line argument and so on. These are automatically set by the
system when we run our script so all we need to do is refer to them.

Let's look at an example.

copyscript.sh

1. #!/bin/bash

2. # A simple copy script

3. cp $1 $2

4. # Let's verify the copy worked

5. echo Details for $2

6. ls -lh $2

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
imran@DevOps:.../testcopy$ mkdir dir1 dir2

imran@DevOps:.../testcopy$ touch dir1/tesla

imran@DevOps:.../testcopy$ ./copyscript.sh dir1/tesla dir2/

Details for dir2/

total 0

-rw-rw-r-- 1 imran imran 0 Jun 19 23:01 tesla

Explanation:

Line 3 - run the command cp with the first command line argument as the source and the second
command line argument as the destination.

Line 5 - run the command echo to print a message.

Line 6 - After the copy has completed, run the command ls for the destination just to verify it worked.
We have included the options l to show us extra information and h to make the size human readable so
we may verify it copied correctly.

Some System Variables

There are a few other variables that the system sets for you to use as well.

$0 - The name of the Bash script.

$1 - $9 - The first 9 arguments to the Bash script. (As mentioned above.)

$# - How many arguments were passed to the Bash script.

$@ - All the arguments supplied to the Bash script.

$? - The exit status of the most recently run process.

$$ - The process ID of the current script.

$USER - The username of the user running the script.

$HOSTNAME - The hostname of the machine the script is running on.

$SECONDS - The number of seconds since the script was started.

$RANDOM - Returns a different random number each time is it referred to.

$LINENO - Returns the current line number in the Bash script.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Setting your own variables

imran@DevOps:~/.../bash_scripts$ cat 1_print.sh

intA=20

floatB=20.20

stringA="first_string"

DIR_PATH="/tmp"

echo

echo "#########################"

echo "Value of integer A is $intA"

echo "#########################"

echo "Value of Float B is $intB"

echo "#########################"

echo "Value of string A is $stringA"

echo "#########################"

echo "Directory path is $DIR_PATH"

echo "#########################"

echo "Content of TMP directory."

echo "#########################"

ls $DIR_PATH

Quotes

Storing a single word in a variable works fine without quotes, but if we want to store a sentence and
also want to store special characters like $,%,@ etc our normal variable assignment will not work.

imran@DevOps:.../bash$ myvar=Hello World

World: command not found

imran@DevOps:.../bash$

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
When we want variables to store more complex values however, we need to make use of quotes. This is because
under normal circumstances Bash uses a space to determine separate items.

When we enclose our content in quotes we are indicating to Bash that the contents should be considered as a
single item. You may use single quotes ( ' ) or double quotes ( " ).

Single quotes will treat every character literally.

Double quotes will allow you to do substitution (that is include variables within the setting of the value).

imran@DevOps:.../bash$ myvar='Hello World'

imran@DevOps:.../bash$ echo $myvar

Hello World

imran@DevOps:.../bash$ newvar="More $myvar"

imran@DevOps:.../bash$ echo $newvar

More Hello World

imran@DevOps:.../bash$ newvar='More $myvar'

imran@DevOps:.../bash$ echo $newvar

More $myvar

imran@DevOps:.../bash$

Command Substitution
We have how to store a string/text into a variable but sometimes you want to store output of a
command to a variable. Like you may need to store of ls command output to a variable. For this we
use Command Substitution. There are two syntax for doing this.
1.
imran@DevOps:.../testcopy$ file=`ls`

imran@DevOps:.../testcopy$ echo $file

copyscript.sh dir1 dir2

2.
imran@DevOps:.../testcopy$ files=$(ls)

imran@DevOps:.../testcopy$ echo $files

copyscript.sh dir1 dir2

imran@DevOps:.../bash$ myvar=$( ls /etc | wc -l )

imran@DevOps:.../bash$ echo There are $myvar entries in the directory /etc

There are 249 entries in the directory /etc

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Exporting Variables
Variable defined in the script leave with it and dies after the script dies or completes. If we want to
define a variable that is accessible to all the scripts from your current shell we need to export it.
Export a variable from bash shell as mentioned below.

imran@DevOps:.../bash$ var1=foo

imran@DevOps:.../bash$ echo $var1

foo

imran@DevOps:.../bash$ export var1

Create a script which prints exported and local variable

imran@DevOps:.../bash$ vi script1.sh

#!/bin/bash

# demonstrate variable scope

var2=foobar

echo "Printing exported varible from bash shell"

echo $var1

echo "Printing varible defined in the script"

echo $var2

Execute the script to see the results

imran@DevOps:.../bash$ ./script1.sh

Printing exported variable from bash shell

foo

Printing variable defined in the script

foobar

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Environment Variables or Exporting Variables Permanently
To export variables permanently, you can add the export command in any of the following start-up
files :
~/.profile
~/.bashrc
/etc/profile

.profile and .bashrc lives in the users home directory so they are accessible only for that user.
/etc/profile is for global access for all the variables.

For example:
imran@DevOps:~$ cat .bashrc | grep export

#export
GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'

export tesla='Alternate current'

export EDITOR=vim

export name=Cassini

Summary
$1, $2, ...
The first, second, etc command line arguments to the script.
variable=value
To set a value for a variable. Remember, no spaces on either side of =
Quotes " '
Double will do variable substitution, single will not.
variable=$(command )
Save the output of a command into a variable
export var1
Make the variable var1 available to child processes.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 3. User Input

Interactive Scripts

Ask the User for Input

Taking input from the user while executing the script, storing it into a variable and then using that variable in our
script. We would be taking inputs from user like IP addresses, usernames, passwords or confirmation Y/N to do
this we use command called read. This command takes the input and will save it into a variable.

read var1

Let's look at a simple example:

input.sh

1. #!/bin/bash

2. # Ask the username

3. echo “Please enter the username”

4. read varname

5. echo “Welcome back $varname”

imran@DevOps:.../bashscripts$ ./input.sh

Please enter the username

DevTestOps

Welcome back DevTestOps

Explanation:

Line 3 - Print a message asking the user for input.

Line 4 - Run the command read and save the users response into the variable varname

Line 5 – echo another message just to verify the read command worked. Note: I had to put a backslash ( \
) in front of the ' so that it was escaped.

You are able to alter the behaviour of read with a variety of command line options. (See the man page for read to
see all of them.) Two commonly used options however are -p which allows you to specify a prompt and -s which
makes the input silent. This can make it easy to ask for a username and password combination like the example
below:

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Login.sh

#!/bin/bash

# Ask the user for login details

read -p 'Username: ' uservar

read -sp 'Password: ' passvar

echo

echo Thankyou $uservar we now have your login details

So far we have looked at a single word as input. We can do more than that however.

multiinput.sh

#!/bin/bash

# Demonstrate how read actually works

echo Enter your Name, Profession & Interests in same order seprated by a space?

read name profession interest

echo Your entered name is: $name

echo Your profession is: $profession

echo Your are interested in: $interest

imran@DevOps:.../bashscripts$ ./multiinput.sh

Enter your Name, Profession & Interests in same order seprated by a space?

Imran DevOps Hiking

Your entered name is: Imran

Your profession is: DevOps

Your are interested in: Hiking

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 4. If Statements
Scripts making decisions.

Basic If Statements
If you use bash for scripting you will undoubtedly have to use conditions a lot. Based on a condition
you decide if you should execute some commands on the system or not.
A basic if statement effectively says, if a particular test is true, then perform a given set of actions. If it is not true
then don't perform those actions. If follows the format below:

if [ <some test> ]
then
<commands>
fi

Anything between then and fi (if backwards) will be executed only if the test (between the square brackets) is
true.

Let's look at a simple example:

if_example.sh

#!/bin/bash

# Basic if statement

if [ $1 -gt 100 ]

then

echo Hey that’s a large number.

pwd

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 fi

date

Explanation:

Line 3 - Let's see if the first command line argument is greater than 100

Line 5 and 6 - Will only get run if the test on line 4 returns true. You can have as many commands here
as you like.

Line 7 - fi signals the end of the if statement. All commands after this will be run as normal.

Line 8 - Because this command is outside the if statement it will be run regardless of the outcome of the
if statement.

imran@DevOps:.../bash$ ./if_example.sh 150

Hey that’s a large number.

/tmp/bash

Sun Oct 30 16:19:28 IST 2016

Sample if/else condition script

If condition is true we execute a block of code but if its false we can execute some other block of
code by using else command.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
#!/bin/bash

intA=20

intB=30

if [ intA==intB ];

then

echo "intA is equal to intB"

else

echo "Not Equal"

fi

if [ -f hosts ];

then

echo "File exists!"

else

echo "Does not exist"

fi

Test
The square brackets ( [ ] ) in the if statement above are actually a reference to the command test. This means that
all of the operators that test allows may be used here as well. Look up the man page for test to see all of the
possible operators (there are quite a few) but some of the more common ones are listed below.

Operator Description
! EXPRESSION The EXPRESSION is false.
-n STRING The length of STRING is greater than zero.
-z STRING The length of STRING is zero (ie it is empty).
STRING1 = STRING2 STRING1 is equal to STRING2
STRING1 != STRING2 STRING1 is not equal to STRING2
INTEGER1 -eq INTEGER2 INTEGER1 is numerically equal to INTEGER2
INTEGER1 -gt INTEGER2 INTEGER1 is numerically greater than INTEGER2
INTEGER1 -lt INTEGER2 INTEGER1 is numerically less than INTEGER2
-d FILE FILE exists and is a directory.
-e FILE FILE exists.
-r FILE FILE exists and the read permission is granted.
-s FILE FILE exists and it's size is greater than zero (ie. it is not empty).
-w FILE FILE exists and the write permission is granted.
-x FILE FILE exists and the execute permission is granted.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
A few points to note:

= is slightly different to -eq. [ 001 = 1 ] will return false as = does a string comparison (ie. character for
character the same) whereas -eq does a numerical comparison meaning [ 001 -eq 1 ] will return true.

When we refer to FILE above we are actually meaning a path. Remember that a path may be absolute or
relative and may refer to a file or a directory.

Because [ 5 is just a reference to the command test we may experiment and trouble shoot with test on the
command line to make sure our understanding of its behaviour is correct.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 5. Loops!
Execute it again and again and again....

Loops allow us to take a series of commands and keep re-running them until a particular situation is reached. They
are useful for automating repetitive tasks.
For Loop
A ‘for loop’ is a bash programming language statement which allows code to be repeatedly executed. A for loop is
classified as an iteration statement i.e. it is the repetition of a process within a bash script. For example, you can
run Linux command or task 5 times or read and process list of files using a for loop. A for loop can be used at a
shell prompt or within a shell script itself.

for var in <list>

do
<commands>
done

The for loop will take each item in the list (in order, one after the other), assign that item as the
value of the variable var, execute the commands between do and done then go back to the top, grab
the next item in the list and repeat over.

The list is defined as a series of strings, separated by spaces.

For loops iterate for as many arguments given:

Example:

The contents of $Variable is printed three times.

#!/bin/bash

for Variable in {1..3}

do

echo "$Variable"

done

Or write it the "traditional for loop" way:

for ((a=1; a <= 3; a++))

do

echo $a

done

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
They can also be used to act on files.
This will run the command 'cat' on file1 and file2

for Variable in file1 file2

do

cat "$Variable"

done

or the output from a command

This will cat the output from ls.

for Output in $(ls)

do

cat "$Output"

done

While loop
The bash while loop is a control flow statement that allows code or commands to be executed repeatedly
based on a given condition. For example, run echo command 5 times or read text file line by line or evaluate
the options passed on the command line for a script.

syntax
The syntax is as follows:

while [ condition ]

do

command1

command2

command3

done

command1 to command3 will be executed repeatedly till condition is true. The argument for a while loop
can be any Boolean expression. Infinite loops occur when the conditional never evaluates to false. For
example, following while loop will print welcome 5 times on screen:

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
#!/bin/bash

a=1

while [ $a -le 5 ]

do

echo "Number $a"

x=$(( $x + 1 ))

done

Printing list of host IP addresses from hosts file.

Create a file named hosts
$ vi hosts
192.168.1.10
192.168.1.11
192.168.1.12
192.168.1.13

#!/bin/bash
• for i in `cat hosts`;do
• echo "Printing list of hosts."
• echo $i
• done

Explanation
Line 1. For command substitution we are using backticks `` . It’s different from single quote ‘’
`cat hosts` will return the content of hosts file line by line, which will be stored in variable “i”.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 6. Real time use cases
Bash script to install Apache, MYSQL and PHP for Ubuntu OS.

Colouring your script.

Check below mentioned link for information on colouring your echo output.
http://misc.flogisoft.com/bash/tip_colors_and_formatting

The ANSI/VT100 terminals and terminal emulators are not just able to display black and white text
; they can display colours and formatted texts thanks to escape sequences. Those sequences are
composed of the Escape character (often represented by ”^[” or ”<Esc>”) followed by some other
characters: ”<Esc>[FormatCodem”.
In Bash, the <Esc> character can be obtained with the following syntaxes:
❖ \e
❖ \033
❖ \x1B

#!/bin/bash

#COLORS

# Reset

Color_Off='\033[0m' # Text Reset

# Regular Colors

Red='\033[0;31m' # Red

Green='\033[0;32m' # Green

Yellow='\033[0;33m' # Yellow

Purple='\033[0;35m' # Purple

Cyan='\033[0;36m' # Cyan

# Update packages and Upgrade system

echo -e "$Cyan \n Updating System.. $Color_Off"

sudo apt-get update -y && sudo apt-get upgrade -y

## Install AMP

echo -e "$Cyan \n Installing Apache2 $Color_Off"

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
sudo apt-get install apache2 apache2-doc apache2-mpm-prefork apache2-utils
libexpat1 ssl-cert -y

echo -e "$Cyan \n Installing PHP & Requirements $Color_Off"

sudo apt-get install libapache2-mod-php5 php5 php5-common php5-curl php5-dev

php5-gd php5-idn php-pear php5-imagick php5-mcrypt php5-mysql php5-ps php5-
pspell php5-recode php5-xsl -y

echo -e "$Cyan \n Installing MySQL $Color_Off"

sudo apt-get install mysql-server mysql-client libmysqlclient15.dev -y

echo -e "$Cyan \n Installing phpMyAdmin $Color_Off"

sudo apt-get install phpmyadmin -y

echo -e "$Cyan \n Verifying installs$Color_Off"

sudo apt-get install apache2 libapache2-mod-php5 php5 mysql-server php-pear

php5-mysql mysql-client mysql-server php5-mysql php5-gd -y

## TWEAKS and Settings

# Permissions

echo -e "$Cyan \n Permissions for /var/www $Color_Off"

sudo chown -R www-data:www-data /var/www

echo -e "$Green \n Permissions have been set $Color_Off"

# Enabling Mod Rewrite, required for WordPress permalinks and .htaccess files

echo -e "$Cyan \n Enabling Modules $Color_Off"

sudo a2enmod rewrite

sudo php5enmod mcrypt

# Restart Apache

echo -e "$Cyan \n Restarting Apache $Color_Off"

sudo service apache2 restart

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Backup scripts
Being working with systems you may need to take backup of files, directories, log files etc.
Below mention scenario shows you how you can automate the backup procedures.

In this example, we will create a file and mention the name of log files that needs to be backup up
with tar command. Before taking backup, our script will also tell us if the log file exists or not. It
will skip the backup procedure if the file does not exists. After all there is no point running backup
command if the log file does not exist.
Create directory for storing log files
$ mkdir -p /tmp/scripts/logs

Put some files in the logs directory.

$ cd /tmp/scripts/logs

$ touch ansible.log apache.log mysql.log nagios.log

You can choose to put some content in the log files, touch will just create empty files.

$ cd /tmp/scripts

Create a file where you place the name of the files that you want to backup.
$ vi backup_files.txt

apache.log

mysql.log

nagios.log

ansible.log

chef.log

There is one extra filename chef.log which is not present in our logs directory /tmp/scripts/logs.
We will see how we will handle it in our script

$ vi backup.sh

#!/bin/bash

LOG_DIR='/tmp/scripts/logs'

BACKUP_DIR='/tmp/scripts/logs_backup'

mkdir -p $BACKUP_DIR

for i in `cat backup_files.txt`; do

if [ -f $LOG_DIR/$i ];

then

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 echo "Copying $i to logs_backup directory."

cp $LOG_DIR/$i $BACKUP_DIR

else

echo "$i log file does exist, skipping."

fi

done

echo

echo

echo "Zipping log files"

tar -czvf logs_backup.tgz logs_backup

echo

echo

echo "Backup completed successfully."

Mysql Database Backup Script

mysqldump command is used to take the db dump for mysql. In the script, we’re
taking the dbdump and sending it to a target directory in a zipped format. We
are also removing 8 days old dbbackup file by using find command. This process
is called a purging or purging old backup/log files.

#!/bin/sh

TIME_NOW="$(date +'%d_%m_%Y_%H_%M_%S')"

BACKUPFILE="db_backup_$TIME_NOW".gz

BACKUP_DIR="/opt/db_backup_dir"

PATHOFBACKUPFILE="$BACKUP_DIR/$BACKUPFILE"

LOG_FILE="$BACKUP_DIR/"backup_log_"$(date +'%Y_%m')".txt

echo "mysqldump started at $(date +'%d-%m-%Y %H:%M:%S')" >> "$LOG_FILE"

mysqldump --user=dbuser--password=dbpass --default-character-set=utf8 mydatabase

| gzip > "$PATHOFBACKUPFILE"

echo "mysqldump finished at $(date +'%d-%m-%Y %H:%M:%S')" >> "$LOG_FILE"

chown myuser "$PATHOFBACKUPFILE"

chown myuser "$LOG_FILE"

echo "file permission changed" >> "$LOG_FILE"

find "$BACKUP_DIR" -name db_backup_* -mtime +8 -exec rm {} \;

echo "old files deleted" >> "$LOG_FILE"

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
echo "operation finished at $(date +'%d-%m-%Y %H:%M:%S')" >> "$LOG_FILE"

echo "*****************" >> "$LOG_FILE"

exit 0

Running command on remote servers/nodes

Sometimes we need to run a command or set of commands on multiple nodes/server.
We use ssh to login to these nodes and run that command individually and manually on all the nodes/servers.
But it’s a very time consuming and mundane work if you have to do it many times.
We will write a bash script to do that.

For this exercise, we will choose to run “yum install httpd” on three nodes.
Assumptions:
1. Three centos VMs
2. VMs have internet connection to download and install software.
3. All VMs have same username to connect.

We will create a file named “hosts-dev” and add ip address of all three nodes in that.
$ vi hosts-dev

192.168.2.5

192.168.2.6

192.168.2.7

We will write a script which will read the ip address from the hosts-dev file, do ssh to all of them one by one and run
yum install httpd command over ssh.

$ vi install.sh

#!/bin/bash

for hosts in `cat hosts-dev`

do

ssh vagrant@$hosts sudo yum install httpd -y

done

Let’s break it down

Line 2: for loop will run over the content on hosts-dev file one by one, which are the ip addresses of the VMs. Notice
we have used backticks `` and not single quotes ‘’ to read the hosts-dev file ( `cat hosts-dev` ).

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Line 4: we are running sudo yum install httpd -y command over ssh. $hosts variable will hold the ip
address and will establish ssh connection with vagrant user( ssh vagrant@$hosts).
This loop will run until we exhaust all the entries in the hosts-dev file, if you have lets say 50 nodes you can add ip
address of all the 50 nodes in this file.

Every time it logs into the vm’s/sever/nodes it will ask you a password, it will be painful if you have lot of servers that
you manage to enter password manually. In the next section, we will deal with this issue by doing ssh key exchange.

The above script will install httpd package on all the three nodes but will also
ask you password everytime it does a ssh login. To avoid this we can do key
based login which is discussed in next chapter.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 7. How To Set Up SSH Keys

About SSH Keys

SSH keys provide a more secure way of logging into a virtual private server with SSH than using a
password alone. While a password can eventually be cracked with a brute force attack, SSH keys
are nearly impossible to decipher by brute force alone. Generating a key pair provides you with two
long string of characters: a public and a private key. You can place the public key on any server,
and then unlock it by connecting to it with a client that already has the private key. When the two
match up, the system unlocks without the need for a password. You can increase security even more
by protecting the private key with a passphrase.

Step One—Create the RSA Key Pair

The first step is to create the key pair on the client machine (there is a good chance that this will just be your computer):

$ ssh-keygen -t rsa

Step Two—Store the Keys and Passphrase

Once you have entered the Gen Key command, you will get a few more questions:

Enter file in which to save the key (/home/demo/.ssh/id_rsa):

You can press enter here, saving the file to the user home (in this case, my example user is called
demo).

Enter passphrase (empty for no passphrase):

It's up to you whether you want to use a passphrase. Entering a passphrase does have its benefits:
the security of a key, no matter how encrypted, still depends on the fact that it is not visible to
anyone else. Should a passphrase-protected private key fall into an unauthorized user’s possession,
they will be unable to log in to its associated accounts until they figure out the passphrase, buying
the hacked user some extra time. The only downside, of course, to having a passphrase, is then
having to type it in each time you use the Key Pair.

The entire key generation process looks like this:

$ ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/demo/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/demo/.ssh/id_rsa.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Your public key has been saved in /home/demo/.ssh/id_rsa.pub.

The key fingerprint is:

4a:dd:0a:c6:35:4e:3f:ed:27:38:8c:74:44:4d:93:67 demo@a

The key's randomart image is:

+--[ RSA 2048]----+

| .oo. |

| . o.E |

| + . o |

| . = = . |

| = S = . |

| o + = + |

| . o + o . |

| . o |

| |

+-----------------+

The public key is now located in /home/demo/.ssh/id_rsa.pub The private key (identification) is now located in
/home/demo/.ssh/id_rsa

Step Three—Copy the Public Key

Once the key pair is generated, it's time to place the public key on the virtual server that we want to
use.

You can copy the public key into the new machine's authorized keys file with the ssh-copy-id
command. Make sure to replace the example username and IP address below.

$ ssh-copy-id [email protected]

Alternatively, you can paste in the keys using SSH:

$ cat ~/.ssh/id_rsa.pub | ssh [email protected] "mkdir -p ~/.ssh && cat >>

~/.ssh/authorized_keys"

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
No matter which command you chose, you should see something like:

The authenticity of host '192.168.2.5 (192.168.2.5)' can't be established.RSA

key fingerprint is b1:2d:33:67:ce:35:4d:5f:f3:a8:cd:c0:c4:48:86:12.

Are you sure you want to continue connecting (yes/no)? Yes

Warning: Permanently added '12.34.56.78' (RSA) to the list of known hosts.

[email protected]'s password:

Now try logging into the machine, with "ssh '[email protected]'", and check in:

~/.ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

Now you can go ahead and log into [email protected] and you will not be prompted for a
password. However, if you set a passphrase, you will be asked to enter the passphrase at that time
(and whenever else you log in in the future).

Exercise:

- Create 4 centos vm’s. One among these four vm one will be the automation box from where we
will run our scripts.

- Do SSH key exchange from automation box to rest of the three vm’s.

- Write a script which will install Apache & mysql server, start and enable both services and check
status of apache, mysql & iptables. Execute this script from automation box for all three vm and
validate.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 8. Few Sample scripts
A script to automate below mentioned task on Centos

1. Install mlocate search tool, update mlocate database

2. Install, start & enable httpd service.

3. Find files with permission 0777 and delete it.

4. Check hard disk free space and alerts if its running low on disk space.

#!/bin/bash

# Sample script to automate tasks:

# -Update local file database:

echo -e "\e[4;32mInstalling mlocate\e[0m"

sudo yum install mlocate -y

echo ""

echo -e "\e[4;32mUPDATING LOCAL FILE DATABASE\e[0m"

sudo updatedb

if [ $? == 0 ]; then

echo "The local file database was updated correctly."

else

echo "The local file database was not updated correctly."

fi

echo ""

# Installing and staring HTTPD service

echo -e "\e[4;32mInstalling HTTPD package\e[0m"

sudo yum install httpd -y

echo ""

echo -e "\e[4;32mStarting and enabling HTTPD package\e[0m"

sudo /etc/init.d/httpd start && chkconfig httpd on

echo ""

# For CentOS 7

#sudo systemctl start httpd && sudo systemctl enable httpd

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
# -Find and / or delete files with 777 permissions.

echo -e "\e[4;32mLOOKING FOR FILES WITH 777 PERMISSIONS\e[0m"

# Enable either option (comment out the other line), but not both.

# Option 1: Delete files without prompting for confirmation. Assumes GNU version
of find.

#find -type f -perm 0777 -delete

# Option 2: Ask for confirmation before deleting files. More portable across
systems.

find -type f -perm 0777 -exec rm -f {} +;

echo ""

# -Alert when file system usage surpasses a defined limit

echo -e "\e[4;32mCHECKING FILE SYSTEM USAGE\e[0m"

THRESHOLD=10

while read line; do

# This variable stores the file system path as a string

FILESYSTEM=$(echo $line | awk '{print $1}')

# This variable stores the use percentage (XX%)

PERCENTAGE=$(echo $line | awk '{print $5}')

# Use percentage without the % sign.

USAGE=${PERCENTAGE%?}

if [ $USAGE -gt $THRESHOLD ]; then

echo "The remaining available space in $FILESYSTEM is critically

low. Used: $PERCENTAGE"

fi

done < <(df -h --total | grep -vi filesystem)

Automatic maintenance script for nginx service.

This Script starts nginx service if its dead.
Place it under cronjob and schedule to run every 2 minutes which acts as a monitoring and
maintenance script.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Assumptions: Nginx is already installed on the system.
cat nginstart.sh
#!/bin/bash

if [ -f /var/run/nginx.pid ]

then

echo "Nginx is running."

else

echo "Starting nginx service."

service nginx start

fi

# crontab -e

* * * * * /opt/scripts/nginstart.sh

Example Script for automating Jenkins setup.

#!/bin/bash

# Author: Pavan Kumar Ranjit

## LOGIC TO CHECK THE TYPE OF DISTRIBUTION (REDHAT OR DEBIAN)

yum --help >> /tmp/log1

if [ $? -eq 0 ]

## "$?" STORES THE EXIT CODE OF THE MOST RECENT COMMAND

then

echo "RPM Based OS Detected"

echo "Installing Java-JDK,Jenkins,Maven"

sleep 3

sudo yum install java-1.8.0-openjdk -y

sudo yum install java-1.8.0-openjdk-devel -y

sudo yum install wget -y

sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-

ci.org/redhat/jenkins.repo

sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key

sudo yum install Jenkins -y

sudo yum install maven -y

sudo yum install git -y

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
 echo "Configuring services.... Please Wait"

sleep 5

sudo service iptables stop

sudo service Jenkins start

else

echo "Debian Based OS Detected"

sleep 3

echo "Installing Java-JDK,Jenkins,Maven"

sudo apt-get update

sudo apt-get install openjdk-8-jdk -y

sudo apt-get install openjdk-8-jre -y

sudo apt-get install maven -y

sudo apt-get install wget -y

wget -q -O - https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key

add -

sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ >

/etc/apt/sources.list.d/jenkins.list'

sudo apt-get update -y

sudo apt-get install jenkins -y

sudo apt-get install git -y

echo "Configuring services.... Please Wait"

sleep 5

sudo systemctl stop ufw

sudo systemctl start jenkins

fi

Summary:

• Scripting is required for doing system tasks automatically without manual intervention.

• Bash scripting is used by Linux System Admin for ages to automation Linux tasks.

• Variables, Condition's, loops etc are important aspects of scripting language which helps us
automate complex tasks.

• Try all the sample and real-time use case scripts to get hold of system automation.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.
Conclusion:
There are so many advanced options in Bash Scripting like functions, list constructs, Regular
Expressions etc, which you may be interested in and feel tempted to use them.
You can check Advanced Bash Scripting guide http://tldp.org/LDP/abs/html/ for advanced options.
Learning all those advanced options are great and make you a Scripting Guru if practised.
There are also so many limitations to Bash script, like its only for Linux systems and there is so
much of syntax in it.

Python should be your next choice of scripting language which is easier to read and write and is
also versatile, it can be used to automate tasks on windows platform and cloud services also.
Python has very less syntax for example check below “if condition” of bash vs python.

Bash way
#!/bin/bash

a=5

if [ $a -lt 10 ]

then

echo "Variable is smaller than 10"

exit 1

fi

Python way
#!/usr/bin/python

a=10

if a < 10:

print “variable is less than 10”

Both have their upsides and downside, you should make your choices wisely. But making choices
comes with experience and practice. There is no perfect way of doing anything.
Practice makes improvement.

Visualpath Training & Consulting.

Flat no: 205, Nilgiri Block,Aditya Enclave, Ameerpet, Hyderabad, Phone No: - +91-970 445 5959, 961 824 5689 E-
Mail ID : [email protected], Website : www.visualpath.in.