Unix and Linux

Essentials for
Developers
Introduction to Linux and Unix

Author: Ranjit Kumar Rai

Information Security Level 1 – Confidential

© 2012 – Proprietary and Confidential Information of Amdocs
 Objectives
● Introduction of Unix and history.
● Introduction to Linux and history.
● Kernel Architecture.
● Common Linux Distributions Overview.

Information Security Level 1 – Confidential

2 © 2012 – Proprietary and Confidential Information of Amdocs
 Unix Introduction.
● Operating System.
● Base program that controls the system and peripherals.
● Open Source OS.
● Multi User.
● Multi Tasking.
● Very Stable.
● Portability.
● Powerful networking capabilities.
● Base mechanisms (Booting, logging in, running applications, file
handling etc).

Information Security Level 1 – Confidential

3 © 2012 – Proprietary and Confidential Information of Amdocs
 Unix History.

Reference

Information Security Level 1 – Confidential

4 © 2012 – Proprietary and Confidential Information of Amdocs
 Unix Flavors.
Since Unix had licenses, people
starting developing their flavors
of Unix, which were open
source.

Most popular Freeware is

Linux.

Information Security Level 1 – Confidential

5 © 2012 – Proprietary and Confidential Information of Amdocs
 Unix Flavors.

Information Security Level 1 – Confidential

6 © 2012 – Proprietary and Confidential Information of Amdocs
 Linux History.

Information Security Level 1 – Confidential

7 © 2012 – Proprietary and Confidential Information of Amdocs
 How does Linux work?

Files

Hardware

Processes
Kernel
Shell
User

Hardware:
Is the base hardware on which Linux OS is installed.

Information Security Level 1 – Confidential

8 © 2012 – Proprietary and Confidential Information of Amdocs
 How does Linux work?

Files

Hardware

Processes
Kernel
Shell
User

Kernel:
Hub of OS, allocates time and memory to programs.
handles file stores and communications in response to the system calls.,
Linux uses Monolithic kernel – Direct access to H/W, process communication easier.
Information Security Level 1 – Confidential
9 © 2012 – Proprietary and Confidential Information of Amdocs
 How does Linux work?

Files

Hardware

Processes
Kernel
Shell
User
Shell:
Interface between User and Kernel.
After login, the program checks for user id and password.
starts the command line interpreter.
Interprets the commands written by the user.
Information Security Level 1 – Confidential
10 © 2012 – Proprietary and Confidential Information of Amdocs
 How does Linux work?

Files

Hardware

Processes
Kernel
Shell
User

User:
Since it’s a multiuser OS, different users needs to have different User id
and passwords to connect to the OS.
This can be done by terminal emulator like Putty etc.
Information Security Level 1 – Confidential
11 © 2012 – Proprietary and Confidential Information of Amdocs
 Linux Distributions

Information Security Level 1 – Confidential

12 © 2012 – Proprietary and Confidential Information of Amdocs
 What happens when you login?

User/client login

Check /etc/passwd Check /etc/shadow

Entered Password encrypted and

checked.

Group recognition /etc/group

Execute profile files.

Information Security Level 1 – Confidential

13 © 2012 – Proprietary and Confidential Information of Amdocs
 Different Shell.
● Bourne shell(sh) named after Steve bourne is the most common shell. It was
excellent for Shell programming.
● C shell (csh) came from Berkley having advance features like interactive use,
job control & history.
● Korn Shell named after David Korn arrived in mid-80’s having compatible
with Bourne shell & having advance feature like history
● T-Shell (tcsh) which has most of feature of C-shell

● Bourne again shell (bash) is from free software foundation. It is very much
similar to Ksh & has C shell features plus command line editing & built in help
command.
● Z shell (zsh) is hybrid of all shell which is having feature of almost all the shell
combine.

Information Security Level 1 – Confidential

14 © 2012 – Proprietary and Confidential Information of Amdocs
 Pipe & filters
● Using Pipe and Tee.
● Head and Tail.
● Cut
● Sort
● Uniq
● Grep.
● Tr.

Information Security Level 1 – Confidential

15 © 2012 – Proprietary and Confidential Information of Amdocs
 Using Pipes.
● Pipes: Standard output of the command at the left is given as an
input to the command on the right side.

<command_1> | <command_2>

● Using redirection operator will create a new file with the name given
on the right.

<command_1> > <command_2>

Demo for the Pipe command. Comparison of Pipe and redirection.

Information Security Level 1 – Confidential

16 © 2012 – Proprietary and Confidential Information of Amdocs
 Using the Tee command.
● Tee command: Used to store and view the output of the
other command at the same time.
● Example:
● $ls - Shows the output on the screen.
● $ls > file - Copies the output to the file but doesn’t show
on the screen.
● $ls | tee file - Shows the output on the screen as well as
copies the output to the file.

● Option ‘tee -a’ will append to the output file.

Information Security Level 1 – Confidential

17 © 2012 – Proprietary and Confidential Information of Amdocs
 Head and Tail Options.
● Head prints the first N number of lines from the given
input files. Default is 10 lines.
● $head -n 5 <filename> - prints the first 5 lines.
● $head -5 <filename> - prints the first 5 lines.
● $head -n -5 <filename> - prints the file without last 5 lines.
● $head -c 5 <filename> - prints the first 5 bytes of a file.

● Combination:
● $ls | head

● Tail prints the last N number of lines from the given input
files. Default is 10 lines.
Information Security Level 1 – Confidential
18 © 2012 – Proprietary and Confidential Information of Amdocs
 Head and Tail Options.
● Tail is effective for viewing logs on the screen.
● $tail -n 5 <filename> - prints the last 5 lines.
● $tail -f <filename> - prints the appended lines as the file grows.

● Can be used to tail the process output till the process is

alive.
● $tail -f <filename> -pid=<pid_id>

Information Security Level 1 – Confidential

19 © 2012 – Proprietary and Confidential Information of Amdocs
 Using Cut.
● Cut: Used to print characters in a line by specifying the position of the
characters.
● Example:
● $cat <filename> - Display the file contents.
● $cut -c4 <filename> - Show the 4th character from each line.
● $cut -c4,6 <filename> - Show the 4th to 6th character from each line.
● $cut -c4-7 <filename> - Show from 4th to 7th character from each line.
● $cut -c-6 <filename> - Show from 1st to 6th character from each line.
● $cut -c10- <filename> - Show from 10th till last character from each line.
● $cut -c- <filename> - Show all the characters from each line.
● $cut -d ‘ ‘ -f2 <filename> - Show 2nd delimited object from each line.
● $cut -d ‘ ‘ -f2,3 <filename> - Show 2nd and 3rd delimited object from each line.
● $cut -d ‘ ‘ -f1-3 <filename> - Show 1st till 3rd delimited object from each line.
● $cut -d ‘:’ -f1 /etc/passwd - Show the first delimited object from /etc/passwd

Information Security Level 1 – Confidential

20 © 2012 – Proprietary and Confidential Information of Amdocs
 Using Sort.
● Sort: Used for forward and reverse alphabetical and numeric sorting
of an input list.
● Example:
● $cat <filename> | sort -- alphabetical and numeric sorting.
● $cat <filename> | sort -u -- Used for Unique sorting.
● $cat <filename> | sort -n -- Numeric sorting, ex. 20, 200.
● $cat <filename> | sort -r -- Reverse sorting.
● $sort -n <filename1> <filename2> -- Numeric sorting of multiple files.
● $sort -nu <filename1> <filename2> -- Sort, merge and remove duplicates.
● $sort -t “,” -k1,1 <filename> -- sort on the basis on first field.
● $sort -t “,” -k2,2 <filename> -- sorting on the basis of 2nd field.

● Reference: http://www.theunixschool.com/2012/08/linux-sort-command-examples.html

Information Security Level 1 – Confidential

21 © 2012 – Proprietary and Confidential Information of Amdocs
 Using Uniq.
● Uniq option is used to compare the consecutive lines and prints the unique lines in
a sorted file.
● Example:
● $sort <filename> | uniq -- Display all lines, show unique files once.
● $sort <filename> | uniq -u -- Unique lines only, eliminate duplicates.
● $sort <filename> | uniq -d -- Only the duplicate lines, eliminates single.
● $sort <filename> | uniq -c -- Statistics of occurrences.
● $sort <filename> | uniq -D -- All duplicate lines multiple times, eliminates single.
● $uniq -D <filename> -- Can be used like this too.

Information Security Level 1 – Confidential

22 © 2012 – Proprietary and Confidential Information of Amdocs
 Using grep.
● Grep will search the input files for lines containing a match to a given
patterned list.
● Examples:
●$grep <pattern> <filename> -- simple syntax.
●$grep string demo_* -- with wildcards.
●$grep -i <pattern> <filename> -- case insensitive search.
●$grep <pattern> * -- searches pattern in pwd.
●$grep -r <pattern> * -- searches pattern in all the subdirectories.
●$grep –w <pattern> * -- Search exact WORD from pwd.
●$grep -v <pattern> <filename> -- Inverse pattern search.
●$grep -c <pattern> <filename> -- counting the number of matches.
●$grep –A 3 “root” /etcpasswd -- prints matched line, and 3 lines after it.
●$grep –B 3 “root” /etc/passwd -- prints matched line and 3 lines before it.
●$grep –C 3 “root” /etc/passwd -- prints matched line and 3 lines around it.
●$grep –l root /etc/passwd -- display the file name which matches the string.
●$grep –o “is.*line” <filename> -- display lines which has is and line.
●$grep –n root /etc/passwd -- show line number of the file with line matched.

Information Security Level 1 – Confidential

23 © 2012 – Proprietary and Confidential Information of Amdocs
 Using tr.
● Tr command is used for translation.
● Example:
● $echo ‘linux’ | tr “[:lower:]” “[:upper:]”
● $echo ‘linux’ | tr “a-z” “A-Z”
● $tr -cd “[:print:]” < <filename>
● -- c for complement. D for delete.
deletes all non printable chars from
the input file.
● $echo “Unix Essentials for Developers” > tr –s ‘ ‘ ‘ ‘
● -- s for squeeze. Removes consecutive blank

Information Security Level 1 – Confidential

24 © 2012 – Proprietary and Confidential Information of Amdocs
 What is a shell ?

● Shell is the interface between end user and the Unix

system (kernel), similar to the command prompt in
Windows.
● The korn, bash, C shell are versions of shell used with
Unix.
● We enter shell commands that interact with the

UNIX kernel.
● E.g. date, ls, echo

Information Security Level 1 – Confidential

25 © 2012 – Proprietary and Confidential Information of Amdocs
 What is a shell environment ?
● When you login to the system, the shell undergoes a phase called
initialization to set up the environment.

● This involves the shell reading a special file called .profile and

running all instructions inside this file.

● This will include aliases, path variables etc. which is what we call
the environment.

● E.g. alias bin='cd $HOME/bin‘

export JAVA_HOME=/opt/shared/java
export PATH=$JAVA_HOME/bin:$PATH

Information Security Level 1 – Confidential

26 © 2012 – Proprietary and Confidential Information of Amdocs
 What is a Shell Script ?

● A text file containing commands which could have been

typed directly into the shell. Like “echo” or “date” . We
also specify the shell to run these commands.

● The power comes from using it as a "glue" language to

combine the standard Unix utilities, to produce a tool
more useful than the component parts.

Information Security Level 1 – Confidential

27 © 2012 – Proprietary and Confidential Information of Amdocs
 A simple shell script called script.sh

#!/bin/ksh
# This is a comment

echo “Hello World”

Information Security Level 1 – Confidential

28 © 2012 – Proprietary and Confidential Information of Amdocs
 Execution of UNIX shell commands.

6 Ways to run a shell script

1. /export/home/user/script.sh
2. ./script.sh &

3. ksh script.sh (If you want to run it in another shell)

4. crontab (add an entry for job)
5. Using at command (Just like cron)

6. Add the entry in the .profile, bashrc

Information Security Level 1 – Confidential

29 © 2012 – Proprietary and Confidential Information of Amdocs
 Redirection of Input and Output

Name Command Description Example

Take all output from
standard out and place it ls > filename
into filename. Note
Redirect to a file > filename
using >>will append to ls >> filename
the file, rather than
overwrite it.
Copy all data from the
Read from a file < filename file to the standard input echo < filename
of the program
Take everything from
standard out
Pipe program1 | program2 of program1and pass it ls | more
to standard input
of program2

Information Security Level 1 – Confidential

30 © 2012 – Proprietary and Confidential Information of Amdocs
 Inputs and Outputs of shell scripts
● Shell scripts can take inputs from files and redirect their
outputs to the file or the terminal screen.
● Arrow pointing towards scripts is input

script.ksh < Input.txt

● Arrow pointing away from the scripts is output .

script.ksh > Output.out 2> /dev/null

script.ksh >> Append.out 2>&1

Information Security Level 1 – Confidential

31 © 2012 – Proprietary and Confidential Information of Amdocs
 Parameters to Scripts
● Input redirection means command line arguments are
being passed to a shell script There are special positional
parameters which store these values.

$script.ksh 27 Delhi

#!/bin/ksh

your age : $1
your city : $2

Information Security Level 1 – Confidential

32 © 2012 – Proprietary and Confidential Information of Amdocs
 Command Substitution
● Command substitution is generally used to assign the
output of a command to a variable. 
● When performing command substitution make sure that
you are using the backquote (below Tilde), not the single
quote character.

● E.g.

USERS=`who | cut -d ' ' -f1`

echo "Users Logged in are \n$USERS"

Information Security Level 1 – Confidential

33 © 2012 – Proprietary and Confidential Information of Amdocs
 Test your knowledge!
● Create a file which shows alphabetically sorted list of /etc/passwd file and
also outputs on the command prompt?
● Collect all the registered system users whose name contains “ro” and put in a
file called output.txt and show output.txt on the command prompt too.
● Find a string inside an unknown file in a directory system.

● How will you print the 5th line of /etc/passwd file with the help of head and tail
command combination?
● Find out all the files named “core” from the entire server and remove them.

Information Security Level 1 – Confidential

34 © 2012 – Proprietary and Confidential Information of Amdocs
 Answers…!
● $cat /etc/passwd | sort | tee file1

● $cut -d : -f 1 /etc/passwd | grep ro > output.txt | tee output.txt

● $find . –type f | xargs grep –i <pattern>

● $head -5 /etc/passwd | tail -1

● $find / -name core | xargs rm –f

Information Security Level 1 – Confidential

35 © 2012 – Proprietary and Confidential Information of Amdocs
 Linux File System Structure.

Reference:

Information Security Level 1 – Confidential

36 © 2012 – Proprietary and Confidential Information of Amdocs
 Linux File System Structure.
● Demo on sample environment.

Information Security Level 1 – Confidential

37 © 2012 – Proprietary and Confidential Information of Amdocs
 Manipulating Directories and Files.
Demo and Practice for below commands for Directories:
1) pwd: Present/Current working directory. (pwd)
2) cd: change directory. (cd .) (cd ..) (cd -) (cd) (cd /) (cd /opt) (cd ~/script)
1) Relative PATH.
2) Absolute PATH or Full PATH.

3) ls: List of files and directories. (ls) (ls –a) (ls –ltr) (ls –l) (ls –ld)
4) cp: Copy file or directory. (cp –R <> <>) (cp <> <>)
5) mv: Move file or directory. (mv <> <>)
6) rm/rmdir: Remove a file or directory. (rm <>) (rm –rf <>) (rmdir <>)
7) touch: Create a blank file. (touch <>)
8) cat: concatenate file. (cat <>)
9) less/more: view the file. (less <>) (more <>)
10) File: Writes the type of the file.
Information Security Level 1 – Confidential
38 © 2012 – Proprietary and Confidential Information of Amdocs
 Manipulating Directories and Files.
Demo and Practice for below commands for Directories:
1) mkdir: Make Directory. (mkdir <>) (mkdir –p <>)
2) which: Full PATH of the command. (which <>).
3) whereis: Locates source binary for program. (whereis <>).

Information Security Level 1 – Confidential

39 © 2012 – Proprietary and Confidential Information of Amdocs
 Wildcards!
Wildcards are used for Pattern Matching like in C programming.

1) Asterisk Wildcard(*): Sequence. (ls *.log) (ls *.log*) (ls *log)

2) Question Mark Wildcard(?): Character Substitution. (ls ab?d)
3) Square Brackets Wildcard([ ]): Range of Characters. (ls abc[0-9])
4) Double Quotes(“ “): Suppress Expansion. (touch “ab%c”)
5) Backquotes(`): Command Substitution. (ls –ltr `touch myunix`)
6) Apostrophe Marks(‘): No change. (ls ‘*.log*’)

Information Security Level 1 – Confidential

40 © 2012 – Proprietary and Confidential Information of Amdocs
 Mounting External FS.

Information Security Level 1 – Confidential

41 © 2012 – Proprietary and Confidential Information of Amdocs
 How it looks like?

From client side, looks the same.

Demo:

Information Security Level 1 – Confidential

42 © 2012 – Proprietary and Confidential Information of Amdocs
 The Vi Editor.
Vi Editor is the editor used in Unix and Linux.

Vi Editor modes:
1) Command Mode (Default mode)
2) Insert Mode.
3) Last Line Mode.

Information Security Level 1 – Confidential

43 © 2012 – Proprietary and Confidential Information of Amdocs
 The Vi Editor.
1) Demo for Vi editor.
2) Command mode of Vi editor:
1) I -: Insert at beginning of the line.
2) A -: Insert at the end of the line.
3) i -: insert text to the left of the cursor.
4) a -: appends text to the right of the cursor.
5) O -: opens line above.
6) o -: opens line below.
7) x -: deletes the character.
8) X -: deletes to the left.
9) J -: Joins two lines.
10) yy and p -: Yanking(copy) and paste.
11) u -: Undo last change.
12) dd -: Delete the present line.
13) cw, dw -: Change word, delete word.

Information Security Level 1 – Confidential

44 © 2012 – Proprietary and Confidential Information of Amdocs
 The Vi Editor.
1) Last Line Mode:
1) :w -: write.
2) :w! -: forceful write.
3) :wq -: overwrite or write if new, save and quit.
4) :wq! -: forceful overwrite, save and quit.
5) :wq <filename> -: write to file and quit.
6) :q -: quit – fails when changes are made.
7) :q! -: forceful quit – exit without saving changes.
8) :%s/<old_value>/<new_value> -: Replace.
9) / -: To search any particular string., n to go next, N to go previous.

Reference for Vi Editor: http://www.unix-manuals.com/tutorials/vi/vi-in-10-1.html

Information Security Level 1 – Confidential

45 © 2012 – Proprietary and Confidential Information of Amdocs
 Vi Editor Exercise.
● Create a directory in your home directory called temp.

● Check where the binary of the command you used in the first step is present.

● Copy the file /etc/passwd to your temp directory using Relative PATH.

● Rename it as mywork.

● Long list this file to check details. Check the time stamp.

● Check the file type for mywork file.

● Open this file using Vi Editor.

● Search for root in the mywork from last line mode. Check for occurrences.

● Replace the “root” with “root_new” for all occurrences.

Information Security Level 1 – Confidential

46 © 2012 – Proprietary and Confidential Information of Amdocs
 All Files can be seen?
Files can be visible or hidden.
1) Files with any name – visible.
2) Starting with a dot (.) – invisible.

Information Security Level 1 – Confidential

47 © 2012 – Proprietary and Confidential Information of Amdocs
 Handling File Permissions.
Permissions can be given with:
1) Chmod command – works with ksh shell.

Which permissions?
2) Read. – represented by ‘r’.
3) Write. – represented by ‘w’.
4) Execute. – represented by ‘x’.

Information Security Level 1 – Confidential

48 © 2012 – Proprietary and Confidential Information of Amdocs
 Handling File Permissions.
d rwx rwx rwx --- Permissions of the file.

1) First digit – Type of file, - for regular, d for directory, l for link.
2) First set – Owner.
3) Second set – Group.
4) Third set – Others.

These permissions have number representations too:

r=4.
w=2.
x=1.

Permissions can be summed to set using chmod command.

Information Security Level 1 – Confidential
49 © 2012 – Proprietary and Confidential Information of Amdocs
 Handling File Permissions.
Below is possible… (Demo)
1) chmod 777 <filename>
2) chmod 700 <filename>
3) chmod 755 <filename> -- most common.
4) chmod 600 <filename> -- most common.
5) chmod 400 <filename>
6) chmod +r <filename>
7) chmod -r <filename>
8) chmod +w <filename>
9) chmod –w <filename>
10) chmod +x <filename>
11) chmod -x <filename>

Default file permission: 666

Default directory permission: 777
Information Security Level 1 – Confidential
50 © 2012 – Proprietary and Confidential Information of Amdocs
 I-Node in Unix and Linux.

Boot Block Super Block I-Node Array Data-node

File System

I-Node = Index node

•Unique identification number for a file.
•Consumes only 1% of the total disk space.
•Each entry is 128 bytes in size.
•File Attributes like permissions, owner, size, type, access, change and modification time.,
etc.

Commands to get I-Node information:

$stat <filename>
Reference: http://web.cs.wpi.edu/~rek/DCS/D04/UnixFileSystems.html

Information Security Level 1 – Confidential

51 © 2012 – Proprietary and Confidential Information of Amdocs
 Tricks to make rm safer
● Make rm -i, possibly as an alias
● write a “delete” script that moves “deleted” files to a
temporary directory
● If you want yes always use
● yes | del * or yes n | del *

● rm –rf * most dangerous command

Information Security Level 1 – Confidential

53 © 2012 – Proprietary and Confidential Information of Amdocs
 Hard and Soft Links for Files.

What is a link?
1) File has I-node which stores information about file.
2) Filename is simply a reference to this data.
3) Link is a way to refer to contents of a file.

Ex: file_1 -> /home/myuser/file_3

Two Types:
4) Hard Link.
5) Soft Link.

Information Security Level 1 – Confidential

54 © 2012 – Proprietary and Confidential Information of Amdocs
 Hard Link.
Pointer to the file’s I-node.
Example: a-file.txt is a file and b-file.txt is a hard link.

Command: $ln a-file.txt b-file.txt

Modify contents of a-file.txt, b-file.txt is modified. The files . And .. Are hard
links.

a-file.txt
Data on Disk

b-file.txt

Reference: http://www.ugrad.cs.ubc.ca/~cs219/CourseNotes/Unix/commands-links.html
Information Security Level 1 – Confidential
55 © 2012 – Proprietary and Confidential Information of Amdocs
 Soft Link
Symbolic link: file which contains the name of another file.
Access contents of the file through link.
Command: $ln –s a-file.txt b-file.txt
Cannot modify contents of a-file.txt from b-file.txt.
Deleting b-file.txt doesn’t have any impact on a-file.txt.

a-file.txt
Data on Disk

b-file.txt

Reference: http://www.ugrad.cs.ubc.ca/~cs219/CourseNotes/Unix/commands-links.html

Information Security Level 1 – Confidential

56 © 2012 – Proprietary and Confidential Information of Amdocs
 How to execute?
1) chmod +x to give execute permissions.
2) . ./a-file.sh
3) . a-file.sh

Information Security Level 1 – Confidential

57 © 2012 – Proprietary and Confidential Information of Amdocs
 Variables
● A Unix variable is a character string to which we assign a
value. The value assigned could be a number, text,
filename, device, or any other type of data.
● By convention, Unix Shell variables would have their
names in UPPERCASE.
● Variable names cannot have special characters !,*, or -.

e.g.
NAME=Zara
echo $NAME

Information Security Level 1 – Confidential

58 © 2012 – Proprietary and Confidential Information of Amdocs
 Variable Types

● Local Variables − A local variable is a variable that is present

within the current instance of the shell. It is not available to
programs that are started by the shell. They are set at
command prompt.
● Environment Variables − An environment variable is a variable
that is available to any child process of the shell. Some
programs need environment variables in order to function
correctly. Usually a shell script defines only those environment
variables that are needed by the programs that it runs.
● Shell Variables − A shell variable is a special variable that is
set by the shell and is required by the shell in order to function
correctly. Some of these variables are environment variables
whereas others are local variables.

Information Security Level 1 – Confidential

59 © 2012 – Proprietary and Confidential Information of Amdocs
 Special Variables

Variable Description

$0 The filename of the current script.

$n These variables correspond to the arguments with which a script was invoked. Here n is a positive
decimal number corresponding to the position of an argument (the first argument is $1, the second
argument is $2, and so on).
$# The number of arguments supplied to a script.
$* If a script receives two arguments, $* is equivalent to $1 $2.

$? The exit status of the last command executed.

$$ The process number of the current shell. For shell scripts, this is the process ID under which they
are executing.
$! The process number of the last background command.

Information Security Level 1 – Confidential

60 © 2012 – Proprietary and Confidential Information of Amdocs
 Variable Expansion

● Ability to access and manipulate values of variables and

parameters. Basic expansion is done by preceding the
variable or parameter name with the $ character. 

● Types of expansion can be used to return portions or the

length of variables, use default or alternate values, check
for mandatory setting, and more.

Information Security Level 1 – Confidential

61 © 2012 – Proprietary and Confidential Information of Amdocs
 Examples of variable expansion

NAME=ZaraZara
echo ${#NAME} #length of string
echo ${NAME}Za #Append charecters

echo ${PATH:-/usr/bin} #if var unset use

value
echo ${ID:+Harry} #if var set use value

Information Security Level 1 – Confidential

62 © 2012 – Proprietary and Confidential Information of Amdocs
 Arithmetic Expansion and Operations
● Allows the evaluation of an arithmetic expression and the
substitution of the result. # ways of doing arithmetic
operations.

STATES=8
1. echo $(( STATES += 4 ))

2. STATES=$((STATES + 4))

3. STATES=`expr $STATES + 4`

echo $STATES

Information Security Level 1 – Confidential

63 © 2012 – Proprietary and Confidential Information of Amdocs
 Arithmetic operator
Operator Description Example

+ Addition - Adds values on either side of the operator `expr $a + $b` will give 30

- Subtraction - Subtracts right hand operand from left `expr $a - $b` will give -10
hand operand

* Multiplication - Multiplies values on either side of `expr $a \* $b` will give 200
the operator

/ Division - Divides left hand operand by right hand `expr $b / $a` will give 2
operand

% Modulus - Divides left hand operand by right hand `expr $b % $a` will give 0
operand and returns remainder

= Assignment - Assign right operand in left operand a=$b would assign value of b into a

== Equality - Compares two numbers, if both are same [ $a == $b ] would return false.
then returns true.

!= Not Equality - Compares two numbers, if both are [ $a != $b ] would return true.
different then returns true.

Information Security Level 1 – Confidential

64 © 2012 – Proprietary and Confidential Information of Amdocs
 Relational operators
Operator Description Example

-eq Checks if the value of two operands are equal or not, if yes [ $a -eq $b ] is not true.
then condition becomes true.

-ne Checks if the value of two operands are equal or not, if values [ $a -ne $b ] is true.
are not equal then condition becomes true.

-gt Checks if the value of left operand is greater than the value of [ $a -gt $b ] is not true.
right operand, if yes then condition becomes true.

-lt Checks if the value of left operand is less than the value of [ $a -lt $b ] is true.
right operand, if yes then condition becomes true.

-ge Checks if the value of left operand is greater than or equal to [ $a -ge $b ] is not true.
the value of right operand, if yes then condition becomes
true.

-le Checks if the value of left operand is less than or equal to the [ $a -le $b ] is tr
value of right operand, if yes then condition becomes true.

Information Security Level 1 – Confidential

65 © 2012 – Proprietary and Confidential Information of Amdocs
 File test operators
Operator Description Example
-b file Checks if file is a block special file if yes then condition becomes [ -b $file ] is false.
true.
-c file Checks if file is a character special file if yes then condition [ -c $file ] is false.
becomes true.
-d file Check if file is a directory if yes then condition becomes true. [ -d $file ] is not true.
-f file Check if file is an ordinary file as opposed to a directory or special [ -f $file ] is true.
file if yes then condition becomes true.
-g file Checks if file has its set group ID (SGID) bit set if yes then condition [ -g $file ] is false.
becomes true.
-k file Checks if file has its sticky bit set if yes then condition becomes [ -k $file ] is false.
true.
-p file Checks if file is a named pipe if yes then condition becomes true. [ -p $file ] is false.

-t file Checks if file descriptor is open and associated with a terminal if [ -t $file ] is false.
yes then condition becomes true.
-u file Checks if file has its set user id (SUID) bit set if yes then condition [ -u $file ] is false.
becomes true.
-r file Checks if file is readable if yes then condition becomes true. [ -r $file ] is true.
-w file Check if file is writable if yes then condition becomes true. [ -w $file ] is true.
-x file Check if file is execute if yes then condition becomes true. [ -x $file ] is true.
-s file Check if file has size greater than 0 if yes then condition becomes [ -s $file ] is true.
true.
-e file Check if file exists. Is true even if file is a directory but exists. [ -e $file ] is true.

Information Security Level 1 – Confidential

66 © 2012 – Proprietary and Confidential Information of Amdocs
 Find command
● One of the most important utility while dealing with files.

● Find file that match a given set of parameters, ranging from the file’s name to its
modification date
● Looks at every file, one at a time.
● Uses the information in the file’s inode to evaluate an expression given by the command-line operators.
● Takes the specified action (e.g., printing the file’s name) if the expression’s value is “true.”

● Syntax :
● Find path operators [Action ] – here path is one or more directories in which find will
begin to search & operators tells find which file you are interested in.
Operators description
-name filename Find files with the given filename.
-perm mode Find files with the given access mode. You must give the access mode in octal.
-type c Find the files of the given type, specified by c. c is a one-letter code; for example, f for a plain
file, b for a block special file, l for a symbolic link, and so forth.
-user name Find files belonging to user name
-group name Find files belonging to group name
-size n Find files that are n blocks long. A block usually equals 512 bytes.
-inum n Find files with the inode number n.

Information Security Level 1 – Confidential

67 © 2012 – Proprietary and Confidential Information of Amdocs
 Find command cont.
Operators description
-atime n Find files that were accessed n days ago. + n means “find files that were accessed over n days
ago” (i.e., not accessed in the last n days). - n means “find files that were accessed less than n
days ago” (i.e., accessed in the last n days).

-mtime n Similar to -atime, except that it checks the time the file’s contents were modified.

-ctime n Similar to -atime, except that it checks the time the inode was last changed. “Changed” means
that the file was modified or that one of its attributes (for example, its owner) was changed.

-newer file Find files that have been modified more recently than file.

● Action :
● -print – print the file names on the standard output
● -ls - list the file names on the standard output
● -exec - To include the pathname of the file that’s just been found in command, use the special symbol {}.
command must end with a backslash followed by a semicolon (\;)
● Example : find . -name "*.o" -exec rm -f {} \;
● -ok - Same as -exec, except that find prompts you for permission before executing command

Information Security Level 1 – Confidential

68 © 2012 – Proprietary and Confidential Information of Amdocs
 Find command cont.
● find . –print - print the names of all the files in the directory and all subdirectories

● ls -l `find . -print` - list of file found by find command however this may generate error when the command line is too
large

● An alternate method uses the xargs command. xargs and find work together beautifully. xargs executes its
arguments as commands and reads standard input to specify arguments to that command

● find / -print | xargs ls –lrt – print all the files from root directory

● find . -name \*.sh –print or find . –name ‘*.sh’ –print  to print all script in your home directory

● find . -mtime 7 –print  find file which are 7 day’s old

● find . -type f -atime +30 –print  check the files which has not been read in last 30 days.

● find . -atime 1 -name '*.sh' –print  tell what are the script being modified in last one days

● find . \! \( -atime 1 -name '*.sh' \) –print  just opposite to what we said above

● Note on –mtime, -atime, -ctime

● A number with no sign, for example, 3 (as in -mtime 3 or -atime 3), means the 24-hour period that ended exactly 3 days ago (in other
words, between 96 and 72 hours ago).
● A number with a minus sign (-) refers to the period since that 24-hour period. For example, -3 (as in -mtime -3) is any time between now
and 3 days ago (in other words, between 0 and 72 hours ago).
● Naturally, a number with a plus sign (+) refers to the period before that 24-hour period. For example, +3 (as in -mtime +3) is any time
more than 3 days ago (in other words, more than 96 hours ago).

Information Security Level 1 – Confidential

69 © 2012 – Proprietary and Confidential Information of Amdocs
 Find command cont.
● find . -type f -atime -1 | xargs ls –lrt -> find the files which have been access in last 24 hrs.

● find . -type f -atime 1 | xargs ls –lrt  file which are older then one day.

● Let’s say you want to find the files after a specific time say 10 th April 09:00 AM.
● First you create a file with old time stamp  touch –t mmddhhmm <file name>, touch -t 04091500 past_time, touch -t 04101500
past_time1
● find . -newer ./past_time | xargs ls –lrt  find the files which are created after the past_time creator
● find . \! -newer ./past_time | xargs ls –lrt  find file older then a specified time.
● find . -newer ./past_time \! -newer ./past_time1 | xargs ls –lrt  to find the files between two specified time.

● find . -perm 20 | xargs ls –lrt  find file where group has write permission

● find . -perm -100 –print  find files were owner can execute

● Some time you accidently create a file with a space or some control character in the name which you can not delete
so you use the –inum to find & delete the file

● find . -inum 31246 -exec rm {} \;

● find . -size 4096c –print  to fine the files of specified size (its always advised to give the size in bytes)

● find . size +72c -size -4096c | xargs ls –lrt  find the files which are larger then 72byte & smaller then 4096 bytes.

● find . -group emas | xargs ls –l  find a files which are owned by emas group

● find . -user ranjitkumarr | xargs ls –l  find the files which are owned by user

Information Security Level 1 – Confidential

70 © 2012 – Proprietary and Confidential Information of Amdocs
 Flow of a shell program
● Shell program is interpreted from top to bottom. Shell is a
procedural programming Language. Unlike C/java where
it looks for main function, Here its just reads & executes
code top to bottom.

● The flow can be redirected certain conditions, creating

functions, loops and case statement.

Information Security Level 1 – Confidential

71 © 2012 – Proprietary and Confidential Information of Amdocs
 Conditional expressions
● If-then- else
● Loops (for , while , do-while , until)

● case statement
● break

● exit
● continue
● functions

Information Security Level 1 – Confidential

72 © 2012 – Proprietary and Confidential Information of Amdocs
 If -else condition

if [ -e /etc ]
then
echo “etc directory exists”
else
echo “This will never run”
fi
----------------------------------------------
if [ 22 –ne 11 ]; then
echo “Numbers are not equal”
elif [ `whoami` != ‘root' ]; then
echo “You are not Root”
fi

Information Security Level 1 – Confidential

73 © 2012 – Proprietary and Confidential Information of Amdocs
 For Loop
● Think of it as a for variable in a list  do and then done

for i in 1 2 3 4 5
do
echo "Welcome $i“
done

for shuttle in $(cat spaceshuttles.txt)

do
print "Current Space Shuttle : $shuttle"
done

Information Security Level 1 – Confidential

74 © 2012 – Proprietary and Confidential Information of Amdocs
 While loop

count=0
while [[ $count -lt 10 ]];
do
print "count is $count"
(( count += 1 ))
done

Information Security Level 1 – Confidential

75 © 2012 – Proprietary and Confidential Information of Amdocs
 Untill loop
● Opposite of while. Loop will continue un-till condition does
not get satisfied.

until [ $answer = "yes" ];do

print -n "Please enter \"yes\": "
read answer
print ""
done

Information Security Level 1 – Confidential

76 © 2012 – Proprietary and Confidential Information of Amdocs
 Case

CMG=“Coming”
DEC=“Not Coming”

case $VAR in
john) print $CMG;;
fred) print $CMG;;
martin) print $DEC;;
*) print "Wrong name...";;
esac

Information Security Level 1 – Confidential

77 © 2012 – Proprietary and Confidential Information of Amdocs
 Break and Continue
● They simply used to break out of a loop or to continue to resume with the
next iterating element.

for NAME in john jack sally sam ; do

if [ ${#NAME} == 4]; then
continue
fi
if [ $NAME == ‘sam’ ]
echo “w found sam”
break
fi
done

Information Security Level 1 – Confidential

78 © 2012 – Proprietary and Confidential Information of Amdocs
 Functions

● function print {

echo “you called a function”

}

● print () {

echo “you called a function”

}

#Call a function like this

print

Information Security Level 1 – Confidential

79 © 2012 – Proprietary and Confidential Information of Amdocs
 Function variables
● Variables created inside functions have to typeset for them to be truly local.

subfunc(){
typeset var
var=2
echo “variable is now $var”
}

var=1
echo “var starts as $var, before calling subfunc”
subfunc # call the function
echo “var after function is now $var”

Information Security Level 1 – Confidential

80 © 2012 – Proprietary and Confidential Information of Amdocs
 Parameters to functions
● Functions take arguments jus like shell scripts.

name(){
arg1=$1
arg2=$2
echo $arg1
}
name foo bar

Information Security Level 1 – Confidential

81 © 2012 – Proprietary and Confidential Information of Amdocs
 Awk
● It is an excellent filter and report writer. . AWK is an excellent tool for processing these rows and columns, and is
easier to use AWK than most conventional programming languages.
AWK programs are written as - pattern { action }
awk 'pattern{action}' file
where the pattern indicates the pattern or the condition on which the action is to be executed for every line
matching the pattern. In case of a pattern not being present, the action will be executed for every line of the file. In
case of the action part not being present, the default action of printing the line will be done.
● awk 'BEGIN {print "Hello"}‘  Begin is the beginning of the awk program
● awk ‘{print $1}’ employee.txt  print the first fields from the file
● awk 'NR!=1{print $1}' employee.txt  print the all lines except first line.

● awk '{print $0}' employee.txt  print all the lines of files

● awk '1' employee.txt  other way to print all lines from the files.

● awk -F"," '{print $1}' awk_example  print first fields where fields are separated by comma

● awk '{print $1,$2}' FS="," awk_example  print files with file separator as comma

● awk -F"," 'NR!=1{print $1,$3}' OFS="," awk_example  print output field separated by comma (OFS- output field
separatror)
● awk -v q="'" '{print q $0 q}' awk_example1  to pass argument using awk

● awk '$0 ~ /Neha/{print}' awk_example  ~ is the symbol used for pattern matching.  The / / symbols are used to
specify the pattern. If the line($0) contains(~) the pattern Neha, print the line. 

Information Security Level 1 – Confidential

82 © 2012 – Proprietary and Confidential Information of Amdocs
 Awk contd.
● awk '/Neha/' awk_example  print any line containing search criteria, since print is the default action hence its omitted.

● awk -F, '$1 ~ /Neha/' awk_example  search Neha in the first fields separated by comma

● awk -F" " '$5>6000' employee.txt  print the employee whose salary is more than $6000

● It’s a very powerful tool when it comes to file formatting. It can group data based on a column/field.

● awk -F" " '{x+=$5}END {print x}' employee.txt  to find the sum of all employee salary

● awk -F" " '$4=="Technology" {x+=$5}END {print x}' employee.txt  find the sum of salary of employee who division is technology

● awk -F, '{print > $1}' awk_file_split  split the files into various files depending upon first field

● awk -F, '{print > $1".txt"}' awk_file_split  split the file by having .txt to the new file names.

● awk -F, '{if($2<=500)print > "500L.txt";else print > "500G.txt"}' awk_file_split  split the file depending upon the condition.

● awk 'NR%3==1{x="F"++i;}{print > x}' awk_file_split  split the file in equal lines (The condition does the trick here: NR%3==1 : NR is
the line number of the current record. NR%3 will be equal to 1 for every 3rd line such as 1st, 4th, 7th and so on. And at every 3rd line,
the file name is changed in the variable X)

● awk -F"," '{$1=++i FS $1;}1' OFS=, awk_file_split  insert a new column before the first column ($1=++i FS $1 => Space is used to
concatenate columns in awk. This expression concatenates a new field(++i) with the 1st field along with the delimiter(FS), and
assigns it back to the 1st field($1))

● awk -F"," '/$1/d' awk_file_split  delet the first column

● awk -F, '{$(NF+1)=++i;}1' OFS=, awk_file_split  insert a column after last column in a file ($NF indicates the value of last column.
Hence,by assigning something to $(NF+1), a new field is inserted at the end automatically.)

Information Security Level 1 – Confidential

83 © 2012 – Proprietary and Confidential Information of Amdocs
 Awk contd.
● awk -F, '{$(NF+1)=++i FS "X";}1' OFS=, awk_file_split  Add the two column after the last column

● awk -F, '{$(NF-1)=++i FS $(NF-1);}1' OFS=, awk_file_split  insert the records before the second last two columns (NF-1
points to the 2nd last column. Hence, by concatenating the serial number in the beginning of NF-1 ends up in inserting a
column before the 2nd last)

● awk -F, '{$2+=10;}1' OFS=, awk_file_split  update second column by adding 10 to each variable

● awk -F, '{$1=toupper($1)}1' OFS=, awk_file_split  convert the first column to upper case.

● awk -F, '{$1=substr($1,0,3)}1' OFS=, awk_file_split  extract the first three character from the first field

● awk -F, '{$2="";}1' OFS=, awk_file_split  Empty the value in the second column

● awk -F, '{for(i=1;i<=NF;i++)if(i!=x)f=f?f FS $i:$i;print f;f=""}' x=2 awk_file_split / awk -F" " '{for(i=1;i<=NF;i++)if(i!=x)f=f?f FS $i:
$i;print f;f=""}' x=2 employee.txt  Remove/delete the second column

(By just emptying a particular column, the column stays as is with empty value. To remove a column, all the subsequent
columns from that position, needs to be advanced one position ahead. The for loop loops on all the fields. Using the ternary
operator, every column is concatenated to the variable "f" provided it is not 2nd column using the FS as delimiter. At the end,
the variable "f" is printed which contains the updated record. The column to be removed is passed through the awk variable "x"
and hence just be setting the appropriate number in x, any specific column can be removed.)

Information Security Level 1 – Confidential

84 © 2012 – Proprietary and Confidential Information of Amdocs
 Thank You

Information Security Level 1 – Confidential

85 © 2012 – Proprietary and Confidential Information of Amdocs