Unix - Shell Scripting
Unix - Shell Scripting
Vinodh K Nair
Introduction:
Name Short ID Role Responsibility Experience Area of interest Any previous experience in shell scripting Current rating on shell scripting knowledge (1-10) Expectations from the program?
Ground Rules In order to ensure the productivity of our training, we will need to be.
Pagers and mobile phones off Full participation One speaker at a time Respect the views of others Silence indicates agreement Keep to the break times agreed
Course Agenda:
Shell Basics
1. Introduction to Shell and types of Shells 2. Variables and Keywords 3. Metacharacters 4. Regular Expressions 5. Using grep and egrep 6. Using Quotes
Pre Quiz
Write your name and Employee ID without fail. 25 questions, multiple choices in 15 minutes Use the pre-quiz columns for writing your answers Do not discuss A question can have more than one correct answer, but choose the best one. Please encircle your guessed answers to find how good you are at guessing Chocolates will be distributed for: Pre-quiz highest scorer Post-quiz highest scorer One with maximum difference
8/23/2012 7:52:52 AM 5864_ER_FED_ALT 5
1. 2. 3.
Unix Basics
Structure of Unix
Applications
Shell Kernel
H/W
Shell
Shell is a command interpreter and acts as an interface between a user and kernel
Types of Shells
Bourne Shell /usr/bin/sh Korn Shell /usr/bin/ksh C Shell /usr/bin/csh bash /bin/sh POSIX /usr/bin/sh
stand
bin
lib
dev
usr
tmp
opt
etc
home
sbin
bin
lib
newconfig
user1
user2
user3
user4
owner
group
others
Permission r w x
Weight 4 2 1
Absolute path
Absolute path represents the location of a file or directory starting from the root directory
Relative path
Accessing files and directories from the current directory
VI Editor
bottom
8/23/2012 7:52:52 AM 5864_ER_FED_ALT 13
Shell Basics
Variables
Variables are the general words which are used to store and manipulate information within a shell program. Variables are fully under user control. These can be created and destroyed at anytime. Ex: <Variable_name> = <Variable_Value> TOWN=Delhi ; TOTAL=0 ; DATE= 28Nov05
Local Variables Local Variables are the variables that are presented within the current instance of the Shell. These are not available to the any child processes that are started by the current shell. Local variables are defined by users. Global Variables Global Variables are the Variables that are presented within the current shell and also available to any child processes that are started by the current shell.
Keywords
Keywords are the words which have standard predefined meaning in the shell already. Keywords can not be used as variable names. Keywords are also called as Reserve words.
List of Keywords
Environment Variables
Environment Variables are the special variables that are set by the shell and is required by the shell in order to function correctly. These variables are also called as shell variables.
Defines the path which the shell must search in Stores the default working directory for the user Stores the login name of the user Defines the name of default working shell Defines the name of the terminal Stores the Present working Directory Defines the system prompt Defines the Internal Field Separator
Ex: $ ls a* $ ls a?b? $ ls [aeiou]* $ ls [A-Fv-z]* $ ls [!A-Z]* Lists all files beginning with character a Lists all 4 character filenames whose first character is a and third character is b Lists all the files whose first character is a, e, i, o, u Lists all files whose first character is in the range A to F or v to z Lists all files whose first character is anything other than in the range of A to Z
Regular Expressions
A regular expression is a string that can be used to describe several sequences of characters. ^ $ \ ^$ \< \> \<\> Matches the beginning of a line Matches the end of a line Used to specify patterns that contains wild cards Matches a blank line Matches at the beginning of the word Matches at the end of the word Matches a complete word
ls [-alRF] file-list
a for listing all files including the dot files l for long format R for recrusive. list the all subdirectories. F for listing directories with a trailing /
date [+format]
%date +%h %d, 19%y Oct 1, 1996
8/23/2012 7:52:53 AM 5864_ER_FED_ALT 21
more file-list
Browse through text files one page at a time.
# extract chars 1-5 and 15-20 from each line of foo. # extract field 1 and 3 from each line of moo.
Example:
file1: apple banana banana apple banana # sort fruit | uniq -c apple 2 banana 3 # tr -cs [a-z][A-Z] [\012*] < fileA | sort | uniq # show a list of distinct words in fileA.
8/23/2012 7:52:53 AM 5864_ER_FED_ALT 27
with a
-a lists all files, -s lists the grand total of each dir given
Examples: # du -s print the total disk space used by the files in and under the current dir.
# du -s *
print the disk space used by each file and dir in the current dir.
The grep
The grep stands for globally regular expression print. It lets us to search particular patterns in a file. The general form of grep is shown below grep < word > < file_name > Grep options -i -n -v -l Ex: $ grep ^unix read.txt $ grep system$ read.txt $ grep -l hello * $ grep -v unix read.txt Lists the lines beginning with word unix Lists the lines ending with word system Lists the file name which contains word hello Lists all the lines except those which contains the word unix Ignores the case Lists the line numbers along with matching patters Lists all the lines that do not match the patters Lists only filenames that contain match
Quotes
\ `` Ex: $ echo Hello\;Welcome $ echo $PWD $ echo $PWD $ echo Working Directory is : `pwd` $ OS=HP-UX $ VER=11i $ echo $OS $VER $ echo $OS $VER `hostname` $ echo $OS $VER $ echo $OS $VER `hostname` Used to specify patterns that contains wild cards Used to take string as-is Used when command substitution is needed Used when command evaluation is needed
Shell Initialization
Script Description Variable Declaration Function Definition Main Program Function calling Exit with Return Status
Preparation:
Organize the components into logical order Keep everything simple to start with Write the components in plain English, and it to shell syntax
Simple Program
Consider the following script and observe the output
### Script #1 welcome.sh echo Hello! Welcome to SHELL World echo Shell Scripts are very handy to system admins echo Enjoy the Course $ chmod u+x welcome.sh
$ ./welcome.sh Hello! Welcome to SHELL World Shell Scripts are very handy to system admins Enjoy the Course
Interactive Script
Script that runs interactively. It takes input from user while execution ###Script #2 interactive.sh echo Enter your Name : read name echo Enter your Designation : read design echo Enter your Organization name : read org echo Hello $name!! echo Your Position in $org is $design
Here Document
Here Document provides STDIN to Unix command from lines that follow until delimiter is found at start line. General syntax command << delimiter input command 1 input command 2 delimiter Ex: $ ./interactive.sh << EOF Paul System Administrator CSC EOF
Example: Clear the screen, move the cursor to line 10, column 20 and turn on bold. tput -S <<EOF clear cup 10 20 bold EOF
Shell Arithmetic
expr command is used to perform integer arithmetic operations in shell. General syntax expr < integer 1 > < operand > < integer 2 > expr operands Operand + \* / % Ex: $ TOTAL=`expr 5 + 4 \* 2`
8/23/2012 7:52:53 AM 5864_ER_FED_ALT 43
Escape Sequence
An escape sequence is a special sequence of characters that represents another character. It is used to format the output messages. Escape Sequence \a \b \c \f \n \r \t \\ Ex: $ echo First Tab\tSecond Tab $ echo First Line\nSecond Line Description Beep sound Backspace Suppress Trailing new line Formfeed New line Carriage Return Tab Backslash
Positional Parameters
These are the special variables defined by the shell and are numbered $1 thorough $9. These can be used to pass the additional information to the script as arguments. Values can be set to these variables using set command. Ex: $ set Shell is a command interpreter $ echo $1 $2 $3 $4 $5 Variable $0 $n $# $* $@ $? $$ $! Description Name of the command being executed Variable correspond to the argument ( n is b/w 1-9 ) No of arguments passed Lists all the arguments passed Lists all the arguments passed Exit status of the previous command Process ID of the current Shell Process ID of the last background command
Conditional Statement
Conditional statement is used to implement decision control instruction. Shell uses the keyword if for this purpose. General syntax 1.
if <test_condition> then command set1 else command set2 fi __________________________________________________________________________ EXAMPLES: if test $# -eq 0 if [ $# -eq 0 ] then then echo no positional param! echo no positional param! fi fi
8/23/2012 7:52:54 AM 5864_ER_FED_ALT 51
2.
Operator ! -a -o
Description True if file exists True if file exists True if file exists True if file exists True if file exists True if file exists
and is not a directory and size is greater than 0 and is a directory and has a read permission and has a write permission and has an execute permission
if [ ! condition ] if [ -f $FILENAME ] if [ -d $DIRNAME ] if [ -x $FILENAME ] if [ -w $FILENAME ] if [ -r $FILENAME ] if [ -s $FILENAME ] if [ $VAR1 -ne $VAR2 ] if [ $VAR1 -gt $VAR2 ] if [ $VAR1 -lt $VAR2 ] if [ $VAR1 -eq $VAR2 ] if [ $VAR1 -ge $VAR2 ] if [ $VAR1 -le $VAR2 ] if ["$VAR1"="$VAR2"] if ["$VAR1"!="$VAR2"] if [ ! "$VAR1" ] if [ "$VAR1" ]
If condition is not true If file exists called $FILENAME If directory exists called $DIRNAME If file $FILENAME is executable If file $FILENAME is writable If file $FILENAME is readable If file $FILENAME exists and is of greater than zero size. If VAR1 does not equal VAR2 If VAR1 is greater than VAR2 If VAR1 is less than VAR2 If VAR1 is equal to VAR2 If VAR1 is greater than or equal to VAR2 if VAR1 is less than or equal to VAR2 If VAR1 is equal to VAR2 (for strings) If VAR1 does not equal VAR2 (strings) If VAR1 is null If VAR1 is not null
esac
###Script #3 makechoice.sh echo Hello!! Welcome to the address tool echo 1. CSC Indore echo 2. CSC Noida echo 3. CSC Hyderabad echo \n Choice: \c read choice case $ choice in 1) echo CSC Indore echo Building #, Indore, MP ;; 2) echo CSC Noida echo C-24/25, Sector- 56, Noida, UP ;; 3) echo CSC Hyderabad echo Building 7, Mindspace, Hyderabad, AP ;; *) echo Invalid choice selected ;; esac
8/23/2012 7:52:54 AM 5864_ER_FED_ALT 57
Loops
Loops enable us to execute series of commands multiple times. There are various loops available in shell to perform the repeated execution. The while loop The until loop The for loop While and until loops works in a very similar fashion. While loop executes as long as the test condition remains true whereas the until loop executes as long as the test condition remains false. For loop works slightly different from the while and until. It executes set of commands repeatedly for each item listed in the list. The most common use of for loop is to perform same set of commands for large number of files.
IO redirection
When a command is executed, the result produced is printed to a terminal which is called STDOUT. And the error messages occurred are printed to a special terminal called STDERR. Most of the commands will have STDOUT as their STDERR. Using the IO redirection operators, we can redirect the output or error messages to any files instead of STDOUT or STDERR. All these IO devices are identified by numbers as follows 0 1 2 2>&1 &> < > >> << | Standard In Standard Out Standard Error Redirect standard error to standard out Redirect standard error & standard out Redirect input Redirect output Concatenate output to file Here Is File for scripts Pipe
8/23/2012 7:52:54 AM 5864_ER_FED_ALT 66
Return: The return command is used to return control from a called subroutine to the instruction following the call to that subroutine.
Break: The break command discontinues the execution of loop immediately and transfers control to the command following the done keyword. Continue: The continue command skips the remaining part of the loop and transfers control to the start of the loop for next iteration. Exit: The exit command completely terminates the program. It returns an exit code that is optionally provided as argument in the program.
Functions
A subroutine (function, method, procedure, or subprogram) is a portion of code within a larger program, which performs a specific task and is relatively independent of the remaining code. The syntax of POSIX shell includes support for creating self contained subroutines, and for calling and returning from them. There are many advantages to breaking a program up into subroutines, including: reducing the duplication of code in a program (e.g., by replicating useful functionality, such as mathematical functions), enabling reuse of code across multiple programs, decomposing complex problems into simpler pieces (this improves maintainability and ease of extension), improving readability of a program, hiding or regulating part of the program The components of a subroutine may include: a body of code to be executed when the subroutine is called parameters that are passed to the subroutine from the point where it is called a value that is returned to the point where the call occurs
AWK
The Awk text-processing language is useful for such tasks as: Tallying information from text files and creating reports from the results.
Debug
Sometimes the hardest part about shell programming is finding the bugs in your scripts. This session walks you through the various features of the shell along with some other techniques to help you track down and fix the bugs in your scripts.
Exercise
Exercise
Search exact pattern from /etc/passwd Add 10 users using script Move a list of *.txt to *.doc Mirror (lvextend) 10 lvols using a script Remove all blank lines from a file Remove 10 different lines from a file matching patterns Investigate a given command used by a set of users Write a script to add two values supplied as script arguments
Post Quiz
Write your name and Employee ID without fail. 25 questions, multiple choices in 15 minutes Use the post-quiz columns for writing your answers Do not discuss A question can have more than one correct answer, but choose the best one. Please encircle your guessed answers to find how good you are at guessing Chocolates will be distributed for: Pre-quiz highest scorer Post-quiz highest scorer One with maximum difference
8/23/2012 7:52:56 AM 5864_ER_FED_ALT 93
1. 2. 3.
Score Card
Sl no 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
8/23/2012 7:52:56 AM 5864_ER_FED_ALT 94
Trainee Name
Pre-quiz
Post-quiz
Difference
My Contact:
Vinodh Kombissan [email protected] Mobile: 9940041817 Nortel : +91 44 39878899 Extn 1227
Computer Sciences Corporation, 6th & 7th Floor , No 9 Prince Kushal Towers, Chennai - 600 002. Please provide your valuable feedback and suggestions
8/23/2012 7:52:57 AM 5864_ER_FED_ALT 95
Thank You