LEARNING AWK &
SED
OBJECTIVES
Learn what is sed and awk
Similarities between sed and awk
Working Of sed and awk
How to Use sed and awk with some examples
What is sed?
■ Sed is a stream editor in Unix used for modifying files.It can perform a lot of operations
on files like searching, find, replace ,insertion and deletion as well.
■ The most common use of sed command in Unix is for substitution or for find and
replace.
■ Sed reads the input data stream line by line, applies the specified operations and then
outputs the modified data. It is often used as a filter in pipeline.
■ Sed has it’s origin in ed ,the original Unix line editor.
■ Basic difference between sed and ed is that ed is not stream oriented whereas sed is.
And ed is an interactive editor but sed is not.
WHAT IS AWK
■ Awk is one of the most powerful tools in Unix used for processing rows and
columns in a file.
■ The awk command programming language requires no compiling and allows
users to use variables, numeric functions ,string functions and logical operations.
■ Awk can be described as a pattern matching programming language. Is is
designed for processing text based data either in files or data streams.
■ Synopsis:
awk 'Program' input-file1 input-file2 ... awk -f PROGRAM-FILE input-file1 input-
file2
SIMILARITIES BETWEEN SED AND
AWK
■ They are invoked using similar syntax
$sed ‘program’ /foo/bar
$awk ‘program’ /foo/bar
■ They are both stream-oriented, reading input from text files one line at a time and
directing the result to a standard output.
■ They use regular expressions for matching patterns.
■ They allow users to specify instructions in a script.
How sed and awk works
■ Reads one line at a time from the input file
■ Makes a copy of the input file
■ Executes the given instructions on the copy of input file
■ Output the modified line.
INPUT Output
STREAM Input Line Stream
Instructions
Invoking SED
■ Syntax : /pattern/action
■ Range & Description :
P : Prints the Line.
d : Deletes the line
s/pattern1/pattern2 : Substitutes the first occurrence of pattern1 with pattern2
Examples:
■ 1> Substitution Example : Suppose we have a dat file , salary.dat whose content is as
follows :
■ Now we can replace the string ‘Ron’ with ‘Rene’.
Examples For the sed to perform a global substitution, add the
letter g to the end of the command
Contd… $cat <filename>| sed ‘s/pattern1/pattern2/g’
Replacing with empty spaces.
$cat <filename> | sed ‘s/pattern//g’
For Specifying more than one option ,use –e along
with sed
Using Regular Expressions
■ For deleting the lines starting from the given pattern the syntax is
$cat <filename>| sed ‘/^pattern/d’
■ Similarly for deleting lines ending with a particular pattern,the syntax is
$cat <filename>|sed ‘/Pattern$/d’
Invoking AWK
■ Instructions on command line
$awk ‘instructions’ /foo/bar
■ Enclosing in single quotes prevents the shell from interpreting special characters
$awk ‘/ya/’ /etc/passwd
$awk –F : ‘/ya/ {print $5}’ /etc/passwd
-F is the field separator. This option lets you to change the field delimiter.
■ Multiple instructions can be mentioned separated with a semicolon.
$awk –F : ‘/ya/{print $2;print $6;print $3}’ /etc/passwd
$1,$2,$3…etc represents individual fields and $0 represents the entire line.
Examples
■ To print the line number of a matched pattern.
$awk ‘/pattern/{print NR}’
For the below mentioned file salary.dat, we want to find the line number of the string ‘Kyra’ .For this
case the ouput must be 3.
Output :
To Print non-empty line from a file
$awk ‘NF>0’ <filename>
SOME Functions in AWK
Length(s) : Returns length of string or length of $0 if no string is supplied
Tolower(s) : Translates all uppercase characters to lower case and return the new string.
$awk –F: ‘{print tolower ($5)}’ /etc/passwd/
toupper(s) : Translates all lower case to uppercase characters
Index(s,t) Returns position of substring t in string s, or zero if not present.
THANK YOU
