H

m A

b e

oF e

ur B

te o

e A r o k

B h B

e i l

vs o

e t g

C o

o f

tt

ha

ec

Ads by Google

Linux Command Line

Linux Tutorial

Unix Shell Script

aiooitrnk ru otI tn d A T uw c
by S A
S Ion K A J L A N U A R Y 6 , 2 0 1 0

Sign up for our free email newsletter [email protected] Sign Up

8

t ee T w

Like

15

RSS

Twitter

Facebook

This is the first article on the new awk tutorial series. Well be posting several articles on awk in the upcoming weeks that will cover all features of awk with practical examples. In this article, let us review the fundamental awk working methodology along with 7 practical awk print examples. Note: Make sure you review our earlier Sed Tutorial Series.

Search

Pnnitonk do utA a crI dw

Awk is a programming language which allows easy manipulation of structured data and the generation of formatted reports. Awk stands for the names of its authors Aho, Weinberger, and Kernighan The Awk is mostly used for pattern scanning and processing. It searches one or more files to see if they contain lines that matches with the specified patterns and then perform associated actions. Some of the key features of Awk are: Awk views a text file as records and fields. Like common programming language, Awk has variables, conditionals and loops Awk has arithmetic and string operators. Awk can generate formatted reports Awk reads from a file or from its standard input, and outputs to its standard output. Awk does not get along with non-text files.
E B O O K S

Syntax:

converted by Web2PDFConvert.com

awk '/search pattern1/ {Actions} /search pattern2/ {Actions}' file

In the above awk syntax: search pattern is a regular expression. Actions statement(s) to be performed. several patterns and actions are possible in Awk. file Input file. Single quotes around program is to avoid shell not to interpret any of its special characters.

yldeW gotnk ohgiA Mw r o k

1. Awk reads the input files one line at a time. 2. For each line, it matches with given pattern in the given order, if matches performs the corresponding action. 3. If no pattern matches, no action will be performed. 4. In the above syntax, either search pattern or action are optional, But not both. 5. If the search pattern is not given, then Awk performs the given actions for each line of the input. 6. If the action is not given, print all that lines that matches with the given patterns which is the default action. 7. Empty braces with out any action does nothing. It wont perform default printing operation. 8. Each statement in Actions should be delimited by semicolon. Let us create employee.txt file which has the following content, which will be used in the examples mentioned below.
$cat 100 200 300 400 500 employee.txt Thomas Manager Jason Developer Sanjay Sysadmin Nisha Manager Randy DBA

12 Amazing and Essential Linux Books To Enrich Your Brain and Library 50 UNIX / Linux Sysadmin Tutorials 50 Most Frequently Used UNIX / Linux Commands (With Examples) How To Be Productive and Get Things Done Using GTD

Sales Technology Technology Marketing Technology

$5,000 $5,500 $7,000 $9,500 $6,000

30 Things To Do When you are Bored and have a Computer Linux Directory Structure (File System Structure) Explained with Examples Linux Crontab: 15 Awesome Cron Job Examples

Awk Example 1. Default behavior of Awk

By default Awk prints every line from the file.
$ awk '{print;}' employee.txt 100 Thomas Manager Sales 200 Jason Developer Technology 300 Sanjay Sysadmin Technology 400 Nisha Manager Marketing 500 Randy DBA Technology

Get a Grip on the Grep! 15 Practical Grep Command Examples Unix LS Command: 15 Practical Examples 15 Examples To Master Linux Command Line History Top 10 Open Source Bug Tracking System

$5,000 $5,500 $7,000 $9,500 $6,000

Vi and Vim Macro Tutorial: How To Record and Play Mommy, I found it! -- 15 Practical Linux Find Command Examples 15 Awesome Gmail Tips and Tricks

In the above example pattern is not given. So the actions are applicable to all the lines. Action print with out any argument prints the whole line by default. So it prints all the lines of the file with out fail. Actions has to be enclosed with in the braces.

15 Awesome Google Search Tips and Tricks RAID 0, RAID 1, RAID 5, RAID 10 Explained with Diagrams Can You Top This? 15 Practical Linux Top Command Examples

Awk Example 2. Print the lines which matches with the pattern.
$ awk '/Thomas/ > /Nisha/' employee.txt 100 Thomas Manager Sales 400 Nisha Manager Marketing

Top 5 Best System Monitoring Tools Top 5 Best Linux OS Distributions How To Monitor Remote Linux Host using Nagios 3.0

$5,000 $9,500

Awk Introduction Tutorial 7 Awk Print Examples How to Backup Linux? 15 rsync Command Examples

In the above example it prints all the line which matches with the Thomas or Nisha. It has two patterns. Awk accepts any number of patterns, but each set (patterns and its corresponding actions) has to be separated by newline.

The Ultimate Wget Download Guide With 15 Awesome Examples Top 5 Best Linux Text Editors Packet Analyzer: 15 TCPDUMP Command Examples

Awk Example 3. Print only specific field.

The Ultimate Bash Array Tutorial with 15 Examples

converted by Web2PDFConvert.com

Awk Example 3. Print only specific field.

Awk has number of built in variables. For each record i.e line, it splits the record delimited by whitespace character by default and stores it in the $n variables. If the line has 4 words, it will be stored in $1, $2, $3 and $4. $0 represents whole line. NF is a built in variable which represents total number of fields in a record.
$ awk '{print $2,$5;}' employee.txt Thomas $5,000 Jason $5,500 Sanjay $7,000 Nisha $9,500 Randy $6,000 $ awk '{print $2,$NF;}' employee.txt Thomas $5,000 Jason $5,500 Sanjay $7,000 Nisha $9,500 Randy $6,000

3 Steps to Perform SSH Login Without Password Using ssh-keygen & ssh-copy-id Unix Sed Tutorial: Advanced Sed Substitution Examples UNIX / Linux: 10 Netstat Command Examples The Ultimate Guide for Creating Strong Passwords 6 Steps to Secure Your Home Wireless Network Turbocharge PuTTY with 12 Powerful Add-Ons

In the above example $2 and $5 represents Name and Salary respectively. We can get the Salary using $NF also, where $NF represents last field. In the print statement , is a concatenator.

Awk Example 4. Initialization and Final Action

Awk has two important patterns which are specified by the keyword called BEGIN and END.
Syntax: BEGIN { Actions} {ACTION} # Action for everyline in a file END { Actions } # is for comments in Awk

Actions specified in the BEGIN section will be executed before starts reading the lines from the input. END actions will be performed after completing the reading and processing the lines from the input.
$ awk 'BEGIN {print "Name\tDesignation\tDepartment\tSalary";} > {print $2,"\t",$3,"\t",$4,"\t",$NF;} > END{print "Report Generated\n--------------"; > }' employee.txt Name Designation Department Salary Thomas Manager Sales $5,000 Jason Developer Technology $5,500 Sanjay Sysadmin Technology $7,000 Nisha Manager Marketing $9,500 Randy DBA Technology $6,000 Report Generated --------------

In the above example, it prints headline and last file for the reports.

Awk Example 5. Find the employees who has employee id greater than 200
$ awk '$1 >200' employee.txt 300 Sanjay Sysadmin Technology 400 Nisha Manager Marketing 500 Randy DBA Technology $7,000 $9,500 $6,000

In the above example, first field ($1) is employee id. So if $1 is greater than 200, then just do the default print action to print the whole line.

converted by Web2PDFConvert.com

Awk Example 6. Print the list of employees in Technology department

Now department name is available as a fourth field, so need to check if $4 matches with the string Technology, if yes print the line.
$ awk '$4 ~/Technology/' employee.txt 200 Jason Developer Technology $5,500 300 Sanjay Sysadmin Technology $7,000 500 Randy DBA Technology $6,000

Operator ~ is for comparing with the regular expressions. If it matches the default action i.e print whole line will be performed.

Awk Example 7. Print number of employees in Technology department

The below example, checks if the department is Technology, if it is yes, in the Action, just increment the count variable, which was initialized with zero in the BEGIN section.
$ awk 'BEGIN { count=0;} $4 ~ /Technology/ { count++; } END { print "Number of employees in Technology Dept =",count;}' employee.txt Number of employees in Tehcnology Dept = 3

Then at the end of the process, just print the value of count which gives you the number of employees in Technology department.

Recommended Reading
Sed and Awk 101 Hacks, by Ramesh Natarajan. I spend several hours a day on UNIX / Linux environment dealing with text files (data, config, and log files). I use Sed and Awk for all my my text manipulation work. Based on my Sed and Awk experience, Ive written Sed and Awk 101 Hacks eBook that contains 101 practical examples on various advanced features of Sed and Awk that will enhance your UNIX / Linux life. Even if youve been using Sed and Awk for several years and have not read this book, please do yourself a favor and read this book. Youll be amazed with the capabilities of Sed and Awk utilities.

Additional Awk Articles

Awk User-defined Variables with 3 Practical Examples 8 Powerful Awk Built-in Variables FS, OFS, RS, ORS, NR, NF, FILENAME, FNR 7 Powerful Awk Operators Examples (Unary, Binary, Arithmetic, String, Assignment, Conditional, Reg-Ex Awk Operators) 4 Awk If Statement Examples ( if, if else, if else if, : ? ) Caught In the Loop? Awk While, Do While, For Loop, Break, Continue, Exit Examples

converted by Web2PDFConvert.com

t ee T w

Like

15

Share

Comment

If you enjoyed this article, you might also like..

1. 50 Linux Sysadmin Tutorials 2. 50 Most Frequently Used Linux Commands (With Examples) 3. Top 25 Best Linux Performance Monitoring and Debugging Tools 4. Mommy, I found it! 15 Practical Linux Find Command Examples 5. Linux 101 Hacks 2nd Edition eBook Awk Introduction 7 Awk Print Examples Advanced Sed Substitution Examples 8 Essential Vim Editor Navigation Fundamentals 25 Most Frequently Used Linux IPTables Rules Examples Turbocharge PuTTY with 12 Powerful Add-Ons

Tags: Awk ACTION Block, Awk BEGIN Block, Awk Begin End, Awk END Block, Awk Tutorial Examples, Linux Awk Examples, Linux Awk Tutorial, Unix Awk Examples, Unix Awk Tutorial

{ 27 comments read them below or add one }

Steve Mills
January 6, 2010 at 3:38 am 1

I have only just started reading these articles. So far I think they are well written and the explanations are clearly done with an awareness as to how they might possibly be misunderstood and hence extra layers of detail are presented where that might happen. For example, pointing out that the tilde (~) is used to compare with regular expressions when the reader might have otherwise expected an equals sign without the explanation the reader might have decided that the tilde represented the same thing as an equals sign. I shall be reading more. Thanks for posting these articles. Kind Regards Steve

Daniel Reimann

January 6, 2010 at 6:02 am

Thank you for the post here on awk. I use it frequently, but it is always good to have some updates and reminders. Happy New Year.

Lawrence

January 7, 2010 at 4:34 am

awk is awesome! thanks for your sharing. Best Regards, Lawrence

converted by Web2PDFConvert.com

Knusper

January 9, 2010 at 5:15 pm

Hi Good article now I know what ark is, and what I could use it for well written. I follow you now on twitter!

Harsh

January 10, 2010 at 10:08 pm

Thanks for posting a tutorial on awk with illustrated examples. I Will be expecting other articles on awk

Ramesh Natarajan
@Steve,

January 14, 2010 at 9:23 pm

Yeah. ~ can be little confusing in this context, if not explained properly. Thanks for you comment. @Daniel, Yeah. Most other readers of the blog are in similar position like you. So, we are here to give constant updated and remainders of the functionality that they already know. @Lawrence, Harsh, Thanks for the very nice comment. Im glad you liked this article. @Knusper, Thanks for following us on twitter.

thalafan

March 21, 2010 at 10:24 am

Nandraka Ulladhu!!! I guess the example 2 can be done without a new line like below? Pattern as regex. ~/bin$ awk /Jason|Randy/ employee 200 Jason Developer Technology $5,500 500 Randy DBA Technology $6,000 And also what does the ; stands for? End of Pattern?

Andreia Amaral
Hi,

April 7, 2010 at 5:14 am

I want to use an if else statement like this: if $10>10 print $0 > filename1 else print $0> filename2 but its not working it is not creating filename1 or filename2, how can I do this? thanks?

Ashu Agrawal

August 6, 2010 at 10:31 am

Grt post.Thanks for making me understand the awk working

avinash

October 1, 2010 at 7:30 am

10

converted by Web2PDFConvert.com

hi, this is avinash. suppose u have a emp table like this: id name designation salary 1 avi manager 1000 2 ash manager 1500 3 nash manager 2000 4 varma trainee 500 5 chow trainee 600 6 hemanth trainee 800 using awk command, i hav to print manager total salary and trainee total salary. i need a program.. can any one plz post it

vikas

October 13, 2010 at 1:34 am

11

Hi..@Avinash.. u can try this one. awk BEGIN {man_sal=0;trainee_sal=0;} $3 ~/manager/ {man_sal+=$NF} /trainee/ {trainee_sal+=$NF} END {print Total salary of managers and trainees are=man_sal,trainee_sal} in_file.name

siva

October 15, 2010 at 1:44 am

12

Hello forum members, Thanks for AWK tutorials ,it was very help ful to me.

avinash
@ vikas thanks you

October 19, 2010 at 5:12 am

13

wish

October 21, 2010 at 3:36 am

14

hi all, if i have a issue file like: 101 add open vish iuo if exit and login again i should get the increment of the first field like 102 add open vish iuo

mounika

October 27, 2010 at 10:40 pm

15

its simply superb to understand its is very useful for the beginning learners and its is very help in exams time also so guys read and enjoy wd the unix

Lynda Zhang

November 17, 2010 at 12:36 pm

16

This is very help. How about if I want to print the salary seperated by commas, e.g. 2,000 instead of 2000

Ikem

December 30, 2010 at 8:22 pm

17

converted by Web2PDFConvert.com

Youve made a little typo: > Number of employees in _Tehcnology_ Dept = 3

sudha

February 2, 2011 at 12:38 pm

18

vary vary healpul to every one

eben

March 27, 2011 at 11:28 pm

19

Its very useful for beginers like me

kalandar

April 7, 2011 at 8:58 am

20

Hi, I found this article to be very useful. Anybody who wants to know what an awk is , this will give a fair idea. Looking forward to similar articles on other topics of unix. Thanks

Bhagyaraj
Hi,

April 24, 2011 at 1:38 am

21

Good, Please try teach in Youtube to try differntly. It will be more success. Keep it up, I need to take an exam on awk, let me see how much I can successed.

kernelkid

June 10, 2011 at 6:48 am

22

very simple and easy to understand, thanks a lot, it help me a lot

liju

June 14, 2011 at 3:05 am

23

good simple article

Marija

June 30, 2011 at 9:03 am

24

I have read few geekstuff articles until now, explanations provided are the best I have ever seen so far! Great job Thanks a lot

Muslim
hi,

July 19, 2011 at 12:02 pm

25

i have the question that how to using print command awk to sort or transpose this data from many coloums to 2 columns only #input file NAME JAN FEB MARCH APRIL MAY JUNE JULY - - - BEN 5,000 6,000 7,000 8,000 6,500 7,500 9,000 YONG 4,000 5,500 6,000 5,800 7,000 8,000 8,5000 # output should be as below. BEN 5,000
converted by Web2PDFConvert.com

BEN 6,000 BEN 7,000 BEN 8,000 BEN 6,500 BEN 7,500 BEN 9,000 YONG 4,000 YONG 5,500 YONG 6,000 YONG 5,800 YONG 7,000 YONG 8,000 YONG 8,5000 Anyone can help.thanks @muss

nails carmody

August 9, 2011 at 1:08 pm

26

I know its late, but #!/bin/bash awk { # skip the first two lines if (NR == 1 || NR == 2) continue for(i=2; i<=NF; i++) printf("%s %s\n", $1, $i) } ' datafile.txt Nice site! I learned some new things about sed I didn't know.

Sudhanshu

August 29, 2011 at 1:00 am

27

The article is simply awesome!!!

Leave a Comment
Name E-mail Website

Notify me of followup comments via e-mail

Submit

converted by Web2PDFConvert.com

P N

R E

E X

Can You Top This? 15 Practical O S Top TCommand Examples I O U S P Linux :

6 Steps for P O UbuntuTInstallation Using debootstrap T Minimal S :

About The Geek Stuff

My name is Ramesh Natarajan. I will be posting instruction guides, how-to, troubleshooting tips and tricks on Linux, database, hardware, security and web. My focus is to write articles that will either teach you or help you resolve a problem. Read more about Ramesh Natarajan and the blog.

Contact Us
Email Me : Use this Contact Form to get in touch me with your comments, questions or suggestions about this site. You can also simply drop me a line to say hello!. Follow us on Twitter Become a fan on Facebook

Support Us
Support this blog by purchasing one of my ebooks. Bash 101 Hacks eBook Sed and Awk 101 Hacks eBook Vim 101 Hacks eBook Nagios Core 3 eBook

Copyright 20082011 Ramesh Natarajan. All rights reserved | Terms of Service | Advertise

converted by Web2PDFConvert.com