Shell Scripting – Part 1

Le Yan/Alex Pacheco
HPC User Services @ LSU

2/11/2015 HPC training series Spring 2015

 Shell Scripting
• Part 1 (today)
– Simple topics such as creating and executing
simple shell scripts, arithmetic operations, flow
control, command line arguments and functions.
• Part 2 (March 4th)
– Advanced topics such as regular expressions and
text processing tools (grep, sed, awk etc.)

2/11/2015 HPC training series Spring 2015 1

 Outline
• Recap of Linux 101
• Shell Scripting Basics
• Beyond Basic Shell Scripting
– Arithmetic Operations
– Arrays
– Flow Control
– Command Line Arguments
– Functions
• Advanced Topics Preview

2/11/2015 HPC training series Spring 2015 2

 What Do Operating Systems Do?
Application • Operating systems
Shell work as a bridge
Kernel between hardware
and applications
– Kernel: hardware
Hardware drivers etc.
– Shell: user interface to
kernel
– Some applications
(system utilities)

: Operating System

2/11/2015 HPC training series Spring 2015 3

 Kernel
• Kernel
– The kernel is the core component of most operating systems
– Kernel’s responsibilities include managing the system’s
resources
– It provides the lowest level abstraction layer for the resources
(especially processors and I/O devices) that application software
must control to perform its functions
– It typically makes these facilities available to application
processes through inter‐process communication mechanisms
and system calls

2/11/2015 HPC training series Spring 2015 4

 Shell
• Shell
– The command line interface is the primary user
interface to Linux/Unix operating systems.
– Each shell has varying capabilities and features and
the users should choose the shell that best suits their
needs
– The shell can be deemed as an application running on
top of the kernel and provides a powerful interface to
the system.

2/11/2015 HPC training series Spring 2015 5

 Type of Shell
• sh (Bourne Shell)
– Developed by Stephen Bourne at AT\&T Bell Labs
• csh (C Shell)
– Developed by Bill Joy at University of California, Berkeley
• ksh (Korn Shell)
– Developed by David Korn at AT&T Bell Labs
– Backward‐compatible with the Bourne shell and includes many features of the C shell
• bash (Bourne Again Shell)
– Developed by Brian Fox for the GNU Project as a free software replacement for the Bourne
shell
– Default Shell on Linux and Mac OSX
– The name is also descriptive of what it did, bashing together the features of sh, csh and ksh
• tcsh (TENEX C Shell)
– Developed by Ken Greer at Carnegie Mellon University
– It is essentially the C shell with programmable command line completion, command‐line
editing, and a few other features.

2/11/2015 HPC training series Spring 2015 6

 Shell Comparison
Software sh csh ksh bash tcsh
Programming language y y y y y
Shell variables y y y y y
Command alias n y y y y
Command history n y y y y
Filename autocompletion n y* y* y y
Command line editing n n y* y y
Job control n y y y y

*: not by default

http://www.cis.rit.edu/class/simg211/unixintro/Shell.html

2/11/2015 HPC training series Spring 2015 7

 File Editing
• The two most commonly used editors on Linux/Unix systems are:
– vi or vim (vi improved)
– emacs
• vi/vim is installed by default on Linux/Unix systems and has only
a command line interface (CLI).
• emacs has both a CLI and a graphical user interface (GUI).
– if emacs GUI is installed then use emacs –nw to open file in console
• Other editors you may come across: kate, gedit, gvim,
pico, nano, kwrite
• To use vi or emacs is your choice, but you need to know one of
them
• For this tutorial, we assume that you already know how to edit a
file with a command line editor

2/11/2015 HPC training series Spring 2015 8

 Variables
• Linux allows the use of variables
– Similar to programming languages
• A variable is a named object that contains data
– Number, character or string
• There are two types of variables: ENVIRONMENT and user defined
• Environment variables provide a simple way to share configuration
settings between multiple applications and processes in Linux
– Environment variables are often named using all uppercase letters
– Example: PATH, LD_LIBRARY_PATH, DISPLAY etc.
• To reference a variable, prepend $ to the name of the variable, e.g.
$PATH, $LD_LIBRARY_PATH
– Example: $PATH, $LD_LIBRARY_PATH, $DISPLAY etc.

2/11/2015 HPC training series Spring 2015 9

 Variables Names
• Rules for variable names
– Must start with a letter or underscore
– Number can be used anywhere else
– Do not use special characters such as @,#,%,$
– (again) They are case sensitive
– Example
• Allowed: VARIABLE, VAR1234able, var_name,
_VAR
• Not allowed: 1var, %name, $myvar, var@NAME

2/11/2015 HPC training series Spring 2015 10

 Editing Variables (1)
• How to assign values to variables depends on the
shell
Type sh/ksh/bash csh/tcsh
Shell name=value set name=value
Environment export name=value setenv name=value

• Shell variables is only valid within the current

shell, while environment variables are valid for all
subsequently opened shells.

2/11/2015 HPC training series Spring 2015 11

 Editing Variables (2)
• Example: to add a directory to the PATH variable
sh/ksh/bash: export PATH=/path/to/executable:${PATH}
csh/tcsh: setenv PATH /path/to executable:${PATH}

– sh/ksh/bash: no spaces except between export and

PATH
– csh/tcsh: no “=“ sign
– Use colon to separate different paths
– The order matters: if you have a customized version of a
software say perl in your home directory, if you append
the perl path to PATH at the end, your program will use
the system wide perl not your locally installed version

2/11/2015 HPC training series Spring 2015 12

 Basic Commands
• Command is a directive to a computer program acting as an
interpreter of some kind, in order to perform a specific task
• Command prompt is a sequence of characters used in a
command line interface to indicate readiness to accept
commands
– Its intent is literally to prompt the user to take action
– A prompt usually ends with one of the characters $,%#,:,> and
often includes information such as user name and the current
working directory
• Each command consists of three parts: name, options and
arguments

2/11/2015 HPC training series Spring 2015 13

 List of Basic Commands
Name Function
ls Lists files and directories
cd Changes the working directory
mkdir Creates new directories
rm Deletes files and directories
cp Copies files and directories
mv Moves or renames files and directories
pwd prints the current working directory
echo prints arguments to standard output
cat Prints file content to standard output

2/11/2015 HPC training series Spring 2015 14

 File Permission (1)
• Since *NIX OS's are designed for multi user environment, it is
necessary to restrict access of files to other users on the system.
• In *NIX OS's, you have three types of file permissions
– Read (r)
– Write (w)
– Execute (x)
• for three types of users
– User (u) (owner of the file)
– Group (g) (group owner of the file)
– World (o) (everyone else who is on the system)

2/11/2015 HPC training series Spring 2015 15

 File Permission (2)
[lyan1@mike2 ~]$ ls -al
total 4056
drwxr-xr-x 45 lyan1 Admins 4096 Sep 2 13:30 .
drwxr-xr-x 509 root root 16384 Aug 29 13:31 ..
drwxr-xr-x 3 lyan1 root 4096 Apr 7 13:07 adminscript
drwxr-xr-x 3 lyan1 Admins 4096 Jun 4 2013 allinea
-rw-r--r-- 1 lyan1 Admins 12 Aug 12 13:53 a.m
drwxr-xr-x 5 lyan1 Admins 4096 May 28 10:13 .ansys
-rwxr-xr-x 1 lyan1 Admins 627911 Aug 28 10:13 a.out

• The first column indicates the type of the file

– d for directory
– l for symbolic link
– - for normal file

2/11/2015 HPC training series Spring 2015 16

 File Permission (2)
[lyan1@mike2 ~]$ ls -al
total 4056
drwxr-xr-x 45 lyan1 Admins 4096 Sep 2 13:30 .
drwxr-xr-x 509 root root 16384 Aug 29 13:31 ..
drwxr-xr-x 3 lyan1 root 4096 Apr 7 13:07 adminscript
drwxr-xr-x 3 lyan1 Admins 4096 Jun 4 2013 allinea
-rw-r--r-- 1 lyan1 Admins 12 Aug 12 13:53 a.m
drwxr-xr-x 5 lyan1 Admins 4096 May 28 10:13 .ansys
-rwxr-xr-x 1 lyan1 Admins 627911 Aug 28 10:13 a.out

• The next nine columns can be grouped into

three triads, which indicates what the owner,
the group member and everyone else can do

2/11/2015 HPC training series Spring 2015 17

 File Permission (2)
[lyan1@mike2 ~]$ ls -al
total 4056
drwxr-xr-x 45 lyan1 Admins 4096 Sep 2 13:30 .
drwxr-xr-x 509 root root 16384 Aug 29 13:31 ..
drwxr-xr-x 3 lyan1 root 4096 Apr 7 13:07 adminscript
drwxr-xr-x 3 lyan1 Admins 4096 Jun 4 2013 allinea
-rw-r--r-- 1 lyan1 Admins 12 Aug 12 13:53 a.m
drwxr-xr-x 5 lyan1 Admins 4096 May 28 10:13 .ansys
-rwxr-xr-x 1 lyan1 Admins 627911 Aug 28 10:13 a.out

• We can also use weights to indicate file permission

– r=4, w=2, x=1
– Example: rwx = 4+2+1 = 7, r‐x = 4+1 = 5, r‐‐ = 4
– This allows us to use three numbers to represent the permission
– Example: rwxr‐xr‐w = 755

2/11/2015 HPC training series Spring 2015 18

 Input & Output Commands (1)
• The basis I/O statement are echo for displaying to screen and read
for reading input from screen/keyboard/prompt
• echo
– The echo arguments command will print arguments to screen or
standard output, where arguments can be a single or multiple
variables, string or numbers
• read
– The read statement takes all characters typed until the Enter key is
pressed
– Usage: read <variable name>
– Example: read name

2/11/2015 HPC training series Spring 2015 19

 Input & Output Commands (2)
• Examples
[lyan1@mike2 ~]$ echo $SHELL
/bin/bash
[lyan1@mike2 ~]$ echo Welcome to HPC training
Welcome to HPC training
[lyan1@mike2 ~]$ echo "Welcome to HPC training"
Welcome to HPC training

• By default, echo eliminates redundant whitespaces (multiple

spaces and tabs) and replaces it with a single whitespace between
arguments.
– To include redundant whitespace, enclose the arguments within
double quotes

2/11/2015 HPC training series Spring 2015 20

 I/O Redirection
• There are three file descriptors for I/O streams (remember
everything is a file in Linux)
– STDIN: Standard input
– STDOUT: standard output
– STDERR: standard error
• 1 represents STDOUT and 2 represents STDERR
• I/O redirection allows users to connect applications
– <: connects a file to STDIN of an application
– >: connects STDOUT of an application to a file
– >>: connects STDOUT of an application by appending to a file
– |: connects the STDOUT of an application to STDIN of another
application.

2/11/2015 HPC training series Spring 2015 21

 I/O Redirection Examples
• Write STDOUT to file: ls –l > ls-l.out
• Write STDERR to file: ls –l &2 > ls-
l.err
• Write STDERR to STDOUT: ls –l 2>&1
• Send STDOUT as STDIN for another
application: ls –l | less

2/11/2015 HPC training series Spring 2015 22

 Outline
• Recap of Linux 101
• Shell Scripting Basics
• Beyond Basic Shell Scripting
– Arithmetic Operations
– Arrays
– Flow Control
– Command Line Arguments
– Functions
• Advanced Topics Preview

2/11/2015 HPC training series Spring 2015 23

 Scripting Languages
• A script is a program written for a software environment that
automate the execution of tasks which could alternatively be
executed one‐by‐one by a human operator.
• Shell scripts are a series of shell commands put together in a file
– When the script is executed, it is as if someone type those commands
on the command line
• The majority of script programs are ``quick and dirty'', where the
main goal is to get the program written quickly.
– Compared to programming languages, scripting languages do not
distinguish between data types: integers, real values, strings, etc.
– Might not be as efficient as programs written in C and Fortran, with
which source files need to be compiled to get the executable

2/11/2015 HPC training series Spring 2015 24

 Startup Scripts
• When you login to a *NIX computer, shell scripts are automatically loaded
depending on your default shell
• sh/ksh (in the specified order)
– /etc/profile
– $HOME/.profile
• bash (in the specified order)
– /etc/profile (for login shell)
– /etc/bashrc or /etc/bash/bashrc
– $HOME/.bash_profile (for login shell)
– $HOME/.bashrc
• csh/tcsh (in the specified order)
– /etc/csh.cshrc
– $HOME/.tcshrc
– $HOME/.cshrc (if .tcshrc is not present)
• .bashrc, .tcshrc, .cshrc, .bash_profile are script files where
users can define their own aliases, environment variables, modify paths etc.

2/11/2015 HPC training series Spring 2015 25

 An Example

2/11/2015 HPC training series Spring 2015 26

 Writing and Executing a Script
• Three steps
– Create and edit a text file (hello.sh)

– Set the appropriate permission

– Execute the script

2/11/2015 HPC training series Spring 2015 27

 Components Explained

• The first line is called the "Shebang” line. It tells the OS

which interpreter to use. In the current example, bash
– For tcsh, it would be: #!/bin/tcsh
• The second line is a comment. All comments begin with
"#".
• The third line tells the OS to print "Hello World!" to the
screen.

2/11/2015 HPC training series Spring 2015 28

 Special Characters (1)
# Starts a comment line.
$ Indicates the name of a variable.
\ Escape character to display next character literally
{} Used to enclose name of variable
; Command separator. Permits putting two or more commands on the same
line.
;; Terminator in a case option
. “dot” command. Equivalent to source (for bash only)

2/11/2015 HPC training series Spring 2015 29

 Special Characters (2)
$? Exit status variable.
$$ Process ID variable.
[] Test expression.
[[ ]] Test expression, more flexible than []
$[], Integer expansion
$(())
||, &&, Logical OR, AND and NOT
!

2/11/2015 HPC training series Spring 2015 30

 Quotation
• Double quotation
– Enclosed string is expanded
• Single quotation
– Enclosed string is read literally
• Back quotation
– Enclose string is executed as a command

2/11/2015 HPC training series Spring 2015 31

 Quotation ‐ Examples

[lyan1@mike2 ~]$ str1="I am $USER"

[lyan1@mike2 ~]$ echo $str1
I am lyan1
[lyan1@mike2 ~]$ str2='I am $USER'
[lyan1@mike2 ~]$ echo $str2
I am $USER
[lyan1@mike2 ~]$ str3=`echo $str2`
[lyan1@mike2 ~]$ echo $str3
I am $USER

2/11/2015 HPC training series Spring 2015 32

 Quotation – More Examples

2/11/2015 HPC training series Spring 2015 33

 Outline
• Recap of Linux 101
• Shell Scripting Basics
• Beyond Basic Shell Scripting
– Arithmetic Operations
– Arrays
– Flow Control
– Command Line Arguments
– Functions
• Advanced Topics Preview

2/11/2015 HPC training series Spring 2015 34

 Arithmetic Operations (1)
• You can carry out numeric operations on
integer variables
Operation Operator
Addition +
Subtraction -
Multiplication *
Division /
Exponentiation ** (bash only)
Modulo %

2/11/2015 HPC training series Spring 2015 35

 Arithmetic Operations (2)
• bash
– $((…)) or $[…] commands
• Addition: $((1+2))
• Multiplication: $[$a*$b]
– Or use the let command: let c=$a-$b
– Or use the expr command: c=‘expr $a - $b‘
– You can also use C‐style increment operators:
let c+=1 or let c--

2/11/2015 HPC training series Spring 2015 36

 Arithmetic Operations (3)
• tcsh
– Add two numbers: @ x = 1 + 2
– Divide two numbers: @ x = $a / $b
– You can also use the expr command: set c = ‘expr $a % $b‘
– You can also use C‐style increment operators:
@ x -= 1 or @ x++
• Note the use of space
– bash: space required around operator in the expr command
– tcsh: space required between @ and variable, around = and numeric
operators.

2/11/2015 HPC training series Spring 2015 37

 Arithmetic Operations (4)
• For floating numbers
– You would need an external calculator like the GNU bc
• Add two numbers
echo "3.8 + 4.2" | bc
• Divide two numbers and print result with a precision of 5 digits:
echo "scale=5; 2/5" | bc
• Call bc directly:
bc <<< “scale=5; 2/5”
• Use bc -l to see result in floating point at max scale:
bc -l <<< "2/5"

2/11/2015 HPC training series Spring 2015 38

 Arrays (1)
• bash and tcsh supports one‐dimensional arrays
• Array elements may be initialized with the variable[i] notation:
variable[i]=1
• Initialize an array during declaration
– bash: name=(firstname ’last name’)
– tcsh: set name = (firstname ’last name’)
• Reference an element i of an array name: ${name[i]}
• Print the whole array
– bash: ${name[@]}
– tcsh: ${name}
• Print length of array
– bash: ${#name[@]}
– tcsh: ${#name}

2/11/2015 HPC training series Spring 2015 39

 Arrays (2)
• Print length of element i of array name: ${#name[i]}
– Note: In bash ${#name} prints the length of the first element of the
array
• Add an element to an existing array
– bash name=(title ${name[@]})
– tcsh set name = ( title "${name}")
– In the above tcsh example, title is first element of new array while the
second element is the old array name
• Copy an array name to an array user
– bash user=(${name[@]})
– tcsh set user = ( ${name} )

2/11/2015 HPC training series Spring 2015 40

 Arrays (3)
• Concatenate two arrays
– bash nameuser=(${name[@]} ${user[@]})
– tcsh set nameuser=( “${name}” “${user}” )
• Delete an entire array: unset name
• Remove an element i from an array
– bash unset name[i]
– tcsh @ j = $i – 1
@ k = $i + 1
set name = ( “${name[1-$j]}” “${name[$k-]}” )
• Note
– bash: array index starts from 0
– tcsh: array index starts from 1

2/11/2015 HPC training series Spring 2015 41

 Arrays (4)

2/11/2015 HPC training series Spring 2015 42

 Flow Control
• Shell scripting languages execute commands in
sequence similar to programming languages such as C
and Fortran
– Control constructs can change the order of command
execution
• Control constructs in bash and tcsh are
– Conditionals: if
– Loops: for, while, until
– Switches: case, switch

2/11/2015 HPC training series Spring 2015 43

 if statement
• An if/then construct tests whether the exit status of a list of
commands is 0, and if so, execute one or more commands

• Note the space between condition and the brackets

– bash is very strict about spaces.
– tcsh commands are not so strict about spaces
– tcsh uses the if-then-else if-else-endif similar to
Fortran

2/11/2015 HPC training series Spring 2015 44

 File Tests
Operation bash tcsh
File exists if [ -e .bashrc ] if ( -e .tcshrc )
File is a regular file if [ -f .bashrc ]
File is a directory if [ -d /home ] if ( -d /home )

File is not zero size if [ -s .bashrc ] if ( ! -z .tcshrc )

File has read permission if [ -r .bashrc ] if ( -r .tcshrc )
File has write permission if [ -w .bashrc ] if ( -w .tcshrc )
File has execute permission if [ -x .bashrc ] if ( -x .tcshrc )

2/11/2015 HPC training series Spring 2015 45

 Integer Comparisons
Operation bash tcsh
Equal to if [ 1 –eq 2] if (1 == 2)
Not equal to if [ $a –ne $b ] if ($a != $b)
Greater than if [ $a –gt $b ] if ($a > $b)
Greater than or equal to if [ 1 –ge $b ] if (1 >= $b)
Less than if [ $a –lt 2 ] if ($a < 2)
Less than or equal to if [[ $a –le $b ]] if ($a <= $b)

2/11/2015 HPC training series Spring 2015 46

 String Comparisons
Operation bash tcsh
Equal to if [ $a == $b ] if ($a == $b)
Not equal to if [ $a != $b ] if ($a != $b)
Zero length or null if [ -z $a ] if ($%a == 0)
Non zero length if [ -n $a ] if ($%a > 0)

2/11/2015 HPC training series Spring 2015 47

 Logical Operators
Operation Example
! (NOT) if [ ! -e .bashrc ]
&& (AND) if [ -f .bashrc ] && [ -s .bashrc ]
| (OR) if [[ -f .bashrc || -f .bash_profile ]]
if ( -e /.tcshrc && ! -z /.tcshrc )

2/11/2015 HPC training series Spring 2015 48

 Examples

2/11/2015 HPC training series Spring 2015 49

 Loop Constructs
• A loop is a block of code that iterates a list of
commands as long as the loop control
condition is evaluated to true
• Loop constructs
– bash: for, while and until
– tcsh: foreach and while

2/11/2015 HPC training series Spring 2015 50

 For Loop ‐ bash
• The for loop is the basic looping construct in bash

• The for and do lines can be written on the same line:

for arg in list; do
• for loops can also use C style syntax

2/11/2015 HPC training series Spring 2015 51

 For Loop ‐ tcsh
• The foreach loop is the basic looping
construct in tcsh

2/11/2015 HPC training series Spring 2015 52

 While Loop
• The while construct tests for a condition at the top of a loop and
keeps going as long as that condition is true.
• In contrast to a for loop, a while loop finds use in situations
where the number of loop repetitions is not known beforehand.
• bash

• tcsh

2/11/2015 HPC training series Spring 2015 53

 While Loop ‐ Example

2/11/2015 HPC training series Spring 2015 54

 Until Loop
• The until construct tests for a condition at the top of a
loop, and keeps looping as long as that condition is false
(opposite of while loop)

2/11/2015 HPC training series Spring 2015 55

• for, while, and until loops can be nested,
to exit from the loop use the break command

2/11/2015 HPC training series Spring 2015 56

2/11/2015 HPC training series Spring 2015 57
 Switching Constructs ‐ bash
• The case and select constructs are technically not loops since
they do not iterate the execution of a code block
• Like loops, however, they direct program flow according to
conditions at the top or bottom of the block

2/11/2015 HPC training series Spring 2015 58

 Switching Constructs ‐ tcsh
• tcsh has the switch constructs

2/11/2015 HPC training series Spring 2015 59

2/11/2015 HPC training series Spring 2015 60
2/11/2015 HPC training series Spring 2015 61
 Command Line Arguments (1)
• Similar to programming languages, bash and other shell scripting
languages can also take command line arguments
– Execute: ./myscript arg1 arg2 arg3
– Within the script, the positional parameters $0, $1, $2, $3 correspond
to ./myscript, arg1, arg2, and arg3, respectively.
– $#: number of command line arguments
– $*: all of the positional parameters, seen as a single word
– $@: same as $* but each parameter is a quoted string.
– shift N: shift positional parameters from N+1 to $# are renamed to
variable names from $1 to $# - N + 1
• In csh and tcsh
– An array argv contains the list of arguments with argv[0] set to the name
of the script
– #argv is the number of arguments, i.e. length of argv array

2/11/2015 HPC training series Spring 2015 62

2/11/2015 HPC training series Spring 2015 63
 Declare command
• Use the declare command to set variable and functions
attributes
• Create a constant variable, i.e. read‐only
– declare -r var
– declare -r varName=value
• Create an integer variable
– declare -i var
– declare -i varName=value
• You can carry out arithmetic operations on variables declared as
integers

2/11/2015 HPC training series Spring 2015 64

 Functions (1)
• Like “real” programming languages, bash has functions.
• A function is a code block that implements a set of operations, a
“black box” that performs a specified task.
• Wherever there is repetitive code, when a task repeats with only
slight variations in procedure, then consider using a function.

2/11/2015 HPC training series Spring 2015 65

2/11/2015 HPC training series Spring 2015 66
 Functions (2)
• You can also pass arguments to a function
• All function parameters can be accessed via $1,
$2, $3…
• $0 always point to the shell script name
• $* or $@ holds all parameters passed to a
function
• $# holds the number of positional parameters
passed to the function

2/11/2015 HPC training series Spring 2015 67

 Functions (3)
• Array variable called FUNCNAME contains the names of
all shell functions currently in the execution call stack.
• By default all variables are global.
• Modifying a variable in a function changes it in the
whole script.
• You can create a local variables using the local
command
local var=value
local varName

2/11/2015 HPC training series Spring 2015 68

• A function may recursively call itself even
without use of local variables.

2/11/2015 HPC training series Spring 2015 69

 Outline
• Recap of Linux 101
• Shell Scripting Basics
• Beyond Basic Shell Scripting
– Arithmetic Operations
– Arrays
– Flow Control
– Command Line Arguments
– Functions
• Advanced Topics Preview

2/11/2015 HPC training series Spring 2015 70

 Advanced Topics Preview
• Text processing commands
– grep & egrep
– sed
– awk
• Regular expression (RegEx)

2/11/2015 HPC training series Spring 2015 71

 grep & egrep
• grep is a Unix utility that searches through
either information piped to it or files in the
current directory.
• egrep is extended grep, same as grep -E
• Use zgrep for compressed files.
• Usage:
grep <options> <search pattern> <files>

2/11/2015 HPC training series Spring 2015 72

 sed
• sed ("stream editor") is Unix utility for
parsing and transforming text files.
• sed is line‐oriented
– It operates one line at a time and allows regular
expression matching and substitution.

2/11/2015 HPC training series Spring 2015 73

 awk
• The awk text‐processing language is useful for such tasks as:
– Tallying information from text files and creating reports from the
results.
– Adding additional functions to text editors like "vi".
– Translating files from one format to another.
– Creating small databases.
– Performing mathematical operations on files of numeric data.
• awk has two faces:
– It is a utility for performing simple text‐processing tasks, and
– It is a programming language for performing complex text‐processing
tasks.

2/11/2015 HPC training series Spring 2015 74

 Further Reading
• BASH Programming
http://tldp.org/HOWTO/bash-Prog-Intro-HOWTO.html
• CSH Programming
http://www.grymoire.com/Unix/Csh.html
• csh Programming Considered Harmful
http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/
• Wiki Books
http://en.wikibooks.org/wiki/Subject:Computing

2/11/2015 HPC training series Spring 2015 75

 Next Tutorial ‐ Distributed Job
Execution
• If any of the following fits you, then you might
want come
– I have to run more than one serial job.
– I don’t want to submit multiple job using the serial
queue
– How do I submit one job which can run multiple serial
jobs?
• Date: Feb 25th, 2015

2/11/2015 HPC training series Spring 2015 76

 Getting Help
• User Guides
– LSU HPC: http://www.hpc.lsu.edu/docs/guides.php#hpc
– LONI:http://www.hpc.lsu.edu/docs/guides.php#loni
• Documentation: http://www.hpc.lsu.edu/docs
• Online courses: http://moodle.hpc.lsu.edu
• Contact us
– Email ticket system: sys‐help@loni.org
– Telephone Help Desk: 225‐578‐0900
– Instant Messenger (AIM, Yahoo Messenger, Google Talk)
• Add “lsuhpchelp”

2/11/2015 HPC training series Spring 2015 77

 Questions?

2/11/2015 HPC training series Spring 2015 78

 Exercises
1. Write a shell script to
– Print “Hello world!” to the screen
– Use a variable to store the greeting
2. Write a shell script to
– Take two integers on the command line as arguments
– Print the sum, different, product of those two integers
– Think: what if there are too few or too many arguments? How can you check
that?
3. Write a shell script to read your first and last name to an array
– Add your salutation and suffix to the array
– Drop either the salutation or suffix
– Print the array after each of the three steps above
4. Write a shell script to calculate the factorial and double factorial of an
integer or list of integers

2/11/2015 HPC training series Spring 2015 79