Simple Shell Scripting

for Scientists

Appendix
Bruce Beckles
University of Cambridge Computing Service

1
 Common Unix commands

The following slides provide a

summary of the Unix commands
used in the “Simple Shell
Scripting for Scientists” course.

[email protected] Simple Shell Scripting for Scientists: Appendix 2

For details of the “Unix: Simple Shell Scripting for Scientists” course,
see:
http://www.cam.ac.uk/cs/courses/coursedesc/scicomp.html#scriptsci

2
 Appendix: Unix commands (1)
basename return the filename from a file
path, removing the given
ending (if specified)
> basename /usr/bin/python
python
> basename ~/hello.sh .sh
hello

dirname return the directory name

from a file path
> dirname /usr/bin/python
/usr/bin
[email protected] Simple Shell Scripting for Scientists: Appendix 3

If you have a path to a file, dirname will give you just the directory, removing
the actual filename whilst basename will give you the filename, removing the
directory path. basename can also remove the endings of filenames.

If you need to do more advanced filename (or file) manipulation, then you
should look at the find and xargs commands, which are covered in the “Unix
Systems: Further Commands” course, the notes for which are available here:
http://www-uxsup.csx.cam.ac.uk/courses/Commands/

The find command searches for files in a directory tree, and having found the
specified files, can run a command on each file.

The xargs command builds a command line from a combination of values

read from standard input and arguments specified on the command line, and
then executes that command line a certain number of times.

3
 Appendix: Unix commands (2)
cat Display contents of a file
> cat /etc/motd
Welcome to PWF Linux 2008/2009.

If you have any problems, please email [email protected].

cd change directory
> cd /tmp
> cd

chmod change the mode (permissions) of

a file or directory
> chmod a+r treasure.txt
[email protected] Simple Shell Scripting for Scientists: Appendix 4

If you give the cd command without specifying a directory then it will

change the directory to your home directory (the location of this
directory is specified in the HOME environment variable).

The chmod command changes the permissions of a file or directory

(in this context, the jargon word for “permissions” is “mode”). For
instance, the above example gives read access to the file
treasure.txt for all users on the system. Unix permissions were
covered in the “Unix: Introduction” course, see:
http://www.cam.ac.uk/cs/courses/coursedesc/unix.html#unix

4
 Appendix: Unix commands (3)
cp copy files and/or directories
> cp /etc/motd /tmp/motd-copy
Options:
-p preserve (where possible) files’ owner,
permissions & date
-f if unable to overwrite destination file,
delete it and try again, i.e. forcibly overwrite
destination files
-r copy any directories recursively, i.e. copy their
contents
-i prompt before overwriting anything (be
interactive – ask the user)
> cp –p /etc/motd /tmp/motd-copy
[email protected] Simple Shell Scripting for Scientists: Appendix 5

Note that the cp command has many other options than the four
listed above, but those are the options that will be most useful to
us in this course.

5
 Appendix: Unix commands (4)
date display/set system date and time
> date
Thu Aug 7 11:52:03 GMT 2008

echo display text

> echo "Hello"
Hello

env With no arguments, display

environment variables

[email protected] Simple Shell Scripting for Scientists: Appendix 6

Please note that if you try out the date command, you will get a
different date and time to that shown on this slide (unless your
computer’s clock is wrong or you have fallen into a worm-hole in
the space-time continuum). Also, note that usually only the system
administrator can use date to set the system date and time.

Note that the echo command has a few useful options, but we
won’t be making use of them today, so they aren’t listed.

Note also that the env command is a very powerful command, but
we will not have occasion to use for anything other than displaying
environment variables, so we don’t discuss its other uses.

6
 Appendix: Unix commands (5)
grep find lines in a file that match a given
pattern
> grep 'PWF' /etc/motd
Welcome to PWF Linux 2008/2009.
ln create a link between files (almost
always used with the -s option for
creating symbolic links)
> ln –s /etc/motd /tmp/motd
> cat /etc/motd
Welcome to PWF Linux 2008/2009.

If you have any problems, please email [email protected].

> cat /tmp/motd

Welcome to PWF Linux 2008/2009.

If you have any problems, please email [email protected].

[email protected] Simple Shell Scripting for Scientists: Appendix 7

The patterns that the grep command uses to find text in files are called regular
expressions. We won’t be covering these in this course, but if you are interested,
or if you need to find particular pieces of text amongst a collection of text, then
you may wish to attend the CS “Pattern Matching Using Regular Expressions”
course, details of which are given here:
http://www.cam.ac.uk/cs/courses/coursedesc/scicomp.html#regex

The ln command creates links between files. In the example above, we create a
symbolic link to the file motd in /etc and then use cat to display both the
original file and the symbolic link we’ve created. We see that they are identical.
There are two sort of links: symbolic links (also called soft links or symlinks) and
hard links. A symbolic link is similar to a shortcut in the Microsoft Windows
operating system (if you are familiar with those) – essentially, a symbolic link
points to another file elsewhere on the system. When you try and access the
contents of a symbolic link, you actually get the contents of the file to which that
symbolic link points. Whereas a symbolic link points to another file on the
system, a hard link points to actual data held on the filesystem. These days
almost no one uses ln to create hard links, and on many systems this can only
be done by the system administrator. If you want a more detailed explanation of
symbolic links and hard links, see the following Wikipedia articles:
http://en.wikipedia.org/wiki/Symbolic_link
http://en.wikipedia.org/wiki/Hard_link

7
 Appendix: Unix commands (6)
ls list the contents of a directory
> ls
bin examples gnuplot hello.sh iterator scripts source treasure.txt

Options:
-d List directory name instead of its
contents
-l use a long listing that gives lots of
information about each directory entry
-R list subdirectories Recursively, i.e. list
their contents and the contents of any
subdirectories within them, etc

[email protected] Simple Shell Scripting for Scientists: Appendix 8

If you try out the ls command, please note that its output may not exactly
match what is shown on this slide – in particular, the colours may be slightly
different shades and there may be additional files and/or directories shown.
Note also that the ls command has many, many more options than the three
given on this slide, but these three are the options that will be of most use to us
in this course.

8
 Appendix: Unix commands (7)
less Display a file one screenful of text at a
time
more Display a file one screenful of text at a
time
> more treasure.txt
The Project Gutenberg EBook of Treasure Island, by Robert Louis Stevenson

This eBook is for the use of anyone anywhere at no cost and with
almost no restrictions whatsoever. You may copy it, give it away or
re-use it under the terms of the Project Gutenberg License included
with this eBook or online at www.gutenberg.org

Title: Treasure Island

Author: Robert Louis Stevenson

Release Date: February 25, 2006 [EBook #120]

Language: English

Character set encoding: ASCII

*** START OF THIS PROJECT GUTENBERG EBOOK TREASURE ISLAND ***

--More--(0%)

[email protected] Simple Shell Scripting for Scientists: Appendix 9

(Note that the output of the more command may not exactly match
that shown on this slide – in particular, the number of lines
displayed before the “--More--(0%)” message depends on the
number of lines it takes to fill up the window in which you are
running the more command.)

The more and less commands basically do the same thing: display
a file one screenful of text at a time. Indeed, on some Linux
systems the more command is actually just another name (an alias)
for the less command.
Why are there two commands that do the same thing? On the
original Unix systems, the less command didn’t exist – the
command to display a file one screenful of text at a time was more.
However, the original more command was somewhat limited, so
someone wrote a better version and called it less. These days the
more command is a bit more sophisticated, although the less
command is still much more powerful.
For everyday usage though, many users find the two commands are
equivalent. Use whichever one you feel most comfortable with, but
remember that every Unix/Linux system should have the more
command, whereas some (especially older Unix systems) may not
have the less command.

9
 Appendix: Unix commands (8)
mkdir make directories
> mkdir /tmp/mydir
Options:
-p make any parent directories as required;
also if directory already exists, don’t
consider this an error
> mkdir /tmp/mydir
mkdir: cannot create directory `/tmp/mydir': File exists

> mkdir –p /tmp/mydir

[email protected] Simple Shell Scripting for Scientists: Appendix 10

Note that the mkdir command has other options, but we won’t be
using them in this course.

10
 Appendix: Unix commands (9)
mktemp safely makes temporary files or
directories for you
> mktemp
/tmp/tmp.fmsAr17215
Options:
-d make a directory instead of a file (by
default mktemp creates files)
-t make file or directory in a temporary
directory (usually /tmp)
> mktemp -t -d iterator.XXXXXXXXXX
/tmp/iterator.khhcE30735
[email protected] Simple Shell Scripting for Scientists: Appendix 11

The mktemp command is an extremely useful command that allows users

to safely create temporary files or directories on multi-user systems. It is
very easy to unsafely create a temporary file or directory to work with
from a shell script, and, indeed, if your shell script tries to create its own
temporary files or directories using the normal Unix commands then it is
almost certainly doing so unsafely. Use the mktemp command instead.

Note that if you try the examples above you will almost certainly get files
and directories with different names created for you.

Note also that mktemp has more options than the two listed above, but
we won’t be using them in this course. Note also that if you use a
version of mktemp earlier than version 1.3 (or a version derived from BSD,
such as that shipped with MacOS X) then you can’t use the -t option, and
will have to specify /tmp (or another temporary directory) explicitly, e.g.
mktemp -d /tmp/iterator.XXXXXXXXXX

How do you use mktemp? You give it a “template” which consists of a

name with some number of X’s appended to it (note that is an UPPER
CASE letter X), e.g. iterator.XXXXX. mktemp then replaces the X’s with
random letters and numbers to make the name unique and creates the
requested file or directory. It outputs the name of the file or directory it
has created.

11
 Appendix: Unix commands (10)
mv move or rename files and directories
> mv /tmp/motd-copy /tmp/junk

Options:
-f do not prompt before overwriting files or
directories, i.e. forcibly move or rename the file
or directory; this is the default behaviour
-i prompt before overwriting files or directories
(be interactive – ask the user)
-v show what is being done (be verbose)

[email protected] Simple Shell Scripting for Scientists: Appendix 12

Note that the mv command has other options, but we won’t be

using them in this course. Note also that if you move a file or
directory between different filesystems, mv actually copies the file
or directory to the other filesystem and then deletes the original.

12
 Appendix: Unix commands (11)
pwd print full path of current working
directory
> cd /tmp
> pwd
/tmp
Options:
-P print the full Physical path of the current
working directory (i.e. the path printed
will not contain any symbolic links)

[email protected] Simple Shell Scripting for Scientists: Appendix 13

Note that the pwd command has another option, but we won’t be
using it in this course.

13
 Appendix: Unix commands (12)
rm remove files or directories
> rm /tmp/junk
Options:
-f ignore non-existent files and do not ever
prompt before removing files or directories, i.e.
forcibly remove the file or directory
-i prompt before removing files or directories
(be interactive – ask the user)
--preserve-root do not act recursively on /
-r remove subdirectories (if any) recursively, i.e.
remove subdirectories and their contents
-v show what is being done (be verbose)
[email protected] Simple Shell Scripting for Scientists: Appendix 14

Note that the rm command has other options, but we won’t be

using them in this course.

14
 Appendix: Unix commands (13)
rmdir remove empty directories
> rmdir /tmp/mydir

touch change the timestamp of a file;

if the file doesn’t exist create it
with the specified timestamp
(the default timestamp is the
current date and time)
> touch /tmp/nowfile

[email protected] Simple Shell Scripting for Scientists: Appendix 15

The rmdir and touch commands have various options but we

won’t be using them on this course. If you try out the touch
command with the example above, check that it has really worked
the way we’ve described here by using the ls command as follows:
ls -l /tmp/nowfile
You should see that the file nowfile has a timestamp of the
current time and date.

15