UNIX System Programming

Introduction

1
 User UNIX Interface: SHELL
 Provides command line as an interface between the user
and the system
 Is simply a program that starts automatically when you
login
 Uses a command language
– Allows programming (shell scripting) within the shell
environment
 Uses variables, loops, conditionals, etc.
– Accepts commands and often makes system calls to carry them
out

2
 Various UNIX shells
 sh (Bourne shell)
 ksh (Korn shell)
 csh (C shell)
 tcsh
 bash
 Differences mostly in scripting details

3
 The Korn Shell (ksh)
 We will be using ksh as the standard shell
for examples in this class
 Language is a superset of the Bourne shell
(sh)

4
 Login scripts
 You don’t want to enter aliases, set environment variables,
set up command line editing, etc. each time you log in
 All of these things can be done in a script that is run each
time the shell is started
 For ksh:
– ~/.profile - is read for a login shell
– ~/.kshrc
 For tcsh
– ~/.login
– ~/.cshrc

5
Example .profile (partial)
# set ENV to a file invoked each time sh is started for
interactive use.
ENV=$HOME/.shrc; export ENV
HOSTNAME=`hostname`; export HOSTNAME
PS1="$USER@$HOSTNAME>"

alias 'll'='ls -l'

alias 'la'='ls -la'
alias 'ls'='ls -F'
alias 'rm'='rm -i'
alias 'm'='more'

set -o vi
echo ".profile was read"

6
 stdin, stdout, and stderr
 Each shell (and in fact all programs) automatically
open three “files” when they start up
– Standard input (stdin): Usually from the keyboard
– Standard output (stdout): Usually to the terminal
– Standard error (stderr): Usually to the terminal
 Programs use these three files when reading (e.g.
cin), writing (e.g. cout), or reporting
errors/diagnostics

7
 Redirecting stdout
 Instead of writing to the terminal, you can tell
a program to print its output to another file
using the > operator
 >> operator is used to append to a file
 Examples:
– man ls > ls_help.txt
– Echo $PWD > current_directory
– cat file1 >> file2

8
 Redirecting stderr
 Instead of writing errors to the terminal, you can
tell a program to write them to another file using
the:
– ksh: 2> operator
– tcsh: >& operator
 Examples (suppose j is a file that does not exist)
{ajax} ls j
ls: j: No such file or directory
{ajax} ls j >& hello.txt
{ajax} cat hello.txt
ls: j: No such file or directory 9
 Redirecting stdin
 Instead of reading from the terminal, you
can tell a program to read from another file
using the < operator
 Examples:
– Mail [email protected] < message
– interactive_program <
command_list

10
 Pipes and filters
 Pipe: a way to send the output of one command
to the input of another
 Filter: a program that takes input and transforms
it in some way
– wc - gives a count of words/lines/chars
– grep - searches for lines with a given string
– more
– sort - sorts lines alphabetically or numerically

11
 Examples of filtering
 ls -la | more
 cat file | wc
 man ksh | grep “history”
 ls -l | grep “dkl” | wc
 who | sort > current_users

12
 UNIX Tutorial
 http://www.ee.surrey.ac.uk/Teaching/Unix/
 Google will give you many links

13
 UNIX Filesystem
 The filesystem is your interface to
– physical storage (disks) on your machine
– storage on other machines
– output devices
– etc.
 Everything in UNIX is a file (programs, text,
peripheral devices, terminals, …)
 There are no drive letters in UNIX! The filesystem
provides a logical view of the storage devices

14
 Working directory
 The current directory in which you are
working
 pwd command: outputs the absolute path
(more on this later) of your working
directory
 Unless you specify another directory,
commands will assume you want to operate
on the working directory

15
 Home directory
A special place for each user to store personal
files
 When you log in, your working directory will
be set to your home directory
 Your home directory is represented by the
symbol ~ (tilde)
 The home directory of “user1” is represented
by ~user1

16
UNIX file hierarchy
/

 Directories may bin users tmp

contain plain files
or other dkl kangli
directories
 Leads to a tree
foo.txt csci1730
structure for the
filesystem bar.c abcd
 Root directory: /
17
 Path names /

 Separate directories by / bin users tmp

 Absolute path
– start at root and follow the dkl kangli
tree
– e.g. /users/dkl/foo.txt
foo.txt csci1730
 Relative path
 start at working directory bar.c abcd
 .. refers to level above; . refers to working dir.
 If /users/dkl/csci1730 is working dir, all these
refer to the same file
../foo.txt ~/foo.txt ~dkl/foo.txt
18
 Types of files
 Plain (- in the first bit)
– Most files
– Includes binary and text files
 Directory (d)
– A directory is actually a file
– Points to another set of files
 Link (l): A pointer to another file or directory
 Special: e.g. peripheral devices

19
 Creating links
 ln –s <curr_file> <link_name>
 This command creates a symbolic link
 The file “link_name” will be a pointer to the
“curr_file” which may be in another directory
or even on another physical machine

20
 File permissions
 Permissions used to allow/disallow access
to file/directory contents
 Read (r) 4, write (w) 2, and execute (x) 1
 For owner, group, and world (everyone)
 chmod <mode> <file(s)>
– chmod 700 file.txt (only owner
can read, write, and execute)
– chmod g+rw file.txt

21
 Looking at file contents
 cat <filename(s)>
– “concatenate”
– output the contents of the file all at once
 more <filename(s)>
– Output the contents of a file one screen at a time
– Allows forward and backward scroll and search

22
 Getting help on UNIX
commands
 These notes only give you the tip of the
iceberg for these basic commands
 man <command_name> shows you all
the documentation for a command
 apropos <keyword> shows you all the
commands with the keyword in their
description

23
 The UNIX System
 Kernel – Heart of the OS
– Process scheduling
– I/O control (accesses)
 Shell– Interpreter between the user and the
computer
 Tools and applications
– Accessible from shell
– Can be run independently of shell
24
UNIX System Programming
 Programs make system calls (also called supervisor calls
to invoke kernel.
 A system call is essentially a procedure call into the
operating system
– The procedure call is protected
 Types of system calls
– File I/O
– Process management
– Inter-process communication (IPC)
– Signal handling

25
System Calls (Library calls)
 System calls
– Interface to the kernel
Program
Code
Library fread

User Space read user

read
kernel
Kernel Space
26
 Basic file I/O
 Processes keep a list of open files
 Files can be opened for reading, writing
 Each file is referenced by a file descriptor
(integer)
 Three files are opened automatically
– FD 0: standard input
– FD 1: standard output
– FD 2: standard error

27
 File I/O system call: open()
 fd = open(path, flags, mode)
 path: string, absolute or relative path
 flags:
– O_RDONLY - open for reading
– O_WRONLY - open for writing
– O_RDWR - open for reading and writing
– O_CREAT - create the file if it doesn’t exist
– O_TRUNC - truncate the file if it exists
– O_APPEND - only write at the end of the file
 mode: specify permissions if using O_CREAT

28
 File I/O system call: close()
 retval = close(fd)
 Close an open file descriptor
 Returns 0 on success, -1 on error

29
 File I/O system call: read()
 bytes_read = read(fd, buffer,
count)
 Read up to count bytes from file and place into
buffer
 fd: file descriptor
 buffer: pointer to array
 count: number of bytes to read
 Returns number of bytes read or -1 if error

30
File I/O system call: write()
 bytes_written = write(fd,
buffer, count)
 Write count bytes from buffer to a file
 fd: file descriptor
 buffer: pointer to array
 count: number of bytes to write
 Returns number of bytes written or -1 if error

31
 System call: lseek()
 retval = lseek(fd, offset, whence)
 Move file pointer to new location
 fd: file descriptor
 offset: number of bytes
 whence:
– SEEK_SET - offset from beginning of file
– SEEK_CUR - offset from current offset location
– SEEK_END - offset from end of file
 Returns offset from beginning of file or -1

32
 UNIX File access primitives
 open – open for reading, or writing or create an empty file
 creat - create an empty file
 close –
 read - get info from file
 write - put info in file
 lseek - move to specific byte in file
 unlink - remove a file
 remove - remove a file
 fcntl - control attributes assoc. w/ file

33
 File I/O using FILEs
 Most UNIX programs use higher-level I/O functions
– fopen()
– fclose()
– fread()
– fwrite()
– fseek()
 These use the FILE datatype instead of file
descriptors
 Need to include stdio.h

34
 Using datatypes with file I/O
 All the functions we’ve seen so far use raw
bytes for file I/O, but program data is
usually stored in meaningful datatypes (int,
char, float, etc.)
 fprintf(), fputs(), fputc() -
used to write data to a file
 fscanf(), fgets(), fgetc() -
used to read data from a file

35