Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

UNIX and C Programming (COMP1000)

Lecture 6: Input and Output

Updated: 19
th August, 2018

Department of Computing

Curtin University

Copyright © 2018, Curtin University

CRICOS Provide Code: 00301J

1/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Textbook Reading (Hanly and Koman)

For more information, see the weekly reading list on Blackboard.

I Chapter 11: Text and Binary File Processing

Consider reading Sections 10.1 and 10.2 (in Chapter 10)
beforehand. These introduce structs, which are used in some
of the examples in Chapter 11.

2/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Outline

Intro to I/O

File I/O in C

Errors

Reading/Writing

Binary les

Standard Streams

3/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Introduction to I/O

I Reading from and writing to les is a crucial part of any

programming language.

I Virtually all programs need to do it (in the real world).

I Files provide permanent storage  persistence  unlike

variables in memory.

4/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Opening and Closing Files

I Before reading/writing a le, the le must be opened.

I After reading/writing, the le must be closed.

I If you're reading from a le, the le must exist.

I If you're writing to a le, you can choose whether:
I the le will be created or overwritten, OR
I the le will be appended to.

5/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Streams

I Most input and output uses streams.

I Characters go in one end and out the other  a queue.

I When you read a le:
I Each character in the le is fed into the stream.
I Your program reads and removes characters from the stream.
I Often your program must wait for characters to become

available.

I When you write to a le:

I Your program feeds characters into the stream.
I Characters are progressively removed from the stream and

written to disk.

I A stream is created when you open a le.

6/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Streams  Visualisation

Input stream Output stream

le1.txt Program le2.txt

I Here, program reads from file1.txt and writes to

file2.txt
I However, you can have as many streams as you like!

I Characters ow through each input stream to the program.

I More characters ow through each output stream from the

program.

7/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Buering (1)

I File I/O is slow, compared to memory-based operations.

I Usually, characters to be read/written are stored temporarily in

buers (inside a stream).

I This allows characters to be read/written in large chunks 

more ecient than one-at-a-time.

I Buering often happens transparently, in the middle of a

stream.

8/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Buering (2)

Input Buering

I If you read one character from a le, the OS actually feeds a

much larger chunk of the le into the stream.

I When you want the next character, it's already waiting in the
buer.

Output Buering

I If you write one character to a le, the OS delays the actual

write until enough characters accumulate, or the le is closed.

I If you don't close a le, you'll lose everything still in the

output buer.

9/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Flushing Output Buers

I Output buers can be ushed.

I This (also) happens automatically when a le is closed.

I Flushing an output buer writes the output to disk

immediately.

10/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Seeking

I Normally a le is read/written sequentially.

I At any given moment, the stream points to a particular
location in the le.
I The rst character on the rst line is 0.

I Each subsequent character is one greater than the previous.

I This is the location where characters will next be read or

written.

I Jumping to another location (either forwards or backwards) is

called seeking.

I You must know the exact byte location.

I You cannot jump straight to a particular line number.

11/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Man Pages (Manual Pages)

I You're about to be assaulted with many new C functions.

I The UNIX man utility can display documentation (as

mentioned before).

I This command will display the man page for the strlen()
function:

[user@pc]$ man 3 strlen

I Standard C functions are located in section 3 of the manual.

I You can omit the section number:

[user@pc]$ man strlen

(. . . but you may get a page from the wrong section!)

12/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Man Page Example

STRLEN(3) Linux Programmer's Manual STRLEN(3)

NAME strlen - calculate the length of a string

SYNOPSIS #include <string.h>

size_t strlen(const char *s);

DESCRIPTION
The strlen() function calculates the length of
the string s, not including the terminating '\0'
character.

RETURN VALUE
The strlen() function returns the number of
characters in s.
13/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Man Page Information

As shown on the previous slide, each man page lists:

I NAME  The function name and purpose.

I SYNOPSIS 
I The header le you must #include.
I The function prototype/declaration.

I DESCRIPTION  A description of the function.

I RETURN VALUE  A description of the return value.
More information is also listed, including notes, bugs and related
functions ( SEE ALSO).

14/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Introduction to File I/O in C

I Uses the stdio.h library.

I File I/O is closely related to terminal I/O  printf() and

scanf().
I Make good use of man pages!

15/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

FILE Pointers

I When you open a le in C you get a  FILE pointer (FILE*).

I The FILE* is used to access the stream.

I When you read/write a open le (stream), you must supply

the FILE*.

Note

I You never need to deal with the FILE type itself, only FILE*.
I You just pass the pointer around to various C functions.

16/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

FILE Pointers  Concept and Reality

Reality

I A FILE* points to a FILE (of course)  a chunk of

information in memory used to access a stream.

I A FILE* is really just a normal pointer.

Concept

I You can think of a FILE* as a pointer to a place in a le.

I When you read/write, the pointer automatically moves

forward.

17/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

FILE Pointers  Visualisation

Memory Disk

FILE*

Conceptually,
points here.

Actual
le
Stream Technically
contents
information points here.

18/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

FILE Pointers  Visualisation

Memory Disk

FILE*

Conceptually,
points here. Actual
le
Stream Technically
contents
information points here.

18/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

FILE Pointers  Visualisation

Memory Disk

FILE*

Conceptually, Actual
points here. le
Stream Technically
contents
information points here.

18/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

FILE Pointers  Visualisation

Memory Disk

FILE*

Actual
Conceptually, le
Stream Technically
points here. contents
information points here.

18/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

FILE Pointers  Visualisation

Memory Disk

FILE*

Actual
le
Stream Technically
Conceptually, contents
information points here.
points here.

18/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Opening and Closing Files in C

The fopen() function

I Opens a le.

I Takes two char* (string) parameters  a lename and a

mode string.

I Returns FILE*, or NULL if the le couldn't be opened.

The fclose() function

I Closes a le.

I Takes a FILE* parameter.

I Returns an int indicating whether an error occurred.

19/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Opening and Closing Files  Example

#include <stdio.h>

...

FILE* f;
f = fopen("filename.txt", "r");

... /* Read from the file */

fclose(f);

(Be careful  we haven't checked for errors here!)

20/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

File Modes

I The second parameter to fopen() is the mode string.

I Indicates what you want to do with the le.

I Always a string, even if only 1 character long.

Basic modes
"r" read from a le (starting at the beginning)
"w" write to a le (create or overwrite the le)
"a" write to a le (starting at the end  appending)

21/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Update Modes

I Adding a  + to the mode allows both reading and writing.

I This makes the stream bi-directional.

I You can switch between reading and writing (but you can't do
both simultaneously).

I Going from reading to writing, you must perform a seek.

I Going from writing to reading, you must perform a seek or

ush.

Update modes
"r+" read & write (starting at the beginning)
"w+" read & write (create or overwrite the le)
"a+" read & write (starting at the end)

22/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Text and Binary Modes

I Microsoft Windows distinguishes between text and binary

modes.

I In binary mode, the le is read/written as-is.

I In text mode, Windows ddles with some special characters.

I Text mode is the default, but will corrupt binary les

(under Windows)!
I To select binary mode, place b in the mode string (e.g.
"rb", "wb").
I UNIX (e.g. Linux and OS X) does not need this  the b
ag is accepted but ignored.

23/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Errors

I File I/O is error-prone.

I Possible errors include:
I Trying to read a le that doesn't exist.
I Trying to write to a le when the disk is full.
I Trying to read/write to a le when you don't have permission.

I Disk hardware errors.

I I/O errors are not necessarily your fault (though they might
be).

24/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Error Checking and Handling

I Your program should detect errors and handle them gracefully.

I Newer languages (like Java) deal with I/O errors by throwing

exceptions, which can be caught and handled.

I C leaves it up to you to check when an error occurs.

I Output helpful error messages.

25/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Error Checking: Opening Files

I Recall that fopen() returns NULL if a le can't be opened.

I You should check for this (with an if statement):

FILE* f = fopen("file.txt", "r");

if(f == NULL) {
printf("Error: could not open 'file.txt'\n");
}
else {
... /* Read from the file */

fclose(f);
}

26/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Error Handling  perror()

I Don't leave the user to gure out what went wrong!
I The perror() function determines what the error was and
prints out a relevant message.
I Takes a char*  a prex to the error message.
I Returns void, and prints out the parameter plus the error.

FILE* f = fopen("file.txt", "r");

if(f == NULL) {
perror("Error opening 'file.txt'");
}
else { ... /* Read and close the file */ }

Possible output (say file.txt does not exist):

Error opening 'file.txt': No such file or directory

27/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

More Error Checking  ferror()

I The ferror() function checks whether an error has occurred.
I Takes a FILE* parameter (must be non-NULL).

I Returns an int  zero (no error) or non-zero (error).

FILE* f = fopen("file.txt", "r");

if(f == NULL) { ... }
else {
... /* Read from the file */

if(ferror(f)) {
perror("Error reading from 'file.txt'\n");
}
fclose(f);
}

28/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Reading and Writing

I Reading usually uses mode "r".

I Writing usually uses mode "w".
I Every read or write will move the FILE pointer forward.
I However, there are dierent functions to read/write data,
based on the type of data:
I Strings with embedded values.

I Individual characters.
I Lines of text.
I Binary data.

I You may also need dierent algorithms, based on the le

format.

I You may (or may not!) know how many data elements are
stored in the le.

29/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Checking for the End of File (EOF)

I How big is the le? You don't always know.

I (You can nd the number of bytes, but each data element may

occupy an unknown number of bytes.)

I The read functions  fscanf, fgets, fgetc  will all tell you
when the le has ended.
I That's the trick: you won't know until after you try to read
past the end of le.
I The last read operation always fails (unless you know in

advance how many reads you can do).

30/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Basic File Reading Algorithm

int done = FALSE;

do {
Attempt a read operation
if(the read succeeded)
Store the data (e.g. in an array)
else
done = TRUE;
}
while(!done);

if(ferror(the file pointer))

Handle error and de-allocate any stored data
else
Success

31/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

fprintf() and fscanf()

I Like printf() and scanf(), but used with les.

I Both take an extra FILE* parameter (rst).

fprintf() example

FILE* f = fopen("output.txt", "w");

int number;
...
fprintf(f, "The number is %d\n", number);

32/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

fscanf() continued
I fscanf() returns either:
I the number of items successfully read, OR

I the preprocessor constant EOF (end-of-le), but only if it

doesn't read anything rst.
I Compare the return value to the number you expected:

FILE* f = fopen("input.txt", "r");

int x, y, z, nRead;
... 3 items

nRead = fscanf(f, "%d %d %d", &x, &y, &z);

if(nRead != 3)
/* An error or end of file occurred. */

Note on EOF
I EOF is a preprocessor constant, guaranteed to be some
negative integer.

33/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

fputc() and fgetc()

I fputc() writes one character to an output stream.

I fgetc() reads one character from an input stream.

fputc() example

FILE* f = fopen("output.txt", "w");

char ch;
...
fputc(ch, f);

34/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

fgetc() continued

I fgetc() returns an int (not achar)  why?

I Because, if it fails, it returns EOF.

I EOF is a negative integer, not representable as a char value.

FILE* f = fopen("input.txt", "r");

int ch;
...
ch = fgetc(f);
if(ch == EOF)
/* Error or end-of-file. */
else
/* Success -- typecast ch to a char. */

35/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Writing a line of text  fputs()

I fputs() writes a string, plus a new line ('\n').

I Takes two parameters: char* and FILE*.

Example

FILE* f = fopen("output.txt", "w");

char* str = "Hello world";
...
fputs(str, f);

(Here, the FILE* parameter comes last.)

36/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Reading a line of text  fgets()

I fgets() reads a string, taking 3 parameters:
I char*  an array to store the text.
I int  the size of the array.
I FILE*  an input stream.
I Stops at the next newline, or the size of the array minus 1

(whichever comes rst).

#define INPUT_SIZE 21
...
FILE* f = fopen("input.txt", "r");
char str[INPUT_SIZE];
...
if(fgets(str, INPUT_SIZE, f) == NULL)
/* Error or end-of-file. */
else
/* Success -- you can safely use str. */
37/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Other Reading/Writing Functions

These functions are all essentially redundant:

getc() and putc()

Reads/writes a character (like fputc() but with possible
side-eects).

getchar() and putchar()

Reads/writes a character from/to the terminal.

gets() and puts()

Reads/writes a line of text from/to the terminal.

Warning

gets() should never be used, because there's no way to prevent

buer overows. Usefgets() instead.

38/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Multiple Files Example

This copies input.txt to output.txt, inserting dashes:

FILE *inFile = fopen("input.txt", "r");

FILE *outFile = fopen("output.txt", "w");
int ch;
... /* Error checking */
do {
ch = fgetc(inFile);
if(ch != EOF) {
fputc((char)ch, outFile);
fputc('-', outFile);
}
} while(ch != EOF);
... /* More error checking */
fclose(inFile);
fclose(outFile);
39/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Flushing  fflush()

I An output stream can be ushed with fflush().

I This may be useful in programs that run for a very long time:
I number crunching
I event logging
I databases

I Flushing an output stream can help prevent data loss (if the
program, OS or computer crashes).

40/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Binary les

I Many les are not human-readable  these are often called

binary les. . .
I . . . even though all data in computing is binary!

I All les are made up of bytes  8-bit integers.

I In a text le, bytes represent characters  printable symbols.

Characters in turn form words, numbers, etc.

I In a binary le, bytes directly represent integers, real numbers,

etc. They do not (generally) represent characters.

41/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Binary le formats

I There are no words, lines or paragraphs in a binary le.

I Each data item occupies a xed number of bytes.

I ints, doubles, etc. are stored as they would be in memory.
I A 32-bit int always occupies 32 bits, i.e. 4 bytes (remember
sizeof?).
I Compare this to text les, where a 32-bit int could be 1 byte
(e.g. 2) or 10 bytes (e.g. 2000000000).

I Due to xed sizes, there are no spaces or other delimiters

around data items  no need to separate them.

I This makes it impossible to distinguish between them, unless

you know the precise format in advance.

42/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Reading and Writing Binary Data

I The fread() and fwrite() functions deal with binary data.

I (Under Windows, you'll need the "rb" or "wb" modes.)

I When writing to a binary le, no conversion is done.
I Before writing the int 123 to a text le, it must be converted
to the string "123" (i.e. a sequence of digit characters).

43/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Writing binary data  fwrite()

I Writes an array of data (of any type).
I Takes four parameters:
I void*  a pointer to the data to write.
I int  the size of each data element to write.
I int  the total number of elements to write.
I FILE*  an output stream.

#define LENGTH 5
...
int data[LENGTH] = {10, 20, 30, 40, 50};
FILE* f = fopen("output.txt", "wb");
...
fwrite(data, sizeof(int), LENGTH, f);

(Note:  sizeof(int) gives you the number of bytes in an int.)

44/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Reading binary data  fread()

I Reads an array of data (of any type).
I Takes the same four parameters as fwrite():
I void*  a pointer to an array to read into.
I int  the size of each data element to read.
I int  the maximum number of elements to read.
I FILE*  an input stream.
I Returns the number of elements actually read (like fscanf).

#define MAXLEN 5
...
int data[MAXLEN], length;
FILE* f = fopen("input.txt", "rb");
...
length = fread(data, sizeof(int), MAXLEN, f);
if(length < MAXLEN) ... /* EOF, error or success? */

45/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Seeking  fseek()
I Can position the FILE pointer anywhere in the le.

I Mostly useful with binary les  you know where something

ought to be.
I Can be used in three ways:
I Move the le pointer f to 15 bytes after the start of the le:

fseek(f, 15, SEEK_SET);

I Moves the le pointer f 15 bytes back from its current
location:

fseek(f, -15, SEEK_CUR);

I Moves the le pointer f to 15 bytes before the end of the le:

fseek(f, -15, SEEK_END);

46/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Seeking  rewind() and ftell()

rewind()
I Resets the FILE pointer to the start of the le.

I This may be useful when you need to read a le multiple times.

I rewind(f) is equivalent to fseek(f, 0, SEEK_SET).

ftell()
I Reports the current location within a le.

I Takes a FILE*.
I Returns the location as a long (indicating the number of bytes
from the start of the le).

47/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Standard Streams

I Not all streams are connected to a le

I In C, there are three special, pre-dened FILE pointers:
I stdin (standard input)  reads from the terminal
I stdout (standard output)  writes to the terminal
I stderr (standard error)  also writes to the terminal
I These do not need to be opened or closed.

I fprintf(stdout, ...) is equivalent to printf(...)

I fscanf(stdin, ...) is equivalent to scanf(...)

48/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Standard Streams  Visualisation

The Terminal

Keyboard input Screen output

Input stream Output stream

stdin Program stdout

I To the program, stdin and stdout look like ordinary les. . .

I . . . except they're actually connected to the terminal, not to

the disk.

49/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Redirection

I The UNIX shell (sh, csh, bash, etc.) can redirect the
standard streams  most commonly stdin and stdout:
I Say you normally run  program with parameters  ...:

[user@pc]$ ./program ...

I Standard output can be redirected to a le, instead of the
terminal:

[user@pc]$ ./program ... >output.txt

I Standard input can be redirected so that it comes from a le:

[user@pc]$ ./program ... <input.txt

I You can also do both at once:

[user@pc]$ ./program ... <input.txt >output.txt

50/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Redirecting Stdin  Visualisation

The Terminal

Input stream

stdin Program

I This is standard input, unredirected (i.e. as normal)

51/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Redirecting Stdin  Visualisation

The Terminal

Input stream

stdin Program

input.txt

I Stdin has been redirected from the terminal to input.txt

I The program doesn't know that stdin has changed
51/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Piping

I Piping is a special form of redirection.

I The UNIX shell can pipe the output of one program to the
input of another.

I On the command-line:

[user@pc]$ ./program1 ... | ./program2 ...

(The  | symbol is called the pipe character.)

I This will cause both program1 and program2 to run

simultaneously

I Whatever program1 outputs (e.g. with printf()), program2

can read in (e.g. with scanf())
I Most UNIX commands are designed with this in mind.

52/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Piping  Visualisation

The Terminal

stdin stdout

program1 program2

stdout stdin

The pipe

I Program1 sends data to program2 via the pipe

I Neither program (necessarily) realises this is happening

53/54
Intro to I/O File I/O in C Errors Reading/Writing Binary les Standard Streams

Coming Up

I The next lecture will look at structs and linked lists.

54/54