Chapter 1

Introduction to Interactive Computer Graphics

Introduction
Computer graphics provides methods to generate images using a computer. The word “image”
should be understood in a more abstract sense here. An image can represent a realistic scene from the
real world, but graphics like histograms (statics diagram) or pie charts as well as the graphical user
interface of a software tool are also considered as images. The development of computer graphics
has made computers easier to interact with, and better for understanding and interpreting many types
of data. Developments in computer graphics have had a profound impact on many types of media
and have revolutionized animation, movies and the video game industry.

The term computer graphics has been used in a broad sense to describe "almost everything on
computers that is not text or sound". Typically, the term computer graphics refers to several different
things:

 the representation and manipulation of image data by a computer

 the various technologies used to create and manipulate images
 the images so produced, and
 the sub-field of computer science which studies methods for digitally synthesizing and
manipulating visual content

The aim of computer graphics is to produce realistic and/or useful images on a computer. For some
applications the emphasis will be on realism (e.g. special effects in films), whereas for others it will
simply be on usefulness (e.g. data visualisation)

Today, computers and computer-generated images touch many aspects of daily life. Computer
imagery is found on television, in newspapers, for example in weather reports, or for example in all
kinds of medical investigation and surgical procedures. A well-constructed graph can present
complex statistics in a form that is easier to understand and interpret. In the media "such graphs are
used to illustrate papers, reports, thesis", and other presentation material.

Many powerful tools have been developed to visualize data. Computer generated imagery can be
categorized into several different types: 2D, 3D, 4D, 7D, and animated graphics. As technology has
improved, 3D computer graphics have become more common, but 2D computer graphics are still
widely used. Computer graphics has emerged as a sub-field of computer science which studies
methods for digitally synthesizing and manipulating visual content. Over the past decade, other
specialized fields have been developed like information visualization, and scientific visualization
more concerned with "the visualization of three dimensional phenomena (architectural,
meteorological, medical, biological, etc.), where the emphasis is on realistic renderings of volumes,
surfaces, illumination sources, and so forth, perhaps with a dynamic (time) component".

1|Page
Whatever the application, the final image will be generated from some model of the scene we want to
take a picture of. For example, in Figure.1 we have a representation of a 3-D tetrahedron (the object
model), together with some model of how the scene is lit (the light sources), and we want to produce
an image on the image plane based on the position of the viewer. The object model will typically
represent the geometry of the object together with some information about what type of material it is
made of (i.e. what it looks like). Normally we consider the viewer to be a virtual camera, which
defines how the picture is taken. This process of starting with a model (often a 3-D model) of the
world and taking a picture of it with a virtual camera is known in computer graphics as rendering.
The rendering process can include many stages, such as hidden surface elimination, surface
rendering and clipping, which we will deal with throughout this course.

Figure 1.1: Rendering a 2-D Image from a 3-D Model

Some graphics applications can be described as interactive graphics applications. In these cases,
images need to be generated in real-time and as well as viewing the images the user can interact with
the model of the world using specialised input hardware. For example, a mouse, keyboard, tablet and
stylus, scanner or a virtual reality headset and gloves can all be used as interactive graphics input
devices.

1.1. Brief History of Computer Graphics

1.1.1. The Age of Sutherland

In the early 1960's IBM, Sperry-Rand, Burroughs and a few other computer companies existed. The
computers of the day had a few kilobytes of memory, no operating systems to speak of and no
graphical display monitors. The peripherals were Hollerith punch cards, line printers, and roll-paper
plotters. The only programming languages supported were assembler, FORTRAN, and Algol.
Function graphs and “Snoopy'' calendars were about the only graphics done.
In 1963 Ivan Sutherland presented his paper Sketchpad at the Summer Joint Computer Conference.
Sketchpad allowed interactive design on a vector graphics display monitor with a light pen input
device. Most people mark this event as the origins of computer graphics.

1.1.2. The Middle to Late '60's

Software and Algorithms

Jack Bresenham taught us how to draw lines on a raster device. He later extended this to circles. Anti-
aliased lines and curve drawing is a major topic in computer graphics. Larry Roberts pointed out the
usefulness of homogeneous coordinates, matrices and hidden line detection algorithms. Steve Coons

2|Page
introduced parametric surfaces and develop early computer aided geometric design concepts. The
earlier work of Pierre Bézier on parametric curves and surfaces also became public. Author Appel at
IBM developed hidden surface and shadow algorithms that were pre-cursors to ray tracing. The fast
Fourier transform was discovered by Cooley and Tukey. This algorithm allows us to better
understand signals and is fundamental for developing antialiasing techniques. It is also a precursor to
wavelets.

Hardware and Technology

Doug Englebart invented the mouse at Xerox PARC. The Evans & Sutherland Corporation and
General Electric started building flight simulators with real-time raster graphics. The floppy disk was
invented at IBM and the microprocessor was invented at Intel. The concept of a research network, the
ARPANET, was developed.
1.1.3. The Early '70's
The state of the art in computing was an IBM 360 computer with about 64 KB of memory, a
Tektronix 4014 storage tube, or a vector display with a light pen (but these were very expensive).

Software and Algorithms

Rendering (shading) was discovered by Gouraud and Phong at the University of Utah. Phong also
introduced a reflection model that included specular highlights. Keyframe based animation for 3-D
graphics was demonstrated. Xerox PARC developed a ``paint'' program. Ed Catmull introduced
parametric patch rendering, the z-buffer algorithm, and texture mapping. BASIC, C, and UNIX were
developed atDartmouth and Bell Labs.

Hardware and Technology

An Evans & Sutherland Picture System was the high-end graphics computer. It was a vector display
with hardware support for clipping and perspective. Xerox PARC introduced the Altos personal
computer, and an 8 bit computer was invented at Intel.
1.1.4. The Middle to Late '70's Software and Algorithms
Turned Whitted developed recursive ray tracing and it became the standard for photorealism, living
in a pristine world. Pascal was the programming language everyone learned.

Hardware and Technology

The Apple I and II computers became the first commercial successes for personal computing. The
DEC VAX computer was the mainframe (mini) computer of choice. Arcade games such as Pong and
Pac Mac became popular. Laser printers were invented at Xerox PARC.
1.1.5. The Early '80's

Hardware and Technology

The IBM PC was marketed in 1981 The Apple MacIntosh started production in 1984, and
microprocessors began to take off, with the Intel x86 chipset, but these were still toys. Computers
with a mouse, bitmapped (raster) display, and Ethernet became the standard in academic and science
and engineering settings.
1.1.6. The Middle to Late '80'sSoftware and Algorithms
Jim Blinn introduces blobby models and texture mapping concepts. Binary space partitioning (BSP)
trees were introduced as a data structure, but not many realized how useful they would become.
Loren Carpenter starting exploring fractals in computer graphics. Postscript was developed by
John Warnock and Adobe was formed. Steve Cook introduced stochastic sampling to ray tracing.
Paul Heckbert taught us to ray trace Jello (this is a joke ;) Character animation became the goal for
animators. Radiosity was introduced by the Greenberg and folks at Cornell. Photoshop was marketed
by Adobe. Video arcade games took off, many people/organizations started publishing on the
desktop. UNIX and X windows were the platforms of choice with programming in C and C++, but

3|Page
MS-DOS was starting to rise.

Hardware and Technology

Sun workstations, with the Motorola 680x0 chipset became popular as advanced workstation a in the
mid 80's. The Video Graphics Array (VGA) card was invented at IBM. Silicon Graphics (SGI)
workstations that supported real-time raster line drawing and later polygons became the computer
graphicists desired. The data glove, a precursor to virtual reality, was invented at NASA. VLSI for
special purpose graphics processors and parallel processing became hot research areas.
1.1.7. The Early '90's
The computer to have now was an SGI workstation with at least 16 MB of memory, at 24-bit raster
display with hardware support for Gouraud shading and z-buffering for hidden surface removal.
Laser printers and single frame video recorders were standard. UNIX, X and Silicon Graphics GL
were the operating systems, window system and application programming interface (API) that
graphicist used. Shaded raster graphics were starting to be introduced in motion pictures. PCs started
to get decent, but still they could not support 3-D graphics, so most programmer's wrote software for
scan conversion (rasterization) used the painter's algorithm for hidden surface removal, and
developed “tricks”' for real-time animation.

Software and Algorithms

Mosaic, the first graphical Internet browser was writtenat the University of Illinois, National Center
for Scientific Applications (NCSA). MPEG standards for compressed video began to be promulgated.
Dynamical systems (physically based modeling) that allowed animation with collisions, gravity,
friction, and cause and effects were introduced. In 1992 OpenGL became the standard for graphics
APIs In 1993; the World Wide Web took off. Surface subdivision algorithms were rediscovered.
Wavelets begin to be used in computer graphics.

Hardware and Technology

Hand-held computers were invented at Hewlett-Packard about 1991. Zip drives were invented at
Iomega. The Intel 486 chipset allowed PC to get reasonable floating point performance. In 1994,
Silicon Graphics produced the Reality Engine: It had hardware for real-time texture mapping. The
Ninetendo 64 game console hit the market providing Reality Engine-like graphics for the masses of
games players. Scanners were introduced.
1.1.8. The Middle to Late '90's
The PC market erupts and supercomputers begin to wane. Microsoft grows, Apple collapses, but
begins to come back, SGI collapses, andlots of new startups enter the graphics field.

Software and Algorithms

Image based rendering became the area for research in photo-realistic graphics. Linux and open
source software become popular.

Hardware and Technology

PC graphics cards, for example 3dfx and Nvidia, were introduced. Laptops were introduced to the
market. The Pentium chipset makes PCs almost as powerful as workstations. Motion capture, begun
with the data glove, becomes a primary method for generating animation sequences. 3-D video games
become very popular: DOOM (which uses BSP trees), Quake, Mario Brothers, etc. Graphics effects
in movies become pervasive: Terminator 2, Jurassic Park, Toy Story, Titanic, Star Wars I. Virtual
reality and the Virtual Reality Meta (Markup) Language (VRML) become hot areas for research.
PDA's, the Palm Pilot, and flat panel displays hit the market.
1.1.9. The '00's
Today most graphicist want an Intel PC with at least 256 MB of memory and a 10 GB hard drive.
Their display should have graphics board that supports real-time texture mapping. A flatbed

4|Page
 scanner, color laser printer, digital video camera, DVD, and MPEG encoder/decoder are the
peripherals one wants. The environment for program development is most likely Windows and
Linux, with Direct 3D and OpenGL, but Java 3D might become more important. Programs would
typically be written in C++ or Java.
What will happen in the near future -- difficult to say, but high definition TV (HDTV) is poised to
take off (after years of hype). Ubiquitous, untethered, wireless computing should become
widespread, and audio and gestural input devices should replace some of the functionality of the
keyboard and mouse.
You should expect 3-D modeling and video editing for the masses, computer vision for robotic
devices and capture facial expressions, and realistic rendering of difficult things like a human face,
hair, and water.
1.2. 3D Graphics Techniques and Terminology
 Rendering
The process by which mathematical and image data is transformed into a 3D dimensional image is
called rendering. When used as a verb, it is the process that your computer goes through to create the
three dimensional image. Rendering is also used as a noun, simply to refer to the final image
produced.
 Transformations and Projections
By transforming, or moving the points around, and drawing lines between them we can produce the
illusion of a 3D world on a flat 2D screen. The earliest flight simulators employed technology no
more sophisticated than this
By transforming, or moving the points around, and drawing lines between them we can produce the
illusion of a 3D world on a flat 2D screen. The earliest flight simulators employed technology no
more sophisticated than this.

FIGURE 1.2: A simple wireframe cube and table

The points themselves are called vertices (or vertex in the singular), and they are moved around in
space with a convenient mathematical construct called a transformation matrix. Another matrix, a
projection matrix takes care of the mathematics necessary to turn our 3D coordinates into two-
dimensional screen coordinates, where the final line drawing actually takes place.
 Rasterization
The actual drawing, or filling in of the pixels between each vertex to make the lines is called
rasterization. We can further clarify our 3D intent with transformed and rasterized lines by
employing hidden surface removal.

5|Page
 FIGURE 1.3: Hiding the back sides of solid geometry enhances the 3D illusion.
Although drawing with lines, or wireframe rendering as it is often called, has its uses, most of the
time we render not with lines, but with solid triangles. Triangles and polygons are also rasterized, or
filled in just like lines are.

FIGURE 1.4: Filling in geometry with solid colors is hardly effective.

 Shading
By varying the color values across the surface (between vertices), we can easily create the effect of a
light shining on a red cube.

FIGURE 1.5: Shading the surface creates the illusion of light.

6|Page
  Texture Mapping
The next hardware advance was texture mapping. A texture is simply a picture that we map
to the surface of a triangle or polygon. Textures add a whole new level of realism to our rendering.

FIGURE 1.6: A single texture is worth a thousand triangles.

Textures are fast and efficient on modern hardware, and a single texture can reproduce a
surface that might take thousands or even millions of triangles to represent otherwise.
 Blending
Blending allows us to mix different colors together. This reflection effect is done simply by drawing
the cube upside down first. Then we draw the floor blended over the top of it, followed by the right
side up cube. You really are seeing “through” the floor to the inverted cube below. Your brain just
says, “Oh… a reflection.” Blending is also how we make things look transparent.

FIGURE 1.7: Using blending to create a reflection effect.

1.3. Application Area and Common Uses of Computer Graphics
Before we look in any more detail at how computers can generate graphical images, let us consider
why we would want to do this. Since the early days of computing, the field of computer graphics has

7|Page
become a very popular one because of its wide range of applications. The following sections
summarise the main categories of application.

1.1. Graphs and Charts

Graphs and charts have long been used for visualising data. They are particularly widely used for
visualising relationships and trends in scientific, mathematical, financial and economic data.
Although, in principle, we do not need advanced graphics algorithms to display graphs and charts,
many modern visualisation packages use 3-D effects such as shadowing to make the graph/chart
more visually appealing. For example, Figure 1.8 shows a range of graphs and charts produced by
the free FusionCharts Lite package (http://www.infosoftglobal.com/FusionCharts/Lite). Commercial
packages are also available and include Microsoft Excel.

(a) (b)

(c) (d)

Figure 1.8 - Computer Graphics used for Graphs and Charts

1.2. Computer-Aided Design

Computer-Aided Design (CAD) and Computer-Aided Drafting and Design (CADD) involve using a
computer application to enable designers to construct and visualise 3-D models. They are commonly
used in fields such as architecture, engineering and circuit design. For example, Figure 1.9 shows
two screenshots from CAD applications being used for architecture. One of the most common CAD

8|Page
applications is AutoCAD.
Computer-Aided Manufacturing (CAM) is a similar concept to CAD, except that the design
application is linked to the manufacturing process, i.e. the application will directly control, via a
hardware communication link, the machine that manufactures the object being designed.

(a) (b)

Figure 1.9 - Computer Graphics used for Computer-Aided Design (CAD)

1.3. Data Visualisation

We already saw in Section 1.1 how graphics can be used for visualising data in the form of graphs
and charts. More complex, often multidimensional, datasets can also be visualised using graphics
techniques. In this case, it is often difficult to visualise such datasets without computer graphics.
These complex datasets are particularly common in science (scientific visualisation) and business
(business visualisation).
For example, Figure (a) shows a three-dimensional medical dataset of a patient. Here we use
computer graphics to visualise the paths of blood vessels inside the patient‟s head. Figure 1.10 (b)
shows an image generated from data acquired by the Hubble Space Telescope. This image of a
distant galaxy is allowing us to visualise a four-dimensional dataset – three dimensions are combined
as the red, green and blue components of an optical image, and the fourth (an X-ray image) is
displayed as overlaid contour plots. Figure 1.10 (c) shows another medical dataset, this time acquired
using a Magnetic Resonance Imaging (MRI) scanner. Computer graphics allow us to visualise this
three-dimensional dataset as three slices through the 3-D volume. Finally, Figure 1.10 (d) shows the
use of colour to visualise altitude in a map of the United States.

9|Page
 (a) (b)

(c) (d)

Figure 1.10 - Computer Graphics used for Visualisation of Complex Datasets

1.4. Virtual Reality and Augmented Reality

Virtual Reality (VR) allows users to be immersed in a computer generated world, and to interact with
it as if they were interacting with the real world. Typically, the user will wear special hardware such
as the VR-headset shown in Figure 1.11(a). This allows the computer to completely control the
visual information received by the user, so if realistic computer-generated images such as that shown
in Figure 1.11(b) are displayed in the headset in stereo (i.e. different images for the left and right
eyes) the user will have a sense of physical immersion in the virtual world. In addition, other
specialised hardware such as VR-gloves can be used to interact with the „objects‟ in the virtual
world.
An alternative way of experiencing virtual reality is by using a CAVE (which stands for Cave
Automatic Virtual Environment). An example of this is shown in Figure 1.11(c). Here, the user
stands inside a cubicle which has images projected onto the walls and ceiling. Often there will also
be a means of „moving‟ in the virtual world such as pressure pads.

10 | P a g e
Augmented Reality (AR) combines a computer-generated virtual world with the real world. In AR,
computer-generated images are overlaid onto a user‟s view of the real world. For example, Figure
1.11(d) shows an AR system used for surgery, in which computer-generated images of hidden
features such as blood vessels and tumours are overlaid on the surgeon‟s view of the patient through
a surgical microscope.

(a) (b)

(c) (d)

Figure 1.11 - Computer Graphics used for Virtual Reality and Augmented Reality

1.5. Education and Training

Computer graphics can be very beneficial in education and training. For example, Figure 1.12(a)
shows a screenshot from some educational software that teaches students about human anatomy.
Graphics are used to help students visualise the appearance and location of different organs inside
the body.
Another common application of computer graphics is in training. In many jobs it is difficult for
workers to get direct experience of the environment that they are expected to work in. This may be
because the environment is inaccessible (e.g. space), dangerous (e.g. bomb disposal), expensive (e.g.

11 | P a g e
flying aircraft) or high-risk (e.g. surgery). Computer graphics, combined with specialised hardware,
can be used to simulate the working environment, so that workers can gain the skills required for
their work. For example, Figure 1.12(b) shows a picture of a truck-driving simulator used by the
German army to train their drivers in emergency situations; Figure 1.12(c) shows a helicopter
simulator; and Figure 1.12(d) shows a screenshot from a simulator for laser eye surgery.

(a) (b)

(c) (d)
Figure 1.12 - Computer Graphics used for Education and Training

1.6. Graphical User Interfaces

Another possibility that has been opened up by the improvements in computer graphics technology is
in the area of user interfaces. Figure 1.13 shows the difference in visual appeal between an old text-
based user interface and a modern windows-based user interface. And the difference is not just in
visual appeal: modern user interfaces have made multitasking (doing several things at once) much
easier and applications such as word-processors have become much more powerful.

(a) (b)
Figure 1.13 - Text-Based and Graphical User Interfaces

12 | P a g e
1.7. Entertainment

The applications of computer graphics in entertainment fall into three categories: computer art,
special effects and animations, and games.
A number of artists have taken advantage of the possibilities offered by computer graphics in
producing works of art. For example, Figure 1.14(a) shows a frame from an animated artwork by
William Latham.
Computer-generated images (CGI) have been widely used in the entertainment industry for
producing special effects for films, and also for producing completely computer-generated films.
Figure 1.14(b) shows a frame from the film Final Fantasy: the Spirits Within, which featured 100%
computer-generated images. For applications such as this, highly realistic images can be produced
off-line, i.e. we do not need to generate the images in real-time, they need only be generated once,
frame-by-frame, and then combined into the desired animated sequence.
For computer games, images must be generated in real-time in response to user actions. Therefore
the graphics in games can be less realistic, since the emphasis is on speed of generation rather than
realism. Figure 1.14(c) and (d) show screenshots from the popular computer game Doom.

(a) (b)

(c) (d)
Figure 1.14 - Computer Graphics used in the Entertainment Industry

13 | P a g e