Chapter 2-3

Graphics and Image Data Representations

1
 3.1 Graphics and Image Data Types
• The number of file formats used in multimedia continues to proliferate. For
example, Table 3.1 shows a list of some file formats used in the popular product
Adobe Premiere.

2
 1-Bit Images

• Each pixel is stored as a single bit (0 or 1), so also referred to as binary image.
• Such an image is also called a 1-bit monochrome image since it contains no color.
• The figure shows a 1-bit monochrome image (used multimedia scientists — this is
a standard image used to illustrate many algorithms).

3
 8-bit Gray-level Images
• Each pixel has a gray-value between 0 and 255.
• Each pixel is represented by a single byte; e.g., a dark pixel might have a value of 10,
and a bright one might be 230.
• Bitmap: The two-dimensional array of pixel values that represents the graphics/image
data.
• Image resolution refers to the number of pixels in a digital image (higher resolution
always yields better quality).
- Fairly high resolution for such an image might be 1600 x 1200, whereas lower resolution might be
640 x 480.

4
• Frame buffer: Hardware used to store bitmap.
- Video card (actually a graphics card) is used for this purpose.
- The resolution of the video card does not have to match the desired resolution of
the image, but if not enough video card memory is available then the data has to be
shifted around in RAM for display.
• An 8-bit image can be thought of as a set of 1-bit bit-planes, where each
plane consists of a 1-bit contribution to the image : a bit is turned on if the
image pixel value has a nonzero value for that bit. E.g., if a pixel stores 1
byte, then all the pixels with byte value > 127 have the most significant bit
turned on.
• Fig. 3.2 displays the concept of bit-planes graphically.
5
Fig. 3.2: Bit-planes for 8-bit grayscale image.

6
• Each pixel is usually stored as a byte (a value between 0 to 255), so a 640 x 480
grayscale image requires 300 kB of storage (640 x 480 = 307,200).
• Fig. 3.3 shows the Lena image again, but this time in grayscale.

7
 Image Data Types
• The most common data types for graphics and image file formats — 24-bit color and
8-bit color.
• Some formats are restricted to particular hardware / operating system platforms,
while others are “cross-platform” formats.
• Even if some formats are not cross-platform, there are conversion applications that
will recognize and translate formats from one system to another.
• Most image formats incorporate some variation of a compression technique due to
the large storage size of image files.
• Compression techniques can be classified into either lossless or lossy.

8
 24-bit Color Images
• In a color 24-bit image, each pixel is represented by three bytes, usually
representing RGB.
- This format supports 256 x 256 x 256 possible combined colors, or a total of 16,777,216 possible
colors.
- However such flexibility does result in a storage penalty: A 640 x 480 24-bit color image would
require 921.6 kB of storage without any compression.

• An important point: many 24-bit color images are actually stored as 32-bit images,
with the extra byte of data for each pixel used to store an alpha value representing
special effect information (e.g., transparency).
• Fig. 3.5 shows the image forestfire.bmp: a 24-bit image in Microsoft Windows BMP
format, and the grayscale images for the Red, Green, and Blue channels for this
image.
9
 (c)

Fig. 3.5: High-resolution color and separate R, G, B color channel images. (a): Example of
24-bit color image “forestfire.bmp”. (b, c, d): R, G, and B color channels for this image.
10
 Higher Bit-depth Images
More information about the scene being imaged can be gained by using more accuracy
for pixel depth (64 bits, say); or by using special cameras that view more than just three
colors (i.e., RGB).
• Could use invisible light (e.g., infra-red, ultraviolet) for security cameras: “dark
flash”.
• Use higher-dimensional medical images of skin (> 3-D) to diagnose skin
carcinoma.
• In satellite imaging, use high-D to obtain types of crop growth, etc.
Such images are called multispectral (more than 3 colors) or hyperspectral (a great
many image planes, say 224 colors for satellite imaging).
11
 8-Bit Color Images

• Many systems can make use of 8 bits of color information (the so-called “256
colors”) in producing a screen image.
• Such image files use the concept of a lookup table to store color information.
- Basically, the image stores not color, but instead just a set of bytes, each of which
is actually an index into a table with 3-byte values that specify the color for a
pixel with that lookup table index.

12
• Fig. 3.7 shows the resulting 8-bit image, in GIF format.

Fig. 3.7: Example of 8-bit color image.

• Note the great savings in space for 8-bit images, over 24-bit ones: a 640 x 480 8-bit
color image only requires 300 kB of storage, compared to 921.6 kB for a color image
(again, without any compression applied).
13
 Color Look-up Tables (LUTs)
• The idea used in 8-bit color images is to store only the index, or code value, for each
pixel. Then, e.g., if a pixel stores the value 25, the meaning is to go to row 25 in a
color look-up table (LUT).

14
• A Color-picker consists of an array of fairly large blocks of color (or a semi-continuous
range of colors) such that a mouse-click will select the color indicated.
- In reality, a color-picker displays the palette colors associated with index values
from 0 to 255.
- Fig. 3.9 displays the concept of a color-picker: if the user selects the color block
with index value 2, then the color meant is cyan, with RGB values (0, 255, 255).

15
 Popular File Formats

• 8-bit GIF: one of the most popular formats because of its historical connection to
the WWW and HTML markup language as the first image type recognized by net
browsers.
• JPEG: currently the most important common file format.
• PNG: most popular lossless image format.
• TIFF: flexible file format due to the addition of tags.
• BMP: standard image file format for Windows.
• PS and PDF: vector based language, popular in publishing and academia
• Many others
16
 GIF
• GIF standard: (We examine GIF standard because it is so simple, yet contains many common
elements.)
Limited to 8-bit (256) color images only, which, while producing acceptable color images, is
best suited for images with few distinctive colors (e.g., graphics or drawing).
• GIF standard supports interlacing — successive display of pixels in widely-spaced rows by a
4-pass display process.

• GIF actually comes in two flavors:

1.GIF87a: The original specification.

2.GIF89a: The later version. Supports simple animation via a Graphics Control Extension block in the data,
provides simple control over delay time, a transparency index, etc.

17
 GIF87
• For the standard specification, the general file format of a GIF87 file is as in Fig. 3.12.

Fig. 3.12: GIF file format.

18
• Screen Descriptor comprises a set of attributes that belong to every image in the
file. According to the GIF87 standard, it is defined as in Fig. 3.13.

Fig. 3.13: GIF screen descriptor

19
• Color Map is set up in a very simple fashion as in Fig. 3.14. However, the actual length of
the table equals 2(pixel+1) as given in the Screen Descriptor.

Fig. 3.14: GIF color map.

20
• Each image in the file has its own Image Descriptor, defined as in Fig. 3.15.

Fig. 3.15: GIF image descriptor.

21
• If the “interlace” bit is set in the local Image Descriptor, then the rows of the image are
displayed in a four-pass sequence (Fig.3.16).

Fig. 3.16: GIF 4-pass interlace display row order.

22
• We can investigate how the file header works in practice by having a look at a particular
GIF image. Fig. 3.7 on page is an 8-bit color GIF image, in UNIX, issue the command:
od -c forestfire.gif | head -2

and we see the first 32 bytes interpreted as characters:

G I F 8 7 a \208 \2 \188 \1 \247 \0 \0 \6 \3 \5
J \132 \24 | ) \7 \198 \195 \ \128 U \27 \196 \166 & T

• To decipher the remainder of the file header (after “GIF87a”), we use hexadecimal:
od -x forestfire.gif | head -2

with the result

4749 4638 3761 d002 bc01 f700 0006 0305 ae84 187c 2907 c6c3 5c80
551b c4a6 2654

23
 JPEG
• JPEG: The most important current standard for image compression.
• The human vision system has some specific limitations and JPEG takes advantage of
these to achieve high rates of compression.
• JPEG allows the user to set a desired level of quality, or compression ratio (input
divided by output).
• As an example, Fig. 3.17 shows our forestfire image, with a quality factor Q=10.
- This image is a mere 1.5% of the original size.
- In comparison, a JPEG image with Q=75 yields an image size 5.6% of the original,
whereas a GIF version of this image compresses down to 23.0% of uncompressed
image size.
24
Fig. 3.17: JPEG image with low quality specified by user.

25
 PNG
• PNG: stands for Portable Network Graphics — meant to supersede the GIF standard, and
extends it in important ways.

• Special features of PNG files include:

1. Support for up to 48 bits of color information — a large increase.

2. Files may contain gamma-correction information for correct display of color images, as
well as alpha-channel information for such uses as control of transparency.

3. The display progressively displays pixels in a 2-dimensional fashion by showing a few

pixels at a time over seven passes through each 8 x 8 block of an image.

26
 TIFF
• TIFF: stands for Tagged Image File Format.

• The support for attachment of additional information (referred to as “tags”) provides a

great deal of flexibility.

1. The most important tag is a format signifier: what type of compression etc. is in use in
the stored image.
2. TIFF can store many different types of image: 1-bit, grayscale, 8-bit color, 24-bit RGB,
etc.
3. TIFF was originally a lossless format but now a JPEG tag allows one to opt for JPEG
compression.

27
 Windows BMP

• BitMap (BMP), aka. Device Independent Bitmap (DIB) is a major system standard
graphics file format for Microsoft Windows, recognized by many programs.
• It uses raster graphics.
• BMP supports many pixel formats, including indexed color (up to 8 bits per pixel), and
16, 24, and 32-bit color images.
• There are many sub-variants within the BMP standard.

28
 PS and PDF

• Postscript is an important language for typesetting, and many high-end printers have a
Postscript interpreter built into them.
• Postscript is a vector-based picture language, rather than pixel-based: page element
definitions are essentially in terms of vectors.

1. Postscript includes text as well as vector/structured graphics.

2. Bit-mapped images can be included in output files.

3. Encapsulated Postscript files (.EPS) add some additional information for inclusion of
Postscript files in another document.

29
 4. Postscript page description language itself does not provide compression; in fact,
Postscript files are just stored as ASCII.

• Another text + figures language has superseded or at least paralleled Postscript: Adobe
Systems Inc. includes LZW compression in its Portable Document Format (PDF) file format.

- PDF files that do not include images have about the same compression ratio, 2:1 or 3:1,
as do files compressed with other LZW-based compression tools.

30
 Multimedia Tasks and Concerns

• More basic concerns will impact multimedia as it now appears in products e.g., we
expect impact from Computer Vision: a camera owner might be encouraged to
think like a computer scientist and ask:
- “What is going on in an image?”
- A less high-level question is “Where has this image been taken?” (scene recognition)
- “Does the image contain a particular object?” (object classification)
- “Where is an object of interest?” (object detection)
- “Which object does each pixel belong to?” (image segmentation)
== a classic Computer Vision hierarchy of high-level to detailed description of an
image, with scene recognition at the top and image segmentation at the bottom.

31
 Data Compression
One of the most evident and2.3 Datachallenges
important Compression
of using multimedia is the necessity to
compress data. The following table shows some values for standard definition and for
highdefinition broadcast video.

32
In the JPEG image compression standard, the amount of compression is controlled by a
value qf (Quality Factor) in the range 1–100.

Fig. 2.9: JPEG compression: (a) original uncompressed image; (b)

JPEG compression with Quality Factor qf = 75 (the typical default).
33
Fig. 2.9: JPEG compression: (c) Quality Factor qf = 25 and (d)
Quality Factor qf = 5.

34
35
• How expensive image and video processing is in terms of CPU cycles?
• Suppose we have an image whose pixels we wish to darken, by a factor of 2:

On a RISC machine, the loop amounts to one increment, one check, and one branch instruction. There
are also three loads, three shifts, and three stores. This makes a total of 12 instructions per pixel, i.e., per
3 bytes. So, we have four instructions per image byte. For standard-definition video, we have 28 MB/s,
meaning 28 × 4 =112 mega instructions per second. For high definition, at 187 MB/s, we need 748 mega
instructions per second.
36