Video
Rawesak Tanawongsuwan
[email protected]
7 191–192
Types of video signals
• Component video
• Composite video
• S-video
7
Component video
• Higher-end video systems make use of three separate
video signals for the red, green, and blue image
planes. Each color channel is sent as a separate video
signal
• Most computer systems use Component Video, with
separate signals for R, G, and B signals
• For any color separation scheme, Component Video
gives the best color reproduction since there is no
“crosstalk” between the three channels
• Component video, however, requires more bandwidth
and good synchronization of the three components
7
Composite video
• Color (chrominance) and intensity (luminance)
signals are mixed into a single carrier wave
• Chrominance is a composition of two color
components (I and Q, or U and V)
• The chrominance and luminance components can
be separated at the receiver end and then the
two color components can be further received
• Composite video uses only one wire and video
color signals are mixed, not sent separately
• Since color and intensity are wrapped into the same
signal, some interference between the luminance
and chrominance signals is inevitable
7
S-video
• S-video uses two wires, one for luminance and
another for a composite chrominance signals
• There is less crosstalk between the color information
and the crucial gray-scale information
• The reason for placing luminance into its own part of
the signal is that black-and-white information is most
crucial for visual perception
• Humans are able to differentiate spatial resolution
in gray-scale images with a much higher acuity
than for the color part of the color images
• We can then send less accurate color information
than for intensity information
7
Analog Video
• An analog signal f(t) samples a time-varying image. So
called “progressive” scanning traces through a complete
picture (a frame) row-wise for each time interval
• In TV, and in some monitors and multimedia standards as
well, another system, called “interlaced” scanning is used
• The odd-numbered lines are traced rst, and then the
even-numbered lines are traced. This results in “odd” and
“even” fields two fields make up one frame
• In fact, the odd lines (starting from 1) end up at the
middle of a line at the end of the odd field, and the even
scan starts at a half-way point.
7
Interlaced raster scan
• First the solid (odd) lines are
traced, P to Q, then R to S, etc.,
ending at T; then the even field
starts at U and ends at V
• The jump from Q to R is called the
horizontal retrace, during which
the electronic beam in the CRT is
blanked. The jump from T to U or
V to P is called the vertical
retrace
7
Interlaced raster scan
• Interlaced scan produces two fields for each frame. (a) The
video frame, (b) Field 1, (c) Field 2, (d) Difference of fields
7
Comb-like effect
• Because of interlacing, the odd and even lines are
displaced in time from each other (can be noticed when
very fast action is taking place)
7
De-interlace
• Since it is sometimes necessary to change the frame rate,
resize, or even produce stills from an interlaced source
video, various schemes are used to “de-interlace” it
• The simplest de-interlacing method consists of discarding
one field and duplicating the scan lines of the other field.
The information in one field is lost completely using this
simple technique
• Other more complicated methods that retain information
from both fields are also possible
7
NTSC Video
• NTSC (National Television System Committee) TV
standard is mostly used in North America and Japan. It
uses the familiar 4:3 aspect ratio (i.e., the ratio of
picture width to its height) and uses 525 scan lines per
frame at 30 frames per second (fps).
• NTSC follows the interlaced scanning system, and each
frame is divided into two fields, with 262.5 lines/field.
• NTSC uses the YIQ color model
7
Video raster, including
retrace and sync data
• The effect of “vertical retrace & sync” and “horizontal retrace &
sync” on the NTSC video raster
• Vertical retrace takes place during 20 lines reserved for control
information at the beginning of each field. Hence, the number
of active video lines per frame is only 485
• It is known that pixels often fall in-between the scan lines.
Therefore, even with non-interlaced scan, NTSC TV is only
capable of showing about 340 (visually distinct) lines, i.e.,
about 70% of the 485 specied active lines. With interlaced
scan, this could be as low as 50%
7
Samples per line
• NTSC video is an analog signal with no fixed horizontal
resolution. Therefore one must decide how many times to
sample the signal for display: each sample corresponds to
one pixel output
• Different video formats provide different numbers of
samples per line
7
PAL Video
• PAL (Phase Alternating Line) is a TV standard
widely used in Western Europe, China, India, and
many other parts of the world.
• PAL uses 625 scan lines per frame, at 25
frames/second, with a 4:3 aspect ratio and interlaced
fields.
• PAL uses the YUV color model.
7
SECAM Video
• SECAM stands for Systeme Electronique Couleur Avec
Memoire, the third major broadcast TV standard
• SECAM also uses 625 scan lines per frame, at 25 frames per
second, with a 4:3 aspect ratio and interlaced fields.
• SECAM and PAL are very similar. They dffier slightly in their
color coding scheme
• In SECAM, U and V signals are modified and sent differently
7
Digital Video
• The advantages of digital representation for video are
many. For example:
• (a) Video can be stored on digital devices or in
memory, ready to be processed (noise removal, cut and
paste, etc.), and integrated to various multimedia
applications;
• (b) Direct access is possible, which makes
nonlinear video editing achievable as a simple, rather than
a complex, task;
• (c) Repeated recording does not degrade image
quality;
• (d) Ease of encryption and better tolerance to
channel noise
7
Digitizing video
• A video sequence consists of a number of frames, each of
which is a single image produced by digitizing the time-varying
signal generated by the sensors in a video camera
• For example, the size of the image produced for each frame of
NTSC video is 640x480
PAL uncompressed NTSC uncompressed
768x576 pixels per frame 640x480 pixels per frame
x 3 bytes per pixel (24 bit x 3 bytes per pixel (24 bit
color) color)
x 25 frames per second x 30 frames per second
(approx)
≈ 31 MB per second ≈ 26 MB per second
≈ 1.85 GB per minute ≈ 1.6 GB per minute
7
Digitization
• In the camera – DV + Firewire
• In the computer – video capture card
• Digitization in camera (DV) means less noise
• Less noise allows better compression
7 197
Streamed Video
• Play back a video stream as it arrives over a
network (like broadcast TV), instead of
downloading an entire video clip and playing it from
disk (like renting a DVD)
7 198
HTTP Streaming
• Start playing a downloaded clip as soon as enough
of it has arrived
• Starts when the (estimated) time to download the
rest is equal to the duration of the clip
7
Digital video standards
• Digital video devices must conform to standards
• Digital standards must maintain compatibility with
older analog standards for broadcast TV
• Digital video standard is called Rec. ITU-R BT.601
• 720 luminance samples (Y), 2x360 color difference
samples (B−Y and R−Y) per line
• PAL 720x576 pixels; NTSC 720x480 pixels
• Pixels are not square
7
Chroma Subsampling
• Since humans see color with much less spatial resolution
than they see black and white, it makes sense to
“decimate” the chrominance signal
• Interesting (but not necessarily informative!) names have
arisen to label the different schemes used
• To begin with, numbers are given stating how many pixel
values, per four original pixels, are actually sent
• The chroma subsampling scheme 4:4:4 indicates that no
chroma subsampling is used: each pixel’s Y, Cb and Cr
values are transmitted, 4 for each of Y, Cb, Cr
7
Chroma Subsampling
• The scheme 4:2:2 indicates horizontal subsampling of the Cb,
Cr signals by a factor of 2. Four pixels horizontally labelled as
0 to 3, all four Ys are sent, and every two Cb’s and two Cr’s
are sent, as (Cb0,Y0), (Cr0,Y1), (Cb2,Y2), (Cr2,Y3), (Cb4,Y4),
and so on (or averaging is used)
• The scheme 4:1:1 subsamples horizontally by a factor of 4
• The scheme 4:2:0 subsamples in both the horizontal and
vertical dimensions by a factor of 2. Theoretically, an average
chroma pixel is positioned between the rows and columns
• Scheme 4:2:0 along with other schemes is commonly used in
JPEG and MPEG
7
Chroma Subsampling
7 203
4:2:2 sub-sampling
• Used in CCIR 601
• Twice as many Y
samples as each of the
colour difference
samples
• Co-sited: same pixel
is used for all three
samples
• Reduces data rate to
just over 20MB per
second
7 204
DV
• Consumer format, also known as mini-DV
• DVCAM, DVPRO use different tape formats, but
generate the same data stream
• 4:1:1 chrominance sub-sampling
• Data rate constant 25Mbits per second
• Compression ratio 5:1
7 210
DV sub-sampling
• PAL DV 4:2:0
chrominance sub-
sampling
7 210
DV sub-sampling
• NTSC DV 4:1:1
chrominance sub-
sampling
7 204–206
MPEG
• ISO/IEC Motion Picture Experts Group
• Series of standards including
• MPEG-1 intended for video CD
• MPEG-2 used in DVD and broadcast
• MPEG-4 for low bitrate multimedia
7 205
MPEG-2 Profiles & Levels
• MPEG-2 Main Profile at Main Level (MP@ML)
used for DVD video
• CCIR 601 scanning
• 4:2:0 chrominance sub-sampling
• 15 Mbits per second
• Most elaborate representation of MPEG-2
compressed data
7 205–206
MPEG-4
• Designed to support a range of multimedia
data at bit rates from 10kbps to >1.8Mbps
• Applications from mobile phones to HDTV
• Video codec becoming popular for Internet use,
is incorporated in QuickTime, RealMedia and
DivX
7 216–219
Older Codecs
• Cinepak – Longest established, high compression
ratio, takes much longer to compress than to
decompress
• Intel Indeo – Similar to Cinepak, but roughly 30%
faster compression
• Sorenson – More recent, higher quality and better
compression ratios than other two
• All three based on vector quantization
• Quality of all three inferior to MPEG-4
7 223–230
Editing
• Making a constructed whole from a collection of
parts
• Selection, trimming and organization of raw
footage
• Apply transitions (e.g. dissolves) between
shots
• Combination of picture with sound
• No changes made to the footage itself
7 230–236
Post-Production
• Changing or adding to the material
• Most changes are generalizations of image
manipulation operations (e.g. colour correction,
blurring and sharpening,…)
• Compositing – combining elements from different shots
into a composite sequence
• Animating elements and combining animation with live
action
7 236–237
Preparing for Delivery
• Compromises required to bring resource requirements
of video within capabilities of delivery media (e.g.
networks) and low-end machines
• Reduce frame size (e.g. downsample to quarter
frame)
• Reduce frame rate (12fps is OK for smooth motion,
flicker not a problem on computer)
• Reduce colour depth
7
HDTV
• The main thrust of HDTV (High Denition TV) is not to increase the
“definition” in each unit area, but rather to increase the visual field
especially in its width
• The first generation of HDTV was based on an analog technology
developed by Sony and NHK in Japan in the late 1970s
• MUSE (MUltiple sub-Nyquist Sampling Encoding) was an improved
NHK HDTV with hybrid analog/digital technologies that was put in
use in the 1990s. It has 1,125 scan lines, interlaced (60 fields per
second), and 16:9 aspect ratio
• Since uncompressed HDTV will easily demand more bandwidth,
various compression techniques are being investigated
• It is also anticipated that high quality HDTV signals will be
transmitted using more than one channel even after compression
7
A brief history of HDTV evolution
• In 1987, the FCC decided that HDTV standards must be compatible
with the existing NTSC standard and be conned to the existing VHF
(Very High Frequency) and UHF (Ultra High Frequency) bands
• In 1990, the FCC announced a very different initiative, i.e., its
preference for a full-resolution HDTV, and it was decided that HDTV
would be simultaneously broadcast with the existing NTSC TV and
eventually replace it
• Witnessing a boom of proposals for digital HDTV, the FCC made a key
decision to go all-digital in 1993. A grand alliance was formed that
included four main proposals, by General Instruments, MIT, Zenith,
and AT&T, and by Thomson, Philips, Sarno and others.
• This eventually led to the formation of the ATSC (Advanced Television
Systems Committee) | responsible for the standard for TV
broadcasting of HDTV.
• In 1995 the U.S. FCC Advisory Committee on Advanced Television
Service recommended that the ATSC Digital Television Standard be
adopted
7
Advanced Digital TV formats
supported by ATSC
7
HDTV details
• For video, MPEG-2 is chosen as the compression standard
• For audio, AC-3 is the standard. It supports the so-called 5.1
channel Dolby surround sound, i.e., five surround channels
plus a subwoofer channel
• The salient difference between conventional TV and HDTV:
• HDTV has a much wider aspect ratio of 16:9 instead of 4:3
• HDTV moves toward progressive (non-interlaced) scan. The
rationale is that interlacing introduces serrated edges to
moving objects and flickers along horizontal edges
7
HDTV details
• The FCC has planned to replace all analog broadcast services
with digital TV broadcasting by the year 2006. The services
provided will include:
• SDTV (Standard Denition TV): the current NTSC TV or
higher
• EDTV (Enhanced Denition TV): 480 active lines or higher
• HDTV (High Denition TV): 720 active lines or higher
