User Guide

EXpert IPTV Test Tools

IPTV Test Tools for FTB-200
Copyright © 2011-2012 EXFO Inc. All rights reserved. No part of this publication may be reproduced,
stored in a retrieval system or transmitted in any form, be it electronically, mechanically, or by
any other means such as photocopying, recording or otherwise, without the prior written
permission of EXFO Inc. (EXFO).

Information provided by EXFO is believed to be accurate and reliable. However, no

responsibility is assumed by EXFO for its use nor for any infringements of patents or other
rights of third parties that may result from its use. No license is granted by implication or
otherwise under any patent rights of EXFO.

EXFO’s Commerce And Government Entities (CAGE) code under the North Atlantic Treaty
Organization (NATO) is 0L8C3.

The information contained in this publication is subject to change without notice.

Trademarks

EXFO’s trademarks have been identified as such. However, the presence or absence of such
identification does not affect the legal status of any trademark.

Units of Measurement

Units of measurement in this publication conform to SI standards and practices.

April 27, 2012

Version number: 2.0.0

2
License Agreement and Warranty
IMPORTANT: CAREFULLY READ THE FOLLOWING LICENSE AGREEMENT BEFORE OPENING THIS
INSTALLATION PACKAGE. BY OPENING THIS PACKAGE AND USING THE SOFTWARE WHETHER
INCORPORATED OR NOT IN AN EXFO INC. ("EXFO") PRODUCT, YOU INDICATE YOUR
ACCEPTANCE TO BE BOUND BY THE TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU
DO NOT ACCEPT THE TERMS AND CONDITIONS OF THIS LICENSE AGREEMENT, DO NOT OPEN
THIS PACKAGE AND PROMPTLY RETURN THE PRODUCT OR SOFTWARE WITH YOUR PROOF OF
PAYMENT, WHEREUPON YOUR MONEY WILL BE REFUNDED.

THE PRODUCT OR SOFTWARE YOU ORDERED MIGHT INCLUDE SOURCE CODE AND/OR
SOFTWARE COMPONENTS, IT IS PROVIDED FOR YOUR CONVENIENCE IN MODIFYING THE
PRODUCT OR SOFTWARE TO SUIT YOUR SPECIFIC NEEDS, OR TO CREATE DERIVATIVE WORKS
INCLUDING SOURCE AND LIBRARY FILES IF APPLICABLE.

THE PRODUCT AND THE SOFTWARE MUST BE USED ONLY FOR YOUR INTERNAL BUSINESS
OPERATION AND ITS INTENDED APPLICATION. YOU MAY NOT COPY OR USE THE SOURCE
CODE OR THE SOFTWARE COMPONENTS TO PRODUCE OTHER SOFTWARE DEVELOPMENT
TOOLS FOR DISTRIBUTION AND RESALE WITHOUT EXPRESS WRITTEN PERMISSION FROM
EXFO. EXFO RETAINS ALL RIGHTS TO THE SOURCE CODE, THE SOFTWARE COMPONENTS AND
ALL MODIFICATIONS THEREOF. YOU SHALL RESPECT AND COMPLY WITH ANY OF THE
PROVISIONS LISTED BELOW WHICH MAY ALSO BENEFIT ANY GIVEN THIRD PARTY BENEFICIARY
AS DEFINED HEREIN.

1. DEFINITIONS: The following definitions apply to the terms in the Agreement.

"Documentation" means the user’s manual and other printed materials accompanying the
Software.

"Product" means the EXFO instrument designed for use with the Software, as the case may
be.

"Software" means the computer programs, source code and software components contained
therein and all updates and upgrades thereto. The term also includes all copies of any part of
the computer program, source code or software components.

3
2. GRANT OF LICENSE: EXFO grants to you, the purchaser of the enclosed Software, a limited,
restricted, non‐exclusive license. You shall use the Software only in conjunction with its
purpose or in conjunction with the Product, subject to the limitations on use and disclosure
contained herein and in the Documentation. You may:

 Use the Software on a network, file service or virtual disk; provided that access is
limited to one user at a time and that you have the original copy of the
Documentation and Software media.
 Make one (1) copy of the Software for backup or modification purposes in support of
the use of the Software on a single computer.
 Merge the Software or incorporate the same into another program provided that such
a program will, for so long as the Software is included therein, be subject to all of the
terms and conditions of this Agreement.
 You may not:
Make copies of the Documentation.

Assign, give or transfer the Software, any services or interests in the Software, to another
individual or entity. Sublicense, lease, time‐share, service bureau, lend, use for subscription
service or rental use any portion of the Product, the Software, or your rights under this
Agreement.

 Reverse engineering, disassemble or decompile in whole or in part the Software or the

Product.
 Publish any result of benchmark tests run on the Software or the Product.
 YOU RECOGNIZE THAT THE SOURCE CODE AND THE SOFTWARE COMPONENTS
COMPRISING THE SOFTWARE ARE HIGHLY VALUABLE TRADE SECRETS OF EXFO OR ANY
GIVEN THIRD PARTY BENEFICIARY AND THAT EXFO WISHES TO PREVENT THEIR
DISCLOSURE TO YOU, OR ANY THIRD PARTY.

3. SOFTWARE OWNERSHIP: The Software is licensed, not sold. Title to the Software shall not
be passed to you or to any other party. All applicable rights to patents, copyrights, trademarks
and trade secrets in the Software, or any modifications made at your request, are and shall
remain the property of EXFO.

4. AUDITS: EXFO reserves its right to audit, at its convenience, your use of the Software.

4
5. TERMS OF TERMINATION: This Agreement shall remain in full force and effect until you
discontinue use of the Software or the Product, until the end of the life of the Product or
Software or until this Agreement is terminated, whichever occurs earlier. Without prejudice to
its other rights, EXFO may terminate this Agreement if you fail to comply with the terms
thereof. In such event, you must destroy or return all copies of the Software and
Documentation as requested in writing by EXFO. You shall be liable for all damages to EXFO as
a result of the breach whether or not you were notify of the likelihood of such damages.

EXFO retains all rights to the Software not expressly granted hereunder. Nothing in this
Agreement constitutes a waiver of the rights of EXFO or any Third Party Beneficiary.

6. LIMITED WARRANTY: EXFO warrants the media on which the Software is distributed to be
free from defects in material and workmanship and that the Software will perform
substantially in accordance with the Documentation. EXFO will replace defective media or
Documentation at no charge, provided you return the item with dated proof of payment to
EXFO within (60) days of the date of delivery. THESE ARE YOUR SOLE REMEDIES FOR BREACH
OF WARRANTY. EXCEPT AS SPECIFICALLY PROVIDED ABOVE, EXFO MAKES NO WARRANTY OR
REPRESENTATION, EITHER EXPRESS OR IMPLIED WITH RESPECT TO THE SOFTWARE OR
DOCUMENTATION INCLUDING THEIR QUALITY, PERFORMANCE, MERCHANTABILITY, OR
FITNESS FOR A PARTICULAR PURPOSE.

7. LIMITATIONS OF LIABILITY: Because software is inherently complex and may not be

completely free of errors, you are advised to verify your work. IN NO EVENT WILL EXFO, ITS
DEALERS, DISTRIBUTORS, RESELLERS, OR THIRD PARTY BENEFICIARY, BE LIABLE FOR DIRECT,
INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS,
DOWNTIME OR DAMAGES TO PROPERTY ARISING OUT OF THE USE OF OR INABILITY TO USE
THE SOFTWARE OR DOCUMENTATION, even if advised of the possibility of such damages. THE
WARRANTY AND REMEDIES SET FORTH ABOVE ARE EXCLUSIVE AND IN LIEU OF ALL OTHERS,
ORAL OR WRITTEN, EXPRESS OR IMPLIED. No dealer, distributor, agent or employee is
authorized to make any modification or addition to this warranty. Some US States or
applicable local legislation do not allow the exclusion or limitation of implied warranties or
limitation of liability for incidental or consequential damages, so the above limitation or
exclusion may not apply to you.

5
8. U.S. GOVERNMENT RESTRICTED RIGHTS: The Software and Documentation are provided
with RESTRICTED RIGHTS. Use, duplication, or disclosure by the Government is subject to
restrictions as set forth in subparagraph (c) (1) (ii) of The Rights in Technical Data and
Computer Software clause at 52.227‐7013. EXFO headquarters are located at 400 Godin
Avenue, Quebec, Quebec, G1M 2K2, Canada.

9. EXPORT AND IMPORT LAWS: You must comply fully with all applicable export and import
laws including, but not limited to, export laws and regulations of the United States of
America. No Programs, Products or Software shall be exported, directly or indirectly, in
violation of laws.

10. THIRD PARTY BENEFICIARY: EXFO may designate, from time to time, any Third Party
Beneficiary, with respect to any Product or Software, which has duly executed a prior writing
agreement with EXFO. Third Party Beneficiaries may include, but are not limited to, namely
ORACLE, Business Objects, Advance Fiber Optics, and MapInfo. The terms of this Agreement
also governs any source code that may be provided in some programs by Third Party
Beneficiary, such as the one mentioned above. This Agreement is not intended to be for the
benefit of and shall not be enforceable by any given Third Party Beneficiary without a prior
written agreement duly executed with EXFO.

11. GENERAL: This Agreement constitutes the entire agreement between you and EXFO as
concerns the subject matter hereof and supersedes any prior agreement as to such subject
matter. If any provision of this Agreement shall be deemed to be invalid, illegal or
unenforceable, the validity, legality and enforceability of the remaining portions of this
Agreement shall not be affected or impaired thereby. This Agreement shall be governed by
and construed in accordance with the laws applicable in the province of Quebec, Canada
without regards to its conflict of laws provisions.

BY USING THE SOFTWARE, YOU ACKNOWLEDGE THAT YOU HAVE READ THIS AGREEMENT,
THAT YOU UNDERSTAND IT, AND YOU AGREE TO BE BOUND BY ITS TERMS AND CONDITIONS.

If you have any questions regarding this Agreement, you may contact the Legal Department at
EXFO at (418) 683‐0211.

6
Table of Contents
EXFO ........................................................................................................................................ 1
EXPERT IPTV TEST TOOLS ............................................................................................................ 1

LICENSE AGREEMENT AND WARRANTY ....................................................................................3

TABLE OF CONTENTS ................................................................................................................7

CHAPTER 1 INTRODUCING THE EXPERT IPTV TEST TOOLS .................................................... 10

CHAPTER 2 SAFETY INFORMATION ..................................................................................... 14

CHAPTER 3 GETTING STARTED ............................................................................................ 15

STARTING APPLICATION ............................................................................................................... 15

CHAPTER 4 PHYSICAL INTERFACE ........................................................................................ 16

ETHERNET 10/100/1000 MBITS/S ETHERNET PORT ......................................................................... 16

CHAPTER 5 USING THE GRAPHICAL USER INTERFACE .......................................................... 17

MAIN WINDOW......................................................................................................................... 17

CHAPTER 6 SETTING THE IPTV TEST .................................................................................... 18

6.1 SETUP INTERFACE .............................................................................................................. 18

6.2 SETUP THRESHOLDS .......................................................................................................... 18
6.3 MANAGE CHANNEL LIST..................................................................................................... 19

CHAPTER 7 RUNNING IPTV TESTS ....................................................................................... 22

7
7.1 SIMULTANEOUS TESTING OF MULTIPLE CHANNELS ................................................................. 22
7.2 PASSIVE TEST .................................................................................................................. 22
7.3 SET TOP BOX EMULATION TEST .......................................................................................... 24
7.4 ETHERNET INTERFACE STATUS .............................................................................................. 25

CHAPTER 8 GETTING IPTV METRICS .................................................................................... 26

8.1 BASIC IPTV METRICS TEST ................................................................................................ 26

Definitions: ............................................................................................................................ 27
8.2 LIVE STREAM PREVIEW ....................................................................................................... 28
8.3 ADVANCED IPTV METRICS TEST ........................................................................................... 29
8.3.1 VIDEO PERCEPTUAL QUALITY METRICS ............................................................................... 29
8.3.2 TRANSPORT PACKET METRICS ............................................................................................... 31
8.3.3. MPEG‐TS TR 101290 PRIORITY ..................................................................................... 33
8.3.4 AUDIO DESCRIPTION AND PERCEPTUAL METRICS INFORMATION ............................................... 35
8.3.5 VIDEO FRAME METRICS .................................................................................................. 37
8.3.6. VIDEO BANDWIDTH ....................................................................................................... 39
8.3.7 METRICS IN GRAPHICAL FORM‐ HISTOGRAMS ...................................................................... 40

CHAPTER 9 GENERATING AND VIEWING TEST RESULTS ...................................................... 41

9.1 TEST RESULTS LOG ............................................................................................................. 41

9.1.1 ACCESSING TEST RESULTS ................................................................................................. 41

APPENDIX A ......................................................................................................................... 47

A.1 UNDERSTANDING IPTV VIDEO PERCEPTUAL QUALITY * ............................................................ 47

A.1.1 TRANSMISSION‐RELATED IMPAIRMENTS ............................................................................. 48
A.1.2.2 Performance of Video Coders ................................................................................. 55
A.2 VQMON/HD QUALITY ANALYSIS ALGORITHM * ..................................................................... 57
A.2.1 VIDEO STREAM ANALYSIS ................................................................................................ 57
A.2.1.1 TR 101 290 Metrics Collection ................................................................................. 58
A.2.1.2 Playout Buffer Emulator ......................................................................................... 58
A.2.1.3 Content Analysis ..................................................................................................... 58
A.2.1.4 Frame Type Detection ............................................................................................ 59
A.2.1.5 Per‐frame Quality Analysis ..................................................................................... 59

8
A.2.1.6 Perceptual Quality Model ....................................................................................... 59
A.2.1.7 VQmon Markov Model (VMM) ............................................................................... 59
A.2.2 AUDIO STREAM ANALYSIS................................................................................................ 61
A.2.3 TELCHEMY VIDEO QUALITY METRICS .................................................................................. 62
A.2.3.1 Mean Opinion Scores (MOS) ................................................................................... 63
A.2.3.2 VQmon/HD Video Service Transmission Quality Metric ‐ VSTQ ............................... 64
A.2.3.3 Degradation Factors ............................................................................................... 65
A.3 IPTV PROTOCOLS .............................................................................................................. 66
A.4 PCR JITTER ...................................................................................................................... 70
A.5 IGMP LATENCY VS. ZAP TIME .............................................................................................. 71

APPENDIX B TECHNICAL SPECIFICATION ............................................................................... 74

TROUBLESHOOTING ............................................................................................................... 77

GLOSSARY ......................................................................................................................... 78

9
Chapter 1 Introducing the EXpert IPTV Test Tools

The EXpert IPTV Test Tools is a software FTB‐200 based IPTV quality analyzer. It consists
of several components:

 Test Logger with multistream detection and synchronizer via the FTB‐200
10/100/1000 Ethernet interface
 Set Top Box(STB) emulator for multicast and unicast streams
 Video Previewer
 Video Metrics
 Packet statistics
 TR 101290 Metrics
 Audio Metrics
 Video Frame Metrics
 Video Bandwidth Metrics
 Charts

When used as a STB emulator The EXpert IPTV Test Tools is terminating the customer
modem or GPON adapter.

10
When used as a passive monitoring device The EXpert IPTV Test Tools is connected via a
manageable switch or aggregating tap to monitor bidirectionally the IGMP, RTSP and
other protocols, and IPTV streams from the video server.

In both connection modes video and audio quality assessment is provided by VQMON
algorithm.

Test results are continuously accumulated and saved on the FTB‐200 hard drive. Live test
results can be watched on the screen. Saved test results can be viewed in the report
format, pdf format or be exported to a file.

Tested metrics can be sorted and viewed in text or graphical formats.

11
The example below shows The EXperrt IPTV Test Tools during analysiss of four IPTV streeams
with
h individual previiew screens.

12
 Conventions
Before using the product described in this guide, you should understand the following
conventions:

WARNING
Indicates a potentially hazardous situation which, if not avoided, could
result in death or serious injury. Do not proceed unless you
understand and meet the required conditions.

CAUTION
Indicates a potentially hazardous situation which, if not avoided, may
result in minor or moderate injury. Do not proceed unless you
understand and meet the required conditions.

CAUTION
Indicates a potentially hazardous situation which, if not avoided, may
result in component damage. Do not proceed unless you understand
and meet the required conditions.

IMPORTANT
Refers to information about this product you should not overlook.

13
Chapter 2 Safety Information

Safety Information
Laser Safety Warnings

WARNING
Do not install or terminate fibers while a laser source is active. Never
look directly into a live fiber, and ensure that your eyes are protected
at all times.

Laser Safety Information

The EXpert IPTV Test Tools software is not provided with any hardware
components. However, it may be used with your platform or modules which may
contain laser components. Refer to the user guides of your platform or modules
for further laser safety details and instructions.

14
Chapter 3 Getting Started
The EXpert IPTV Test Tools application is preinstalled on the FTB‐200. If The EXpert
IPTV Test Tools is not installed, refer to the FTB‐200 User Guide for more information
on how to install the application.

Starting Application

You can access many tools directly from your unit.

To start an application:

1. From Compact Toolbox, select the Test Tools tab.

2. Click on the Expert IPTV Test Tools.
OR
Select from the START menu EXFO, Tools and the Expert IPTV Test Tools.

15
Chapter 4 Physical Interface
The EXpert IPTV Test Tools uses the GigE port of the FTB‐200.

Ethernet 10/100/1000 Mbits/s Ethernet Port

 Connect the 10/100/1000 electrical signal with the IPTV to the port with
the RJ‐45 connector.

Note: Refer to the FTB‐200 Compact Modular Platform User Guide for more information.

16
Chapter 5 Using the Graphical User Interface
This chapter describes the graphical user interface of your Expert IPTV Test Tools
application.

The EXpert IPTV Test Tools allows you to start a test, connect and disconnect to IPTV streams,
program and use Set Top Box, view information about the stream, view IPTV metrics and
statistics, view or print test results, and more.

Main Window

Start the EXpert Test Tools IPTV application. The application allows to view the first
level tabs and buttons for test setup and results:

17
Chapter 6 S
Setting the
e IPTV Test

6.1 Setup
S Interfface

The Expeert IPTV Test Too

ols setup has facttory defaults.
Changingg these defaults is necessary onlyy if any of the
followingg parameters is required
r for login
n process:

 MAC C address
 Statiic IP address
 or DHCP client‐Vendo or Class ID
 Channges in the Thressholds settings

The defaault Ethernet inte

erface is:

 Local
L Area Conne
ection Realtek
PCIe
P GBE Family Controller‐packe
et
Scheduler Minip
port
 No
N MAC address
 IGMPv2
I
 DHCP
D Client

6.2 Setup
S Thressholds
Failed,
F Warned and
a Passed thresh
hold
parameters
p are preset
p for:

Average
A Audio/Video MOS, Max PCRP Jitter,
Video
V Packet Losss Ratio, IGMP Lattency and
TR101290.
T The TRR101290 allows configuration
c
or
o Priority 1 and 2 parameters from a separate
menu
m as shown to the right.

18
 These
T color coded
thresholds
t will th
hen
trigger
t the
corresponding
c flaags
in the Summary
pan.
p Unchecking
any
a parameters
removes
r it from thet
active
a metrics.
Unchecking
U all
parameters
p will
result
r in flagg
for
f the TR101290 0
Errors
E of Summarry.

6.3 Manage
M Cha
annel List

Clickk on
Chan nnel Setup
and click again
on Manage
M
chan nnel list.

19
 Enter
E New Groupp
name
n as a channel
list.
l

 Select
S either
Multicast
M channe
els
or
o VoD streams.

For Multicast
M channeels:
 Enter
E New Chann nel
Number,
N Name,
Multicast
M addresss
and
a Port.
 Press
P OK.

Forr VoD streams:

 Enter Neww
Channel
Number, NName
and rtsp urri and
press OK.

 Repeat thee
above step p for
each additional
channel (or rtsp
uri) to be aadded
to the multicast
(or VoD) grroup.

20
When Grroups are exportted a file with
on “.ExfoChannels” are created.
extensio

Groups can
c be imported only from files
with thiss “.ExfoChannels”” extension.

 To
T import a Grou up click on
“Import
“ all Group
ps” then select
t file form the desired folder
the
and
a open it.

 To
T export a Grou
up click on “Exporrt all Groups” theen type a file nam
me and Save it.

21
Chapter 7 Running IPTV Tests

7.1 Simultaneous Testing of Multiple Channels

7.2 Passive Test

The Passive Test allows monitoring of any IPTV streams present at the interface, for quality of
service and video preview. The Passive test can be performed via a 3‐ way Tap ( or a
manageable switch) as shown below:

22
 The Passive Test starts automatically after
depressing the and upon detection
of a valid IPTV stream(s). All detected valid
streams will be displayed in the Test Log box and
marked as Passive.

 Click on the channel to see its current metrics and

preview
 Channel whose metrics and preview are currently
displayed is underlined
 To view metrics and preview of another channel click
on it in the Test Log box.
 Check the Summary metrics for a quick signal quality
check.
 Click on the tab of a specific metrics of interest to
view details.
 To stop the test on all channels press
button or disconnect the cable carrying the stream or
wait until the stream(s) expires.

23
7.3 Set Top Box Emulation Test

The Set Top Box Emulation Test allows IGMP or RTSP join& leave requests to one or multiple
IPTV streams.
Once the stream(s) become detected and synchronized to the test metrics start and live video
preview is available on non‐encrypted streams.

Number of channels that can be tested simultaneously is limited to 10. Exceeding this number
will generate an ERROR message.
Live Preview is limited to one stream.

To start the Set Top Box Emulation Test:

 Assuming that Setup has been already configured or default is uses start test press
.

 From the STB channels pulldown menu select Multicast , VoD or Run auto test:

For Multicast streams:

 Choose the channel to be tested from
the Channel List in the STB

 Click button of the chosen

channel in the Set Top Box. This will
initiate IGMP request for this multicast
channel. Once the stream is detected it
will show up in the Test Log box as active.
 Repeat this process for all channels to be tested.

24
For VoD streams (uri’s):
 Choose the RTSP uri to be tested from the
Channel List in the STB

 Click on Play button of the chosen channel in

the Set Top Box. This will initiate RTSP join
request for this unicast channel. Once the
stream in this is detected it will show up in the
Test Log box as active. Repeat this process for
all channels to be tested.

To stop the test of the specific stream press button, or press either
or disconnect the Ethernet cable.

7.4 Ethernet Interface Status

The Interface panel shows the status of the

selected Ethernet link.

This information is useful to check if the link is

up or down.

The Downstream Rate give the aggregate kbits/s for the all the streams present in the link.

25
Chapter 8 Getting IPTV Metrics
The channel whose metrics are currently displayed is underlined in the Test Log.

Active Terminated # of
streams streams passed # of tests
count count tests with
warnings

# of failed
tests

8.1 Basic IPTV Metrics Test

 Check the Video

Description Information
for the stream
characteristic and
reference data.

Video Description
Information provides
reference information
extracted from the stream
MPEG frames headers such
as:

• IP Address
• Encapsulation
Protocols
• Code Type
• GOP Details
• Frame Details
• Frame Details
• Reference Clock Summary pan displays the most
• Bandwidth and important metrics along with their
Packet Size Pass, Warning or Fail flags.
26
  Check Summary metrics for a quick signal quality. Summary provides a list of industry
most used parameters for quick evaluation of IPTV service quality such as MOS Score,
PCR Jitter, Video Packet Loss, TR 101290 and IGMP Latency( zap time).

Definitions:

Tab Category Metric Name Range Description

Summary Summary Measurement Time 0‐12 hrs Total duration of test in seconds from the time START was
pressed to the time STOP was pressed
Average Audio/Video 1.0‐5.0 The average MOS‐AV across all completed audio/video
MOS streams monitored on the logical
interface. (This does not include active streams.)
Max PCR Jitter ms The PCR jitter in milliseconds
Video Packet Loss Rate % Total proportion of stream transport packets lost or
discarded (minus corrected)
TR 101 290 Errors NA The occurrence of a TR101290 Priority 1 error, TR101290
Priority 2 error or both.
IGMP Latency 0‐3 sec The time from IGMP Join request to the stream arrival
time
Video IP Source Address NA The source IP address
Description IP Destination Address NA The destination IP address
Information Source Port NA The source transport port.
Destination Port NA The destination transport port
Protocols NA The encapsulation of the stream, for example UDP/RTP,
UDP, UDP/TCP, etc.
Codec Type NA The video codec used for the MPEG‐TS frames in the
stream
Group of Picture Type NA The GOP structure expressed as a series of ‘I’, ‘B’, ‘P’
characters describing the frame type series in the
structure.
Current GOP Length NA The current GOP length, in frames.
Average GOP Length NA The average GOP length, in frames.
Frame Width Pixels The video frame image width, in pixels.
Frame Height Pixels The video frame image height, in pixels.
Frame Rate Frames per The video frame rate, in frames per one thousand
sec seconds – e.g. 29,970 equals 29.97 frames per second.
Interlaced 0 OR 1 A Boolean value indicating whether the stream is
interlaced (TRUE), with alternating scanning of even and
odd scan lines or progressive (FALSE), with scanning line
by line in order.
Average Number of Slices The average number of slice contained in each I‐frame
in I‐frame
Reference Clock Rate Khz The 33‐bit value of the MP2TS program clock reference
(PCR) 90kHz portion.
Average Video Mbits/s The average bandwidth of the video stream in
Bandwidth Mbits/second.
Average Packet Size The average packet size of the video stream.

27
8.2 Live Stream Preview

 Video and Audio preview of the channel whose metrics are currently displayed can be
enabled at anytime in the main test window. The preview can be toggled to full screen
size or stopped all together and audio can be turned on or muted.

Start/Stop Audio Full size

Video preview On/Off screen

28
8.3 Advanced IPTV Metrics Test
8.3.1 Video Perceptual Quality Metrics

As described in the technical reference section Appendix A.1, the Perceptual Quality
Metrics are calculated on the selected video stream to provide Mean Opinion Score (MOS)
and related parameters.

Video Perceptual Video Jitter

Quality provides the Metrics provide
most essential QoE amount of
scoring metrics such a
Video MOS scores that Average Frame
results from and I‐frame Inter
computation of a Arrival Jitter
complex formula (see
Technical Overview for Average and Peak
details). Frame Arrival
Delay in
milliseconds

Degradation Factors
provide insight into the
nature of MOS score Video Scene
degradation by listing Analysis Metrics
10 contributing factors provide
with their weight proportions of
score. the stream with
Detail, Panning,
Motion and Static
Image.

These factors
weight on the
overall Video
MOS score.

29
Definitions:

Tab Category Metric Name Range Description

Video Video Instantaneous Absolute MOS 1.0-5.0 The instantaneous absolute Video Mean Opinion Score, a 1-5 score that
Perceptual measures the impact of the video codec, image size, frame rate, packet loss
Quality distribution, GoP structure, content, and frame loss concealment on viewing
Metrics quality
Minimal Absolute MOS 1.0-5.0 The minimum absolute Video Mean Opinion Score, a 1-5 score that
measures the impact of the video codec, image size, frame rate, packet loss
distribution, GoP structure, content, and frame loss concealment on viewing
quality
Average Absolute MOS 1.0-5.0 The average absolute Video Mean Opinion Score, a 1-5 score that
measures the impact of the video codec, image size, frame rate, packet loss
distribution, GoP structure, content, and frame loss concealment on viewing
quality
Maximum Absolute MOS 1.0-5.0 The maximum absolute Video Mean Opinion Score, a 1-5 score that
measures the impact of the video codec, image size, frame rate, packet loss
distribution, GoP structure, content, and frame loss concealment on viewing
quality
MOS Below Threshold 0-100% The proportion of the stream duration where the Absolute MOS-V fell below
the configured analysis threshold.
Transmission Quality 0-50.0 The codec-independent video service transmission quality rating
EPSNR dB Estimated Peak Signal to Noise Ratio (PSNR) expressed in dB. This is an
estimate of the distortion that has occurred between the source video stream
and the output video stream.
Degradation Factors : 0-10 Severity of perceptual quality degradation caused by (each of) the following
factors: Loss, Discard, Codec Quantization Level, Frame Resolution, Frame
Resolution, Frame Rate, GOP Length, Available Network Bandwidth,
Audio/Video Synchronization, Recency
Video Instantaneous detail level 0-100 Instantaneous detail level
Scene Instantaneous motion level 0-100 Instantaneous motion level
Analysis Instantaneous panning level 0-100 Instantaneous panning level
Metrics Static Image Proportions % The proportion of the video stream that contains static image[s]
High Detail Proportions % The proportion of the video stream that contains high detail
Low Detail Proportions % The proportion of the video stream that contains low detail
High Panning Proportions % The proportion of the video stream that contains a high degree of panning
Low Panning Proportions % The proportion of the video stream that contains a low degree of panning
High Motion Proportions % The proportion of the video stream that contains high motion
Low Motion Proportions % The proportion of the video stream that contains low motion
Video Jitter Frame Inter-arrival Jitter ms Average frame inter-arrival jitter in milliseconds. The inter-arrival jitter is
Metrics computed relative to the expected arrival time based on the frame rate.
I Frame Inter-arrival Jitter ms Average I frame inter-arrival jitter in milliseconds. The inter-arrival jitter is
computed relative to the expected arrival time based on the frame rate.
Average Frame Arrival Delay ms Average frame arrival delay, in milliseconds.
Peak Frame Arrival Delay ms Peak frame arrival delay, in milliseconds. The peak maintains a hold time of
approx. 10 seconds

30
8.3.2 Transport Packet Metrics

Transport Packets are

Jitter Metrics
MPEG‐TS packets that
focus on the PCR
carry Packet
Jitter which is
Elementary Stream
defined as Time
(PES).
Stamp arrival
time jitter.
Statistics of these
MPEG‐TS Packets such
PCR Jitter is a
as lost, discarded,
major contributor
corrected, out‐of‐
to loss of frames
sequence and
due to router or
duplicated packets
STB buffer
allow detection of this
overflow.
major source of video
picture degradation.

RTP Packet Metrics (if available) are statistics of RTP protocol

encapsulation. RTP encapsulation is optional and is used to provide
independent time stamping to video and audio. But it also adds overhead
to the transmission.

Tracking RTP statistics allows problem isolation to the RTP encapsulating

device.

31
Definitions:

Tab Category Metric Name Range Description

Packets Video Packets Received NA Number of video transport packets received properly for playout during the
Transport interval by the end system or observed at the monitoring point, excluding
Packet any duplicate packets.
Metrics
Packets Lost NA Number of video transport packets identified as lost at the endpoint or
monitoring point during the interval
Packets Discarded NA Number of video transport packets discarded by the receiving jitter buffer
during the interval or an estimate of this value made at the monitoring point.
Packets Out of NA Number of video transport packets arriving out of sequence during the
Sequence interval.
Packets Duplicated NA Number of duplicate video transport packets received during the interval.
Lost% 0-100% Total proportion of stream transport packets lost or discarded (minus
corrected)
Corrected and Lost % 0-100% The proportion of stream transport packets lost in the transport network after
error correction algorithms have been applied, e.g. FEC or Reliable UDP.
Discarded % 0-100% The proportion of stream transport packets discarded due to late arrival.
Out Of Sequence % 0-100% The proportion of stream transport packets arriving out-of-sequence.
Duplicated % 0-100% The proportion of stream transport packets arriving as duplicates of
previously received
packets.
Burst Count NA The number of transport packet loss/discard burst occurrences over the
duration of the
stream.
Burst Loss Rate 0-100% The proportion of packets lost (before application of error correction
algorithms)
during burst periods.
Average Burst Length packets The average length of a packet loss burst, in transport packets.
Gaps Count NA The number of transport packet loss/discard gap occurrences over the
duration of the
stream.
Gaps Loss Rate 0-100% The proportion of packets lost (before application of error correction
algorithms)
during gap periods.
Average Gap Length packets The average length of a gap, in transport packets.
Jitter Metrics PCR Jitter ms The PCR jitter in milliseconds
Max PCR Jitter ms The maximum PCR Jitter in ms
RTP Packet Packets Received NA A count of a number of stream RTP transport packets received during the
Metrics interval
Packets Corrected NA A count of a number of stream RTP transport packets corrected by FEC
during the interval
Packets Lost NA A count of a number of stream RTP transport packets lost during the interval
Packets Discarded NA A count of a number of stream RTP transport packets discarded lost during
the interval
Packets Out of NA A count of a number of stream RTP transport packets out-of-sequence lost
Sequence during the interval
Packets Duplicated NA A count of a number of stream RTP transport packets duplicated lost during
the interval
Packets w/Timestamp NA A count of a number of stream RTP transport packets with a timestamp error
Errors during the interval

32
8.3.3. MPEG‐TS TR 101290 Priority

TR 101290 Second Priority lists Error Counters

TR 101290 First Priority lists Loss and Error
recommended for continuous or periodic
counters that are essential for video de‐
monitoring.
codability.

33
Definitions: MPEG_2 Transport Stream TR 101290 Priority 1 Metrics

Tab Category Metric Name Range Description

TR101290 TR101290 Transport Stream Sync NA Number of occurrences of transport stream sync loss, i.e.,
Priority 1 Loss Count two or more consecutive
sync bytes
Sync Byte Error Count NA Number of occurrences of sync byte error.
PAT Error Count NA Number of occurrences of Program Association Table (PAT)
error.
PAT 2 Error Count NA Number of occurrences of PAT 2 error.
Continuity Error Count NA Number of occurrences of continuity counter error, i.e.,
incorrect packet order,
duplicate packet, or lost packet.
PMT Error Count NA Number of occurrences of Program Map Table (PMT) error.
PMT 2 Error Count NA Number of occurrences of PMT 2 error.
PID Error Count NA Number of occurrences of Process IDentifier (PID) error.
TR101290 Transport Stream Error NA Indicators of the MP2TS errors that have occurred since this
Priority 2 Indicators metrics block was last
retrieved.
Transport Error Count NA Number of occurrences of packet with transport error bit set.
CRC Error Count NA Number of occurrences of PAT and/or PMT cyclic redundancy
check (CRC) error.
PCR Error Count NA Number of occurrences of Program Clock Reference (PCR)
error
PCR Repetition Error NA Number of times the interval between two consecutive PCR
Count values is greater than
40 milliseconds.
PCR Discontinuity Error NA Number of times the difference between two consecutive
Count PCR values is outside the
range of 0 to 100 milliseconds.
PCR Accuracy Error Count NA Number of occurrences of PCR accuracy error.
PTS Error Count NA Number of times the presentation timestamp (PTS)
repetition period is greater than
700 milliseconds.
CAT Error Count NA The number of occurrences of Conditional Access Table (CAT)
error.

34
8.3.4 Audio Description and Perceptual Metrics Information

Audio Description
Information is
similar to Video
Description
Information Audio
providing essential Perceptual
reference Quality Metrics
information about are MOS scores
the stream. with subjective
test content
calculated with
a specialized
VQMON audio
algorithm.

Degradation
Factors provide
severity of
degradation
due to Packet
Loss, Packet
Discard, Codec,
a Recency of
Audio Bandwidth Metrics provides bandwidth burst packet
average and peak rate with and without loss
headers.

35
Definitions:

Tab Category Metric Name Range Description

Audio Audio IP Source Address NA The stream source IP address
Description IP Destination Address NA The stream destination IP address
Information Source Port NA The stream source TCP/UDP port number
Destination Port NA The stream destination TCP/UDP port number
Type NA The stream type‐ audio, video, voice
Protocols NA The stream (s) transport protocols
Codec Type NA The audio codec type for the audio stream
Number of Channels NA The number of audio channels encoded in the audio stream
Reference Clock Hz The reference clock (RTP timestamp or MPT2S PCR/PTS) rate,
in Hz
Audio Average Audio Kbit/s Average audio bandwidth in bits/second, excluding transport
Bandwidth Bandwidth packet overhead,
Metrics FEC, and retransmissions
Peak Audio Bandwidth Kbit/s Peak audio bandwidth in bits/second, excluding transport
packet overhead, FEC,
and retransmissions.
Average Receive Kbit/s Average bandwidth of transport packets received, in
Bandwidth bits/second.
Peak Receive Bandwidth Kbit/s Peak bandwidth of transport packets received, in bits/second
Audio Minimal MOS 1.0‐5.0 Minimal Audio Mean Opinion Score, a 1‐5 score that
Perceptual measures the impact of the audio
Quality codec, bit rate, sample rate, and packet loss on audio quality.
Metrics Average MOS 1.0‐5.0 Average Audio Mean Opinion Score, a 1‐5 score that
measures the impact of the audio
codec, bit rate, sample rate, and packet loss on audio quality.
Maximum MOS 1.0‐5.0 Maximum Audio Mean Opinion Score, a 1‐5 score that
measures the impact of the audio
codec, bit rate, sample rate, and packet loss on audio quality.
Instantaneous MOS 1.0‐5.0 Instantaneous Audio Mean Opinion Score, a 1‐5 score that
measures the impact of the audio codec, bit rate, sample
rate, and packet loss on audio quality.
MOS Below Failed 0‐100% The proportion of the stream duration where the MOS‐A
Threshold value falls below the
configured analysis threshold
Degradation Factors: 0‐10 Severity of perceptual quality degradation caused by (each
Loss, Discard, Codec, of) the following
Recency factors: packet loss, packet discard (jitter), codec distortion,
and the recency of
burst packet loss.

36
8.3.5 Video Frame Metrics

Video Frames
(MPEG‐TS) statistics
provide the total
number of Received
and Impaired
frames I, P, B and
optional SI and SP.
This breakdown
allows better
diagnostics of the
distorted video.

Packet statistics
such as Packet Lost,
Packets Discarded
and Packets
Impaired
correspond to the
MPTEG‐TS frame
statistics and they
allow more detail
breakdown of the
diagnostics.

37
Definitions:

Tab Category Metric Name Range Description

Video Video I Frames Received NA Number of I Frames received
Frames Frame I Frames Impaired NA Number of I Frames impaired by packet loss/discard
Metrics % of I Frames Impaired 0‐100% Proportion of I Frames impaired by packet loss/discard
P Frames Received NA Number of P Frames received.
P Frames Impaired NA Number of P Frames impaired by packet loss/discard
% of P Frames Impaired 0‐100% Proportion of P Frames impaired by packet loss/discard.
B Frames Received NA Number of B Frames received
B Frames Impaired NA Number of B Frames impaired by packet loss/discard
% of B Frames Impaired 0‐100% Proportion of B Frames impaired by packet loss/discard
SI Frames Received NA Number of SI Frames received.
SI Frames Impaired NA Number of SI Frames impaired by packet loss/discard.
% of SI Frames Impaired 0‐100% Proportion of SI Frames impaired by packet loss/discard.
SP Frames Received NA Number of SP Frames received.
SP Frames Impaired NA Number of SP Frames impaired by packet loss/discard.
% of SP Frames Impaired 0‐100% Proportion of SP Frames impaired by packet loss/discard.
I Frame Packets Received NA Number of transport packets received containing video I
frame information
I Frame Packets Lost NA Number of transport packets lost containing video I frame
information
I Frame Packets Lost NA Number of transport packets discarded by the playout (jitter)
buffer containing video
I frame information.
% of I Frame Packets 0‐100% Proportion of transport packets impaired by loss/discard
Impaired containing video I frame
information.
P Frame Packets NA Number of transport packets received containing video P
Received frame information
P Frame Packets Lost NA Number of transport packets lost containing video P frame
information.

38
8.3.6. Video Bandwidth

Video Bandwidth of I, P,
B, SI and SP frames
provides a useful
information for traffic
engineering. Looking
and MOS and other
metrics one can
determine how much
less or more bandwidth
to add or reduce to
maintain the quality
goal and bandwidth
utilizations at the same
time.

Definitions:

Tab Category Metric Name Range Description

Video Video I frame Average Video Mb/s The average bandwidth of I‐frame video content transmitted,
Bandwidth Bandwidth bandwidth in Mbits/second.
Metrics I Frame Max Video Mb/s The maximum bandwidth of I‐frame video content
Bandwidth transmitted, in Mbits/second.
P frame Average Video Mb/s The average bandwidth of P‐frame video content
bandwidth transmitted, in Mbits/second.
P Frame Max Video Mb/s The maximum bandwidth of P‐frame video content
Bandwidth transmitted, in Mbits/second.
B frame Average Video Mb/s The average bandwidth of B‐frame video content
bandwidth transmitted, in Mbits/second.
B frame Max Video Mb/s The maximum bandwidth of B‐frame video content
bandwidth transmitted, in Mbits/second.

39
8.3.7 Metrics in Graphical Form‐ Histograms

Any two metrcis can be displayed as histograms allowing correlation of events for diagnostics.

40
Chapter 9 Generating and Viewing Test Results

9.1 Test Results Log

9.1.1 Accessing Test Results

Access to the Test Results is available from 2 locations:

1. Results button shown to the right

OR
2. Button on the right top side of the
screen

41
Test Results generated
by Resuults button or

button
b produce

the follo
owing log:

Double‐clicking
D on
n the
seelected test reco
ord
produces
p the mettrics
sccreen where all ttest
metrics
m are accesssible
frrom the tabs:

42
 Here iss an example of tthe
Charts screen produced d for
the single test record.

• To
T generate a pd df
report
r click on

button
b and wait for
the
t Summary PD DF
report
r to be
generated.
g

43
 ort start with the
The repo e summary page followed
f with teest metrics four pages long as sho
own
below:

44
45
46
Appendix A
The IPTV Video Stream is subjected to multilevel analysis as shown here:

A.1 Understanding IPTV Video Perceptual Quality *

The perceptual quality of video transmitted across IPTV networks is susceptible to

degradation from a number of transmission network sources including, frame errors caused
by packet loss, discard of packets due to excessive delay/jitter, and discard of packets due to
arrival sequencing errors. Simply relying on packet loss statistics, however, is not an accurate
way to measure video quality as perceived by viewers. The same degree of packet loss may

47
cause obvious distortion or may not even be noticed by the end user, depending on which
video frame types are impaired.

In addition, impairments can be introduced during the encoding/decoding process, by the

codec itself or an inappropriately low bitrates. The video content (e.g., level of detail and
motion onscreen) can also have a significant impact on the visibility of problems.
Furthermore, perceptual quality is affected by subjective factors including human reaction
time and the ‘recency effect’. Coupled with the type of content, e.g., fast motion, high detail,
or frequent scene changes, the quality of experience for the viewer will vary even under the
same impairment conditions.

Each of these objective and subjective factors must be taken into consideration in order to
accurately estimate IPTV video perceptual quality.

A.1.1 Transmission‐Related Impairments

Packet‐based video can be very sensitive to network impairments. Packet loss can
cause sections of frames or complete frames to be corrupted or deleted. For example, the
MPEG compression algorithm uses block‐based motion compensation for the reduction of
temporal redundancy and Discrete Cosine Transform (DCT)‐based compression for the
reduction of spatial redundancy. An MPEG encoder may generate three types of frame: Intra‐
coded (I), Predictive (P), and Bi‐directional (B) frames.

Figure2‐1. Packet
vs. Frame Loss
Rates for MPEG

48
As a frame often spans multiple packets, and a typical video stream includes interpolated
frames (P‐frames and B‐frames), a given packet loss rate can result in a frame loss rate six
times higher [9]. See Figure 2‐1 above.

In order to accommodate IPTV transmission network delay and low levels of delay variation,
a playout buffer is used to temporarily store incoming frames. For streaming video, such as
DVB or IPTV, it is permissible to apply arbitrary delays, and hence the playout buffer can be
quite large. Interactive videoconferencing requires a relatively low delay, since a participant
needs to respond immediately to questions or opinions of the other participants. Packets
arriving too late may be discarded and will appear as lost packets.

Traditionally, video quality is measured by comparing the video sequence that has been
processed by a video system to the original reference video sequence. It is not feasible to
implement a mid‐stream real‐time monitoring system using this approach. This is because it
needs to directly access the video source and decode video content, and has high
computational complexity.

To overcome these disadvantages, it is therefore desirable to monitor transmission

impairments in order to ensure good video quality. VQmon/HD is a non‐intrusive monitoring
technology that can passively measure the characteristics of live packet‐based video streams
and report quality scores in real‐time. VQmon/HD models the way that time‐varying
impairments, most notably burst packet loss and possible playout buffer discards, affect
perceived video quality.

A.1.1.1 Impact of Packet Loss and Jitter on Video Quality

Common video codecs, such as MPEG and H.263 implementations, use a combination
of intra‐ and inter‐frame coding methods. For intra‐frame encoding (I frame) the image frame
is divided into blocks, a Discrete Cosine Transform is used to convert each block to a set of
coefficients and then variable length coding is applied. A group of blocks are combined into a
single entity (slice), which can be carried within a single packet. If a transmission error occurs
then the whole group may be lost, creating a “stripe” within the decoded image. For example,
because the DC coefficients within each block are predicatively encoded from the first block in
the slice, an error makes this information unusable for the remainder of the slice. Some errors
may damage the frame structure and render the whole frame unusable.

49
For inter‐frame or motion‐based coding (P and B frames), motion vectors are determined for
each block and encoded. As for intra‐frame coding, errors can render a whole slice or frame
unusable. In simple inter‐frame coding systems, the loss of one I or P frame can make all
subsequent frames unusable until the next I frame is received—resulting in a significant
period of degraded, frozen, or blank video.

Note: the H.264

(MPEG‐4 AVC) codec
standard introduces
two new frame types,

Figure 2‐2. Error

propagation through
frames in a typical
GOP

“Switching I” (SI) and

“Switching P” (SP), which are designed to enable the decoder to more easily switch between
video streams with different bitrates. VQmon/HD reports SI and SP frame metrics when
H.264 is used.

Figure 2‐2 shows the impact of encoding errors on various frame types (I, B, and P) in a typical
Group of Pictures (GOP).

In most cases, the standards for video coding provide considerable flexibility to both the
encoder and decoder, allowing a range of cost/performance tradeoffs to be made. This can
make it difficult to precisely assess the impact of network impairments without knowledge of
the exact implementation.

50
A.1.1.2 Impact of Packet Loss on Specific Video Codecs

As shown in Figure 2‐3, a simple non‐robust video stream can be severely degraded
with even low levels of packet loss due to the error propagation effects described above.
Peak Signal‐to‐Noise Ratio (PSNR) is an objective measurement of video service quality
comparing the maximum power of the video signal to the power of corrupting noise affecting
the signal. Generally, a PSNR of under 20dB is regarded as unwatchable, and this level is
reached for MPEG video with a loss rate under 1 percent.

Figure 2‐3.
Estimated PSNR
for Three
Common Codecs

Error mitigation algorithms are being increasingly applied to help to compensate for packet
loss [11][12]. Methods include:

Forward Error Correction (FEC) ‐ redundancy is applied to the data stream to allow some
proportion of lost or errored packets to be replaced
51
Interleaving ‐ in which the video stream is split into alternate frames and each encoded
separately

Macro‐block error concealment ‐ spatially corresponding macro‐blocks are copied from the
previous frame

These approaches can help considerably with tolerance to packet loss.

A.1.1.3 Playout Buffer Configuration

It is assumed that both videoconferencing and streaming video systems provide a

playout buffer; however, the configuration of these is quite different. It is highly desirable to
determine whether an application is streaming or interactive, since the playout buffer
configuration is typically very different for these two applications.

Following are examples for the playout buffer settings:

Videoconferencing ‐ nominal 100ms (0.1 seconds), adaptive

Streaming Video ‐ nominal 3,000ms (3 seconds), fixed

A.1.2 Encoding/Decoding Impairments

While transmission impairments such as packet loss are a common source of video
quality degradation, quality can also be affected by the encoding/decoding process itself; i.e.,
some distortion can be introduced by the bitrate and codec used.

A.1.2.1 Impact of Coding Bitrate on Video Quality

For a typical MPEG‐2 encoded video stream with standard resolution 720x486, GOP
sequence IBBPBBPBBPBBPBB, at 30 frames per second, the mean square error (MSE) due to
bitrate can be approximated by:

MSEbr = 0.00001 + 1.5 / (B + B2 / 30000).

Where B is the bitrate in kilobits per second. The corresponding peak signal‐to‐noise ratio

52
(PSNR) value for the frame can be computed by

PSNRbr = ‐10 log10MSEbr.

Figure 2‐4 shows the estimated PSNR value on coding bitrate for MPEG‐2 encoded video.

Figure 2‐4

Estimated
PSNR on
Coding Bitrate
for Standard
Definition
MPEG‐2 Video
at 30 Frames
per Second

The following examples illustrate the impact of the coding bitrate on video quality. Both
images show the same frame from a video sequence encoded with MPEG‐2 at a resolution of
720x480, at 30 frames per second. In Figure 2‐5 the sequence was encoded at a bitrate of
5000 Kbps (610KB/s).

53
 Figure 2 ‐5

Still frame from

MPEG‐2 video
sequence encoded
at 5000Kbps

Figure 2‐6.

Still frame from

MPEG‐2 video
sequence encoded
at 1127Kbps

In Figure 2‐6, the sequence was encoded at a bitrate of 1127Kbps (138KB/s). The use of
coarser quantization conserves bandwidth, but creates distortion that impairs overall image
quality throughout the entire video sequence.
54
A.1.2.2 Performance of Video Coders

There are many standardized video coding algorithms, such as ITU‐T H.261, H.263,
H.264, ISO/IEC MPEG‐1, MPEG‐2 and MPEG‐4, AVS, VC‐1, etc. These standards do not explicitly
define codecs; they only define the syntax of an encoded video bitstream together with the
methods of decoding the bitstream. The consequence is that there might be significant quality
differences between codecs conforming to the same standard [16]. The Windows Media 9
(VC‐1) of Microsoft and RealVideo of RealNetworks are widely used proprietary video coding
algorithms for video streaming applications.

As a rule of thumb, ITU‐T H.263 reduces the coding bitrate by about 50% against H.261 for
comparable video quality. ITU‐T H.264/AVC significantly outperforms all other listed
standards. Average bit‐savings of more than 60% relative to MPEG‐2 are reported. H.264/AVC
Main Profile provides more than 1/3 bitrate saving relative to its competitors, MPEG‐4
Advanced Simple Profile (ASP) and H.263 Conversational High Compression (CHC) Profiles [25].

Reports show that WMV 9 (VC‐1) achieved similar quality to MPEG‐2 and MPEG‐4 with only
1/3

and 1/2 of the bitrate, respectively [21]. A paper by Bennet and Bock [13] concludes that
comparing H.264/AVC and VC‐1, there is very little performance difference between them.

A.1.2.3 Effective Coding Bitrate

The bitrate used in estimating the mean squared error as referenced in the beginning
of section 2.2.1 can be adjusted to an effective bitrate based on performance of a video codec
described above. The effective bitrate also needs to be adjusted based on the frame
resolution and frame rate of the video signal. Videoconferencing applications generally
support low to medium bitrates and frame resolution. The most commonly used resolutions
are QCIF (176x144) and CIF (352x288) at 10 to 30 frames per second. Entertainment‐quality
IPTV applications generally support video encoded with SD resolutions of 720x486 (30 Hz) or
720x576 (25 Hz), or HD resolutions of 1280x720 (720p) or 1920x1080 (1080i) at an average
bitrate of 3 Mbits/sec or higher.

Typically, the number of bits spent on coding a P‐frame is about 20% of the number required
for an I‐frame, whereas a B‐frame takes about 5% of the bitrate consumption of the I‐frame.
This implies that group of pictures (GOP) structure has a big impact on quality of encoded

55
video at a given bitrate under no loss condition, as well as under loss conditions as described
below. VQmon/HD automatically adjusts the video stream effective bitrate based on the GOP
structure of the encoded video stream.

A.1.3 Impact of Subjective Factors on Perceptual Quality

The degree to which viewers find video impairments annoying—or notice them at
all— depends in part on the severity and duration of the impairment events, but also on
certain inherent characteristics of human perception. The same type of impairment may be
extremely irritating or barely noticed, depending on factors such as the scene content when
the error takes place, and whether it occurs alone or simultaneously with other impairments.

A.1.3.1 Video Content

The visibility of video problems depends partially on the scene content; for example,
frame freezes tend to be much more noticeable in sequences containing high levels of motion
than in relatively static scenes, such as footage of a television news anchor. VQmon/HD
performs content and scene analysis, detecting levels of detail, motion, and panning, and can
detect and provide alerts for content problems such as noise/snow and frame freezes. Scene
analysis data is leveraged to increase the accuracy of VQmon/HD’s estimated perceptual
quality scores.

A.1.3.2 Temporal Phenomena: Reaction Time, Masking and Recency Effects

Perceptual quality is affected somewhat by a natural delay in human reaction time

when impairments occur, or when quality improves after a period of degradation; i.e., the
viewer’s reaction to either event is not immediate. As with audio, when two or more
impairments occur simultaneously (or in rapid succession), there can be a “masking” effect
that affects the way viewers perceive the severity of quality degradation. In addition, a
“recency” phenomenon exists, wherein viewers tend to perceive impairments as more severe
when they occurred recently, but are willing to “forgive” them to some extent as time passes.

VQmon/HD’s quality analysis algorithm considers these temporal phenomena in order to

calculate perceptual quality scores that correlate as accurately as possible to scores obtained
from subjective tests of live viewers.

56
A.2 VQmon/HD Quality Analysis Algorithm *

This section describes VQmon/HD’s video and audio quality analysis algorithms and
lists some of the key metrics reported by VQmon/HD as part of the Telchemy Video Quality
Metrics (TVQM) data set.

A.2.1 Video Stream Analysis

VQmon/HD’s video quality analysis algorithm analyzes RTP or MPEG‐2 Transport

video streams and generates real‐time perceptual quality scores and other diagnostic metrics.

Video codecs supported by VQmon/HD include:

 Motion JPEG
 MPEG‐1
 MPEG‐2
 H.261
 H.263/263+
 H.264
 MPEG‐4
 VC‐1

Figure 2‐7 depicts the various components of the VQmon/HD video quality analysis algorithm
and the metrics produced by each component.

57
 Figure 2‐7.

VQmon/H
D video
stream
quality
analysis
algorithm

A.2.1.1 TR 101 290 Metrics Collection

VQmon/HD collects and reports the full set of Priority 1 and 2 metrics described in TR
101 290 for the measurement and analysis of MPEG‐2 Transport streams.

A.2.1.2 Playout Buffer Emulator

VQmon/HD’s playout buffer emulator component detects lost, duplicate, and out‐of‐
sequence packets and measures packet‐to‐packet delay (jitter) levels, reporting PPDV
(RFC3550) and MAPDV (ITU‐T G.1020). Detailed IPTV packet loss/discard statistics are
provided, along with metrics evaluating the effectiveness of Forward Error Correction (FEC), if
used. If Reliable UDP is in use, VQmon/HD reports the proportion of retransmitted packets
and the impact on bandwidth due to retransmission.

A.2.1.3 Content Analysis

VQmon/HD performs high‐level content analysis to detect levels of detail, motion, and
panning, and can also detect some scene changes. VQmon/HD also recognizes abnormal
content conditions (including no content, noise/snow, and frozen image) and can generate
alerts if they are detected.

58
A.2.1.4 Frame Type Detection

VQmon/HD identifies individual I, P, and B frames in the GoP and measures the packet
loss rate and loss distribution occurring in each frame type. For unencrypted video streams,
VQmon/HD performs picture header decoding to identify individual frames, GoP size, and
frame rate. For encrypted/scrambled streams, heuristic algorithms are applied in order to
detect frame boundaries and measure frame size.

As mentioned in Section 2.1.1.1, the GoP structure has impact on both the efficiency of video
encoding and the robustness of encoded video. VQmon/HD takes the different I, P, and B
frame packet loss/discard rates into account when calculating perceptual video quality
metrics.

A.2.1.5 Per‐frame Quality Analysis

VQmon/HD performs per‐frame quality calculation using the frame type, frame size,
codec type, video bandwidth, and packet loss data. The proportion of each frame type
impaired by loss/discard is reported, along with the proportion of B and P frames impaired
due to the propagation of errors from earlier reference (I or P) frames in the GoP.

A.2.1.6 Perceptual Quality Model

VQmon/HD’s perceptual quality model calculates estimated perceptual quality (MOS)
scores using the per‐frame quality metrics and content analysis as inputs. The calculation
model considers the sensitivity of the content to quality degradation (e.g., that frame freezes
occurring during a high‐motion scene will be more visible and annoying than those occurring
during a static scene) and other subjective factors such as viewer reaction time, recency, and
temporal masking (see Section 2.1.3.2).

A.2.1.7 VQmon Markov Model (VMM)

VQmon/HD uses a four‐state Markov Model to gather and report packet loss statistics
for “burst” periods (where quality is significantly degraded) and “gap” periods (periods
between each burst interval when quality is relatively unimpaired).

59
VQmon/HD calculates quality metrics in these burst and gap states, and then combines them
to generate the overall quality score on user experience. These calculations are based on the
results of numerous subjective voice/video quality analysis tests that indicate there is not an
instantaneous change in perceived quality when a transition between the gap and burst states
occurs, but rather that the perceived quality exponentially “decays” from one level to
another. For example, a 100‐millisecond burst of “noise” is much less annoying than a 10
second burst of “noise”.

Figure 2‐8 illustrates the VQmon 4‐state Markov model. The VQmon 4‐state Markov model is
defined as having the following states and associated transitions:

Table 2‐1. VQmon 4‐state Markov model state transition descriptions

60
 Figure 2‐8.

VQmon Four‐state Markov Model

diagram

A.2.2 Audio Stream Analysis

VQmon/HD’s audio quality analysis algorithm performs real‐time analysis of audio

stream packets and generates perceptual quality scores (MOS‐A) and other metrics
corresponding to those produced by the video quality analysis algorithm. Like the video
stream analysis algorithm, the audio quality analysis algorithm calculates the impact of time‐
varying impairments (i.e., burst packet loss and jitter) and recency on perceptual quality.

VQmon/HD calculates overall multimedia quality (MOS‐AV) by measuring both audio and
video perceptual quality and the accuracy of the audio‐video synchronization.

High‐fidelity audio codecs supported by VQmon/HD include:

 MPEG‐1 Layer 1, 2, and 3

 MPEG‐2 Advanced Audio Coding (AAC)
 AC‐3
 MPEG‐4 AAC, Low Delay AAC, and High Efficiency AAC

61
A.2.3 Telchemy Video Quality Metrics

VQmon/HD provides real‐time perceptual quality scores, performance

statistics, and extensive diagnostic data for monitored video streams in the form of
the TVQM™ (Telchemy Video Quality Metrics) data set.
TVQM metrics reported by VQmon/HD fall into three main categories:

a) Perceptual Quality Metrics – including Mean Opinion Scores (MOS) for picture
quality (MOS‐V), audio quality (MOS‐A), and combined audio‐video quality (MOSAV),
expressed in a range of 1 to 5, with 5 being best. For picture quality, both “Relative”
MOS (which does not consider the resolution of the display, frame rate, or progressive
vs. interlaced scanning) and “Absolute” MOS (which includes consideration of these
factors) are reported.

TVQM perceptual quality metrics also include an Estimated Peak Signal‐to‐Noise Ratio
(ESPR) in dB, and a set of metrics indicating the severity level (on scale of 010) of
several degradation factors including packet loss, jitter, codec type, etc.

b) Video Stream Metrics – including video stream description (image size, codec type,
frame rate, etc.); content and scene analysis (detail and motion level) metrics; frame
statistics indicating the number and proportion of each frame type (I, B, P, SI, and SP)
received/impaired/lost/discarded; average and maximum bandwidth for each frame
type and for the stream overall; video stream jitter and delay metrics; and interval
metrics.

c) Transport Metrics –including VSTQ (Video Service Transmission Quality), a 0‐50

codec‐independent score that measures the ability of the IPTV network to carry
reliable video; packet transport metrics (packets received/discarded/duplicate/out‐of‐
sequence, along with burst and gap statistics); packet jitter metrics including PPDV
(Packet‐to‐Packet Delay Variation); FEC (Forward Error Correction) and Reliable UDP
metrics; and MPEG‐2 Transport Stream (ETSI TR 101 290) metrics.

62
Table below lists some of the perceptual quality metrics reported by VQmon/HD,
including acceptable ranges for each. (A complete list and description of the TVQM
Video Quality Metrics reported by VQmon/HD is provided in Appendix C.)

A.2.3.1 Mean Opinion Scores (MOS)

VQmon/HD reports estimated Mean Opinion Scores (MOS) for picture quality
(MOS‐V), audio quality (MOS‐A), and multimedia quality (MOS‐AV) for each
monitored video stream. MOS scores range from 1 to 5, with 1 considered
“Unacceptable” and 5 “Excellent.” MOS scores are reported as instantaneous (per‐
frame), minimum, maximum, and average values. Interval MOS scores are also
reported as instantaneous and average values.
For picture quality, VQmon/HD reports both Relative MOS‐V and Absolute MOS‐V
scores:

63
 Relative MOS‐V is an estimated perceptual quality score that considers the
effects of codec/quantization level, the impact of IPTV impairments (e.g., packet loss)
on the GoP structure and video content, and the effectiveness of loss concealment
methods—but does not consider the image size/resolution, frame rate, or scanning
method (interlaced vs. progressive).

 Absolute MOS‐V is an estimated perceptual quality score that considers all the
above mentioned factors as well as image resolution, frame rate, and the use of
progressive vs. interlaced scanning.

Some video formats offer inherently higher perceptual quality than others—for
example, in unimpaired conditions, the quality of an HD broadcast will be higher than
that of SD; 1080p better than 1080i or 720p; 60 frames per second better than 30 fps,
etc. VQmon/HD’s Absolute MOS‐V score takes these factors into consideration, and
thus provides an accurate estimate of “overall” perceptual quality.

Because it is independent of image resolution/frame rate, the Relative MOS‐V score

helps provide an indication of video quality relative to the ideal for a given video
format.

Example: An IPTV service provider offers 480i SD, 720p HD, and 1080p HD broadcasts.
Assuming unimpaired conditions, the services might receive the following scores
(values provided for example only):
Absolute MOS‐V: 480i = 4.0 720p = 4.3 1080p = 4.7
Relative MOS‐V: 480i = 4.5 720p = 4.5 1080p = 4.5

Relative MOS‐V can be used to simplify alert thresholding in cases where multiple
video formats are in use.

A.2.3.2 VQmon/HD Video Service Transmission Quality Metric ‐ VSTQ

VQmon/HD produces a video services quality metric, VSTQ (Video Service

Transmission Quality), which is a codec‐independent measure of the ability of the

64
bearer channel to support reliable video. This video service quality metric is expressed
in the range of 0.0 to 50.0, as defined in [14].
VSTQ can be calculated by a mapping function from Peak Signal‐to‐Noise Ratio PSNR
values as follows:

VSTQ = max(0, min(50, (PSNR ‐ 12) * 1.8))

Of course, VQmon/HD uses a more sophisticated algorithm to calculate the resulting

VSTQ
score, which takes time‐varying distributions of network impairments and recency
into account.

Note: The VSTQ score that VQmon/HD’s API function produces actually ranges from 0
to 100. This value is scaled by 2 to get half‐point accuracy for the score.
A.2.3.3 Degradation Factors
To facilitate troubleshooting, VQmon/HD has the ability to obtain a breakdown of
factors contributing to quality degradation. VQmon/HD reports the severity of quality
degradation (on a scale of 0‐10, with 0 indicating no degradation and 10 indicating the
most severe level of degradation) attributable to each of the following factors:

 Packet loss
 Audio‐video sync
 Codec quantization
 GOP length
 Codec bandwidth restriction
 Frame resolution
 Packet discards due to jitter
 Frame rate
 (One‐way) delay
 Recency

*Note: Information in this document is proprietary to Telchemy, Incorporated and was

published with its permission.

65
A.3 IPTV Protocols

 IPTV uses a multilayer protocol stack to deliver the media contents.

 The first 3 layers are typical for IPTV transmission.
 The UDP/RTP session layer is responsible for transferring packets between the
sender and the receiver.
 The MPEG‐TS transport stream layer combines multiple media streams ( video, audio,
data) into a single program transport stream.
 The PES layer assigns video and audio to specific packet streams.
 The application layer performs encoding and decoding of the video and audio using
MPEG‐2, MPEG‐4, VC‐1 or other formats.

66
MPEG frame is fixed at 188 byte, with 184 Data Payload and 4 Byte Header. The 13‐bit PID
indentifies Packet Elementary Stream (PES).

The PES is much longer than the MPEG frame so it is divided into segments over multiple
MPEG frames.

67
The Packetized Elementary Stream is converted into the Video Elementary Stream ( MPEG‐2
transport stream)

68
 PES header contain information about the contents of the PES packet.
 Variable length packets typically up to 64 Kbyte but may be longer
 PTS and DTS allow a decoder to reconstruct the video from I, Band P frames sent by
encoder.
 If header information is corrupted, entire PES packet will be lost.
 Fixed Length packets, multiplexes many PES packets.
 Program Identifier (PID) contains information to find, identify and reconstruct
program contents
 Continuity Counter (CC) – 4 bit counter to detect loss or out of sequence packets
 Program Clock Reference (PCR) – clock reference to time sync the video and audio
 Sync byte set the start of a TS packet and allows synchronization
 Transport Error Indicator indicates a TS packet error

 PAT lists all programs available in the transport stream with their program ID (PID).
 Each program has a PMT that lists the elementary streams for that program.

69
  Errors in PMT, PAT, PID/PSI(Packet Identifier/ Program Specific Information) Errors
will cause the set top box lose contact with the stream. These errors may originate at
any of the MPEG transport multiplexers when new streams are inserted.

A.4 PCR Jitter

PCR (program counter reference) timestamps is generated by the MPEG‐2 encoder

and received throughout the subsequent links in the network at least 100 milliseconds apart.
The decoder uses the PCR to lock its own 27 MHz clock to the encoder system time clock (STC).
This 27Mhz clock is very sensitive to variations caused primarily by packet transmissions
delays. Therefore, measurement of the PCR timestamp arrival time variation called commonly
“PCR jitter” is essential. The 27 Mhz oscillator itself may not exceed 500nsec of internal jitter
per TR 101290.

70
A.5 IGMP Latency vs. Zap time

The zap time is the total duration from the time viewer presses the channel change
button, to the point the picture of the new channel is displayed, along with corresponding
audio. These kind of delays exist in all television systems, but they are greater in digital
television and systems that use the Internet like IPTV. Human interaction with the system is
completely ignored in these measurements, so zap time is not the same as channel surfing.

Key Factors affecting Zap Time

 Encoding
 Network
 Set top Box (leave and joins)
71
Examples:

In this section some typical values of zap time are shown. Actually, in IPTV television these
delays are greater than in other technologies:

 Analog (Cable) ~ 1s
 Analog (off‐air) ~ 1 – 3s
 MPEG2 over QAM ~ 1.2 – 3s
 MPEG2 over QPSK ~ 2 – 4s
 MPEG2 over IPTV Multicast ~ 1.5 – 3.5s
 H.264 over IPTV Multicast ~ 1.7 – 4s

IGMP is the signaling protocol used to access broadcast video services that use a multicast
network design to efficiently manage network bandwidth. In this implementation, a join
message is sent from the STB to the network.

72
The join message asks the network to send the requested program or channel to the STB by
joining a multicast group carrying the desired broadcast channel.

IGMP latency, then, is the time between when the join message is sent and the first video
packet is received by the STB.

This parameter measures network performance, but not the end user's experience with
regard to channel changing time.

The IGMP latency plus the time it takes to fill the decode buffer and to decode and display the
content is the total user experience time. However, the buffer fill time and the decode time
are functions of the network architecture and are not variables. This total time is called Zap
Time.

73
Appendix B Technical Specification
Physical Layers Supported ‐ Average GOP length
10/100/1000 Ethernet, Wi‐Fi 802.11 b/g/n. Max GOP length
Average gap of I frame
Recognized Video Compression Standards Codecs ‐ Frame width
unknown video Frame height
motion JPEG video Frame rate
MPEG‐1 video interlaced
MPEG‐2 video Average number of slices in I frame
ITU‐T H.261 video Reference clock rate
ITU‐T 1996 version of H.263 video Average video bandwidth
ITU‐T 1998 version of H.263+ video Average packet size
ITU‐T H.264 video
MPEG‐4 video Summary Metrics
Microsoft VC1 video Measurement time
MPEG2, MPEG4 part 2&10 (H.264). Average audio/video MOS
Max PCR Jitter
Recognized Audio Compression Standards Codecs – Video packet loss rate
unknown audio TR 101290 errors priority 1 and 2
MPEG‐1 Layer 1 audio IGMP Latency
MPEG‐1 Layer 2 audio
MPEG‐1 Layer 3 audio Video ‐
MPEG‐2 Advanced Audio Coding Video Perceptual Quality Metrics
AC‐3 audio Instantaneous absolute MOS
MPEG‐4 Advanced Audio Coding Minimal absolute MOS
MPEG‐4 Low Delay Advanced Audio Coding Average absolute MOS
MPEG‐4 High Efficiency Advanced Audio Coding Maximum absolute MOS
Instantaneous relative MOS
Signaling Protocols ‐ Minimal relative MOS
Average relative MOS
IGMP versions 2 & 3, RTSP
Maximum relative MOS
Instantaneous audio/video MOS
STB Emulatiom ‐
Minimal audio/video MOS
IGMP Join and Leave request
Average audio/video MOS
IGMP latency
Maximum audio/video MOS
VoD RTSP Play and Stop
Transmission quality:
Connects up to 10 channels(streams)
EPSNR
Monitors up to 10 channels
EPSNR(ATIS)
Video preview ( live full frame SD and HD)
Degradation Factors:
Lost
Summary –
Discarded
Video Description Information
Codec quantization level
IP source address
Codec bandwidth restrictions
IP destination address
Frame resolution
Source port
Frame rate
Destination port
GOP length
Protocols
Available network bandwidth
Codec type
Audio/video synchronization
Group of picture type
Recency
Current GOP length

74
 Video Jitter MPEG2‐TS TR101290 ‐
Frame inter arrival jitter Priority 1 last errors
I frame inter arrival jitter TS sync loss count
Average frame arrival delay Sync byte error count
Peak frame arrival delay PAT error count
PAT2 error count
Video Scene Analysis Metrics Continuity error count
Instantaneous detail level PMT error count
Instantaneous motion level PMT2 error count
Instantaneous panning level PID error count
Static image proportion
High detail proportion MPEG2‐TS TR101290 –
Low detail proportion Priority 2 last errors
High panning proportion Transport error count
Low panning proportion CRC error count
High motion proportion PCR error count
Low motion proportion PCR repetition error count
PCR discontinuity error count
Packets – PCR accuracy error count
Transport Packets Metrics ‐
Packets received Audio –
Lost Audio Description Information
Discarded IP source address
Corrected IP destination address
Out of sequence Source port
Duplicated Destination port
Lost % Type
Corrected % Transport protocol
Discarded % Codec Type
Out of sequence % Number of channels
Duplicated % Reference clock
Burst count Audio Perceptual Quality Metrics
Burst loss rate Minimal MOS
Average burst length [pkts} Average MOS
Gap count Maximum MOS
Gaps loss rate Instantaneous MOS
Average gap length [pkts] Degradation factors
Lost
Jitter Metrics (RFC3550) ‐ Discarded
Packet to packet delay variation Codec
Max packet to packet delay variation Recency
Audio bandwidth metrics
RTP Packets Metrics (Carrying MPEG2‐TS)‐ Avg audio bandwidth
Packets Received Peak audio bandwidth
Corrected Avg audio bandwidth( incl headers)
Lost Peak audio bandwidth( incl headers)
Discarded
Out of sequence
Duplicated
With timestamp errors

75
Video Frames I,P,B,SI,SP – Video bandwidth of I,P,B, SI and SP frames –
Frames received Average video bandwidth
Frames impaired Max video bandwidth
% frames impaired
Packets received Histograms (charts)
Packet lost Up to two charts from any metrics
Packet discarded
% packets impaired LAN Port ‐
%packets impaired by error propagation IP address
Except for I and SI Downstream rate
Upstream rate

76
Troubleshooting
Contacting the Technical Support GroupTo obtain after‐sales service or technical
support for this product, contact EXFO at one of the following numbers. The Technical Support
Group is available to take your calls from Monday to Friday, 8:00 a.m. to 7:00 p.m. (Eastern
Time in North America).

For detailed information about technical support, visit the EXFO Web site at www.exfo.com.

Technical Support Group

400 Godin Avenue 1 866 683‐0155 (USA and Canada)
Quebec (Quebec) G1M 2K2 Tel.: 1 418 683‐5498
CANADA Fax: 1 418 683‐9224
[email protected]

77
Glossary

Perceptual Quality Metrics

Metric Description
MOS‐V Video MOS, a 1‐5 score that considers the effect of the video codec,
frame rate, packet loss distribution and GoP structure on viewing quality
MOS‐A1 and MOS‐A2 for 2 codecs that Audio MOS, a 1‐5 score that considers the effect of the audio codec, bit
are present in some streams rate, sample rate and packet loss on viewing quality
MOS‐A1V Audio‐Video MOS – a 1‐5 score that considers the effect of picture &
audio quality and audio‐video synchronization on overall user experience
Video Service Transmission Quality Transmission quality, a 0‐50 codec independent score measuring the
(VSTQ) ability of the IPTV network to carry reliable video
Video Service Picture Quality (VSPQ) Picture quality, a 0‐50 score that considers the effect of the video codec,
frame rate, packet loss distribution and GoP structure
Gap VSPQ Picture quality during “good” periods when little or no degradation is
occurring
Burst VSPQ Picture quality during “bad” periods when significant degradation is
occurring
Video Service Audio Quality for 2 Audio quality, a 0‐50 score that considers the effect of the audio codec,
codecs (VSA1Q and VSA2Q) bit rate, sample rate and packet loss on viewing quality
Video Service Multimedia Quality Multimedia (Audio‐Video) quality, a 0‐50 score that considers the effect
(VSMQ) of picture & audio quality and audio‐video synchronization on overall
user experience
Estimated PSNR (EPSNR) Estimated Peak Signal to Noise Ratio (PSNR) expressed in dB. This is an
estimate of the distortion that has occurred between the source video
stream and the output video stream.
Degradation Factors Percentage degradation in quality due to (each of) Loss, Discard, Codec
type, Audio‐Video Sync, Delay and Recency.

78
 Video Stream Metrics

The Video Stream Description provides information on the type of codec being used, Group of
Pictures structure and length, image size and other key factors.

Video Stream Description

Metric Description
Codec type Type of codec (e.g. MPEG4)
GoP type Group of Pictures type (e.g. IBBP…)
GoP length Number of frames in Group of Pictures

Image size Image size in pixels (X x Y) (autodetected *) and number of frames per
second

Video Stream Metrics provide insight into the proportion of different type of video frame that
are impacted by packet loss and discard, and to the overall video bandwidth.

Video Stream Metrics

Metric Description
I, P, B frame packets received Counts of the numbers of I, P and B frame packets received
I, P, B frame packets lost Counts of the numbers of I, P and B frame packets
I, P, B frame packets discarded Counts of the numbers of I, P and B frame packets

79
Transport Metrics

Packet Loss Metrics provide essential data on IPTV packet loss before and after the effects of
error correction (such as FEC or Reliable UDP). Burst and gap statistics provide valuable
insight into the time distribution of lost and discarded packets.

MPEG-TS Packet Loss Metrics

Metric Description
Packet Loss Rate Percentage of MPEG‐TS packets lost in the network
Packet Discard Rate Percentage of packets discarded due to late arrival
Out of Sequence Packet Rate Percentage of packets arriving out of sequence
Duplicate Packet Rate Percentage of duplicate packets
Burst Loss Rate Percentage of packets lost within burst periods
Burst Length Average length of burst periods
Gap Loss Rate Percentage of packets lost within gap periods
Gap Length Average length of gaps between bursts
PCR (Program Clock Reference) Jitter Metrics
Metric Description
PCR Jitter Program Clock Reference Jitter
MDI (Media Delivery Index) Metrics
Metric Description
MDI Delay Factor Media Delivery Index Delay Factor is the maximum difference, observed
at the end of each media stream packet, between the arrival of media
data and the drain of media data
MDI Media Loss Rate Media Delivery Index Loss Rate is the count of lost or out‐of‐order packets
carrying streaming application information over a selected time interval,

80
TR101 290 metrics provide green LED ON/OFF information on certain key error types that
occur with MPEG Transport protocols, and are useful in identifying and resulting these error
conditions.

TR 101 290 MPEG Metrics

Metric Description
TS_sync_loss Loss of synchronization at MPEG transport layer
Sync_byte_error Invalid MPEG transport sync byte
Continuity_count_error Incorrect packet order, duplicate packet or lost packet
Transport_error Transport error indicator in MPEG transport header set

PCR_repetition_error Time interval between two successive PCR values more than 40ms
PCR_discontinuity_indicator_error Difference between two consecutive PCR values is over 100ms
without discontinuity bit set
PTS_error Interval between presentation time stamps more than 700ms

81
PN: 1062992

www.EXFO.com · [email protected]
CORPORATE HEADQUARTERS 400 Godin Avenue Quebec (Quebec) G1M 2K2 CANADA
Tel.: 1 418 683-0211 · Fax: 1 418 683-2170

EXFO AMERICA 3400 Waterview Parkway Suite 100 Richardson, TX 75080 USA
Tel.: 1 972-761-927 · Fax: 1 972-761-9067

EXFO EUROPE Omega Enterprise Park, Chandlers Ford, Hampshire S053 4SE ENGLAND
Electron Way Tel.: +44 2380 246810 · Fax: +44 2380 246801

EXFO ASIA-PACIFIC 100 Beach Road, SINGAPORE 189702

#22-01/03 Shaw Tower Tel.: +65 6333 8241 · Fax: +65 6333 8242

EXFO CHINA Beijing Global Trade Center, Tower C, Beijing 100013 P. R. CHINA
Room 1207, 36 North Third Ring Road Tel.: +86 (10) 5825 7755 · Fax: +86 (10) 5825 7722
East, Dongcheng District

EXFO SERVICE ASSURANCE 270 Billerica Road Chelmsford MA, 01824 USA
Tel.: 1 978 367-5600 · Fax: 1 978 367-5700

EXFO NETHAWK Elektroniikkatie 2 FI-90590 Oulu, FINLAND

Tel.: +358 (0) 403 010 300 · Fax: +358 (0) 8 564 5203

TOLL-FREE (USA and Canada) 1 800 663-3936

© 2012 EXFO Inc. All rights reserved.

Printed in Canada (2012-04)

