Manual - Swissqual Kpi Users Guide
Manual - Swissqual Kpi Users Guide
Manual - Swissqual Kpi Users Guide
Manual
August 2009
SwissQual License AG
Allmendweg 8
CH-4528 Zuchwil
Switzerland
Internet: http://www.swissqual.com
Office: +41 32 686 65 65
Fax: +41 32 686 65 66
No part of this publication may be copied, distributed, transmitted, transcribed, stored in a retrieval system,
or translated into any human or computer language without the prior written permission of SwissQual AG.
SwissQual has made every effort to ensure that eventual instructions contained in the document are
adequate and free of errors and omissions. SwissQual will, if necessary, explain issues which may not be
covered by the documents. SwissQuals liability for any errors in the documents is limited to the correction of
errors and the aforementioned advisory services.
When you refer to a SwissQual technology or product, you must acknowledge the respective text or logo
trademark somewhere in your text.
SwissQual, Seven.Five, SQuad, QualiPoc, NetQual, VQuad as well as the following logos are
registered trademarks of SwissQual AG.
SwissQual acknowledges the following trademarks for company names and products:
Adobe, Adobe Acrobat, and Adobe Postscript are trademarks of Adobe Systems Incorporated.
Intel, Intel Itanium, Intel Pentium, and Intel Xeon are trademarks or registered trademarks of Intel
Corporation.
Microsoft, Microsoft Windows, Microsoft Windows NT, and Windows Vista are either registered
trademarks or trademarks of Microsoft Corporation in the United States and/or other countries U.S.
Contents
1 Introduction........................................................................................................................ 1
Related Documentation ....................................................................................................... 1
KPI Overview ....................................................................................................................... 2
Accessibility..................................................................................................................... 2
Retainability................................................................................................................... 10
Integrity.......................................................................................................................... 12
2 Services ............................................................................................................................ 20
Voice.................................................................................................................................. 20
GSM/UMTS ...................................................................................................................20
CDMA ............................................................................................................................ 21
Data ................................................................................................................................... 21
TCP/IP ........................................................................................................................... 21
Email.............................................................................................................................. 22
FTP................................................................................................................................ 23
HTTP ............................................................................................................................. 24
Ping ............................................................................................................................... 25
UDPDL ..........................................................................................................................25
WAP .............................................................................................................................. 26
Messages .......................................................................................................................... 26
MMS .............................................................................................................................. 26
SMS............................................................................................................................... 27
Mobile TV........................................................................................................................... 27
DVB-H ........................................................................................................................... 27
3 Indicators on Layer 3 ......................................................................................................28
Data (PS) ........................................................................................................................... 28
Voice (CS) ......................................................................................................................... 29
2G.................................................................................................................................. 29
3G.................................................................................................................................. 29
Active set update...........................................................................................................29
Cell Update....................................................................................................................31
4 Indicators on Layer 1 ......................................................................................................32
Scanner Based Measurements ......................................................................................... 32
Scrambling Code Coverage .......................................................................................... 32
Pilot Pollution................................................................................................................. 32
Radio based Measurements..............................................................................................32
Radio Link .....................................................................................................................32
Power Control................................................................................................................33
ii | Contents
NQDI KPIs Manual
2000 - 2009 SwissQual AG
A Appendix .......................................................................................................................... 34
KPI Values ......................................................................................................................... 34
KPI Results Table..........................................................................................................34
Accessibility Records.........................................................................................................34
Voice.............................................................................................................................. 34
Data Services ................................................................................................................ 39
Messages ......................................................................................................................51
Data call setup...............................................................................................................56
Security (CS) .................................................................................................................61
Channels .......................................................................................................................62
Coverage.......................................................................................................................66
Retainability Records.........................................................................................................67
Voice.............................................................................................................................. 67
Data Services ................................................................................................................ 68
Messages ......................................................................................................................78
Data Call........................................................................................................................ 79
Resource .......................................................................................................................81
Integrity Records................................................................................................................ 82
Telephony......................................................................................................................82
Data ............................................................................................................................... 82
Messages ......................................................................................................................91
Handover.......................................................................................................................94
Resource .....................................................................................................................101
Mobility ........................................................................................................................ 102
Resource .....................................................................................................................105
WCDMA.......................................................................................................................105
Status............................................................................................................................... 106
B Appendix ........................................................................................................................ 108
KPI Calculation ................................................................................................................108
Success Rate ..............................................................................................................108
Rejection Rate.............................................................................................................108
Time............................................................................................................................. 108
PDF ............................................................................................................................. 108
CDF ............................................................................................................................. 109
95% Cut-Off point........................................................................................................109
C Appendix ........................................................................................................................ 110
References ...................................................................................................................... 110
Figures
Figure 2-1 Trace and Graph of a CS call on UMTS .......................................................................20
Figure 2-2 Trace and Graph of a CS call on GSM .........................................................................20
Contents | iii
NQDI KPIs Manual
2000 - 2009 SwissQual AG
iv | Contents
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Contents | v
NQDI KPIs Manual
2000 - 2009 SwissQual AG
1 Introduction
This document contains a description of all the KPIs (Key Performance Indicators) measured and
reported with Diversity and QualiPoc data in NQDI. This list of KPIs represents a summary from
inputs from the Experts within ETSI, SwissQual KPI definitions and various customer requests.
THhe structure of the KPIs follow the principles of the document ITU P-800 where the
accessibility, retainability and integrity are the key aspects of the assessment of quality in
telecommunications.
The concept of accessibility has been enlarged to accommodate customer requests for KPIs
considering the completion of a service. It spans from the request of a service to its successful
completion. Since the access to the service is essential to its completion these KPIs are regarded
as belonging to the accessibility group.
The described KPIs are to be found on the KPI sheet of the NQDI Analysis tab and as a report
item in the Report Generator. The usage of the Report Generator is described in detail in the
NQDI manual.
The calculation of the KPIs is automatically when the trigger points are available. Depending on
the data collection application type and mode used (Diversity, QualiPoc) maybe only a limited
choice of KPIs is available.
Related Documentation
For a description of the NQDI Database Structure, see: Manual-NQDI database structure.pdf
For a description of the Voice Test Measurement Results, see: Manual-Result Description.pdf
For a description of the Data Test Measurement Results, see: Manual-Data Test Result
description.pdf
Chapter 1 Introduction | 1
NQDI KPIs Manual
2000 - 2009 SwissQual AG
KPI Overview
The following legend is for the KPI tables in this section.
Item Description
IP Protocol
Layer 3 Message
Scanner
Accessibility
Accessibility Parameter ETSI Parameter
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
Channel Request /
RRCConnectionRequest
10100 Telephony (MTC Dial) CC: Alerting SuccessRate Duration SA-T ST-T
Voice (CS)
Service
Telephony
10102 Service CM Service Request CC: Alerting SuccessRate Duration
Video Call
10105 Video Setup RRCConnectionRequest Call Marker Video Connect SuccessRate Duration
2 | Chapter 1 Introduction
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
Paging Delay
10141 MTC Dial Paging/RRC Paging Type 1 SuccessRate Duration
Telephony
10150 MTC Paging CC: Alerting SuccessRate Duration
Telephony
MTC Channel Request /
10151 intermediate RRCConnectionRequest CC: Alerting SuccessRate Duration
Telephony
MTC Request
10152 Delay Paging Channel Request / RRCConnectionRequest SuccessRate Duration
Telephony
Random
10153 Access Dial Channel Request/RRCConnection Request SuccessRate Duration
Telephony
Completion
10160 MTC Dial Channel Release / RRC Connection Release SuccessRate Duration
Telephony
Completion
10165 MTC Paging Paging Channel Release / RRC Connection Release SuccessRate Duration
IPST-
Data
10200 Email send > 25 [SYN] Message Body (after CommandReset) SuccessRate Duration IPSA-PSD PSD
IPST-
10201 Email receive > 110 [SYN] Response:+ OK and octets or bytes SuccessRate Duration IPSA-PSD PSD
Chapter 1 Introduction | 3
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
IPST-
10300 FTP UL > 21 [SYN] First FTP Data SuccessRate Duration IPSA-PSD PSD
IPST-
10307 FTP UL (B) > 21 [SYN] Reception of [Ack] from the [Syn;Ack] SuccessRate Duration IPSA-PSD PSD
FTP UL IPST-
10310 Completion > 21 [SYN] Reception of [Fin;Ack] SuccessRate Duration IPSA-PSD PSD
IPST-
10301 FTP DL > 21 [SYN] First FTP Data SuccessRate Duration IPSA-PSD PSD
IPST-
10308 FTP DL (B) > 21 [SYN] Reception of [Ack] from the [Syn;Ack] SuccessRate Duration IPSA-PSD PSD
FTP DL IPST-
10311 Completion > 21 [SYN] Last FTP Data SuccessRate Duration IPSA-PSD PSD
IPST-
10400 HTTP Browser > 80 [SYN] HTTP/1.1200 OK or HTTP/1.0 200 OK SuccessRate Duration IPSA-PSD PSD
HTTP Transfer
10402 Put > 80 [SYN] PUT SuccessRate Duration
4 | Chapter 1 Introduction
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
HTTP Transfer
GET
10411 Completion > 80 [SYN] Last Continuation SuccessRate Duration
HTTP Transfer
PUT
10412 Completion > 80 [SYN] Reception of [Fin;Ack] SuccessRate Duration
Service Access
HTTP Browser IPST-
10420 (B) Start RAS Dialup Sending of the first GET command SuccessRate Duration IPSA-PSD PSD
Service HTTP
Transfer GET IPST-
10421 (B) Start RAS Dialup Sending of the first GET command SuccessRate Duration IPSA-PSD PSD
Network
Access WAP Activate PDP Context
10430 Portal Request Last HTTP/1.n 200 OK SuccessRate Duration QP
Channel Request
RRCConnectionRequest
HTTP Network
10450 DL Cycle HTTP Get HTTP/1.n complete SuccessRate Duration QP
HTTP Page DL
10451 Cycle HTTP Get Last HTTP/1.n complete SuccessRate Duration QP
HTTP File DL
10452 Cycle HTTP Get Last HTTP/1.n complete before new HTTP Get SuccessRate Duration QP
Chapter 1 Introduction | 5
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
Network
Access Video Activate PDP Context
10610 Streaming Request First Packet SuccessRate Duration QP
10700 WAP WSP Connect WSP Reply/ WSP Result SuccessRate Duration
WAP
10710 Completion WSP Connect Last WSP Reply/ WSP Result SuccessRate Duration
NOTIFICATION
10801 MMS Receive RECEIVED TRANSMISSION STARTED SuccessRate Duration
AD
SMS-
10900 SMS Send START SEND SMS START SuccessRate Duration SA SMS MO MO
6 | Chapter 1 Introduction
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
SIP Call
11010 Access SIP INVITE SIP 200 OK SuccessRate Duration
Mobile TV
11600 DVB-H Menu Service Request Service available (DVB-H Service Available) SuccessRate Duration
DVB-H
Mobile TV
DVB-H
11610 Channel Channel Request Streaming Started (DVB-H Channel added) SuccessRate Duration
ST-
15000 Data (CS) Telephony AT-Command Valid Server Response SuccessRate Duration SA-CSD CSD *
Attach
Data (PS)
15100 Attach Attach Request Attach Accept or Reject SuccessRate Duration Attach FR ST
Attach
15102 Completion Attach Request Attach Complete SuccessRate Duration
Chapter 1 Introduction | 7
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
Channel Request /
Deactivate RRCConnectionRequest Deactivate PDP Context Accept
PDP Context
15203 Completion SuccessRate Duration
15400 Ciphpering Ciphering Mode command Ciphering Mode complete SuccessRate Duration
15500 Authentification Authentification Request Authentification Response or Failure or Reject SuccessRate Duration
15600 Traffic Channel Assignment Command Assignment Complete or Failure SuccessRate Duration
15700 Signal Channel Channel Request Immediate Assignment or IA Reject SuccessRate Duration
Call Access
CDMA
16310 Traffic Channel Origination Service Connect Completion SuccessRate Duration
8 | Chapter 1 Introduction
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
DCCH-Security Mode
15900 Security Command DCCH-Security Mode Complete or Failure SuccessRate Duration
3G
16000 Signal Channel RRCConnectionRequest RRCConnectionSetupComplete SuccessRate Duration
Signal Channel
16050 Completion RRCConnectionRequest RRCConnectionRelease SuccessRate Duration
NA CS /
16100 Coverage C1 > 0 SuccessRate NA PS *
Layer 1 2G
GPRS
16200 Availability Selection of new Cell SI 13 on BCCH SuccessRate Unavailabilty
Chapter 1 Introduction | 9
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Retainability
Retainability Parameter ETSI Parameter
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
20200 Data Email send First Message Body Response: Message accepted - half roundtrip SuccessRate Duration
20201 Email receive Octets - half roundtrip Last Continuation SuccessRate Duration
20205 Email send First Message Body Response: Message accepted SuccessRate Duration DTCoR-PSD
(DTCoR-
20300 FTP UL FTP Data [FIN;ACK] - half roundtrip SuccessRate Duration PSD)
Service
20301 FTP DL FTP Data - half roundtrip Last FTP DATA SuccessRate Duration DTCoR-PSD
(DTCoR-
20305 FTP UL(V) FTP Data First [FIN;ACK] after Last FTP DATA SuccessRate Duration PSD)
20306 FTP DL(V) FTP Data Last FTP DATA SuccessRate Duration DTCoR-PSD
10 | Chapter 1 Introduction
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
HTTP Browser
20405 (V) HTTP/1.1 200 OK Last Continuation SuccessRate Duration DTCoR-PSD
Messag (MMS
e NOTIFICATION MMS RFR RT
20801 MMS Receive RECEIVED TRANSMISSION FINISHED SuccessRate Duration (MT) (MT))
20900 SMS SEND SMS START SEND SMS FINISHED SuccessRate Duration
Echo (Ping)
21000 Data request Echo (Ping) reply SuccessRate Duration RTT Ping
Chapter 1 Introduction | 11
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Success
KPI ID Stratum Type Function Start Trigger Event End Trigger Event Status Value 1 Time
Rate
Data Telephony
25000 (CS) (Data CS) Valid Server Response Intentional Termination SuccessRate Duration CR-CSD *
Data
PDP Context Activate PDP Context
(PS)
25201 End Accept Dectivate PDP Context Accept SuccessRate Duration
RRCConnectionSetupCo
3G
25400 Network mplete RRCConnection Release SuccessRate Duration
HSDPA RadioBearerSetup
3.5G Activity
RadioBearerReconfigurati RRCConnectionRelase, CellChangeOrder
26100 on formUTRAN, RadioBearerReconfiguration
Integrity
Integrity Parameter ETSI Parameter
Voice
(CS)
12 | Chapter 1 Introduction
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Transmission Duration
30201 Email receive 20201 Transferred Data Throughput MDR-PSD
Transmission Duration
30205 Email send 20205 Transferred Data Throughput MDR-PSD
Transmission Duration
30206 Email receive 20206 Transferred Data Throughput MDR-PSD
Transmission Duration
30306 FTP DL(V) 20306 Transferred Data Throughput MDR-PSD
Transmission Duration
30308 FTP DL(B) 20308 Transferred Data Throughput MDR-PSD
Chapter 1 Introduction | 13
NQDI KPIs Manual
2000 - 2009 SwissQual AG
[SYN] Displaying
Video Streaming
30605 First Frame Success Rate Duration
[SYN] Resize
Video Streaming
30607 Setup Buffering Success Rate Duration
TRANSMISSION MMS
30803 MMS Sent Time STARTED (MO) TRANSMISSION FINISHED (MO) Success Rate Duration MMS NFR NT
14 | Chapter 1 Introduction
NQDI KPIs Manual
2000 - 2009 SwissQual AG
[SYN] [SYN,ACK]
31000 TCP RTT Duration
DNS Host
31100 resolution Standard query A% Standard response A% Success Rate Duration
TMSIReallocationComma
Layer3
34050 Handover Handover Command Handover Complete or Failure Success Rate Duration
Channel Modify
34060 2G 2G Channel Mode Modify Channel Mode Modify Acknowledge Success Rate Duration
Intracell
34070 Handover Assignment Command Assignment Complete or Failure Success Rate Duration
35003 2G/3G Location Location Update Request Location Updating Accept or Reject Success Rate Duration
Chapter 1 Introduction | 15
NQDI KPIs Manual
2000 - 2009 SwissQual AG
All tgps_Status
deactivate/HandoverFromUTRANCommand-
35010 Compress Mode tgps_Status activate GSM Success Rate Duration
InterSystemHO CellChangeOrderFromUT
35041 RAU RAN Routing Area Update Complete or Reject Success Rate Duration
InterSystemHO CellChangeOrderFromUT
35042 Interruption RAN First IP Packet Success Rate Duration *
Last GSM
SystemInformation before
35050 InterSystemHO CellChange CellChange Success Rate Duration *
Last GSM
SystemInformation or
35060 InterSystemHO Paging before CellChange First UMTS RRC Message Success Rate Duration
PTMSIReallocationComm
35080 P-TMSI and PTMSIReallocatioinComplete Success Rate Duration
16 | Chapter 1 Introduction
NQDI KPIs Manual
2000 - 2009 SwissQual AG
3G- Inter
Frequency
35105 Reselection Last old UARFCN report First new UARFCN report Success Rate Duration
3G Inter PhysicalChannel/Transpor
Frequency tChannel/RadioBeareReco PhysicalChannel/TransportChannel/RadioBear
35106 Handover nfiguration eReconfigurationComplete Success Rate Duration
RadioBearerReconfigurati
on with RadioBearerReconfigurationComplete
HSPA Cell HSDSCH_RL_indicator
35110 Change falseand true Success Rate Duration
RadioBearerReconfigurati RadioBearerReconfigurationComplete or
35200 Resource on Failure Success Rate Duration
Routing Area
35300 Update URAUpdate URAUpdateConfirm Success Rate Duration
Physical PhysicalChannelReconfig
35400 Channel uration PhysicalChannelReconfigurationComplete Success Rate Duration *
Signaling SignalingChannelReconfig
35500 Channel uration SignalingChannelReconfigurationComplete Success Rate Duration *
Transport TransportChannelReconfi
35600 Channel guration TransportChannelReconfigurationComplete Success Rate Duration *
Chapter 1 Introduction | 17
NQDI KPIs Manual
2000 - 2009 SwissQual AG
EDGE Coding
Scheme Coding Scheme Duration Change Coding Scheme Duration +
Current Server
C/I Measurement Sample Value +
Current Server
RxLev Measurement Sample Value +
Best Server
RxLev Measurement Sample Value +
18 | Chapter 1 Introduction
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Coverage area
per SC Measurement Sample Value +
Pilot pollution +
Chapter 1 Introduction | 19
NQDI KPIs Manual
2000 - 2009 SwissQual AG
2 Services
Voice
The performance KPIs for circuit switched (voice) calls are obtained with the help of a set of
trigger points that can be found in the layer 3 communication between the MS and the network or
as Call markers in the case of a call from the land unit.
GSM/UMTS
The call setup success rate and time is calculated for UMTS (GSM) by using the
RRCConectionRequest (Channel Request in GSM) and the alerting message for the MOCs. The
call retained rate and duration is calculated using the alerting and the disconnect message.
20 | Chapter 2 Services
NQDI KPIs Manual
2000 - 2009 SwissQual AG
from different vendors. To keep the comparability between different models the start trigger point
is the first request for a signalisation channel.
CDMA
In CDMA there is no layer 3 signalisation that the call is connected with the called party therefore
there is no call setup KPI. However there is one for the access. But since the sequence of layer 3
messages in CDMA voice service differs from UMTS/GSM it has separate KPIs with respective it
trigger points.
Data
The data tests make use of three different protocols: TCP, UDP and WAP. Predominant is the
TCP or TCP/IP protocol that supports various known applications. UDP is tested on the protocol
level and with WAP the application and the protocol is identical.
TCP/IP
The TCP/IP connection consists in general of three different phases:
Establishment, transfer and disconnect. In the Data tests individual applications are investigated
(such as FTP up- and download, Email sent and receive and HTTP). While FTP has an
establishment as well as a disconnect phase, the others maybe doesnt have either.
The connection establishment on the TCP level is shown in Figure 2-4 where the three phases
are clearly distinguishable. Every TCP connection has an establishment phase. However a HTTP
server can end a connection without the exchange of packets with the FIN flag set.
Chapter 2 Services | 21
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Email
The process of the test is traced on the protocol level, which is presented in the protocol info tab
in NQDI. This trace represents the basis of the evaluation of the test. The steps of the Email send
test are shown below.
22 | Chapter 2 Services
NQDI KPIs Manual
2000 - 2009 SwissQual AG
The establishment of the TCP connection is triggered by the invoke of the protocol. E.g. > telnet
110
n the contrary because SMTP is running on the server, the start of the application is the actual
upload command to the server.
To obtain the correct TCP download duration, it is necessary to add a half-roundtrip time. This is
due to the consideration, that the last packet needs a finite time to arrive at its destination. This
addition may not match the exact delay of the packet, but it renders the result more accurate as if
there were no additional time considered.
For the Email Receive this additional time is half the time between the request and the first
packet.
FTP
During the FTP tests a secondary data connection is established. (In the figure below, the trace of
this secondary connection is shown with smaller arrows)
The items in brackets like (ftp) reflect commands that may not occur in the trace but have to be
executed to trigger the establishment of the TCP connection. In the FTP uplink test the last packet
containing data has the FIN flag set. This indicates that the client closes the connection.
The trigger points to calculate the TCP throughput are indicated in the following picture. It is
calculated from the first to the last FTP Data packet and in addition the time, which the first packet
needs to arrive at the client.
Chapter 2 Services | 23
NQDI KPIs Manual
2000 - 2009 SwissQual AG
HTTP
According to the RFC 2126 a HTTP server does not need to close the TCP connection with the
setting the FIN flag. The browser can identify the end of the HTTP packet sending within the
packet itself.
24 | Chapter 2 Services
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Ping
The ping does not use the TCP protocol but is a command of the ICMP protocol. The round trip
time of a ping represents the time elapsed between the sending of an ICMP echo request and the
answer of the destination. In the protocol trace, where these events are taken from, it is calculated
by the difference between a request and the subsequent reply.
UDPDL
The UDPDL test is a pure protocol test. There is no known application used in this test. The
access duration is the time between sending the SQUDPDL command to the server and the
arrival of the first packet.
Chapter 2 Services | 25
NQDI KPIs Manual
2000 - 2009 SwissQual AG
WAP
Contrary to the applications described above the WAP (wireless application protocol) test has no
separate transport protocol.
Messages
MMS
The multi media messaging service (MMS) is an application of WAP and SMS. The WAP
capabilities are used to transfer the data of the message while the SMS serves as the notification.
26 | Chapter 2 Services
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Although is consists of known functionalities, this service is highly specific for mobile equipment
and is therefore tested individually.
SMS
Like MMS, a SMS is separated in a sending and retrieving part. For the reception it is less evident
since we cannot speak of a process here. With the indication of a SMS the actual text is retrieved.
There is no download- time available.
Mobile TV
DVB-H
Digital Video Broadcast (DVB) is a transmission system to delivier digital TV. The H in the name
is standing for the handheld variant, which implements a lot of special techniques particularly
addressing the limitations of mobile. Although DVB-H uses IP as a tranport layer there is no
direct access to with the consequence that the application layer information items are used as
trigger events for DVB-H KPIs
Chapter 2 Services | 27
NQDI KPIs Manual
2000 - 2009 SwissQual AG
3 Indicators on Layer 3
The desired times and the success rates calculating the layer 3 KPIs are based on the messages
that are exchanged between the MS and the network. The phone gives a time wise and exact
trace of these messages as they are sent and received by the mobile station. The messages can
be seen in detail in the collection of data as well as in their post processing.
Data (PS)
The KPIs for data calls are obtained by using the resource request (RRCConnectionRequest for
UMTS and Channel Request for GSM), the PDP context activation accept message and the PDP
deactivation accept message. PDP context setup time is the difference between the
RRCConnectionRequest and the PDP context activation accept message, while the PDP context
holding time is the difference between the occurrence of the PDP context activation accept
message and the PDP context deactivation accept message.
Voice (CS)
2G
The accessibility parameter for the GSM system is the acquisition of the traffic channel and the
Signal channel
Figure 3-4 Signal channel and traffic channel assignment and release on GSM
Besides the establishment messages and the retain ability indicators there are other KPIs for the
integrity of a call.
3G
Active set update
To change the active set of the MS the network orders an active set update over the DCCH. In
case of a successful update the MS station sends an active set update complete.
Cell Update
The cell update, the location update and the routing area update procedures are also traced via
the RRC messages.
4 Indicators on Layer 1
The indicators from layer 1 are used for items of the report generator they are most of the times
not explicit KPIs with their own ID but measurements.
Pilot Pollution
If at a given time more than three CPICHs are stronger than the best CPICH minus a 15 dB
threshold the situation is considered pilot pollution. The report item shows the percentage of time
there was pilot pollution.
The information about the changes in the active set is be found the RRC messages where the
ActiveSetUpdate message indicates the type radio link change ordered by the network. In case a
addition information list and a removal information list is present in the message the order is a
radio link replacement if only the former is present its a radio link addition if only the latter the
network orders a radio link removal
Power Control
The phone provides information about the UE Tx Power several times a second and about the
block error rate (BLER)in lower frequency. The distribution of the obtained values is calculated
using all measured samples.
A Appendix
KPI Values
All KPI values are stored in the NQDI Database in the KPI Result Table.
Accessibility Records
Voice
Field Value Description
34 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 35
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Sum 0 Unused
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Name of the start event
Value4 Name of the end event
Value5 Unused
36 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 37
NQDI KPIs Manual
2000 - 2009 SwissQual AG
38 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Data Services
Field Value Description
Appendix A | 39
NQDI KPIs Manual
2000 - 2009 SwissQual AG
40 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 41
NQDI KPIs Manual
2000 - 2009 SwissQual AG
42 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 43
NQDI KPIs Manual
2000 - 2009 SwissQual AG
44 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 45
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value1 Unused
Value2 Unused
Value3 Varchar Name of the start event
Value4 Varchar Name of the end event
Value5 Unused
46 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 47
NQDI KPIs Manual
2000 - 2009 SwissQual AG
48 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 49
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar Name of the start event
Value4 Varchar Name of the end event
Value5 Unused
50 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Messages
For the detailed mapping between the events in the log and the trigger points in the application
see Data and Video Test Scenario Manual section MMS send and MMS receive.
StartTime Timestamp of the log message START, which indicates the begin
of an MMS Send Test.
EndTime Timestamp of the log message TRANSMISSION STARTED, which
indicates the begin of an MMS transmission
Duration Time in ms between StartTime and EndTime
Status 0 = OK, see Status in the appendix
Sum 0 Unused
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar StartTime event message
Value4 Varchar EndTime event message
Value5 Unused
Appendix A | 51
NQDI KPIs Manual
2000 - 2009 SwissQual AG
9 o---------------------------wtp ACK---------------------->>> 10
11 o---------------------MMS m-send.req------------------->> 12
14 <<<-----------------------wtp ACK-------------------------- 13
15 o-----------------------MMS m-send.req----------------->> 16
18 <<<-------------------MMS m-send.conf-------------------o 17
52 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 53
NQDI KPIs Manual
2000 - 2009 SwissQual AG
54 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 55
NQDI KPIs Manual
2000 - 2009 SwissQual AG
56 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 57
NQDI KPIs Manual
2000 - 2009 SwissQual AG
58 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar Unused
Value4 Varchar Name of the end event
Value5 Unused
Appendix A | 59
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value2 Unused
Value3 Varchar Unused
Value4 Varchar Unused
Value5 Unused
60 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Security (CS)
Field Value Description
Appendix A | 61
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Channels
Field Value Description
62 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 63
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value5 Unused
64 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 65
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Coverage
Field Value Description
66 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Retainability Records
Voice
Field Value Description
Appendix A | 67
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Data Services
Field Value Description
68 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 69
NQDI KPIs Manual
2000 - 2009 SwissQual AG
70 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
StartTime Timestamp of the first FTP Data message which indicates the first
transmitted data packet
EndTime Timestamp of the message 21 [FIN;ACK]
Duration Time in ms between StartTime and EndTime
ErrorCode 0 = OK, see Status in the appendix
Sum 0 Unused
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar Unused
Value4 Varchar Name of the end event
Value5 Unused
Appendix A | 71
NQDI KPIs Manual
2000 - 2009 SwissQual AG
72 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 73
NQDI KPIs Manual
2000 - 2009 SwissQual AG
StartTime Timestamp of the first Source port message which is the connect
ack message for the SwissQual UDPDL test
EndTime Reception of the last packet Source port of the test or timeout
exceeded
Duration Time in ms between StartTime and EndTime
74 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 75
NQDI KPIs Manual
2000 - 2009 SwissQual AG
StartTime Timestamp of the WSP Get message plus the halftime between
WSP Get and the WSP: Reply or WSP: Result message which
is the connect ack message on the WSP/WTP port
EndTime Timestamp of the last WSP Reply message
Duration Time in ms between StartTime and EndTime
ErrorCode 0 = OK, see Status in the appendix
Sum 0 Unused
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar Name of the half time message
Value4 Varchar Name of the end event
Value5 Unused
76 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 77
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Messages
For the detailed mapping between the events in the log and the trigger points in the application
see Data and Video Test Scenario Manual section MMS send and MMS receive.
78 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value1 Unused
Value2 Unused
Value3 Varchar Name of the start event
Value4 Varchar Name of the end event
Value5 Unused
Data Call
Field Value Description
KPIId 25100 Data (PS), Attach
StartTime Timestamp of the GMM message Attach Accept
EndTime Timestamp of the GMM message Detach Request
Duration Time in ms between StartTime and EndTime
Appendix A | 79
NQDI KPIs Manual
2000 - 2009 SwissQual AG
80 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value2 Unused
Value3 Varchar Unused
Value4 Varchar Unused
Value5 Unused
Resource
Field Value Description
Appendix A | 81
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value5 Unused
Integrity Records
Telephony
Field Value Description
Data
Field Value Description
82 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value5 Unused
Appendix A | 83
NQDI KPIs Manual
2000 - 2009 SwissQual AG
84 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Counter 0 Unused
Value1 Throughput [Bytes/sec]
Value2 Unused
Value3 Varchar Size of file
Value4 Varchar Unused
Value5 Unused
Appendix A | 85
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value5 Unused
86 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 87
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Counter 0 Unused
Value1 Throughput [Bytes/sec]
Value2 Unused
Value3 Varchar Size of file
Value4 Varchar Unused
Value5 Unused
88 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value5 Unused
Appendix A | 89
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value5 Unused
StartTime 554[SYN]
EndTime Resized or Buffering or Loading
Duration Time in ms between StartTime and EndTime
ErrorCode 0 = OK, see Status in the appendix
Sum 0 Unused
Counter 0 Unused
Value1 Unused]
Value2 Unused
Value3 Varchar Start Message
Value4 Varchar End Message
Value5 Unused
90 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Messages
The following Messages KPIs are not implemented as explicit KPIs but calculated on the
respective tab sheet and report. For the detailed mapping between the events in the log and the
trigger points in the application see Data and Video Test Scenario Manual section MMS send and
MMS receive.
Appendix A | 91
NQDI KPIs Manual
2000 - 2009 SwissQual AG
StartTime Timestamp of the log message START, which indicates the begin
92 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
StartTime [SYN]
EndTime [SYN;ACK]
Duration Time in ms between StartTime and EndTime
ErrorCode 0 = OK, see Status in the appendix
Sum 0 Unused
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar Unused
Value4 Varchar Unused
Value5 Unused
Appendix A | 93
NQDI KPIs Manual
2000 - 2009 SwissQual AG
StartTime SQ < 0
EndTime SQ > 0
Duration Time in ms between StartTime and EndTime
ErrorCode 0 = OK, see Status in the appendix
Sum 0 Unused
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar Unused
Value4 Varchar Unused
Value5 Unused
Handover
Field Value Description
94 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 95
NQDI KPIs Manual
2000 - 2009 SwissQual AG
96 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Value1 Unused
Value2 Unused
Value3 Varchar Unused
Value4 Varchar Unused
Value5 Unused
Appendix A | 97
NQDI KPIs Manual
2000 - 2009 SwissQual AG
98 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
StartTime CellUpdate
Appendix A | 99
NQDI KPIs Manual
2000 - 2009 SwissQual AG
StartTime PhysicalChannelReconfiguration
RadioBearerReconfiguration
TransportChannelReconfiguration
only when at least one servingHSDSCH_RL_indicator true
EndTime PhysicalChannelReconfigurationComplete
RadioBearerReconfigurationComplete
TransportChannelReconfigurationComplete
Duration Time in ms between StartTime and EndTime
ErrorCode 0 = OK, see Status in the appendix
Sum 0 Unused
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar Start Message
Value4 Varchar End Message
Value5 Unused
100 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
StartTime PhysicalChannelReconfiguration
RadioBearerReconfiguration
TransportChannelReconfiguration
only when all servingHSDSCH_RL_indicator are false, but the
servingHSDSCH_RL_indicator is present or a delete hsdsch is
found
EndTime PhysicalChannelReconfigurationComplete
RadioBearerReconfigurationComplete
TransportChannelReconfigurationComplete
Duration Time in ms between StartTime and EndTime
ErrorCode 0 = OK, see Status in the appendix
Sum 0 Unused
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar Start Message
Value4 Varchar End Message
Value5 Unused
Resource
Field Value Description
Appendix A | 101
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Mobility
Field Value Description
102 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 103
NQDI KPIs Manual
2000 - 2009 SwissQual AG
104 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Counter 0 Unused
Value1 Unused
Value2 Unused
Value3 Varchar Unused
Value4 Varchar Unused
Value5 Unused
Resource
Field Value Description
WCDMA
This KPI is not implemented in the current version
Appendix A | 105
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Status
The result of the KPI measurements can to be found in the ErrorCode field of the record. The
status is OK or successful whenever the service or a resource is obtained as expected.
Otherwise, if that is not the case e.g. a PDP context request rejected then the status field will
show 108001. Also, if there is e.g. a ActiveSetUpdateFailure sent then the status field is set to
108002. Some end trigger points are expected to come within a limited time (this is a time based
trigger). If that time is exceeded and no end trigger point is detected then the status field is set to
108003. In the reports, the last three cases described above are all marked as Failed since the
process was not successfully concluded.
If a KPI measurement is based on session or test and one of the trigger point is missing then the
status field is set to 108004 or 108005 depending on what is missing the Status field in the report
will show Incomplete.
(With the Qualcomm trace phone the rate of missing layer 3 messages is particularly high at this
time so that whenever a message is missing for a complete procedure the status is set to 108100,
108101, 108102 or 108103 to mark that as a special event. The different types are listed in the
table below).
106 | Appendix A
NQDI KPIs Manual
2000 - 2009 SwissQual AG
Appendix A | 107
NQDI KPIs Manual
2000 - 2009 SwissQual AG
B Appendix
KPI Calculation
The presentation as described in this appendixr is the fashion the KPIs are calculated and
presented in the reports generated with the Report Generator. For details see NQDI Manual
Success Rate
The success rate is in general the number of successful attempts divided by the total number of
attempts expressed in percent:
SuccessRate = # successful attempts/ total # of attempts * 100%
Table 4-3 Table for rates and averages
This rule is applied throughout with the exception of RRC Connection Request where sometimes
a second request occurs although the first is successful. In such a case the success rate
calculated as if there is only one attempt.
Rejection Rate
The rejection rate is only required for the PDP context activation where it is defined as the number
of rejected attempts divided by the total number of attempts expressed in percent.
RejectionRate = # rejected attempts/ total # of attempts * 100%
Time
The time that is given for the individual processes is the time elapsed between two trigger points
given in Section 2 and 3. With the exception of retainability durations with time in seconds (s) the
time is given in milliseconds (ms).
PDF
The probability density diagram is a histogram where the frequency of occurrence of the values in
the given intervals is shown.
108 | Appendix B
NQDI KPIs Manual
2000 - 2009 SwissQual AG
CDF
The cumulative density function is a histogram where the percentages of the measured values
that are equal or bigger than the respective value point on the x axis is shown.
Appendix B | 109
NQDI KPIs Manual
2000 - 2009 SwissQual AG
C Appendix
References
ETSI TS 102 250 -2
110 | Appendix C