Characterization of A 3G EV-DO Network - A Measurement Study
Characterization of A 3G EV-DO Network - A Measurement Study
Characterization of A 3G EV-DO Network - A Measurement Study
a Measurement Study
Zhe Zhou, Mark Claypool and Robert Kinicki
Computer Science Department at Worcester Polytechnic Institute
Worcester, MA USA
{jeffz,claypool,rek}@cs.wpi.edu
Abstract—The growth of mobile streaming applications has overhead, frequency and duration of transmission data rates,
increased the importance of a better understanding of 3G EV- EV-DO packet fragmentation and encapsulation, EV-DO and
DO performance. This paper provides performance measure- higher layer loss characteristics, and observed signal strengths.
ments for a series of experiments that offer an in-depth, cross-
layer understanding of EV-DO. Insights are provided for data While providing a general understanding of a deployed EV-DO
rate control, packet encapsulation/combination, early completion, network, these low-level characteristics facilitate the develop-
packet interarrival time, packet loss pattern and queue manage- ment of an EV-DO simulator or they can be used by end hosts
ment. Key results are: data rates for non-mobile EV-DO users to improve application performance over EV-DO.
are more stable than previously reported with little variance Our experiments show high variability in network conditions
over short time scales and more random behavior over long
time scales; low-level packet encapsulation/combination happens when the client location changes. However, measured station-
frequently; and custom service provider configurations (e.g., ary EV-DO client reception characteristics have less variability
advanced queue management), and operating system differences than previously reported [3]. Over time scales of a few min-
(e.g., clock granularity for device drivers) can obfuscate network utes, throughput at EV-DO clients shows little variance, while
behavior relative to EV-DO specifications. over interval of tens of minutes, EV-DO client throughput
Index Terms—3G, EV-DO, bandwidth estimation
changes unpredictably. Only relatively short sampling periods
(less than 1 second) are needed for a consistent estimate of the
I. I NTRODUCTION
standard deviation of network bandwidth. Operating systems
The emergence of smart phones as a popular Internet mobile provide timing granularities that are smaller (1 millisecond in
platform and the increased use of cellular access cards on Windows) and larger (increments of 4 milliseconds in Linux)
laptops has fueled the pressure for cellular network providers than those provided by EV-DO packet transmissions (1.667
to develop technologies that enable higher capacity to support milliseconds). EV-DO packet errors are infrequent but when
the plethora of new streaming applications designed for mobile they do occur, they are bursty. The effects of upper layer queue
wireless clients. management (e.g., RED [4]) often obfuscate EV-DO network
Effective mobile video streaming requires video servers that behavior beyond what can be inferred in the specification.
adapt quickly and efficiently to the variability in wireless The paper is organized as follows: Section II provides
transmission conditions, particularly the transmission rates. an overview of previous related research; Section III intro-
Bandwidth estimation has been proposed and investigated as duces EV-DO networks; Section IV details our measurement
a technique to optimally adjust streamed video transmission methodology and experiment design; Section V presents and
rates to match the available bandwidth over the Internet path. analyzes experimental results and Section VI concludes by
However, historically, bandwidth estimation explorations have summarizing our main findings.
only considered cases where the last hop is either a wired
LAN or an IEEE 802.11 wireless access point. Bandwidth II. R ELATED W ORK
estimation research for cellular networks is challenging, with To the best of our knowledge, there are no bandwidth
only sparse published measurements, due to: the volatile nature estimation techniques designed specifically for 3G EV-DO
of mobile wireless transmissions, limited access by external networks. Due to the fundamental differences between EV-DO
researchers to proprietary cellular network infrastructures, networks and wireless 802.11 or wireline networks, existing
difficulties in managing controlled measurements over cellu- bandwidth estimation tools are likely to be inaccurate in mea-
lar network infrastructures, and no proven, publicly-available suring the bandwidth for EV-DO networks. The fundamental
cellular network simulators [1], [2]. challenges for bandwidth estimation over EV-DO include pos-
This paper presents measurement studies of the widely sible packet fragmentation and multiple packet encapsulation,
deployed 3G EV-DO Rev. A cellular network technology. By the proportional fair scheduler, a time-division multiplexing
investigating low-level EV-DO characteristics, this research slotted forward link, and highly variable transmission rates.
assists in the ultimate goal of developing effective band- A handful of literature explores the feasibility of directly
width estimation techniques for mobile streaming applications. applying existing bandwidth estimation techniques designed
Important EV-DO details uncovered include EV-DO packet for wireline and wireless IEEE 802.11 to EV-DO networks.
For example, Castellanos et al. [2] compare three bandwidth to an EV-DO packet, whereas protocol together with packet
estimation tools (TOPP, SLoPS and pathChirp) in mobile refers to packets at other network layers (e.g., IP packet).
networks. The authors do not find any of these estimation tools Figure 1 illustrates a typical EV-DO network. Access Termi-
to be more outstanding than the others, but the results show nals (ATs) connect to an Access Point (AP), also known as a
that pathChirp performs the best. However, the experiments Radio Node (RN). APs are managed by Radio Control Nodes
only use NS-2 based simulations, and the NS-2 configurations (RNCs) that offer services including resource and mobility
are not presented. The paper does not provide details about management. Packet Data Serving Nodes (PDSNs) connect
the mobile network studied (for example, whether it is 1xRTT, RNCs into the Internet and manage Point-to-Point Protocol
1xEV, or UMTS). (PPP) sessions for the ATs.
Claypool et al. [5] compare TCP throughput with the
bandwidth estimation results from a tool called npath [6].
The estimation results remain steady at about 525 kbps, but
the TCP throughput varies from 300 to 350 kbps. While the
authors do not comment on the difference, the result shows
npath does not estimate the achievable throughput accurately.
Koutsonikolas and Hu [1] use WBest [7] to estimate the
achievable throughput of an EV-DO terminal. Their experi-
mental results show that WBest fails to measure the achievable Fig. 1. Typical EV-DO Network Architecture
throughput defined either by the maximum TCP throughput
or the maximum UDP throughput. The study concludes that An EV-DO Rev. A AP transmits data to ATs at a variable
bandwidth estimation in EV-DO networks is not feasible rate that depends upon the received DRC (Data Rate Control)
because of the high variability in transmission rates exhibited index. Every EV-DO time slot (1.667 milliseconds), each AT
at small time scales. estimates the highest possible DRC1 for individual packet
An interesting phenomena is that the measured TCP reception based on its measured Signal to Interference and
throughput in [5] and [1] exhibit different properties. For Noise Ratio (SINR) and sends the DRC index to the AP. Better
example, [5] observes TCP throughput being in the 300 wireless channel conditions yield higher DRCs where DRC 14
to 350 kbps range and reasonably stable over a 24 hour corresponds to the highest data rate and DRC 0 corresponds
period, while [1] sees TCP throughput changes much more to the lowest data rate.
drastically (e.g., for one specific location, TCP throughput is The AP maps an AT’s DRC into a specific EV-DO MAC
measured at 700 kbps at 4am and about 350 kbps at 10am). layer transmission format that consists of a packet size (in
A possible explanation of this difference is the year the two bits), a duration (in time slots) and a preamble length (in
sets of measurement are taken, 2006 for [5] and 2009 for chips). Generally, a higher DRC implies a larger packet size
[1]. The worldwide number of EV-DO subscribers had grown and/or a shorter duration, which is intended to produce a
from 65 million in 2006 [8] to 127 million in 2009 [9]. higher data rate.
Although service providers may have built additional network The duration, measured in time slots, can be understood
infrastructure to accommodate the increased number of users, as the number of time slots a packet is scheduled to be
the increased density of the EV-DO user population may have transmitted. If the SINR at reception time is good, the AT
caused the throughput variability observed in [1]. can receive and decode the packet in the first time slot the
Despite the difference between observed achievable packet is transmitted, even when the packet could have two
throughput, both [5] and [1] find that the achievable throughput time slots scheduled. This event, known as early completion,
measurement by sending TCP/UDP flows stays relatively helps compensate for an AT’s inaccurate DRC estimation and
steady for several minutes at a time, suggesting streaming typical fluctuations in channel signal strengths.
applications may benefit from using appropriate bandwidth An EV-DO network maintains PPP sessions for each AT.
estimators. For instance, a bandwidth estimate valid for several Internet IP packets are encapsulated into PPP packets and
minutes can help choose a suitable transmission rate for a sent through several EV-DO layers to the appropriate AT.
YouTube streaming video since about 98% of YouTube videos The formation of PPP packets does not preserve IP packet
are less than 10 minutes long [10]. More generally, an accurate boundaries, meaning that IP packets can be combined or
bandwidth estimation can allow for initial configuration of fragmented by the EV-DO network layers.
application network parameters (e.g., initial video encoding EV-DO APs use Proportional Fairness (PF) scheduling
rate). Subsequent adjustment of bandwidth estimation can when transferring data to multiple ATs. The PF scheduler
be inferred from the statistics of actual application packets essentially uses a round-robin mechanism across ATs with
received, or by rerunning the bandwidth estimation. an adjustment that gives priority to ATs that report to the
III. BACKGROUND AP a current DRC that is better than its previous weighted
average. While a PF scheduler tends to improve overall system
This section briefly introduces fundamental EV-DO con-
cepts and terminology. In this paper, Packet used alone refers 1 With an objective of no more than a 1% packet error rate
TABLE II
throughput [11], it increases the variability of IP packet S OFTWARE TOOLS
response times. Tool Author Purpose
Packet error rate (PER) is the error rate of EV-DO packets. EV-DO Sniffer Airvana Record EV-DO traffic trace for
An EV-DO packet can fail to be received after the scheduled DRC, SINR and PER
number of transmission attempts (as opposed to early comple- QCAT Qualcomm Parse EV-DO traffic trace for
DRCs and packet transmission
tion) and all retransmissions attempted by the EV-DO Radio formats (every second)
Link Protocol. Higher PERs increase delays in transmissions Keep-Alive Tool WPI Send dummy traffic periodically
and possibly packet loss at the IP layer. to keep aircard connected
Traffic Generation WPI Send from server to client with
IV. M ETHODOLOGY Tool controlled packet rates and sizes
SINR (dBm)
Fraction
0.6
DRC
-4
3 0.5
-6 0.4
2 -8 0.3
-10 0.2
1
-12 0.1
0 -14 0
0 0.2 0.4 0.6 0.8 1 0 2 4 6 8 10 12 14
Time (seconds) DRC indices
Fig. 3. DRC and SINR correlation over 1 second intervals (mobile) Fig. 4. CDF of DRCs
1
We are aware that the limited scale of our experiments may
0.9
not yield ubiquitously representative results. However, previ-
0.8
ous mobile network data studied in large scale has been pre-
0.7
sented mostly only from the vantage of the network provider,
Fraction
0.6
likely due to the manual process needed to gather data at the
AT. Some of our experiments (e.g., throughput versus sample 0.5
time) are run at locations with different DRC ranges providing 0.4
system) are generally applicable. Overall, the results offer a 0.1 Mobile
Station
practical instance of an AT-side perspective of a commercial 0
0 1000 2000 3000 4000 5000 6000
EV-DO network, while highlighting network aspects that could Stability Duration (slots)
be studied at a larger scale in future work.
Fig. 5. CDF of DRC steadiness
A. DRC Analysis
DRC indices are forward-link rate requests made per EV-
DO time slot from an AT to the AP. Our experiments examine DRC fluctuation has a direct impact on AP transmission
DRC distributions from both stationary and mobile EV-DO rate variability, which in turn impacts streaming applications
endpoints. Previous work studying 3G networks has shown and bandwidth estimation. Generally, a more stable DRC
DRCs to be a key attribute in determining EV-DO end-device distribution should yield steadier streaming data rates, more
performance [3]. accurate bandwidth estimation and reduced estimation times.
DRC requests are based on the SINR computed at the
To examine this variation, AT DRC requests made every EV-
AT. Figure 3 shows an example of the correlation between
DO time slot (1.667 milliseconds) are recorded over a period
DRC and SINR over a one-second period for a mobile device.
of 800 seconds. From this data, the size of continuous bursts
Although the DRC fluctuates among 2, 3, and 5, there is no
of non-changing DRC values (referred to as ”steady region”
visual correlation between SINR and DRC over this time scale.
in the rest of the text) are extracted and then computed as a
Thus, we provide no further analysis of performance based on
fraction of the total number of DRC values. Figure 5 graphs
SINR.
the CDF of this DRC steadiness for the same stationary and
Figure 4 depicts the CDF of DRCs reported over two
mobile experiments.
90-minute experiments. The mobile experiment is run in a
car moving at 30 mile per hour as aforementioned and the In the stationary experiment, DRC values remain constant
stationary experiment is run at Location B. Both cases yield over relatively long time periods. For the stationary AT, over
nearly a full-range of DRC values, with DRC values of 1-13 half the time the DRC stays in steady regions of 1000+ time
seen clearly. The DRC distribution is slightly broader during slots (approximately 1.5 seconds), exhibiting more steadiness
the mobile experiment. There are few DRC 14 values reported than reported previously [3]. Almost 10% of the time, the
in the mobile case, and none in the stationary case.3 DRCs belong to steady regions of 3000+ time slots (more
than 5 seconds). Even in the mobile AT case, DRCs remain in
3 Reported DRC 0 values appear to be mode changes by the AT and not steady regions of 600+ time slots (more than a second) over
actual requests for data transmission rates. half the time.
B. Throughput versus Sample Time 3 620
800
DRC 14, EV-DO sends 640 bytes per slot and Figure 7 implies
the EV-DO header is 24 bytes.
600
To ascertain EV-DO overhead for other DRC values, ex-
periments with three distinct IP packet sizes (50 bytes, 400
400
bytes, and 750 bytes) are conducted at Location B and C. Each
experiment has the server send packets to the AT at a fixed
200
packet rate for 10 seconds, followed by a three-second quiet
period where no packets are sent, then doubling the packet
0
1 2 3 4 5 6
rate and repeating transmissions up to a saturating rate (higher
Sample Interval (seconds) than the EV-DO maximum). Each experiment is repeated 5
times for each packet size and location combination. Figure 8
Fig. 6. Standard deviation versus sampling interval shows the EV-DO packets arrivals for one such experiment
(packet size 50 bytes, Location C). For example, at around
90 seconds, the previous batch of IP packets are received (as
C. EV-DO Packet Analysis EV-DO packets), a 3 seconds pause is executed, and the next
IP packets can be fragmented during transfer to EV-DO batch of IP packets (sent at double the previous packet rate)
ATs. To better understand how and when packet fragmentation start to be received. In correspondence to the doubling of IP
occurs within EV-DO, experiments run at Location A that packets sending rate, the EV-DO packet arrival rate increases
elicited predominantly a 14 DRC are examined over a range every 13 seconds up to the saturation point, with no noticeable
of IP packet sizes. UDP packets are sent from the server to the drop-off in packet arrival rates once saturation is reached. EV-
AT in ten-packet batches. All packets in a batch are the same DO packet arrival rate seems to increase less when packet
size sent at a rate of 1 packet per 3 seconds. After sending one rate is higher (e.g., for the two batches at time 128 and 256),
batch, the server pauses 10 seconds, increases the packet size suggesting EV-DO is combining IP packets.
by 1 byte, and sends the next batch. The 3 second interval is Figure 9 presents the ratio of the number of bytes of the
chosen since it is long enough to distinguish packet(s) sent at EV-DO traffic to that of IP layer traffic (including IP header)
the EV-DO layer for each UDP packet, but short enough to for all experimental runs. Some experiments finshed earlier
avoid the AT going dormant and disconnecting. If there is no than others because those experiments use the larger packet
data transmission in 5 seconds the AT will enter a dormant sizes and reach the saturation faster. The EV-DO overhead
state which affects the statistics of the next packet. converges from about 15:1 at a low packet per second rate, to
Figure 7 shows an experimental segment where the UDP close to 1 at a high packet per second rate. The high overhead
packet size varies from 612 bytes to 620 bytes (shown by for small IP packets at low sending rate is because EV-DO
EV-DO Packet Arrival Rate (per Second)
1000 1
0.9
0.8
0.7
100
Fraction
0.6
0.5
0.4
10
0.3
0.2
0.1
1 0
0 20 40 60 80 100 120 140 160 1 2 3 4 6 8 10 12 1416
Time (seconds) Packet Interarrival Time (ms)
1
0.8
0.7
does not have a backlog from which the IP packets can be
Fraction
0.6
combined. At high sending rate, EV-DO combines IP packet 0.5
to help keep overhead low. 0.4
0.3
0
4 8 12 16
Packet Interarrival Time (ms)
10
Ratio
Fig. 10. Interarrival time of application packets (Windows top, Linux bottom)
Throughput (kbps)
0.7
800
Fraction
0.6 600
0.5 600
0.4 400
400
0.3
0.2 200
200
0.1
0 0 0
0 0.2 0.4 0.6 0.8 1 0 20 40 60 80 100 120 140
Packet Error Rate (Fraction) Time (seconds)
1 1800 1200
Throughput
1600 Loss (packets)
1000
Throughput (kbps)
1200 800
Fraction
1000
0.01 600
800
600 400
0.001
400
200
200
0.0001 0 0
0.1 0.2 0.4 0.6 0.8 1 0 20 40 60 80 100 120 140
Packet Error Rate (Fraction) Time (seconds)
Fig. 11. CDF of PER (top), CCDF of PER (bottom) Fig. 12. Packet loss with increasing throughput (top: 33 byte packets, bottom:
1500 byte packets)
TABLE IV
dip in throughput around 100 seconds. With large, 1500-byte PACKET TRANSMISSION FORMAT REPORT
IP packets (bottom), there is a similar increase in rate until DRC Size Slots slot1 slot2 slot3 slot4 PER
saturation, but a noticeable increase in throughput around 60 0 1024 16 0 7 0 0 0
seconds before the rate decreases to the saturation point. 2 128 8 12 0 0 0 0
2 512 8 5 0 0 0 0
Application sequence numbers can be used to determine 2 1024 8 14 12 0 0 0
lost packets, but the analysis is made difficult by out-of-order 3 128 4 4 0 0 0 0
arrivals. To infer out-of-order packets caused by the EV-DO 3 512 4 5 0 0 0 0
network, two experiments are run. Using 33 byte packets, the 3 1024 4 113 17 0 0 0
4 1024 2 33 1 0
first experiment sends data at 100 kbps from the server to the 5 2048 4 14 937 60 0 0.4%
AT (achieved rate of 85 kbps) and the second experiment sends 6 1024 1 31 0
data at 750 kbps (achieved rate of 585 kbps). Both experiments 7 2048 2 1475 18373 0.3%
run for three minutes. In the first experiment, there are no 8 3072 2 0 26774 0.7%
9 2048 1 4 0
out-of-order packets, while in the second experiment there are 10 4096 2 0 4984 1.4%
167 out-of-order packets, accounting for 0.04% of all 379198 13 5120 2 0 94 0
packets that are received.
Examining traceroute output before and after the experi-
ments indicates routing changes occurred in both the first and
at Location A (the receiving rate is 377 kbps). Transmission
second experiments. These route changes, and not the EV-DO
formats which have no packets transmitted are omitted from
link, could be the cause of the out-of-order packets observed
the table. From the table, early completion is prevalent for low
in the second experiment.
DRCs.4 For example, DRC 3 with 1024 bit EV-DO packets
F. Early completion has 113 packets complete in slot 1 and 17 in slot 2 and no
EV-DO early completion can be observed in Table IV. In the packets in the subsequent slots. However, for higher DRCs,
table, each DRC implies one or more combinations of packet all DRC slots are needed for transmission. For example, DRC
sizes (in bits) and time slots scheduled to send those packets. 10 has all EV-DO packet completions during slot 2.
This experiment runs for 278 seconds, with the server sending 4 DRC 0 and 2 have more than 4 slots but no EV-DO packets are sent in
33-byte packets at a rate of 1 Mbps to the EV-DO end device these slots.
In most cases the PER is less than 1% [13], though at DRC VI. C ONCLUSIONS
10 the PER is a little higher. The PER exhibits an increasing Though wide-spread in deployment, low-level 3G EV-DO
trend with the DRC going up, implying that packets are more analysis suitable for simulation and bandwidth estimation tech-
likely to be lost at higher DRC for the same location. niques for streaming video are not well-established. Through
G. Queue management careful experimentation and measurements with an EV-DO
Saturating traffic is sent from our server to the EV-DO end sniffer, Wireshark and application-level tools, this paper pro-
device, with the packet loss pattern is plotted in Figure 13. vides insights into EV-DO performance not previously re-
The experiment is conducted at Location A, with a fixed ported or demonstrated. The results should be useful for those
packet size of 33 bytes. All losses are in bursts of 3 or building EV-DO simulators or building bandwidth estimation
more packets, with most losses in bursts 400-1000 packets tools for streaming applications over EV-DO – both areas of
in a row. Figure 14 shows the corresponding CDF of the our future work.
time between two consecutive packet losses. Loss intervals Our preliminary investigations show Data Rate Controls
are mostly between 30 to 100 milliseconds. The bursty nature (DRCs) requested by the EV-DO ATs are centered around a
of the packet drops suggests queue management, likely above few values in the case of stationary end-hosts and somewhat
the EV-DO layer, perhaps employing an algorithm such as more scattered in case of mobile end-hosts. In all cases,
Random Early Detection (RED) [4]. DRCs observed are steadier than reported in previous work.
Measured throughputs over these DRCs exhibit consistent
10000 variance, suggesting accurate bandwidth predictions may be
made after relatively few samples.
EV-DO constructs forward link packets based on the max-
Number of Packet Losses
1000 imum packet size the current DRC allows regardless of the
original IP packet boundaries. IP packets are observed being
combined and fragmented to best fit into each EV-DO packet.
100
DRC 14 packets have 24-byte headers, but the exact size of
the EV-DO headers appears to vary with different DRCs.
10 The interarrival times of application packets are multiples
of a few milliseconds that differ by operating system on the
EV-DO end-host. At such granularities, the interarrival times
1
0 0.5 1 1.5 2 2.5 3 3.5 4 observed at the application layer are not necessarily a direct
Time (seconds) reflection of the interarrival times at the EV-DO layer.
Normal packet error rates (PERs) are less than 1%, but
Fig. 13. Packet loss over time losses tend to be bursty. When losses do occur, per-second
loss rates typically range from 20-40%.
There is no significant packet loss in Figure 13 in the initial Early completion can achieve significant gains. For DRCs
1.2 seconds, suggesting a downlink queue at the AP. Using the that have a nominal transmission duration longer than 1 time
method in [14], we calculate the queue size to be about 235 slot, in some cases most of the packets are successfully
bytes. This size does not change significantly with the packet transmitted in the first time slot. In contrast, in some cases,
size which implies the queue is a per-byte queue, rather than particularly for higher DRCs, none of the packets are received
a per-packet queue. in the first time slot.
EV-DO networks may use queue management to stop ac-
1 cepting packets into the queue when the network is congested.
0.9 When an EV-DO network receives incoming packets at a rate
0.8 higher then EV-DO can forward, the packets are dropped in
0.7 a regular discrete fashion rather than a continuous fashion as
would be the case with a drop-tail queue.
Fraction
0.6
0
1 10 100 1000
Time between Packet Losses (Milliseconds)