1

Measuring Impact of DoS Attacks

Jelena Mirkovic
University of Delaware
Newark, DE
Roshan Thomas
SPARTA, Inc.
Centreville, VA

Sonia Fahmy
Purdue University
West Lafayette, IN
Alefiya Hussain
SPARTA, Inc.
El Segundo, CA

Abstract Denial of service attacks are an increasing threat to

the Internets availability and reliability. To evaluate a variety of
defenses proposed against this threat we must be able to precisely
measure impact of an ongoing attack on a network. The effectiveness
of a defense can then be calculated with regard to how quickly and
how completely it eliminates this DoS impact.
We propose a DoS impact measure which divides legitimate traffic
in the network into higher-level user tasks, called transactions, and
classifies these transactions into application-level categories. For each
category, we define quality-of-service requirements that have to be
met for a satisfactory service. DoS impact is then measured as a
percentage of transactions in each category that have not met their
QoS requirements.

I. I NTRODUCTION
Denial-of-service attacks have been a serious Internet threat for
at least a decade. Recently, this threat has grown more imminent
because people are increasingly relying on network services for
everyday communication, business tasks and even critical services
such as vessel navigation, emergency service coordination, etc.
Accurately measuring a denial-of-service impact is essential for
evaluation of potential DoS defenses. A defense is only valuable
if it provably prevents or eliminates the denial-of-service impact,
making DoS attacks transparent to Internet users. If we could
measure which services were denied by the attack with and
without the defense we could: (1) understand and express severity
of various attacks (e.g., attack A denied all the services in the
network, whereas attack B only denied HTTP service to new
users, (2) characterize the effectiveness of proposed defenses
(e.g., defense X eliminated 90% of DoS impact after 1 minute)
and compare defenses to understand a price/performance tradeoff.
An overall impact of DoS attacks on a target is that they
slow down or stop some service needed by legitimate users.
Historically, DoS researchers used several measures to capture
this effect: percentage of legitimate packets that received no
service, division of resources between legitimate and attack traffic,
throughput or goodput of TCP connections, and the overall
transaction duration. While these measures capture the essence of
a DoS impact they show a deviation of a measured parameter
during the attack from the same parameter without the attack
they do not really measure if some service was denied. This
is because Internet applications have very different quality-ofservice requirements. Online games and audio conversations are
sensitive to even a very small delay of 100 ms, while bulk file
transfer can endure multiple-minute delays. Audio applications
are sensitive to delay variation (jitter) while other applications
are not. Media and online games are also sensitive to packet

Peter Reiher
University of California Los Angeles
Los Angeles, CA

Steven Schwab
SPARTA, Inc.
El Segundo, CA

Calvin Ko
SPARTA, Inc.
El Segundo, CA

loss, while TCP-based applications can tolerate and recover from

large amounts of packet loss. Clearly, whether some amount of
delay, delay variation and packet loss cause a denial-of-service
impact or not depends on the quality of service requirements of
the applications that experience these factors.
We propose a DoS impact metric that speaks to the heart of the
problem: it measures if the legitimate clients receive acceptable
service or not during an attack. This metric requires capturing the
legitimate traffic trace at the source and the destination. Within
this trace, we recognize transactions that represent applicationspecific tasks such as downloading a file, browsing a Web page
or having a VoIP conversation. For each transaction, we measure
five parameters: (1) one-way delay, (2) request/response delay, (3)
packet loss, (4) overall transaction duration and (5) delay variation (jitter). Depending on the quality-of-service requirements
of each application in the trace, we classify transactions into
several application categories. We then apply category-specific
thresholds to measured parameter values to determine if each
transaction succeeded or failed, and calculate the percentage of
failed transactions (pft) in each application category. The overall
DoS impact can then be specified as a weighted average of pft
values, or as a histogram across categories. To capture the time
dimension we can also split the attack time into several windows
and measure a DoS impact in each window.
The main challenge of the proposed DoS impact metric lies in
categorizing applications by their QoS requirements, and specifying realistic, objective and measurable criteria for success (or
failure) for each application category. An additional challenge lies
in the ability of some applications to hide a network-introduced
delay, delay variation or loss using variable buffering (streaming
media applications [18]) or extrapolation (online games [8], [5]).
Since it would be non-scalable to define transaction success or
failure for each existing application, we must decide to either
consider all applications or of a certain kind (e.g., streaming
audio) or none of them, as capable of masking some specific
range of delay, jitter and loss.
We acknowledge that definition of universally acceptable QoS
criteria for each application category is going to be a major
undertaking, and will require participation of a large research and
commercial community. However difficult, we believe that this
effort is necessary for objective evaluation of DoS defenses and
their comparison. In Section II-A we propose a set of application
categories and their success criteria. In this we largely borrow
from 3GPPs specification of QoS performance requirements

Application Category
email (server to server)
NNTP (Usenet)
Chat
Web, e-commerce
FTP, file-sharing
FPS Games
RTS Games
Telnet
E-mail (user to server)
DNS
ICMP
Audio, conversational
Audio, voice-messaging
Audio, streaming
Videophone
Video, streaming

QoS Classification
Non real-time
Non real-time
Non real-time
Real-time, block
Real-time, block
Real-time, block
Real-time, block
Real-time, block
Real-time, block
Real-time, block
Real-time, block
Real-time
Real-time
Real-time
Real-time
Real-time

One-way delay

Request/response delay
whole < 4 hours
whole < 4 hours

Loss

Duration

Jitter

< 30 s
any < 4 s or maxRTT < 4 s
any < 10 s or maxRTT < 10 s
< 150 ms
< 500 ms

< 150 ms (media)

< 2 s (media)
< 10 s (media)
< 150 ms (media)
< 10 s (media)

< 300%
< 300%
<3%

any < 250 ms or maxRTT < 250 ms

any < 4 s or maxRTT < 4 s
whole < 4 s
whole < 4 s
whole < 4 s or maxRTT < 4 s (control)
whole < 4 s or maxRTT < 4 s (control)
whole < 4 s or maxRTT < 4 s (control)
whole < 4 s or maxRTT < 4 s (control)
whole < 4 s or maxRTT < 4 s (control)

< 300%
< 300%

<
<
<
<
<

3%
3%
1%
3%
1%

< 50 ms
< 50 ms
< 50 ms

TABLE I
A PPLICATION CATEGORIES AND THEIR REQUIREMENTS

for Universal Mobile Telecommunications System, that defines

acceptable service quality for various applications. 3GPP is a
collaboration agreement which brings together a number of
telecommunications standards bodies [1] from all over the world,
in an effort to produce globally applicable Technical Specifications and Technical Reports for a 3rd Generation Mobile System
[1]. Thus, the proposed set of QoS specifications has an advantage
of being already accepted by the worlds large standards bodies.
Our proposed DoS impact metric requires measurement of
legitimate traffic at various points in the Internet: at the legitimate
senders and at the traffic destinations. As such, it is suitable for
testbed experimentation where we can capture traffic at any point,
but it would not be applicable to DoS impact measurement at the
victim end during real-world attacks. We discuss how to measure
required parameters in DoS experiments in Section II-B, and how
to aggregate pft measures into a DoS impact metric in Section
II-C We illustrate our metric with small-scale experiments in
DETER testbed [4] in Section III and discuss open issues and
future directions in Section V.
Another question that relates to objective DoS defense evaluation concerns the design of realistic and comprehensive test
scenarios, i.e., DDoS defense benchmarks. While this question is
an object of our current research, it is out of scope of this paper.
II. D O S I MPACT M ETRIC
Our proposed DoS impact metrics considers a set of transactions, and categorizes them based on their QoS requirements into
several application categories. For each transaction, we measure
five parameters, specified in Section I, and evaluate success
or failure based on how well the measured parameters meet
QoS criteria for this transactions application category. We now
specify application categories and their QoS requirements, and
then we explain how we define transactions and measure required
parameters, and how we aggregate transaction success and failure
data into an overall DoS impact metric.
A. Application Categories and Success Criteria
Table I lists the application categories we propose, mostly
borrowed from [16], and the corresponding QoS requirements.
We also modified several QoS requirements from [16], to: (1)
account for jitter elimination in audio applications using variable
size buffers [2], (2) differentiate between QoS requirements for
first-person shooter (FPS) [3] and real time strategy (RTS) [17]

games, (3) account for receipt of partial but useful response

from a server [6], (4) account for DNS and ICMP services, (5)
formalize extraction of the delay information from TCP dynamics
and request/response dynamics of various applications.
B. Measuring Performance
We only measure performance for traffic on conversations
initiated by a user with the target of a DoS attack. We identify
user-initiated conversations by looking for SYN packets sent from
a legitimate users machine to a DoS target for TCP traffic, UDP
packets sent to a well-known UDP service port for UDP traffic
and ICMP request packets. We capture traffic trace data at the
sender and at the receiver side, and identify transactions in this
data. We define a transaction as some application-specific task
that a user wants to perform. For example, if a user wants to
download 10 files from an FTP server, one by one, this represents
10 transactions. On the other hand, if a user downloads 10 files in
a batch, this represents one transaction. Table II-B shows how we
identify transactions in the traffic trace data. A flow is identified
as all traffic between two fixed IP addresses and port numbers.
Application
email (server to server)
NNTP (Usenet)
Chat
Web, e-commerce
FTP, file-sharing
Games
Telnet
E-mail (user to server)
DNS
ICMP
Audio and video

Transaction
TCP flow
TCP flow
TCP flow and inactive time > 4 s
TCP flow and inactive time > 4 s
TCP flow and inactive time > 4 s
UDP flow
TCP flow and inactive time > 4 s
TCP flow and inactive time > 4 s
One request/response
One request/response
TCP flow and a corresponding UDP flow

TABLE II
T RANSACTION

IDENTIFICATION

We can measure request/response delay and transaction duration using a sender-collected trace, but we need to correlate the
sender/receiver traces to measure one-way delay, loss and jitter.
We do this by matching source IP, port and identification (e.g.,
sequence number, request ID) of the packets at the sender with the
packets at the receiver side, and synchronizing sender and receiver
clocks at the beginning of the experiment. We use tcpdump to
capture a traffic trace during the experiment. Since tcpdump uses
interrupt-based packet capture, it has problems keeping up with
high packet speeds [10] and it will largely overestimate packet
loss for strong attacks. This problem can be handled to a large
extent if tcpdump is combined with device polling [10].

We identify requests and responses using data flow between

senders and receivers. Let A be a client that initiates some
conversation with a server B. A request is identified as all data
packets sent from A to B, before any data packet from B. A reply
is identified as all data packets sent from B to A, before any new
request from A.
Email and Usenet applications have a delay bound of 4 hours
and retry each failed transactions for a limited number of times.
It would be infeasible to create several-hour long experiments so
we need to extrapolate transaction success for these applications
using short experiment data. We do that using an observation
that the DoS impact usually stabilizes after some short time from
the onset of an attack, if the attack is not time-variable. When
a defense system is present, the DoS effect will usually stabilize
after some limited time from its detection by the defense, and the
defenses engagement. if we ensure that the experiment duration
is long enough that the DoS effect stabilizes, we can use the pft
for transactions started after the stabilization point as a predictor
of pft in a longer experiment. Also, since each server can set
its own values for the number and timing of retries, we need to
specify some default values to be used for transaction success
evaluation. Let r be a total number of retries within the 4-hour
delay bound and let s be the stabilized pft for email (or Usenet)
transactions during a short experiment. The predicted pft for a
long experiment is then: pftp = 1 (1 s)r .
Finally, some success criteria require comparing a transaction
duration during an attack with its expected duration without the
attack. If we have perfectly repeatable experiments, i.e., we can
specify a list of transactions to be generated and the timing and
sizes of packets in each direction, then we can measure the
expected transaction duration directly, running the experiment
without the attack. Some traffic generators may have built-in
randomness that prevents repeatable experiments. In this case
we must estimate the expected transaction duration, using the
information about transactions of the same type completed prior
to the start of the attack. Let us observe a transaction T that has
completed in tr seconds, sending B bytes of data, and whose
duration overlaps an attack. Let Th be the average throughput of
transactions generated by the same application as transaction T,
and completed prior to the attacks start. We can then calculate
the expected duration for the transaction T as te = B/T h and
compare it with the measured duration tr .
C. Aggregating Results
Many DoS attacks inflict damage only while they are active,
and the DoS impact ceases when the attack is aborted. It thus
makes sense to calculate transaction success only for transactions
whose duration overlaps the attack.
One way to aggregate transaction success and failure into a
DoS impact measure is to calculate the percentage of failed
transactions pft per application category and aggregate it into a
histogram across categories or across service port numbers. This
is especially useful to capture the effect of attacks that target only
one application, e.g., TCP SYN attack at Web server port 80. We
call this aggregated measure DoS-hist.
Sometimes it may be useful to aggregate DoS-hist into a
single number, called DoS-degree, expressing an overall DoS

LU
100Mbps

VIC
10Mbps

100Mbps
ATT

Fig. 1.

Simple experimental topology

impact. This can be done by calculating

a weighted average
P
of pft values: DoS-degree =
pft(k)

wk , where k goes
k
over all application categories, and wk is a weight associated
with a category k. Applications could be weighted equally or
according to their popularity or importance. Note that DoS-degree
is highly dependent on the chosen set of application weights.
Unless there is a broad consensus on a representative set of
application weights, using DoS-degree for defense performance
comparison could lead to false conclusions, as application weights
can be chosen to drive the results in favor of a chosen defense.
For DoS defense evaluation it is useful to calculate the DoS
impact over time. Since DoS defenses usually experience an
activation delay before responding effectively, the DoS impact
measured at the start of the attack will be much higher than
later, when the defense has engaged. We capture this time-based
change by splitting attack duration into intervals of T seconds and
calculating DoS-hist and DoS-degree measures for each interval
in the following manner: (1) We calculate success or failure
for transactions overlapping the current interval, (2) We use
transaction success data to calculate pft per application category
and aggregate it into a desired measure, (3) A transaction that
has failed in one interval is not used for pft calculation in the
following intervals.
III. E XPERIMENTS
We now illustrate our proposed DoS impact metric in a smallscale experiment in DETER testbed. We use a simple network
topology, shown in Figure 1 and generate the following legitimate
traffic: (1) HTTP traffic with Paretto-distributed file sizes, and
exponentially distributed connection arrivals with 1s mean, (2)
Telnet traffic with Paretto-distributed duration and traffic volume,
and exponentially distributed connection arrivals with 1s mean,
(3) FTP traffic with Paretto-distributed file sizes, and exponentially distributed connection arrivals with 1s mean, (4) DNS traffic
with exponentially distributed request arrivals with 1s mean, and
(5) ICMP traffic with exponentially distributed request arrivals
with 5s mean.
We generate a UDP flood attack that aims to overwhelm the
bottleneck link whose bandwidth is 1.25MBps. Figure 2 shows
the pft measure per application as we vary the attack strength. The
measure clearly reveals the sensitivity of long-lived transactions,
such as generated by FTP and Telnet, to small-rate attacks, while
DNS transactions are only affected when the attack strength is
four-fold the bandwidth of the bottleneck link.
IV. R ELATED W ORK
For brevity we only provide a short overview of the work
related to DoS impact measurement. In the quality of service
field there is an initiative to define a universally accepted set of
QoS requirements for applications. This initiative is lead by the

loss, throughput, fairness, etc. They briefly consider user-based

QoS metrics, but do not specify them in any detail.
V. C ONCLUSIONS

Fig. 2.

Impact of UDP flood attacks on applications

3GPP partnership between large standards bodies from all over

the world [1]. While many of the specified requirements apply to
our work, we extend, modify and formalize these requirements
as explained in Section II-A.
There is a significant body of work in differentiated services
field that separates applications into several categories based
on their sensitivity to delay, loss and jitter. A representative
list of applications includes video, voice, image and data in
conversational, messaging, distribution and retrieval modes [12].
These applications are either inelastic (real time) which require
end-to-end delay bounds, or elastic, which can wait for data to
arrive. Real time applications are further subdivided into those
that are intolerant to delay, and those that are more tolerant,
called delay adaptive. The Internet integrated services framework
mapped these three application types onto three service categories: the guaranteed service for delay intolerant applications,
the controlled load service for delay adaptive applications, and
the currently available best effort service for elastic applications.
The guaranteed service gives firm bounds on the throughput
and delay, while the controlled load service tries to approximate
the performance of an unloaded packet network [7]. Similarly,
the differentiated services (DiffServ) framework standardized a
number of Per-Hop Behaviors (PHBs) employed in the core
routers. In the early 1990s, Asynchronous transfer mode (ATM)
networks were designed to provide six service categories: Constant Bit Rate (CBR), real-time Variable Bit Rate (rt-VBR),
non real-time Variable Bit Rate (nrt-VBR), Available Bit Rate
(ABR), Guaranteed Frame Rate (GFR) and Unspecified Bit Rate
(UBR) [11]. The traffic management specifications [11] defined
methods to measure network-provided service so that users can
ensure they are receiving the service they had paid for.
In [9] researchers measure the effect of a DoS attack on
network traffic. They study the distribution of several parameters:
the throughput of FTP applications, roundtrip times of of FTP and
Web flows, and latency of Web flows and the DNS lookup service.
Our work concerns specifying the acceptable-service thresholds
for these and several other parameters, and for a broader variety
of services.
Recently, IRTFs Transport Modeling Research Group (TMRG)
has been chartered to standardize evaluation of transport protocols
by developing a common testing methodology, including a benchmark suite of tests [13]. In their drafts, they discuss the metrics
for evaluation of congestion control mechanisms, such as delay,

DoS defense field critically needs to formalize its defense

evaluation methods. The most important feature of a defense is
how well it can handle the impact of the attack. We proposed a
DoS impact measure that directly captures the effect of a DoS
attack on the legitimate network traffic, by defining applicationlevel quality of service requirements and measuring if they have
been met during an attack. The effectiveness of a DoS defense
can then easily be evaluated with regard to how quickly and how
completely it minimizes the DoS impact.
Much work remains to be done in tuning the parameters
that define application QoS requirements, refining application
categories, testing the proposed metric in a variety of attack
scenarios, formalizing aggregation methods, and promoting the
proposed metrics in research and commercial communities.
R EFERENCES
[1] 3GPP. The 3rd Generation Partnership Project (3GPP).
[2] J. Ash, M. Dolly, C. Dvorak, A. Morton, P. Taraporte, and Y. E. Mghazli.
Y.1541-qosm y.1541 qos model for networks using y.1541 qos classes.
NSIS Working Group, Internet Draft, Work in progress, May 2006.
[3] T. Beigbeder, R. Coughlan, C. Lusher, J. Plunkett, E. Agu, and M. Claypool.
The effects of loss and latency on user performance in unreal tournament
2003. In In Proceedings of ACM Network and System Support for Games
Workshop (NetGames), September 2004.
[4] T. Benzel, R. Braden, D. Kim, C. Neuman, A. Joseph, K. Sklower,
R. Ostrenga, and S. Schwab. Experiences with deter: A testbed for security
research. In 2nd IEEE Conference on Testbeds and Research Infrastructure
for the Development of Networks and Communities (TridentCom 2006),
March 2006.
[5] A. R. Bharambe, V. N. Padmanabhan, and S. Seshan. Supporting spectators
in online multiplayer games. In In Proceedings of ACM SIGCOMM
Workshop on Hot Topic in Networks, November 2004.
[6] Nina Bhatti, Anna Bouch, and Allan Kuchinsky. Quality is in the eye of
the beholder: Meeting users requirements for internet quality of service.
Technical Report HPL-2000-4, Hewlett Packard, 2000.
[7] R. Braden, D. Clark, and S. Shenker.
Integrated services in
the internet architecture: an overview.
RFC 1633, June 1994.
http://www.ietf.org/rfc/rfc1633.txt.
[8] R. F. Buccheit. Delay compensation in networked computer games. M.S.
project, Case Western Reserve University, 2004.
[9] Kun chan Lan, Alefiya Hussain, and Debojyoti Dutta. The Effect of
Malicious Traffic on the Network. In Passive and Active Measurement
Workshop (PAM), April 2003.
[10] Luca Deri. Improving passive packet capture:beyond device polling. In
Proceedings of SANE 2004, October 2004.
[11] The ATM Forum. The ATM forum traffic management specification version
4.0. ftp://ftp.atmforum.com/pub/approved-specs/af-tm-0056.000.ps, April
1996.
[12] M. W. Garrett. Service architecture for ATM: from applications to scheduling. IEEE Network, 10(3):614, May/June 1996.
[13] IRTF TMRG group. The transport modeling research groups web page.
http://www.icir.org/tmrg/.
[14] J. Heinanen, F. Baker, W. Weiss, and J. Wroclawski. Assured Forwarding
PHB Group. RFC 2597, June 1999. http://www.ietf.org/rfc/rfc2597.txt.
[15] V. Jacobson, K. Nichols, and K. Poduri. An Expedited Forwarding PHB.
RFC 2598, June 1999. http://www.ietf.org/rfc/rfc2598.txt.
[16] Nortel Networks. QoS Performance requirements for UMTS. The 3rd Generation Partnership Project (3GPP). http://www.3gpp.org/ftp/tsg
sa/WG1 Serv/TSGS1 03-HCourt/Docs/Docs/s1-99362.pdf.
[17] Nathan Sheldon, Eric Girard, Seth Borg, Mark Claypool, and Emmanuel
Agu. The effect of latency on user performance in warcraft iii. In In
Proceedings of ACM Network and System Support for Games (NetGames),
May 2003.
[18] L.A.R. Yamamoto and J.G. Beerends. Impact of network performance
parameters on the end-to-end perceived speech quality. In In Proceedings
of EXPERT ATM Traffic Symposium, September 1997.