A Checklist Based Approach for the Mitigation of

Buffer Overflow Attacks

S. K. Pandey, K. Mustafa, S. I. Ahson
Department of Computer Science
Jamia Millia Islamia (Central University), New Delhi-1 10025, INDIA
santo.pandaygyahoo. co. in, kmfarookigyahoo. com,drsiahsongyahoo. com
Abstract- Buffer overflows has appear to be one of the most after using the checklists are discussed in Section IV and
common problems in the area of software security. Many of Conclusions are drawn in Section V.
the buffer overflow problems are probably the result of
careless programming, which might have been found and II. BUFFER OVERFLOW ATTACKS
corrected by the developers, before releasing the software.
The work presented in this paper is intended to detect and Buffer overflow attacks exploit a lack of bounds checking
prevent such buffer overflow vulnerabilities. In this paper, on the size of input being stored in a buffer array. By
two separate checklists are proposed to both programmers as . .
well as test engineers for verifying the software during coding writing data past the end of an allocated array, the attacker
and testing phases respectively for building secure software. can make arbitrary changes to program state stored
adjacent to the array. By far, the most common data
Keywords: Software Security, Buffer Overflow, Attacks, structure to corrupt in this fashion is the stack, called a
Checklist "stack smashing attack," which we briefly describe here,
and is described at length elsewhere [10, 11, 12].
I. INTRODUCTION Many C programs have buffer overflow vulnerabilities,
both because the C language lacks array bounds checking,
Buffer overflow attacks aim to alter the execution of a and because the culture of C programmers encourages a
vulnerable program by copying data to a variable in such a performance-oriented style that avoids error checking
way that the original storage capacity is exceeded [1]. This where possible [13, 14, 16]. For instance, many of the
may cause excess data to spill over the unallocated address standard C library functions such as gets and strcpy do not
space and overwrite the pointer to the next instruction after do bounds checking by default [10].
a function call. However, in order to deploy the attack The common form of buffer overflow exploitation is to
successfully, execution must be accurately diverted to attack buffers allocated on the stack. Stack smashing
attacker's arbitrary code. To do so, the attacker might attacks strive to achieve two mutually dependent goals,
develop a program, which can assemble the different illustrated in Figure 1:
components of the malicious buffer. Moreover, because the Process Address Space
location of the vulnerable program in address space is OxFFFF
determined at runtime, certain characteristics of the
malicious buffer should be approximated in the code. Top of Stack
The principal objective of evolving overflow attacks is to Attack Code
access critical applications against buffer overflow
vulnerabilities. However, recent work on vulnerability
testing indicates that intrusion detection systems can detect
a particular instance of an attack, but are generally unable Return Address
to 'generalize' to the class of overflow attacks [2, 3, 4, 5, 6,
7, 8, 9]. Local Variables.
The main contribution of this work is to prepare
checklists for programmers as well as test engineers, to Buffer
take care of these attacks at the time of coding. And if any
thing is missed during coding phase by mistake then test
engineer must catch it during the testing phase by using
another checklist. Such practices may facilitate the Ox0000
mitigation of the attacks, exploiting the ' inherent buffer ~~~~~~~~~~~~~~~Figure
1: Buffer Overflow Scenario
overflow vulnerabilities.
The remainder of this paper is organized as follows. Inject Attack Code. The attacker provides an input string
Section II describes the buffer overflow concepts. that is actually executable, binary code native to the
Checklists methodology is discussed in Section III. Results machine being attacked. Typically this code is simple, and

978-1-4244-1878-7/07/$25.OO 2007 IEEE 115

does something similar to exec ("sh") to produce a root greater than the size of the buffer otherwise an
shell. off-by-one error may occur.
9 Programmers should avoid strcpy( and
Change the Return Address. There is a stack frame for a strcat(.They should use strncpy( and strncat()
currently active function above the buffer being attacked on instead. strcpy() and strcat() are with constant
the stack. The buffer overflow changes the return address strings for the source, but it's probably not safe to
to point to the attack code. When the function returns, assume that a string will always be constant.
instead of jumping back to where it was called from, it . Programmers should use precision specifiers with
jumps to the attack code. the scanf( family of functions (scanf(, fscanf(,
The programs that are attacked using this technique are sscanf(, etc.). Otherwise they will not do any
usually privileged daemons; the programs that run under bounds checking for the software.
the user-ID of root to perform some service. The injected * Programmers should never use variable format
attack code is usually a short sequence of instructions that strings with the printf( family of functions.
spawns a shell, also under the user-ID of root. The effect is * Programmers should watch for off-by-one errors
to give the attacker a shell with root's privileges. with otherwise safe functions such as memcpy(.
If the input to the program is provided from a locally * When using streadd( or strecpy(, it should make
running process, then this class of vulnerability may allow sure that the destination buffer is four times the
any user with a local account to become root. More size of the source buffer.
distressing, if the program input comes from a network . There should be proper care of every library
connection, this class of vulnerability may allow any user function that takes a pointer (especially string
anywhere on the network the ability to become root on the pointers) as input for a place to store its output. If
local host. Thus while new instances of this class of attack there is not another parameter for the buffer size
are not intellectually interesting, they are none the less or string size then that function does not do
critical to practical system security. bounds checking.
Engineering such an attack from scratch is surely non- . Programmers should also be careful and do
trivial. Often, the attacks are based on reverse-engineering bounds checking when using functions like getc(
the attacked program, so as to determine the exact offset and read( in a loop.
from the buffer to the return address in the stack frame, and . Functions like syslog( and getopt( take strings
the offset from the return address to the injected attack as input but, depending on the implementation,
code. However, it is possible to soften these exacting might not check the size of the string before using
requirements [11]: it. Programmers should truncate strings to a
reasonable length before passing pointers to them
1. The location of the return address can be approximated as input to any library function.
by simply repeating the desired return address several times * When using library functions that take buffer size
in the approximate region of the return address. as a parameter, programmers should make sure
that buffer is as big as they say it is.
2. The offset to the attack code can be approximated by
prepending the attack code with an arbitrary number of Here two checklists are proposed, one for Programmers,
NOP instructions. The overwritten return address need only and other for Test Engineers:
jump into the middle of the field of NOPs to hit the target.
A. Checklistfor Programmers
The cook-book descriptions of stack smashing attacks [10,
11, 12] have made construction of buffer overflow exploits 1. Is there bounds checking on arrays?
quite easy. The only remaining work for a would-be 2. Is there bounds checking on pointer arithmetic?
attacker to do is to find a poorly protected buffer in a 3. Is there any provision for checking before copy to,
privileged program, and construct an exploit. Hundreds of format, or send input to a buffer, for making sure that it is
such exploits have been reported in recent years [4]. big enough to hold whatever might be thrown at it?
4. Is there any provision for checking the input because
III. CHECKLIST METHODOLOGY input itself might cause a buffer overflow and not the size
of the input?
Some general guidelines of programming practices are 5. Is there correct use of unsafe library functions?
proposed by experts. Some of them are given as follows 6. Is there any provision for checking of off-by-one
[1l]: errors?
* Programmers should avoid gets(). They should 7. Is this clear that an array declared in C as int A[l00]
use fgets() with stdin as the file instead. fgets() can only be accessed with indices A[0] through A[99]?
will ask for specifying a string size, but it should 8. Is there any provision for checking old code?
be in mind that the size ofthe string should not be 9. Are all the assumptions taken with attacker's point of

116
view? evolutionary hackers," Proceedings of the Genetic and
Io.
u Isit
10. considered
Is lt con that
sl (l ere(l tn at all
al l buffer
DU ller overflows
overil ows are security
are aa security C Evolutionary
omputeComputer Computation
Science, Conference, Lecture Notes in
LNCS 3102, pp 263-274, 2004.
risk? [5] Marti R., "THOR: A tool to test intrusion detection systems by
variations of attacks", Master's Thesis, Swiss Federal Institute of
B. Checklistfor Test Engineers Technology, March 2002.
[6] Rubin S., Jha S., Miller B.P., "Automatic Generation and
Analysis ofNIDS Attacks", 20th Annual Computer Security
1. Is there a checking of all string and buffer inputs by Applications Conference - ACSAC, pp 28-38, 2004.
entering a very long string or too much data? [7] Tan, K.M.C., Killourhy, K.S., Maxion, R.A., "Undermining an
2. Is there any checking for one number in the partition? Anomaly-based Intrusion Detection System using Common
3. there
3. Is there any checking for each partition anychckingforechpatitioofvlueExploits",
of value
Is
or size
5th International Symposium on Recent Advances in
Intrusion Detection - RAID, Lecture Notes in Computer Science,
on any input? LNCS 2516, pp 54-73, 2002.
4. Is there any provision for testing of old code when [8] Vigna, G., Robertson, W., Balzarotti D., "Testing Network Based
it for new things, even if it is Intrusion Detection Signatures Using Mutant Exploits", ACM
programmers are using Conference on Computer Security, 2004.
tested before? [9] Wagner, D., Soto, P., "Mimicry Attacks on Host-based Intrusion
5. Is the code tested on all platforms it was meant to run Detection Systems", ACM Conference on Computer Security, pp
on? 255-264, 2002.
6. Are all the assumptions
.
.
taken with attacker's point of
.
htpdgeh.ow/avisriesBuferOvhtml,w99
~~~~~~~~[10]
"Mudge". How to Write Buffer Overflows.
~~~~~~~~~~~~~~~~~~~~~http
://10pht. com/advisories/bufero.html, 1997
view? [I 1] "Aleph One". Smashing The Stack For Fun And Profit. Phrack,
7. Is it considered that all buffer overflows are a security 7(49), November 1996.
risk? [12] Nathan P. Smith. Stack Smashing vulnerabilities in the UNIX
Operating System. http://millcomm.comFnate/
machines/security/stack-smashing/ nate-buffer.ps, 1997.
IV. RESULTS AFTER USING THE CHECKLISTS [13] Barton P. Miller, David Koski, Cjin Pheow Lee, Vivekananda
Maganty, Ravi Murthy, Ajitkumar Natarajan, and Jeff Steidl.
These checklists were tested in a development project of Fuzz Revisited: A reexamination ofthe Reliabilityof
UNIX Utilities and Services. Report, University of Wisconsin,
an organization and results given by them show that risk of 1995.
such attacks is reduced up to 80%. On the request of the [14] B.P. Miller, L. Fredrikson, and B. So. An Empirical Study of
organization, we are not disclosing the name and details of the Reliability of UNIX Utilities. Communications of the
the project. ACM, 33(12):33-44, December 1990.
[15] Michele Crabb. Curmudgeon's Executive Summary. In Michele
Crabb, editor, The SANS Network Security Digest. SANS,
V. CONCLUSION & FUTURE WORK 1997. Contributing Editors: Matt Bishop, Gene Spafford, Steve
Bellovin, Gene Schultz, Rob Kolstad, Marcus Ranum, Dorothy
A checklist based solution is proposed for prevention and Denning, Dan Geer, Peter Neumann, Peter Galvin, David
detection of buffer overflow attacks. If programmers and Harley, Jean Chouanard.
[16] S. K.Pandey, S. I. Ahson: "A Taxonomy of Software
prevtentiogmas beepdnesatthe timeofcoding
mmantnesting
test engineers keep these points in mind, then the
prevention may be done at the time of coding and testing
Security Vulnerabilities", Proceedings of the National
Conference on Security Issues in e-commerce, Aligarh, India,
phase respectively. In future, we are planning to make a Mar 10, 2007.
security assessment framework in which we will use these
checklist based mitigation mechanisms of defects. This
work will help to software programmers and security
experts for building secure software which is the main
objective of this project.
ACKNOWLEDGEMENT
The research work is funded by DIT, Ministry of
Communications and Information Technology, Govt. of
India, under grant no. 12(51)/05-ESD.
REFERENCES
[1] Foster J.C., Osipov V., Bhalla N., Heinen N., "Buffer Overflow
Attacks: Detect, Exploit, Prevent", Syngress Publishing, ISBN
1-932266-67-4, Ch.5, 2005.
[2] Christodorescu M., Jha S., "Static analysis of executables to
detect malicious patterns", Proceedings ofthe USENIX Security
Symposium, 2003.
[3] Detristan T., Ulenspiegel T., Malcom Y., Underduk M. S.,
"Polymorphic shellcode engine using spectrum analysis",
Phrack Online Magazine, 61, 2003.
[4] Dozier, G., Brown, D., Cain, K., Hurley, J., "Vulnerability
analysis of immunity-based intrusion detection systems using

117