ASIDE: IDE Support for Web Application Security

Jing Xie, Bill Chu, Heather Richter Lipford, and John T. Melton
Department of Software and Information Systems
University of North Carolina at Charlotte
9201 University City Blvd
Charlotte, NC, 28223, USA
jxie2, billchu, [email protected], [email protected]

ABSTRACT today [36]. Unfortunately, many developers have not been

Many of today’s application security vulnerabilities are in- adequately trained in writing secure programs that are re-
troduced by software developers writing insecure code. This sistant from attacks violating program confidentiality, in-
may be due to either a lack of understanding of secure pro- tegrity, and availability. We refer to this concept of writing
gramming practices, and/or developers’ lapses of attention secure code as secure programming [4]. Programmer errors,
on security. Much work on software security has focused on including security ones, are unavoidable even for well-trained
detecting software vulnerabilities through automated anal- developers. One major cause of such errors is software de-
ysis techniques. While they are effective, we believe they velopers’ heavy cognitive load in dealing with a multitude
are not sufficient. We propose to increase developer aware- of issues, such as functional requirements, runtime perfor-
ness and promote practice of secure programming by interac- mance, deadlines, and security. Consider Donald Knuth’s
tively reminding programmers of secure programming prac- analysis of 867 software errors he made while writing TEX
tices inside Integrated Development Environments (IDEs). [19]. It is clear from the log that some of these errors could
We have implemented a proof-of-concept plugin for Eclipse have made TEX vulnerable to security breaches. The follow-
and Java. Initial evaluation results show that this approach ing quote illustrates Knuth’s experience of heavy cognitive
can detect and address common web application vulnerabil- burden as a major source of software errors:
ities and can serve as an effective aid for programmers. Our
“Here I did not remember to do everything I had
approach can also effectively complement existing software
intended when I actually got around to writing
security best practices and significantly increase developer
a particular part of the code. It was a simple
productivity.
error of omission, rather than commission. . . This
seems to be one of my favorite mistakes: I often
Categories and Subject Descriptors forget the most obvious things” [19].
D.2 [Software Engineering]: [Software Security]; D.2.4
[Software/Program Verification]: [Correctness Proofs]; Current tool support for secure programming, both from
D.2.6 [Programming Environments]: [Integrated Envi- tool vendors as well as within the research community, fo-
ronments] cuses on catching security errors after the program is writ-
ten. Static and dynamic analyzers work in a similar way as
early compilers: developers must first run the tool, obtain
General Terms and analyze results, diagnose programs, and finally fix the
Security code if necessary. Thus, these tools tend to be used to find
vulnerabilities at the end of the development lifecycle. How-
Keywords ever, their popularity does not guarantee utilization; other
business priorities may take precedence. Moreover, using
Secure programming, application security, interactive sup- such tools often requires some security expertise and can
port, secure software development be costly. If programmers are removed from this analysis
process, these tools will also not help prevent them from
1. INTRODUCTION continuing to write insecure code.
Software vulnerabilities originating from insecure code are We believe these vulnerability detection tools could be
one of the leading causes of security problems people face complemented by interactive support that reminds develop-
ers of good secure programming practices in situ, helping
them to either close the secure programming knowledge gap
or overcome attention/memory lapses. This approach can
Permission to make digital or hard copies of all or part of this work for be justified based on cognitive theories of programmer errors
personal or classroom use is granted without fee provided that copies are [20, 21, 38]. Our hypothesis is that by providing effective
not made or distributed for profit or commercial advantage and that copies reminder support in an IDE, one can effectively reduce com-
bear this notice and the full citation on the first page. To copy otherwise, to mon security vulnerabilities. Our approach is analogous to
republish, to post on servers or to redistribute to lists, requires prior specific word processor spelling and grammar support. While people
permission and/or a fee.
ACSAC ’11 Dec. 5-9, 2011, Orlando, Florida USA can run spelling and grammar checks after they have writ-
Copyright 2011 ACM 978-1-4503-0672-0/11/12 ...$10.00. ten a document, today’s word processors also provide visual


cues – colored lines drawn underneath potential errors – to as a helpful assistant and does not hinder developers’ cre-
help writers notice and fix problems while they are com- ativity and productivity by dictating a rigid approach to
posing text. Similarly, our approach proposes that an IDE secure programming.
could interactively identify parts of the program where se- Third, an in-situ reminder tool can be an effective train-
curity considerations, such as input validation/encoding or ing aid that either helps novices to learn secure program-
Cross-site Request Forgery (CRSF) protection, are needed ming practices or reinforces developers’ secure programming
while programmers are writing code. training, making security a first class concern throughout
In this paper, we describe our interactive reminder ap- the development process. This will help developers learn to
proach through the Application Security IDE (ASIDE), an reduce their programmer errors over time, reducing costly
Eclipse plugin for Java. ASIDE uses two key techniques to analysis and testing after implementation.
help programmers prevent errors: interactive code refactor- Finally, we want to support sharing secure programming
ing and interactive code annotation. While we have pre- knowledge and standards amongst development teams. In
viously introduced this idea of an interactive approach to an industrial setting, ASIDE could be configured by an orga-
secure programming [37], in this paper, we present our pro- nization’s software security group (SSG), which is responsi-
totype implementation, along with a detailed evaluation ble for ensuring software security as identified by best indus-
of this plugin on two large-scale open source code bases, try practice [11]. Thus, a SSG could use ASIDE to communi-
demonstrating its potential feasibility and effectiveness at cate and promote organizational and/or application-specific
identifying and preventing important security vulnerabili- programming standards. In addition, the plugin can gener-
ties. We also describe a user evaluation showing that users ate logs of how security considerations were addressed dur-
embrace such unobtrusive and helpful reminder support, and ing construction, providing necessary information for more
actively engage in the interaction with ASIDE. Thus, the effective code review and auditing.
major contribution of this paper is the multi-aspect eval- ASIDE first must detect potential vulnerabilities in pro-
uation of our proposed approach. Through our evaluation gram code, and alert programmers to those vulnerabilities.
process, we also identify directions for future improvements Thus, to be effective, ASIDE must exhibit a certain level
to ASIDE and research. of precision in identifying vulnerable code in order to pro-
vide proper assistance to address it. Based on our prototype
2. ASIDE: APPLICATION SECURITY IN IDE implementation, a vulnerability can be easily eliminated by
either of the two interactive techniques if the correspond-
The ASIDE plugin is a prototype implementation of our
ing vulnerable code is correctly identified. Therefore, a key
interactive approach to reminding and helping programmers
measure of the effectiveness of ASIDE is the measure of its
to perform good secure programming practices. Our ap-
ability to find vulnerable code. For each of the interactive
proach was based on a number of design considerations.
techniques presented below, we present the details of how
First, it is easiest and most cost effective for developers to
effective ASIDE is at detecting the corresponding vulnera-
write secure code and to document security implementation
bilities. The effectiveness of the interactive techniques to ad-
during program construction. This means creating a tool
dress those vulnerabilities heavily depends on how the pro-
that integrates into the programmers’ development environ-
grammer responds to the warning. This we evaluate based
ment is promising. As a starting point, we chose the popu-
on a preliminary user study of programmer behaviors, pre-
lar Eclipse platform, and implemented ASIDE to work with
sented in Section 7.
Java. Our current implementation of ASIDE is primarily
designed to assist Web-based applications and to mitigate
commonly committed vulnerabilities due to improper input 3. INTERACTIVE CODE REFACTORING
validation and/or filtering, broken access control, and Cross- Interactive code refactoring [37] works in a manner similar
site Request Forgery (CSRF). ASIDE, once activated, con- to a word processor that corrects spelling and grammatical
tinuously monitors Eclipse workspace changes in order to re- errors. In this case, ASIDE automatically inserts necessary
spond to newly created projects, as well as modifications to code, with the help of the programmer. We believe code
existing projects. ASIDE performs static analysis incremen- refactoring can be appropriately applied to input validation
tally on a small chunk of code under construction, thus en- and/or filtering types of vulnerabilities. In this section, we
suring prompt response to developers’ immediate code edit- discuss an example concerning input validation to illustrate
ing. Such analysis is carried out “under the hood” without the key concepts.
requiring developers to understand its security implications. A developer is alerted by a marker and highlighted text
Our second consideration is the interface design principle in the edit window when input validation is needed. ASIDE
that recognition is favored over recall [33]. Developers are has a rule-based specification language, which is also XML-
provided with appropriate visual alerts on secure program- based, to specify sources of untrusted inputs which we for-
ming issues and offered assistance to practice secure pro- mally named as trust boundaries. Currently two types of
gramming. ASIDE provides these alerts as a colored icon on rules are supported: Method (API) invocations, for exam-
the left hand side margin of the code editing window, accom- ple, method textttgetParameter(String parameter) in class
panied with highlighted text, see Figure 1. A developer can HttpServletRequest introduces user inputs from clients into
then either click on that icon or hover over the highlighted the system; and Parameter input, for instance, arguments of
text to interact with ASIDE to address the issue. Our two the Java program entrance method main(String[] args).
key mechanisms for this interactive support are interactive With a mouse click, the developer has access to a list of
code refactoring and interactive code annotation, which are possible validation options, such as a file path, URL, date, or
discussed in detail in the following sections. There are no safe text. Upon the selection of an option, appropriate input
additional steps a developer must remember, and they can validation code will be inserted and the red marker will be
interact with ASIDE when they choose. Thus, the tool acts dismissed. Figure 1 shows a screenshot of ASIDE facilitating


a developer to select an appropriate input validation type for composite data type is unbound, ASIDE leaves the marker
an identified untrusted input. The library of input validation shown in Figure 3 throughout the development to serve as
options can be easily reconfigured by an individual developer a continual reminder.
or an organization.

Figure 3: Visually softer marker that marks a

tainted input with composite data type: Map.

ASIDE supports two types of input validation rules: syn-

tactic rules and semantic rules. A syntactic rule defines the
syntax structure of an acceptable input and is often rep-
resented as a regular expression. Examples include valid
Figure 1: The user interactively chooses the type of names, addresses, URLs, filenames, etc. Semantic rules de-
input to be validated using a white-list approach. pend on the application context. For example, restricting
the domain of URLs, files under certain directories, date
range, or extentions for uploaded files. They can also be
Figure 2 illustrates how ASIDE refactors code to perform
used to validate inputs of special data types, such as certain
input validation using OWASP Enterprise Security API (ESAPI)
properties of a sparse matrix. While validation rules can be
Validator [28].
declaratively specified by a SSG, a developer has the option
to address the identified input validation issue by writing
custom routines. This is then documented by ASIDE for
later security audit. A developer can also easily dismiss
ASIDE warnings if they are determined to be irrelevant. In
any case, once the alert has been addressed, the correspond-
ing red marker and text highlights will disappear in order to
reduce programmer distraction and annoyance.
Another benefit of ASIDE is that it can help enforce secure
software development standards across the organization. For
Figure 2: ASIDE validates an input using OWASP example, a company may deploy a validation and/or encod-
ESAPI validator API. ing library and clearly define a set of trust boundaries for
the purpose of performing input validation and/or encod-
Previous works employing refactoring techniques for se- ing. Once appropriatly configured with the defined trust
cure programming use program transformation rules, which boundaries and libraries, ASIDE can collect information as
operate on completed programs, and thus work best on legacy to where in the source code an untrusted input was brought
code. One recognized limitation of the program transforma- into the system and what actions a developer took to address
tion approach is the lack of knowledge of specific business validation and/or encoding. ASIDE can also effectively sup-
logic and context [12]. In contrast, our approach is designed plement the security audit process by generating rules for
to provide interactive support for secure programming and traditional static analyzers. For example, once an untrusted
takes full advantage of developers’ contextual knowledge of input has been validated, customized Fortify rules can be
the application under development. generated to remove taints, thus avoiding unnecessary is-
There are two possible strategies for when to perform in- sues being generated during the auditing process. This can
put validation. One is to validate a variable containing un- significantly reduce the time of a software security audit.
trusted input when it is used in a critical operation, such
as a database update, insertion, or deletion. The other is
to validate an untrusted input as soon as it is read into an 4. EVALUATION OF CODE REFACTORING
application-declared variable. A major disadvantage of the We now evaluate the effectiveness of ASIDE’s vulnerable
first strategy is that it is not always possible to predict what code detection for interactive code refactoring using an open
operations are critical, and thus fails to validate input when source project of realistic scale. Thus, in this section, we
the context of the application evolves. ASIDE promotes focus on input validation and/or encoding vulnerabilities,
what we believe is the best practice for secure programming, as the ones currently supported by code refactoring. Our
validating untrusted inputs at the earliest possible time [8]. goals are to determine: (a) How effective is ASIDE at dis-
Another issue is that untrusted inputs could be of com- covering exploitable software vulnerabilities and preventing
posite data type, such as a List, where input types may be them? and (b) What constitutes false positives for ASIDE?
different for each element of the list. In the current ASIDE The significance of this evaluation is the use of real world
implementation, the developer is warned of taint sources cases that help us understand the effectiveness of ASIDE
of a composite type with a visually softer yellow marker and provide guidance for further research.
as shown in Figure 3. ASIDE uses data flow analysis to
track an untrusted composite object. As soon as the de- 4.1 Establishing a baseline using an open source
veloper retrieves an element that is of primitive data type project
(e.g.java.lang.String), ASIDE alerts the need to perform We selected Apache Roller (a full-featured blog server) [30]
input validation and/or encoding in the same manner as release version 3.0.0 because it is one of the few mature Java
described above. Given that the element retrieval from a EE based open source web applications. A Google search of


“powered by Apache Roller” yielded over 1.8M entries in- the 922 issues of input validation and/or encoding we will
cluding sites such as blogs.sun.com. One of the co-authors, compare to ASIDE.
who is an experienced member of a SSG at a large financial It is common that multiple Fortify issues share the same
service organization, performed a software security audit us- root cause of an untrusted input, referred to as a taint
ing his company’s practices to identify security vulnerabili- source. A single taint source may reach multiple taint sinks,
ties that are exploitable in Roller. exploitable API calls, and thus generates several different
The audit process consisted of two parts: (1) automatic vulnerabilities. For example, a user-entered value might lead
static analysis using Fortify SCA [34], and (2) manual ex- to a Log Forging vulnerability if it is inserted into a log, and
amination of Fortify findings. Default Fortify rules were a SQL Injection if it is used in an operation that executes a
used, followed by manual auditing to eliminate Fortify find- dynamic SQL statement.
ings that are not immediately exploitable. For each issue The 922 Fortify issues are caused by 143 unique taint
reported by Fortify, its source code was reviewed to: sources including both primitive data types (e.g. java.lang
.String) and composite data types (e.g. java.util.Map).
• determine whether appropriate input validation/encoding Variables requiring output encoding are always the result of
has been performed; a taint source. Thus, we exclude them in our analysis to
• determine whether Fortify’s environmental assumption avoid duplications.
is valid. For example, in the case of log forging, whether
the logging mechanism has not been wrapped in any
Table 2: Detail results from security auditing
way that prevents log forging;
against Roller using Fortify SCA.
• determine whether Fortify’s trust boundary assump- Severity Category Name
tion is valid. For instance, whether property files are Cross-Site Scripting: Persistent 2
considered to be trusted, and in this case, data from Cross-Site Scripting: Reflected 2
Critical
property files is untrustworthy; Path Manipulation 19
SQL Injection 11
• scrutinize input validation and encoding routines to Cross-Site Scripting: Persistent 31
make sure they are proper. For example, check if Denial of Service 4
blacklist-based filtering is used. File, LDAP, DB, and Medium Header Manipulation 52
Web all require different encoders or filters because Log Forging 252
different data schemes are used; and Path Manipulation 6
Cross-Site Scripting: Poor Validation 6
• pay close attention to DOS related warnings (e.g. file Log Forging (debug) 531
handles and database connections) as resources may be Low
SQL Injection 3
released in a non-standard way. Often times, warning Trust Boundary Violation 3
are generated even when resources are released in a
Total 922
finally block.
Roller 3.0.0 has over 65K lines of source code. Fortify
reported 3,416 issues in 80 vulnerability categories, out of 4.2 Vulnerability Coverage of ASIDE
which, 1,655 issues were determined to be exploitable vul- We then imported Roller into an Eclipse platform that has
nerabilities. Table 1 summarizes the results of this audit ASIDE installed and configured. ASIDE identified 131 of
process. Based on the evaluator’s experience, Roller’s secu- the 143 (92%) exploitable taint sources. The remaining 12
rity quality is at the average level of what he has evaluated. cases involve JSP files and Struts form beans. The current
According to current work load estimate metrics of his en- ASIDE implementation does not cover these cases, but they
terprise, the analysis work reported here would amount to could be easily handled in future implementations.
2.5 person days. 41 of the 143 are taint sources of composite data returned
from APIs such as org.hibernate.Criteria.list() and
javax.servlet.ServletRequest.getParameterMap(). ASI
Table 1: Results rendered by the industry security -DE performs dataflow tracking within the method where
auditing process on Apache Roller version 3.0.0. untrusted input is read. When a primitive value (e.g. java.
Critical High Medium Low lang.String) is retrieved from the composite data structure
Fortify Issue Categories 8 18 2 52 instance, ASIDE will raise a regular warning and provide as-
Raw Issues 164 653 13 2,597 sistance to validate and/or encode that input, as described
Exploitable Issues 37 397 0 1,221 in Section 3.
While we successfully identified tainted inputs of compos-
ASIDE’s code refactoring is primarily aimed at preventing ite data types in Roller, in many cases, developers did not
vulnerabilities resulting from lack of input validation and/ use the elements in that data object within the immediate
or encoding. Out of the 1,655 Fortify issues that can be method. Since ASIDE only currently performs taint track-
exploited in Roller, 922 (58%) of them are caused by lack ing within the immediate method declaration, future im-
of input validation and/or encoding including most of the plementations of ASIDE will be expanded to support taint
vulnerabilities from the critical bucket. The rest, mostly in tracking for composite objects beyond the scope of the im-
the low security risk category, are related to failure to release mediate method declaration, which would then alert the pro-
resources (e.g. database connection) and other bad coding grammer to all these primitive data type uses.
practices. Table 2 lists the details of the audit findings for Our analysis also raised the issue of delayed binding. An



example of delayed binding is the access methods in a POJO any recognized security vulnerabilities. These often involve
(Plain Old Java Object). For example, setBookTitle() inputs that are highly specific to the application context.
method of a Java class Book.java with a bookTitle at- For example, as illustrated in Figure 5, an input is tested to
tribute of String type. Binding of access methods to input see if it equals to a constant value, determining the applica-
streams can be delayed until after the program is completed. tion flow.
Thus, at the time of writing the program, there is no strong
reason to believe the input is untrusted. After completion of
the application, an integrator may bind an untrusted input
stream directly to a POJO, making the application vulner-
able. Figure 5: Untrusted input is used for logic test.
Delayed binding is a difficult problem for existing static
analysis tools as well. If the binding specification (typically Another such case is shown in Figure 6, where the input is
in XML format) is composed in the same IDE environment, cast into a Boolean value with only two possible outcomes:
which is usually the case for Java EE development, one could true and false, which will not result in any harm to the
extend ASIDE to help developers discover input validation intended application logic.
issues by resolving the binding specifications. Further re-
search is needed on the best approach to address delayed
bindings in ASIDE.
4.2.1 False Positive for ASIDE Figure 6: Untrusted input is parsed into harmless
As we just demonstrated, ASIDE had good coverage of the Boolean value.
input validation/encoding issues in Roller. In this section,
we discuss the additional warnings that ASIDE generated. Because false positive rate often is positively correlated
In analyzing false positives, we only look at taint sources of to accuracy, it is difficult to design a highly accurate tool
primitive data types. Taint sources of composite types are with very low false positive rates. Both traditional analysis
accounted for when elements of the composite object are tools, such as Fortify SCA, and ASIDE will require manual
retrieved and treated as a taint source of a primitive data inspection of warnings to eliminate false positives. However,
type. our analysis of Roller suggests that for vulnerabilities due to
ASIDE reported 118 taint sources of primitive data types improper input validation and/or encoding, ASIDE gener-
that were not identified as exploitable Roller vulnerabilities ates far fewer issues than Fortify, reducing the workload for
by the Fortify software security audit. 94 of them are cases both developers as well as software security auditors.
that are not exploitable at the moment. For example, a Additionally, we believe that it may take less effort to
taint source does not reach any taint sink. Failure to val- recognize and deal with ASIDE false positives compared
idate/encode the untrusted input may not be exploitable to those generated by traditional static analysis. ASIDE’s
in the context of the current application. However, often warnings are generated while the developer is actively en-
times, such untreated inputs will eventually be used, and gaged in the programming process, making it easier to exam-
thus cause exploitable security vulnerability as the software ine and understand the context of the warning. Moreover,
evolves. Therefore, we believe it is still a good secure pro- with a click of a button on ASIDE’s resolution menu, the de-
gramming practice to validate/encode all untrusted inputs, veloper can dismiss a warning as false positive. In contrast,
regardless of whether they will reach a taint sink or not. false positives generated by traditional analysis tools such as
Figure 4 shows another example from Roller, where a Fortify SCA are often dealt with by either software security
tainted request URL is directly passed into an InvalidReques- auditors who typically do not have full application knowl-
tException constructor, and eventually inserted into the er- edge or by developers after the program was completed. In
ror log. Fortify default rules do not acknowledge this code both cases, we believe it will take them longer to fully under-
to be vulnerable. However, if the logs are viewed in a web- stand the impact of a particular warning generated by static
based log viewer such as a web browser, which is common in analysis and to recognize it as a false positive. As excessive
some organizations, this would allow an attacker to launch false positives could have a negative impact on the usability
a Cross-site Scripting attack on the system administrator of any tool, further research is needed to understand how
reviewing the log. false positives in ASIDE impact developer behavior.

5. INTERACTIVE CODE ANNOTATION

Interactive code annotation [37] is a mechanism that helps
developers to avoid more subtle security vulnerabilities where
Figure 4: Untrusted input is logged through an Ex- code refactoring is not feasible, such as broken access con-
ception construction. trol and CSRF. Instead, programmers are asked to indicate
where practices were performed, which is added as an an-
Thus, from a broad secure programming best practice per- notation to the code. This serves as both a reminder to
spective, we believe these 94 cases should be regarded as perform good practices, and enables further analysis and
true positives, and ASIDE’s warnings should still be fol- auditing. We first discuss interactive code annotation using
lowed. However, from a circumscribed perspective of a spe- an example of access control.
cific application, they may be regarded as false positives. Consider an online banking application with three database
The remaining 24 reported taint sources we regard as false tables: user, account, account user, and transaction,
positive, where inputs are used in ways that do not lead to where the tables account and transaction are specified as



requiring authentication in such a way that the subject must We are still in the process of implementation interactive
be authenticated by the key of the user table, referred to as code annotation in ASIDE. Thus, the screenshots in this
an access control table. section are of our design, not a working prototype. We first
Figure 7(b) shows a highlighted line of code in the doGet() discuss the issues in implementing code annotation, before
method of the accounts servlet, which contains a query to ta- moving on to our analysis of its potential effectiveness.
ble account. ASIDE requests the developer to identify the Our implementation of code annotation will take a knowledge-
authentication logic in the program, again by a red marker based approach relying on the specific structure of the target
and highlighted text in the editing window. In this case, the technology, initially Java servlet-based applications. We are
developer highlights a test condition reques.getsession(). first focusing on requesting programmers’ annotations on ac-
getAttribute(“USER”) == null as illustrated in Figure 7(b), cess control logic The key implementation issues are (a) how
which ASIDE saves as an annotation to the query code. The to identify the application context to prompt the developer
annotations can be reviewed and modified in an additional for annotation and (b) what are the forms of acceptable an-
view as shown in Figure 7(b) and Figure 7(c), on which notations.
different information corresponding to different actions the If we know the names of database tables whose access re-
developer has taken is displayed. Thus, the annotation pro- quires authentication, we can easily identify program state-
cess is seamlessly integrated into the IDE without requiring ments that access these tables. Thus, we will provide a
the developer to learn any new annotation language syntax. method for designers or a SSG to specify such tables as
part of configuring ASIDE. However, such database access
statements may not convey important application context,
as they may be part of a utility library used by different web
requests. For example, the account table may be accessed
by the same access routine in multiple web requests (e.g.
“customer account balance”, and “electronic fund transfer”).
Also, different web requests may require different access con-
(a) Developer identifies authentication logic (highlighted trol logic (e.g. two-factor authentication for fund transfer).
text) upon request from the ASIDE (see marker and high- Therefore, we need to identify locations where application
lighted text of Figure 7(b)) and annotates it.
logic takes place by invoking database access operations.
For Java servlet-based web applications, access control an-
notations can be requested in the context of a web request:
as in “where is the authentication logic for accessing the ac-
count table in the customer account balance request?”, and
“where is the authorization logic for accessing the account
table in the electronic fund transfer request?” In this exam-
ple each web request is implemented as a servlet. Therefore,
we will start by tracking doGet() and doPost() methods
within each servlet class (and this can be easily extended
to include JSP pages)1 . They are referred to hereafter as
(b) ASIDE issues a question for proper access control check entry methods. Through static analysis techniques, we will
that grants/denies the access to the highlighted data access
operation. The detail of such request is displayed on the detect cases where an entry method leads to a statement ac-
view called Annotation below the code editing window. cessing a database table which requires access control. The
developer will then be requested to provide annotation of
authentication in the context of the entry method, as illus-
trated in Figure 7(b).
Our annotation design satisfies the following requirements:
(a) it consists of a set of logical tests (e.g. conditional tests
in if statement, cases in switch statements), as shown in
(c) The Annotation view adds the annotated information as Figure 7(a), and (b) each test must be on at least one execu-
a response to the developer’s annotating of the access control tion path, as determined by the control flow, from the start
logic in above Figure 7(a). of the entry method to the identified table access statement.
Annotations may enable more in-depth static analysis.
Figure 7: An example showing how ASIDE interac- We are specifically looking into one type of execution anal-
tive code annotation works. ysis. For example, a broken access control may be detected
if there is an execution path in the entry method leading
to the database access without any identified access control
Several benefits can be derived from this annotation mech-
checks along the path. We believe such an analysis can also
anism. First, the developer is reminded of the need to per-
be used to help prevent CSRF vulnerabilities. Of course,
form authentication, and the annotation process may help
the accuracy of this analysis depends on the accuracy of the
the developer to verify that authentication logic has been in-
annotation. We now move to our evaluation of code anno-
cluded. The developer has an opportunity to add intended
tation against real world cases.
access control logic should that be missing. Second, the
logged annotations provide valuable information for code
review. Third, heuristics-based static analysis can be per-
formed to provide more in-depth analysis of the authentica- 1
Most popular Java frameworks also have structured entry
tion logic. methods, such as controller in Spring MVC



6. EVALUATE INTERACTIVE CODE AN-
NOTATION
We have conceptually tested this approach on real world
open source projects. The security audit did not identify
any broken access control or CSRF issues in Roller. Thus, Figure 9: Annotate access control logics.
we turned to bug tracking records to uncover previously dis-
covered issues. Since Apache Roller only has a small number
of fully documented security patches, we also included secu- access point with an annotated access control check on it,
rity patch information from Moodle [25], a PHP-based open while another path from the same entry point to the same
source project for course management. A Google search data access point has no access control check on it. There-
of “powered by Moodle” yielded over 4.3M sites including fore, ASIDE would be able to provide a warning to the de-
many large universities. A total of 20 fully documented veloper of a potential broken access control vulnerability.
security patches were found for the two projects. Four of
them are due to improper input validation and/or encoding
and can be addressed by ASIDE’s code refactoring support.
Out of the remaining 16 vulnerabilities, 3 (1 broken access
control and 2 CSRF) can be addressed by code annotation
and the path analysis heuristics outlined above.
The broken access control issue is from Roller [31]. The
authenticator, as illustrated in Figure 8, gets a web request
from the client side and checks to see whether the headers
of the request are valid. If they are valid, it extracts the
credentials and verifies the validity of them. If the creden-
tials are valid, the program goes on to access protected data
in the database. If the credentials are not valid, an excep-
tion will be thrown, thus preventing unauthenticated access. Figure 10: Java Servlet code for processing authenti-
However, there is another path from the web entry point to cation request (left) and Access control check graph
the data access point when the headers do not conform to that involves processing the request (right).
the expected format, as shown in the control flow diagram
in Figure 8. Two CSRF vulnerabilities were recorded in Moodle’s bug
tracking system. An effective way to prevent CSRF is to
assign a random token to each instance of a web page that
changes the state of the application. The token will be ver-
ified by the server before the application state is changed.
The Moodle project is clearly well aware of the CRSF prob-
lem and implemented this strategy as a set of standard util-
ity functions, simplifying developers’ tasks. However, devel-
opers still missed using this CSRF protection, introducing
serious vulnerabilities into the software.
Through code annotation, ASIDE can be designed to re-
mind developer of places where CSRF protection is needed,
such as web transactions that change application states.
Whenever a form submission/web request contains an oper-
ation to update (add, delete, modify) database entries, the
form submission needs to be checked for CSRF. We describe
Figure 8: Control flow diagram of how an authenti- one of the CSRF cases in Moodle, MSA-08-0013 [26], in de-
cation request is processed. tail; the other example is similar. This CSRF vulnerability is
based on editing a user’s profile page, edit.php. Since ASIDE
Applying ASIDE’s code annotation approach, when the is currently being designed for the Java EE environment, we
application code accesses the protected database resource to recast this example in equivalent Java terms.
get all users’ information, ASIDE would prompt a request The edit function would have a web entry point such as
for proper access control logic on the path from the web in a Servlet. Function update_record() is called, as high-
request to the data access method call. Considering that lighted in Figure 11, to update selected database tables
the question should be raised on a transaction level, line 52 through database operations. ASIDE would prompt the de-
in the Servlet processing the request would be highlighted, veloper to annotate a logic test that implements CSRF pro-
as shown in Figure 10. In this case, the developer could tection. The request for annotation would be made at the
easily identify the access logic as the logic tests which lie in line where update_record() is called. In this case, CSRF
the method invocation verifyUser(username, password) protection was omitted, so the programmer would be re-
in BasicAuthenticator.java, highlighted in Figure 9. In minded to add such protection. Once the CSRF protection
this case, there are three tests to be annotated. is added with appropriate annotation, ASIDE will apply the
Based on the developer’s annotation, ASIDE would be path analysis heuristics to further check for possible logic er-
able to construct a control flow graph, as illustrated in Fig- rors that may bypass the CSRF protection.
ure 10, that has one path from a web entry point to a data We have not yet conducted an analysis on false positives


 ings? (b) do they use code refactoring functions, and (c) do
warnings and code generation help developers produce more
secure code?
Figure 12 depicts the results of all 9 participants. Over all
9 participants, 101 distinctive ASIDE warnings were gener-
ated, resulting in 11.2 warnings per participant on average.
83 warnings were clicked by participants, 9.2 for each par-
ticipant on average.
Figure 11: A snippet of source code of the web trans-
action that changes user profile.

for code annotation since we have not completed a prototype

of ASIDE’s code annotation functions, however, we would
like to point out a case where false positives may appear.
It is not uncommon for a web application to access differ-
ent tables of a database to respond to one single request.
Without context, ASIDE would raise the same questions for
each and every function call that changes a table, and thus
may produce false positives. Further research is needed to Figure 12: Metrics and results from 9 participants.
make ASIDE more contextual and intelligent about asking
questions in these cases. Out of the 83 warnings clicked, 63 were addressed (7 per
participant on average) by clicking on one of the validation
7. DEVELOPER BEHAVIOR STUDY rules provided by ASIDE, leading to validation/encoding
code being generated. The remaining warnings were dis-
The previous evaluations focused on the ability of ASIDE
missed by participants. All participants used ASIDE to
to detect or fix vulnerable code. However, ASIDE must be
generate validation routines in their development. None of
designed in a way that fits naturally into a developer’s work
them wrote any customized validation or encoding routines.
environment in order to be successful. To gain an under-
Thus, ASIDE’s code refactoring was effective in helping stu-
standing of programmers’ reactions towards real-time secure
dents write more secure code, even though they were not
coding support, we conducted a pilot user study involving
required to do so.
9 graduate students from our college using our current im-
Multiple factors explain why certain warnings were not
plementation of ASIDE in a controlled experimental setting.
clicked or acted upon. Some of the warnings were generated
All 9 participants were recruited from a Java based web ap-
when the participant wrote debugging code, which was soon
plication development course in the Spring 2011 semester.
deleted. Perhaps students ignored security warnings on code
As part of the course, students were briefly introduced to
that they knew was transient. Other cases have to do with
basic secure web programming techniques such as input vali-
a bug in the version of ASIDE used, which falsely warned
dation and encoding. However, project grades were assigned
participants of the need for output encoding. We noticed
only based on functional requirements, not on secure coding
that participants learned this warning was a false positive
practices.
after one or more encounters, and then ignored those warn-
Part of the students’ course work was to build an online
ings thereafter. However, at least in this study, the presence
stock trading system incrementally over 4 projects through-
of a false positive did not seem to cause the participants
out the semester. Our study focused on the last increment
to not pay attention to other ASIDE warnings. Thus, in
of this project where students were asked to implement func-
cases where participants encountered false positives, they
tionality including add a banking account, display stock de-
were able to recognize them quickly and use the options
tails, make a buy/sell transaction, and display transaction
provided by ASIDE to dismiss the false positives.
history. Students added these functions on top of their ex-
We transcribed all interviews into text via Inqscribe [15],
isting work artifacts, which included static web pages, login,
and coded the transcripts using Atlas.ti [2] to identify gen-
logout, and register functionalities.
eral themes and interesting cases. All but one of the par-
We asked students to come to our lab and work on their
ticipants indicated that if they were not given ASIDE, they
assignment for 3 hours, using Eclipse with ASIDE. ASIDE
would not have been aware that some of the inputs needed
was implemented as described in Section 3, generating in-
to be validated before being used. Furthermore, they unan-
put validation and encoding warnings. Participants were
imously agreed that they would not write their own valida-
given a brief tutorial of ASIDE, then told to work on what-
tion routines should ASIDE not provide them in the context
ever aspects of their code they wanted, using ASIDE as
of their assignment. As one participant stated:
they wished. We recorded all their interactions with ASIDE
through screen recording software, and logged all interac- [P1] That (The warning design of ASIDE) was
tions with the ASIDE interface. Immediately after the 3- good because hadn’t it prompted me, I wouldn’t
hour study, participants were interviewed about their expe- have to realize I have to inspect those input val-
riences of using ASIDE. Participation was voluntary and not ues.
related to class grades; students were compensated with a
$30 gift card for his/her time. No one expressed that the ASIDE warnings were annoy-
Our goals with this study were to evaluate the usability of ing, and all seemed to have faith that using ASIDE would
ASIDE and determine: (a) do users pay attention to warn- make their code more secure.


 [P4] No (it does not bother me). It gives me effective in detecting and helping prevent common types of
warnings so that I can write secure code. web application vulnerabilities, such as lack of proper in-
put validation and/or encoding, CSRF, and broken access
So overall, 8 out of 9 participants expressed a positive control.
impression of ASIDE, with one being neutral. This study No single tool is capable of detecting all software vulner-
demonstrated that ASIDE was usable by our participants. abilities, and ASIDE is no exception. However, we believe
They were able to utilize ASIDE quickly, in the context of that ASIDE can be most effective in supplementing static
their own development, potentially improving the security of analysis tools and increasing the productivity of current best
their code. We plan to expand upon this study to examine software security practices. For input validation and encod-
the use of ASIDE by professional developers to more deeply ing issues, which often constitute the largest percentage of
investigate the impact on programmer behavior in a variety issues in a typical web application, ASIDE can significantly
of contexts. reduce the number of issues generated by static analysis
by helping programmers to prevent them in the first place.
8. RELATED WORK Thus, ASIDE can help improve software security and reduce
Research into tool support for software security focuses the number of security fixes needed after static analysis, as
heavily on machine-related issues, such as technique ad- well as saving time for the software security audit by 50%,
vancements for vulnerability detection effectiveness, accu- based on our Roller case study.
racy, and vulnerability coverage, with very little concern Our user study results suggest that ASIDE is effective
with human factors issues. The two prominent techniques in helping novice developers/students to write more secure
are static and dynamic program analyses. Static analysis code when using code refactoring. They appear to pay atten-
typically is based on taint tracking [27, 17, 21, 7] and dy- tion to ASIDE warnings and follow ASIDE advice to perform
namic analyses are often based on model checking [14, 9, input validation and/or encoding. Further studies, including
22] and symbolic execution [6, 10, 32]. As both approaches studies involving experienced developers, are needed to show
have their advantages and disadvantages, a variety of work whether ASIDE can improve developers’ understanding and
has explored the combination of these two techniques in an practice of secure programming in more contexts. Prelim-
attempt to achieve better performance [3, 23, 13] . inary evidence appears to suggest that users can quickly
Software developer education and training has also been recognize false positives in ASIDE and take easy action to
directed at achieving better software security. Most efforts dismiss them. More research on false positives and how they
have been spent on developing educational material and impact on developers using ASIDE is needed.
guidelines for the best secure programming practices [5, 16, Finally, our approach also provides a tool platform to pro-
1, 18, 35]. However, the mere existence of such abundant in- vide additional support for the secure software development
formation does not guarantee its use by programmers [38]. lifecycle in the areas of enforcing standards, information col-
Our work takes human factors into consideration and fills lection for secure coding metrics, and capturing developer
this missing gap, complementing program analysis tools to rationale for code review or in depth program analysis. In
help developers write more secure code. an enterprise environment, ASIDE can serve as a medium
Our work is in part motivated by studies on human errors that communicates a Software Security Group’s secure cod-
[29], and programmer errors in particular [20], which demon- ing knowledge to developers. As we continue the develop-
strate that programmer errors contribute a great deal to ment of ASIDE, we plan to further investigate the use of
software flaws. Many such errors are caused by three types these features and how they can contribute to secure appli-
of cognitive breakdowns: skill-based breakdowns where pro- cations.
grammers fail to perform routine actions at critical times; ACKNOWLEDGMENTS This work is supported in part
rule-based breakdowns where programmers fail to do an ac- by a grant from the National Science Foundation 0830624,
tion in a new context; and knowledge-based breakdowns where and a research/educational license from HP Fortify Inc. We
a programmer’s knowledge is insufficient. We believe that would like to thank Will Stranathan for his contribution in
ASIDE’s contextualized reminders can help address many of the formation of this idea and critiques and suggestions on
these issues by filling in knowledge gaps and reminding pro- the intial prototyping of ASIDE. We also want to thank all
grammers of secure programming issues within their current our participants for their time.
context.
Prior work on code annotation (e.g. [24]) used textual 10. REFERENCES
extensions of programming languages, such as C and Java, [1] S. Ardi, D. Byers, P. H. Meland, I. A. Tondel, and
which demands developers to learn and use yet another type N. Shahmehri. How can the developer benefit from
of language. In contrast, our approach leverages GUI sup- security modeling? In The Second International
port to make the process more intuitive, direct and easy to Conference on Availability, Reliability and Security,
use. Furthermore, soliciting security information from the 2007, pages 1017 –1025, april 2007.
developer through annotation could open up new ways to [2] Atlas.ti. Atlas.ti, 2011. www.atlasti.com.
detect software vulnerabilities. [3] D. Balzarotti, M. Cova, V. Felmetsger, N. Jovanovic,
E. Kirda, C. Kruegel, and G. Vigna. Saner:
9. CONCLUSIONS Composing static and dynamic analysis to validate
The main contribution of our work is to augment IDE sanitization in web applications. In Proceedings of the
tools to intelligently recognize software security issues and 2008 IEEE Symposium on Security and Privacy, pages
remind and assist developers with possible mitigation ac- 387–401. IEEE Computer Society, 2008.
tions. We evaluated this approach against mature open [4] M. Bishop and B. J. Orvis. A clinic to teach good
source projects. Our results suggest that this approach is programming practices. In Proceedings from the Tenth


 Colloquium on Information Systems Security [21] V. B. Livshits and M. S. Lam. Finding security errors
Education, pages 168–174, June 2006. in Java programs with static analysis. In Proceedings
[5] CERT. CERT Secure Coding, 2011. of the 14th Usenix Security Symposium, pages
www.cert.org/secure-coding. 271–286, August 2005.
[6] A. Chaudhuri and J. S. Foster. Symbolic security [22] M. Martin and M. S. Lam. Automatic generation of
analysis of ruby-on-rails web applications. In xss and sql injection attacks with goal-directed model
Proceedings of the 17th ACM conference on Computer checking. In Proceedings of the 17th conference on
and communications security, CCS ’10, pages 585–594. Security symposium, pages 31–43. USENIX
ACM, 2010. Association, 2008.
[7] B. Chess and G. McGraw. Static analysis for security. [23] M. Martin, B. Livshits, and M. S. Lam. Finding
IEEE Security and Privacy, 2:76–79, November 2004. application errors and security flaws using PQL: a
[8] B. Chess and J. West. Secure programming with static program query language. In OOPSLA ’05:
analysis. Addison-Wesley Professional, first edition, Proceedings of the 20th annual ACM SIGPLAN
2007. conference on Object oriented programming systems
[9] V. Felmetsger, L. Cavedon, C. Kruegel, and G. Vigna. languages and applications, pages 365–383, 2005.
Toward automated detection of logic vulnerabilities in [24] Microsoft. Microsoft SAL Annotations, 2011. http:
web applications. In Proceedings of the 19th USENIX //msdn.microsoft.com/en-us/library/ms235402.aspx.
conference on Security, USENIX Security’10, pages [25] Moodle. Moodle, 2011. http://moodle.org.
10–10. USENIX Association, 2010. [26] Moodle. MSA-08-0013, 2011.
[10] X. Fu and K. Qian. Safeli: Sql injection scanner using http://moodle.org/mod/forum/discuss.php?d=101405.
symbolic execution. In Proceedings of the 2008 [27] G. Naumovich and P. Centonze. Static analysis of
workshop on Testing, analysis, and verification of web role-based access control in j2ee applications.
services and applications, TAV-WEB ’08, pages 34–39. SIGSOFT Softw. Eng. Notes, 29:1–10, September
ACM, 2008. 2004.
[11] B. C. G. McGraw and S. Migues. Building security in [28] OWASP. ESAPI Validator API, 2011.
maturity model, 2011. www.bsimm2.com. http://owasp-esapi-java.googlecode.com/svn/trunk\
[12] M. Hafiz, P. Adamczyk, and R. Johnson. _doc/latest/org/owasp/esapi/Validator.html.
Systematically eradicating data injection attacks using [29] J. Reason. Human Error. Cambridge University Press,
security-oriented program transformations. In Cambridge, UK, 1990.
Proceedings of the 1st International Symposium on [30] A. Roller. Apache Roller, 2011.
Engineering Secure Software and Systems, ESSoS ’09, http://roller.apache.org.
pages 75–90. Springer-Verlag, 2009. [31] A. Roller. ROL-1766, 2011.
[13] Y.-W. Huang, F. Yu, C. Hang, C.-H. Tsai, D.-T. Lee, https://issues.apache.org/jira/browse/ROL-1766.
and S.-Y. Kuo. Securing web application code by [32] P. Saxena, D. Akhawe, S. Hanna, F. Mao,
static analysis and runtime protection. In Proceedings S. McCamant, and D. Song. A symbolic execution
of the 13th international conference on World Wide framework for javascript. Technical Report
Web, WWW ’04, pages 40–52. ACM, 2004. UCB/EECS-2010-26, EECS Department, University
[14] Y.-W. Huang, F. Yu, C. Hang, C.-H. Tsai, D. T. Lee, of California, Berkeley, Mar 2010.
and S.-Y. Kuo. Verifying web applications using [33] H. Sharp, Y. Rogers, and J. Preece. Interaction
bounded model checking. In Proceedings of the 2004 Design: Beyond Human-Computer Interaction. Wiley,
International Conference on Dependable Systems and 2 edition, 2007.
Networks, pages 199–, Washington, DC, USA, 2004. [34] F. Software. Fortify SCA, 2011. https://www.fortify.
IEEE Computer Society. com/products/fortify360/source-code-analyzer.html.
[15] Inqscribe. Inqscribe, 2011. www.inqscribe.com. [35] B. Taylor and S. Azadegan. Moving beyond security
[16] S. Institute. SANS Institute, 2011. www.sans.org. tracks: integrating security in cs0 and cs1. In
[17] N. Jovanovic, C. Kruegel, and E. Kirda. Pixy: a static Proceedings of the 39th SIGCSE technical symposium
analysis tool for detecting web application on Computer science education, SIGCSE ’08, pages
vulnerabilities. In Security and Privacy, 2006 IEEE 320–324. ACM, 2008.
Symposium on, pages 6 pp. –263, may 2006. [36] VERACODE. State of Software Security Report
[18] K. Karppinen, L. Yonkwa, and M. Lindvall. Why Volume 1, 2, and 3, 2011.
developers insert security vulnerabilities into their http://www.veracode.com/reports/index.html.
code. In Proceedings of the 2009 Second International [37] J. Xie, B. Chu, and H. R. Lipford. Idea: interactive
Conferences on Advances in Computer-Human support for secure software development. In
Interactions, ACHI ’09, pages 289–294. IEEE Proceedings of the Third international conference on
Computer Society, 2009. Engineering secure software and systems, ESSoS’11,
[19] D. E. Knuth. The errors of tex. Softw. Pract. Exper., pages 248–255. Springer-Verlag, 2011.
19:607–685, July 1989. [38] J. Xie, H. R. Lipford, and B. Chu. Why do
[20] A. J. Ko and B. A. Myers. A framework and programmers make security errors? In Proceedings of
methodology for studying the causes of software errors 2011 IEEE Symposium on Visual Languages and
in programming systems. J. Vis. Lang. Comput., Human Centric Computing, pages 161–164, 2011.
16:41–84, February 2005.