BU Y ER’S GUIDE

Application Security
Buyer’s Guide
15 questions to ask
yourself and your DAST vendor
TA B L E O F CO N T E N T S

An Introduction to the AppSec Market 3

Dynamic Application Security Testing Requirements 4
DAST Requirements for Advanced Application Security
Programs and DevSecOps 7
Go Forth and Scan Repeatedly 8
Discover a DAST solution built to help you
address modern application security challenges 9
About Rapid7 10

Application Security 2
An Introduction to the AppSec Market

In today’s world of complex, modern web applications, accurate

and automated Dynamic Application Security Testing (DAST) tools are
rare, but do exist. What characteristics should you look for in a
DAST tool to give you greater accuracy and ease of use?

Given the various dimensions upon which you can compare vendors, finding the right DAST tool for you isn’t always a walk
in the park. The following are common challenges during the buying process:

• Often, when organizations are looking to purchase a

DAST tool, they are doing so under a very compressed
• Results from DAST tools will naturally differ quite a
bit, due to the differences in configuration, scanning
time frame. techniques, and reported findings.

• You need to test a real application with known

vulnerabilities; otherwise, it can be difficult to compare
• You will be, in certain instances, forced to rely on
the word of the DAST vendor. Why? The technology
one solution’s effectiveness against that of another’s. that underlies DAST tools can be a black box.

• Ittotakes
By knowing the vulnerabilities present in advance,
a lot of time (time that not many teams have
you can determine which DAST tool is more apt to
spare) to check and re-check reports for accuracy.
identifying any and all vulnerabilities. When possible,
work with your development team to seed a test
application with a variety of SQL injections, XSS, and
other vulnerabilities.

In this guide, we comprehensively outline the major features and capabilities you should be looking for when selecting a
DAST tool. To help you cover all your bases, we’ve also included some questions and techniques you can leverage to get the
most out of your evaluation period. Our goal? To equip you to select the best application security for your organization — one
that is automated, accurate, and easy to use. For more advanced application security programs, we’ve included a few other
considerations that will not only improve the effectiveness of your DAST solution, but also its ability to fold seamlessly into
the workflows of your development counterparts.

Application Security 3
 Dynamic Application
01 Security Testing Requirements

1 COVERAGE OF MODERN
WEB TECHNOLOGIES
Coverage is the first step of accuracy. A DAST tool can’t
3 QUICK START CAPABILITIES
The best pen testers love to do things by hand,
leading to a comprehensive yet slow, manual process.
test what it can’t find or doesn’t understand. Most DAST The reality is that you need those smart pen testers to
tools were built to scan HTML, and they do so quite cover the work that can’t be done by automation. A good
effectively. But times have changed, and in reality very DAST tool’s real value is in its capacity for automation,
few applications are built solely in HTML. Today’s thus reducing the need for manual testing. The best tool
applications have gone beyond static pages to involve is the one that will work well in a “point and shoot” mode.
advanced web clients and web services that make use In many cases, that is all that’s possible for understaffed
of new technologies. These applications are powered by security teams.
JavaScript and AJAX on the client-side, and often have
interfaces built in JSON, REST, and SOAP with CSRF
protection thrown in for good measure. Thus, you need Make sure the tool you choose offers a simple,
a tool that is built to scan apps utilizing modern web “point and shoot” approach, allowing you to maximize
technologies, on top of just basic HTML. the most of your limited time and resources

Speak with your development team and understand

4
which technologies are used to build the applications
ARCHITECTURE AND SCALABILITY
you are tasked to secure. Look for DAST tools that
THAT MEETS YOUR NEEDS
explicitly support these technologies out of the
box — without a significant amount of training. DAST tools can be deployed in a number of ways: on-
premise, in the cloud, or as managed services. In
addition to where security data is hosted, scalability is
also an important component; some organizations may

2
manage just a few dozen web applications, while some
FUTURE-PROOF STRATEGY organizations can have thousands to tens of thousands
Modern DAST tools need to understand and of web applications that all need to be secured. Finally,
adapt to new application technologies as they become not all organizations are able to execute application
popular. Inevitably, we will continue to see an increase security programs in-house due to the lack of staffing,
in application complexity with the emergence of new necessitating external consultants to run and manage
technologies. While most DAST tools continuously work scans as well as validate vulnerability findings.
toward understanding and attacking classic web apps
of the past, modern DAST tools need to be architected
so that new technologies can be bolted on like drill bits Ask your vendors about their different deployment
on a drill. options for DAST, and which would be best for your
scalability, staffing, and data handling requirements.

Ask your vendors how their architecture provides

the flexibility to handle new technologies.as well, as
you upgrade or swap out various tools in your stack.

Application Security 4
5 AUTHENTICATION AND SESSION
MANAGEMENT (DEVELOPER’S
FUNZONE, SECURITY’S NIGHTMARE)
8 REDUNDANT FALSE
POSITIVE CHECKING
False positives are simultaneously the bane of automated
Developers seem to revel in creating innovative, complex, scanning and a time suck for security teams. Web
and difficult-to-automate schemes for authentication applications often behave in mysterious ways, and per the
and session management. Your DAST tool needs to have nature of the beast, smart DAST tools must check and
advanced capabilities to authenticate automatically and recheck findings to avoid false positives.
have backup plans (macros and advanced settings) to
tweak, in case there is a clever edge case.
Seek out solutions that provide findings you can be
confident in, as well as vendors who are committed
Make sure your tool can automate the login process to improving the quality of their results based on your
and maintain a session on your applications. It’s also feedback.
important to make sure the solution has a macro recorder
that supports user events so that you can better handle
complex authentication scenarios.

9 RELEVANT DATA INPUT

During automated scans, there are usually two

6
phases: crawl and attack. During the crawl phase, it is
CUSTOMER SUPPORT imperative that a tool provide valid data for each input
AND CUSTOMIZATION field as expected by the application. For example, if a
The reality of application scanning today is that your form is asking for a shipping address, some tools enter
applications are highly customized, making it extremely random values into each input instead of the expected
difficult for DAST tools to address 100% of cases. Each values. Certain fields such as the ZIP code would be
custom application uses unique technical approaches invalid, and the application would subsequently reject the
that can trip tools up and cause them to crash. You need request. In this case, the scan is actually halted, resulting
to seek out a solution that is flexible and responsive in the in a less comprehensive scan and the potential for missed
face of unique, complex applications. vulnerabilities.

Find a vendor that has a proven track record of quickly Ask vendors what kind of data they use in their attack
responding to customers’ needs and tweaking DAST phase to determine if they are using both expected
functionalities to improve testing coverage—especially and unexpected datasets. Furthermore, ask if they are
of complex applications. attacking one input at a time.

7 SOPHISTICATED ATTACK TECHNIQUES

All DAST tools must find a balance between
comprehensiveness and performance. In order to
10 INCLUSION OF EVERY
The point of automation is to handle the
repetitive tasks against every input, but this can also lead
improve performance, some DAST tools randomly limit to slower scan times. To save time, some web application
the set of attacks to send based on proprietary choices. security solutions only check the first several parameters
Others intelligently profile applications to determine on each page. However, each parameter could use
which attacks are useful, and dynamically adjust attacks different filters. Why is this important? Tools could be
for each input. This latter approach increases not only arbitrarily missing vulnerabilities for the sole sake of
the efficiency of the scan, but also its ability to find valid saving time. Our take? Time savings may not be worth the
vulnerabilities. increased risk.

Be sure you understand how your DAST tool selects its Make sure the solution you choose checks every
attacks, and how configurable the attacks are to fit your parameter on every page for comprehensive
needs. identification of application vulnerabilities.

Application Security 5
11 SCAN SCHEDULING
AND BLACKOUTPERIODS
Continuously assessing your web applications for
13 ATTACK REPLAY
When developers are handed a list of
security bugs in their applications, they’re often skeptical
vulnerabilities is more critical than ever in today’s world that the bugs truly exist and aren’t just false positives.
of rapid development release cycles. In response, Some DAST tools offer “Attack Replay” or “Validate”
automated scan scheduling can be leveraged to help features that enable developers to replay attacks directly
your program stay on top of the vulnerabilities that appear within exported vulnerability findings reports. This is
in constantly-evolving applications. Blackout periods game-changing, as developers can now validate that
can also be useful to ensure scans don’t run during times security bugs truly exist, and also test potential source
of high activity on an application, and in turn prevent code patches for the vulnerabilities without running
potential negative user impacts. another DAST scan.

Check your DAST solution for flexible scan scheduling When evaluating DAST tools, make sure an “Attack
and blackout period capabilities. Replay” capability is available to reduce friction between
security and development teams, and streamline
remediation efforts.

12 INTERACTIVE AND

14
USABLE REPORTING
COMPLIANCE REPORTING
As you know all too well, “reporting” in most tools takes
the form of very, very long PDF files that are difficult to Many organizations will launch application
work with. Your team doesn’t want to send them, and security initiatives in response to regulatory compliance
those in charge of remediation most definitely don’t want requirements like PCI, HIPAA, and SOX. Often, there are
to open (let alone read) them. A good DAST solution security compliance requirements to adhere to as well,
provides you with results that can be used by auditors such as the OWASP Top 10. In order to make your life
and developers alike. Reports should be easy to navigate easier, your application security solution should facilitate
through, and allow you to reproduce the attacks with your journey towards compliance.
a few clicks. It should also be easy to understand the
context around issues, with the ability to read summaries,
drill into details, and view the information in different The DAST solution you choose should have
ways. The unfortunate reality is that developers with the ability to generate reports specifically organized and
limited security training often have a difficult time designed around the compliance requirements to which
replicating vulnerabilities, thus slowing down or stopping your organization is subject, making it easier for auditors
remediation. and business stakeholders to understand the compliance
risk of your organization’s proprietary applications.

When choosing a solution, make sure the reports are

interactive, easy to use, and useful for review and

15
remediation — across all of your stakeholder groups. CUSTOM MOBILE APPLICATIONS,
Custom mobile applications are the new
frontier for security teams. They provide native mobile
interfaces, but then communicate with web services or
APIs (JSON, REST/XML, AMF, etc.) that have the same
range of potential vulnerabilities (SQLi, authentication,
and session management weaknesses) that web
applications have.

Be sure your DAST tool is capable of testing back-end

interfaces or APIs: This is where the real weaknesses are
likely to be found.

Application Security 6
 DAST Requirements for
02 Advanced Application Security
Programs and DevSecOps

1 CONTINUOUS INTEGRATION (CI)

Many organizations are pushing development
teams and teams working under the DevSecOps mentality
2 WAF/IPS LINKING WITH CUSTOM
RULES AND QUICK RE-TEST
Due to the volume of applications and vulnerabilities,
to use Continuous Integration solutions (whether off-the- most organizations are relying on WAFs and IPS devices
shelf or home grown) to streamline QA efforts and reduce to protect themselves against vulnerabilities that haven’t
time-to-market. Security teams are wise to find ways to yet been patched. These WAFs and IPS devices come
plug their scanning activities into the CI to ensure every with default rules, which will not give your custom
build is security tested before it goes into production. application all the protection it needs. You will likely
This requires a tool that works well in “point and shoot” require a custom rule that combines knowledge of both
mode (see #3), and offers open APIs for running scans. the WAF/IPS device and the application.

If your organization is ready to start integrating Be sure you have a sufficient understanding of how
application security assessment into the Software rules are created and applied. Look for a solution that
Development Lifecycle (SDLC), speak with your vendors goes beyond turning on a default rule from a WAF/IPS to
about how their DAST solutions would fit into your creating truly custom rules for your custom application.
development team’s CI toolchain.

Application Security 7
Go Forth and Scan Repeatedly
Each of the recommendations and considerations noted in this guide works toward a simple goal: making sure your DAST
solution is lightening your load, not adding onto it. That means automating and streamlining the scanning and reporting
process as much as possible. In due time, you’ll gain a profound level of visibility including what’s wrong and where, see faster
remediation times, and most importantly, ensure that applications get and stay secure.

Application Security 8
Discover a DAST solution built
to help you address modern
application security challenges.
Built upon Rapid7’s Insight platform, InsightAppSec combines ease-of-use with powerful crawling and attack capabilities.
Get unparalleled visibility into your application vulnerabilities within minutes.

Test your own application with a free 30-day trial of InsightAppSec:

Visit www.rapid7.com/try/InsightAppSec to get started today.

Application Security 9
About Rapid7
Rapid7 (Nasdaq: RPD) is advancing security with visibility, analytics, and automation delivered through our Insight cloud.
Our solutions simplify the complex, allowing security teams to work more effectively with IT and development to reduce
vulnerabilities, monitor for malicious behavior, investigate and shut down attacks, and automate routine tasks. Customers
around the globe rely on Rapid7 technology, services, and research to improve security outcomes and securely advance their
organizations. For more information, visit our website, check out our blog, or follow us on Twitter.

To learn more about Rapid7 or

get involved in our threat research,
visit www.rapid7.com.

Application Security 10