Security Testing For RESTful

Applications
Ofer Shezaf, HP Enterprise Security Products
[email protected]

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
What I do for a living?
• Product Manager, Security Solutions, HP ArcSight
• Led security research and product
management at Breach Security & HP Fortify

I am passionate about security after hours as well:

• OWASP leader and founder of the Israeli chapter
• Leads the Web Application Firewall Evaluation Criteria project
• Wrote the ModSecurity Core Rule Set

Fun fact: the closest airport to my house is in Damascus, Syria

2 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
In this Presentation

About RESTful Web Services

RESTful WS in the Wild
Security of RESTful WS
Pen-testing RESTful WS
Automated security testing of RESTful WS

3 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
About RESTful Web Services
• About RESTful Web Services
• RESTful WS in the Wild
• Security of RESTful WS
• Pen-testing RESTful WS
• Automated security testing of RESTful WS

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Web Services

Employing web technology (i.e. HTTP)

for machine to machine communication

Used for:
• Inter application communication
• Web 2.0 and Mashups
• Think client applications
• Phone applications

5 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
SOAP Web Services: example

Highly defined

Parameters are sent as a well

formed XML

Isn’t this a rather complex way

to send a single parameter?

6 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
SOAP Web Services
Commonly used protocol set for Web Services

The theory:
• Structures and well defined
• Robust
• Secure (?*)

However:
• Complex and heavy, especially for phone and Web 2.0
• Not the HTTP way: Designed to work on any protocol
including SMTP

* See WS-Attacks.org for an alternative view

7 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
 The REST design pattern
Essentially what the Web always was

Client/Server •Clients are separated from servers by a uniform interface.

Stateless •The client–server communication is further constrained by no client context being stored on the
server between requests*.

Cacheable •Responses must therefore, implicitly or explicitly, define themselves as cacheable or not

Layered •A client cannot ordinarily tell whether it is connected directly to the end server, or to an
intermediary along the way.

Uniform •A uniform interface between clients and servers simplifies and decouples the architecture.

Code on demand •Servers are able to temporarily extend or customize the functionality of a client by transferring
logic to it that it can execute.

* The server can be stateful; this constraint merely requires that server-side state be addressable by URL as a resource.
8 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

8
RESTful Web Services

Are:
• A common practice for using REST design patterns
for Web Services

Are Not:
• A well defined protocol
• A set of software libraries or frameworks

9 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
RESTful Web Services: example

Isn’t this much simpler?

GET /InStock/HP

10 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Common RESTful WS Practices
Use of HTTP methods to indicate action
CRUD: GET /InStock/HP
• Create (PUT),
• Read (GET),
• Update (POST),
• Delete (DELETE)

11 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Common RESTful WS Practices
None standard parameters specifications
• As part of the URL GET /InStock/HP
• None standard request parameters
• In headers
• Serialized as JSON in a parameter value of
request body

12 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Common RESTful WS Practices
Structured parameters and responses
• JSON and XML both widely used http://api.geonames.org/earthquakesJSON
?north=44.1&south=-9.9&east=-22.4&
• Parameter: west=55.2&username=demo
• In the request body
• Embedded in the value of a single parameter {"earthquakes":
[
• Response usually in the response body {"eqid":"c0001xgp","magnitude":8.8,"lng":142.369,"src
":"us","datetime":"2011-03-11
04:46:23","depth":24.4,"lat":38.322},
{"eqid":"2007hear","magnitude":8.4,"lng":101.3815,"sr
c":"us","datetime":"2007-09-12 09:10:26","depth":30,"lat":-
4.5172},
{"eqid":"2007aqbk","magnitude":8,"lng":156.9567,"src
":"us","datetime":"2007-04-01 18:39:56","depth":10,"lat":-
8.4528},
...

13 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Common RESTful WS Practices
Custom authentication and session management
• Commonly use security token/tickets
• While pure REST calls for URL based
tokens, this is not secure and headers are
often used.

14 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
RESTful services Documentation

• No common documentation format

similar to WSDL.
• WADL (Web Application Definition
Languages) is a standard proposal:
• Not approved
• Not widely used

15 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
RESTful WS in the Wild
• About RESTful Web Services
• RESTful WS in the Wild
• Security of RESTful WS
• Pen-testing RESTful WS
• Automated security testing of RESTful WS

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
It’s Up and Coming!

17 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Everybody uses REST

18 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Security of RESTful WS
• About RESTful Web Services
• RESTful WS in the Wild
• Security of RESTful WS
• Pen-testing RESTful WS
• Automated security testing of RESTful WS

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
You Already Know This Part

REST is just Web

REST Security is just Web application security

20 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Key issues to keep in mind

Some common design flaws associated

with REST:
No standard security mechanism Proprietary authentication and session •Overreliance on SSL
similar to SOAP Web Services (WS-*) management. •Session IDs used in the URL
•Using basic HTTP Authentication
•Bad implementation of SSO

21 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Pen-testing RESTful WS
• About RESTful Web Services
• RESTful WS in the Wild
• Security of RESTful WS
• Pen-testing RESTful WS
• Automated security testing of RESTful WS

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Challenges
Detecting Attack Surface

Inspecting the application does not

reveal application attack surface

Not all Web Service Requests are often

None Web
functionality actually dynamically created,
applications
used by application Web 2.0 style.

23 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Challenges
Mega fuzzing

None
Standard
Parameters

A very large
number of
parameters
to fuzz
Serialized
JSON/XML
paramters

24 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Challenges
Session management

Custom authentication and

session management requires
adjustment in every pen test.

Need to follow custom SSO

processes and session
management breaks common
cookie sharing practices.

25 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Solutions
Use Documentation

Determine:

• Available services
• Use of HTTP methods
• Use of parameters

Potential Sources:

• WADL
• Programing guides
• Configuration information
• Application source

26 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Solutions
Use Documentation

Determine:

• Available services
• Use of HTTP methods
• Use of parameters

Potential Sources:

• WADL
• Programing guides
• Configuration information
• Application source

27 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Solutions
Use a proxy

Extremely helpful for:

• None web applications
Determining attack surface • Dynamically generated requests
when no documentation exists

Useful also when

documentation exists to
determine initial fuzzing values

28 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Solutions
Guessing parameters

Look for none standard headers

Determine if URL segments have a pattern

•Numerical values
•Well known templates such as date

Look for structures in parameter values

•JSON, XML, YAML or other

URLs with not extension

29 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Automated security testing of RESTful WS
• About RESTful Web Services
• RESTful WS in the Wild
• Security of RESTful WS
• Pen-testing RESTful WS
• Automated security testing of RESTful WS

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How does automated pen-testing work?

Understanding
request
generation (i.e.
Today employ links)

Crawling Determining
attack surface
Historically only
links based
JavaScript
emulation to get
dynamic requests
Pre- Understanding
requisites parameters

Attacking Sending known

attack vectors
Fuzzing
parameters
Session based
Understanding
session
management

31 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
RESTful WS Challenges

Finding attack surface by crawling

Determining what elements of the request to attack

Optimizing fuzzing time while still addressing all potential parameters

Getting initial values for fuzzing

Custom authentication and session management breaks common cookie sharing practices

32 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
One: define rules

Define parameter structure for URL

Use rules when crawling and attacking
Rule can be:
• User defined
• Imported documentation, WADL or
configuration files
• Proxy discovered attack surface, potentially
during crawl.

Or…
Get smart! WebInspect 9.2 REST rule editor

33 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Two: ask the server

A server module communicating with

the scanner can: Java or .NET Virtual <Rule>
Machine
• Identify rewrites
• Send configuration and debug information Target Program Monitor
• Provide file and method structure
• Monitor server based session information Event

Program
Point Handlers <Event
Handler>

Action Log

Fortify Run Time Architecture

34 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Three: Look for highly varying URL segments
http://www.44tips.com:80/svc/Grid.asmx/GetContentItems
http://www.44tips.com:80/js/templates/new/controlPanelSearch.htm?v=2
http://www.44tips.com:80/svc/Grid.asmx/GetRelatedListItems
http://www.44tips.com:80/svc/Grid.asmx/GetContentItems
http://www.44tips.com:80/js/templates/publishSetDialog.htm
http://www.44tips.com:80/svc/grid.asmx/GetUserCollectionInfo
http://www.44tips.com:80/svc/grid.asmx/GetUserSetThumbUrls
http://www.44tips.com:80/svc/grid.asmx/IsCollectionTitleUnique
http://www.44tips.com:80/svc/grid.asmx/InsertCollection
http://www.44tips.com:80/svc/Grid.asmx/GetContentItems
http://www.44tips.com:80/svc/grid.asmx/GetUserSetThumbUrls
http://www.44tips.com:80/svc/Grid.asmx/GetRelatedListItems
http://www.44tips.com:80/svc/Grid.asmx/GetRelatedListItems
http://www.44tips.com:80/svc/grid.asmx/GetUserCollectionInfo
http://www.44tips.com:80/c/k1collection/Sem_Schilt/i72665/Sem_Schilt
http://www.44tips.com:80/c/k1collection/Sem_Schilt/i72662/Mirko_Cro_Cop
_vs__Semmy_Schilt__Video_Game_
http://www.44tips.com:80/c/k1collection/Sem_Schilt/i72661/Josh_Barnett_
vs__Semmy_Schilt_II_Part_1
http://www.44tips.com:80/c/k1collection/Sem_Schilt/i72660/_Part_2__Fedo
r_vs_Semmy_Schilt__PRIDE_21____23_06_2002_
http://www.44tips.com:80/c/k1collection/Sem_Schilt/i72659/Fedor_Emelian
enko_vs_Semmy_Schilt__Part_4_4_
…

svc/Grid.asmx/{param}
c/k1collection/Sem_Schilt/{param1}/{param2}
{param1}/{param2}/{param3}/{param4}/{param5}

35 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Four: examine response codes

404 analysis

Examine if
“folder” access
returns 404

everything
beyond the
“folder” is a
parameter

36 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank You!
Ofer Shezaf, [email protected]

37 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.