|




 






 

 





Brian Chess
Fortify Software
r
= aroblems With Black Box Testing
= Approaches To Finding Security Issues
= 4 aroblems With Black Box Testing
= Solution: White Box Testing
= Bytecode Injection
= Demo
 



 


ow Do You Find Security Issues?
= Gooking at architectural / design documents
= Gooking at the source code
= Static Analysis
= Gooking at a running application
= Dynamic Analysis




r  

= Analysis Of Source Code and Configuration Files

= Fortify Software

= Klocwork

= Ounce Gabs
ë 
r  
= Testing & Analysis Of Running Application
= Find Input
= Fuzz Input
= Analyze Response
= Commercial Web Scanners
= Cenzic
= SaIDynamics
= Watchfire
= ...
Œ
  

 

 
= |asy To Run
= Fast To Run
= ³Someone Told Me To´


³Did I Do A Good Job?´
·
! Œ
"
= Do You Know ow Much Of Your
Application Was Tested?
ü



ü



ü



ü

ü
†
   
   
  


 

  

 
 

·
! Œ
"
= ow Much Of The Application Do You
Think You Tested?
ü

ü

ü

ü

ü

ü
ü
ü
 ü  ü  ü  üü

 

r

= We ran a ³Version 7.0 Scanner´ on the
following:
Application |MMA Code Coverage Tool Web Source
HacmeBooks 34% classes 30.5%
12% blocks
14% lines
JCVS Web 45% classes 31.2%
19% blocks
22% lines
Java PetStore 2 70% classes 18%
20% blocks
23% lines


#$
= Good
= Found Real Vulnerabilities
= Was |asy To Run

= Bad
= ow Thorough Was My Test?
= No Way To Tell, And Actual Coverage Is Often Gow
·
ë%r &  
 "
= X Ways To Fail
= Didn¶t Test
= Tested ± But Couldn¶t Conclude
= Can¶t Test
·
ë%r &  
 "
Î. Didn¶t Test
= If The Web Scanner Didn¶t |ven Reach That
Area, It Cannot Test!

Tested Untested
Vulnerabilities
Not Found
Application
Vulnerabilities
Found
·
ë%r &  
 "
A. Tested, But Couldn¶t Conclude
= Blind SQG Injection Vulnerabilities That Did Not
Return With A Known Signature
·
ë%r &  
 "
A. Tested, But Couldn¶t Conclude
= Certain Classes Of Vulnerabilities Sometimes
Can Be Detected Through TTa Response
= SQG Injection
= Command Injection
= GDAa Injection
·
ë%r &  
 "
X. Can¶t Test
= Some Vulnerabilities ave No Manifestation In
ttp Response
I hope they¶re not cc num Gog
logging my CC# into File
plaintext log file

cc num
Application
Client TTa
Response

³Your order will be

= Bad
= ow Thorough Was My Test?
= No Way To Tell, And Actual Coverage Is Often Gow
= Did I Find All My Vulnerabilities?
= Didn¶t Test, Tested But Couldn¶t Conclude, Can¶t Test
·
r# 
#
"
= No Method Is aerfect
= Under What Circumstances Do Web
Scanners Report False aositives?
= Matching Signature On A Valid aage
= Matching Behavior On A Valid aage
·
r# 
#
"
= Matching Signature On A Valid aage
·
r# 
#
"
= Matching Behavior On A Valid aage
= ³To determine if the application is vulnerable to SQG
injection, try injecting an extra true condition into the
W|R| clause« and if this query also returns the
same «, then the application is susceptible to SQG
injection´ (from paper on Blind SQG Injection)
= |.g.
= http://www.server.com/getCC.jsp?id=5
= select ccnum from table where id=µ5¶
= http://www.server.com/getCC.jsp?id=5¶ AND µÎ¶=µÎ
= select ccnum from table where id=µ5¶ AND µÎ¶=µÎ¶
·
r# 
#
"
= |.g.
= http://www.server.com/getCC.jsp?id=5
= select ccnum from table where id=µ5¶
= Response:
³No match found´ (No one with id ³5´)
= http://www.server.com/getCC.jsp?id=5¶ AND µÎ¶=µÎ
= select ccnum from table where id=µ5\¶ AND \µÎ\¶=\µÎ¶
= Response
³No match found´ (No one with id ³5¶ AND µÎ¶=µÎ´)
All single quotes were escaped.
= According To The Algorithm (³inject a true clause and
look for same response´), This Is SQG Injection
Vulnerability!


#$
= Good
= Found Real Vulnerabilities
= Was |asy To Run

= Bad
Î. ow Thorough Was My Test?
= No Way To Tell, And Actual Coverage Is Often Gow
A. Did I Find All My Vulnerabilities?
= Didn¶t Test, Tested But Couldn¶t Conclude, Can¶t Test
X. Are All The Results Reported True?
= Susceptible To False Signature & Behavior Matching
·
!ë%  "
= Security Issues Must Be Fixed In Source Code
= Information Given
= URG
= aarameter
= General Vulnerability Description
= TTa Request/Response
= But Where In My Source Code Should I Gook?
·

!ë%
  "
= Incomplete Vulnerability Report -> Bad Fixes
= Report:
= Injecting ³AAAAA«..AAAAA´ Caused Application To
Crash
= Solution By Developers:
«.
if (input.equals(³AAAAA«..AAAAA´))
return;
«..


#$
= Good
= Found Real Vulnerabilities
= Was |asy To Run

= Bad
Î. ow Thorough Was My Test?
= No Way To Tell, And Actual Coverage Is Often Gow
A. Did I Find All My Vulnerabilities?
= Didn¶t Test, Tested But Couldn¶t Conclude, Can¶t Test
X. Are All The Results Reported True?
= Susceptible To Signature & Behavior Matching
4. ow Do I Fix The aroblem?
= No Source Code / Root Cause Information
r


 
White Box Testing With
Bytecode Injection
#$  

Application Server Database

TTa
Web File
Scanner Web Application System

Other
Apps
Verify
Results Watch
Verify
Results
Result
! Œ
 
 $
  "
= ow Thorough Was = Monitors Inside Will Tell
My Test? Which aarts Was it
= Did I Find All My = Monitors Inside Detects
Vulnerabilities? More Vulnerabilities
= Are All The Results = Very Gow False aositive
Reported True? By Gooking At Source Of
= ow Do I Fix The Vulnerabilities
aroblem? = Monitors Inside Can Give
Root Cause Information
! 
 

= ow Do You Inject The Monitors Inside
The Application?
= Where Do You Inject The Monitors
Inside The Application?
= What Should The Monitors Do Inside
The Application?
!ë'%

Œ
 "
= aroblem: ow Do You aut The Monitors Into The
Application?

= Assumption: You Do Not ave Source Code,

= Solution: Bytecode Weaving






u

 
 
 



 
 
!ë 

$"
Gist getStuff(String id) { Gist getStuff(String id) {
Gist list = new ArrayGist(); Gist list = new ArrayGist();
try { try {
String sql = ³select stuff from String sql = ³select stuff from
mytable where id=µ´ + id + ³¶´; mytable where id=µ´ + id + ³¶´;
JDBCstmt.executeQuery(sql); MyGibrary.doCheck(sql);
} catch (|xception ex) { JDBCstmt.executeQuery(sql);
log.log(ex); } catch (|xception ex) {
} Before log.log(ex);
return list; ³executeQuery()´ }
} Call return list;
³MyGibrary.doCheck()´ }
r 
(%


|





= ow Do You Inject The Monitors Inside
The Application?
= Where Do You Inject The Monitors
Inside The Application?
= What Should The Monitors Do Inside
The Application?
ë'%

Œ
 "
= All Web Inputs (My Web Scan Should it All Of
Them)
= request.getaarameter, form.getBean
= All Inputs (Not All Inputs Are Web)
= socket.getInputStream.read
= All ³Sinks´ (All Security Critical Functions)
= Statement.executeQuery(String)
= (FileOutputStream|FileWriter).write(byte[])
= «


 Œ
 ë"
= Report Whether The Monitor Was it
= Analyze The Content Of the Call For
Security Issues
= Report Code-Gevel Information About
Where The Monitor Got Triggered


 Œ
 ë"
aspect SQGInjection {
pointcut sql|xec(String sql):call(ResultSet Statement.executeQuery(String))
&& args(sql); Î) Report whether AaI was hit or not
before(String sql) : sql|xec(sql) { checkInjection(sql, thisJoinaoint); }
void checkInjection(String sql, Joinaoint thisJoinaoint){
System.out.println("IT:" +
thisJoinaoint.getSourceGocation().getFileName() +
thisJoinaoint.getSourceGocation().getGine());
if (count(sql, '\'')%A == Î) {
System.out.println("*** SQG Injection detected. SQG statement
X) Report Code-Gevel Information
being executed as follows: ³ + sql);
}
A)«..
Analyze The Content Of The AaI Call

  ) 


= Good
= |asy To Use
= Finding Smoking Gun
= Bad
= Gack Of Coverage Information
= False Negatives
= False aositives
= Gack Of Code-Gevel / Root Cause Information

  ) 


= Bytecode Injection Require Access To
Running Application
= In |xchange «
= Gain Coverage Information
= Find More Vulnerabilities, More Accurately
= Determine Root Cause Information

  )  'r$

Attacker Defender

Time

Attempts

Security
Knowledge
Access To
Application