FYP Report VulnScan
FYP Report VulnScan
VulnScan
A Web Vulnerability Scanner
Submitted By,
ANANTIM PATIL (181070006)
VIRAJ YADAV (181070073)
ANIRUDDHA BABAR (191070906)
ABHIJEET RAUT (191070908)
Certificate
This is to certify that Mr. Anantim Patil (181070006), Mr. Viraj Ya-
dav (181070073), Mr. Aniruddha Babar (191070906) and Mr. Abhijeet
Raut (191070908), students of B.Tech (Computer Engineering), Veermata
Jijabai Technological Institute (VJTI), Mumbai have successfully completed
the Project Dissertation on “VulnScan” to our satisfaction.
Project Guide
Prof. Varshapriya J N
Department Of CE And IT
VJTI, Mumbai
Dr. M R Shirole
Head Of Department Of CE And IT
VJTI, Mumbai
2
Certificate
Project Guide
Prof. Varshapriya J N
Department Of CE And IT
VJTI, Mumbai
Examiner
3
Acknowledgement
Abstract
Contents
1 Introduction 8
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.3 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . 17
1.4 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2 Literature Review 18
2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.1.1 Web Application Vulnerability . . . . . . . . . . . . . . 18
2.1.2 Web Application Scanners . . . . . . . . . . . . . . . . 19
2.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2.2 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . 24
3 Proposed System 25
3.1 Vulnerability Detection System . . . . . . . . . . . . . . . . . 25
3.1.1 Features of Open Source web scanner . . . . . . . . . . 26
3.2 Attack and Analysis Concepts . . . . . . . . . . . . . . . . . 26
3.2.1 SQL Injection . . . . . . . . . . . . . . . . . . . . . . . 27
3.2.2 Simple Reflected XSS attack . . . . . . . . . . . . . . 29
3.2.3 Encoded Reflected XSS attack . . . . . . . . . . . . . . 30
3.2.4 Stored XSS attack . . . . . . . . . . . . . . . . . . . . 30
3.2.5 Error Based SQL injection . . . . . . . . . . . . . . . . 31
3.2.6 SubDomain and directories scanner . . . . . . . . . . . 32
3.3 Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Characteristics of the System . . . . . . . . . . . . . . . . . . 34
3.4.1 Vulnerability scanner information . . . . . . . . . . . . 34
3.4.2 Easy to Use . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.3 Reduced cost . . . . . . . . . . . . . . . . . . . . . . . 34
3.4.4 Faster Scanning . . . . . . . . . . . . . . . . . . . . . . 34
3.4.5 Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.4.6 Remediating Reports . . . . . . . . . . . . . . . . . . . 35
3.5 Advantages And Disadvantages . . . . . . . . . . . . . . . . . 35
3.5.1 Advantages . . . . . . . . . . . . . . . . . . . . . . . . 35
3.5.2 Disadvantages . . . . . . . . . . . . . . . . . . . . . . . 36
7
5 System Implementation 47
5.1 System Requirements . . . . . . . . . . . . . . . . . . . . . . . 47
5.1.1 Hardware requirements . . . . . . . . . . . . . . . . . . 47
5.1.2 Software requirements . . . . . . . . . . . . . . . . . . 47
5.1.3 Communication Requirements . . . . . . . . . . . . . . 47
5.2 Vulnerability Scanner Implementation . . . . . . . . . . . . . . 47
5.2.1 Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
7 Summary 69
7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
7.2 Future Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
7.2.1 Range . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
7.2.2 Better Reporting . . . . . . . . . . . . . . . . . . . . . 69
7.2.3 Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
8 Bibliography 70
8
1 Introduction
1.1 Background
Over the past decade or so, the web has been embraced by millions of busi-
nesses as an inexpensive channel to communicate and exchange information
with prospects and transactions with customers. In particular, the web pro-
vides a way for marketers to get to know the people visiting their sites and
start communicating with them. One way of doing this is by asking web visi-
tors to subscribe to newsletters, to submit an application form when request-
ing information on products, or provide details to customize their browsing
experience when next visiting a particular website.
Web browsers are software applications that allow users to retrieve data
9
and interact with content located on web pages within a website. Today’s
websites are a far cry from the static text and graphics showcases of the early
and mid-nineties: modern web pages allow personalized dynamic content to
be pulled down by users according to individual preferences and settings.
Furthermore, web pages may also run client-side scripts that change the
Internet browser into an interface for such applications as webmail and in-
teractive mapping software (e.g., Yahoo Mail and Google Maps).
Most importantly, modern web sites allow the capture, processing, stor-
age and transmission of sensitive customer data (e.g., personal details, credit
card numbers, social security information, etc.) for immediate and recurrent
use. And this is done through web applications. Such features as webmail,
login pages, support and product request forms, shopping carts, and content
management systems shape modern websites and provide businesses with the
means necessary to communicate with prospects and customers. These are
all common examples of web applications.
The figure below details the three-layered web application model. The first
layer is normally a web browser or the user interface; the second layer is the
dynamic content generation technology tool such as Java servlets (JSP) or
Active Server Pages (ASP), and the third layer is the database containing
content (e.g., news) and customer data (e.g., usernames and passwords, so-
cial security numbers, and credit card details).
The figure below shows how the initial request is triggered by the user
through the browser over the Internet to the web application server. The
web application accesses the databases servers to perform the requested task
11
updating and retrieving the information lying within the database. The web
application then presents the information to the user through the browser.
Let us now look at types of attacks on web applications. Despite their ad-
vantages, web applications do raise a number of security concerns stemming
from improper coding. Serious weaknesses or vulnerabilities allow criminals
to gain direct and public access to databases in order to churn sensitive data
this is known as a web application attack. Many of these databases con-
tain valuable information (e.g. personal data and financial details) making
them a frequent target of attacks. Although such acts of vandalism (often
performed by the so-called script kiddies) as defacing corporate websites are
still commonplace, nowadays attackers prefer gaining access to the sensitive
data residing on the database server because of the immense pay-offs in sell-
ing the results of data breaches. In the framework described above, it is easy
to see how a criminal can quickly access the data residing on the database
through a dose of creativity and, with luck, negligence or human error, lead-
ing to vulnerabilities in the web applications.
12
public. All modern database systems (e.g. Microsoft SQL Server, Oracle, and
MySQL) may be accessed through specific ports (e.g., port 80 and 443) and
anyone can attempt direct connections to the databases effectively bypassing
the security mechanisms used by the operating system. These ports remain
open to allow communication with legitimate traffic and therefore constitute
a major vulnerability.
Web applications often have direct access to backend data such as cus-
tomer databases and, hence, control valuable data and are much more diffi-
cult to secure. Those that do not have access will have some form of script
that allows data capture and transmission. If an attacker becomes aware
of weaknesses in such a script, they may easily reroute unwitting traffic to
another location and illegitimately hive off personal details.
1.2 Motivation
The gaping security loophole in Web applications is being exploited by hack-
ers worldwide. According to a survey by the Gartner Group, almost three-
fourths of all Internet assaults are targeted at Web applications.
On October 31, 2001, the website of Acme Art Inc. was hacked and all
the credit card numbers from its online store’s database were extracted and
displayed on a Usenet newsgroup. This breach was reported to the public
by the media and the company lost hundreds of thousands of dollars due
to orders withdrawn by wary customers. The company also lost its second
phase of funding by a venture capital firm.
In June 2003, hackers detected that the Web applications of the fashion
label Guess and pet supply retailer PetCo contained SQL injection vulner-
abilities. As a result, the credit card information of almost half a million
customers was stolen.
The only way to combat the Web application security threat is to proac-
tively scan websites and Web applications for vulnerabilities and then fix
them. Implementing a Web application scanning solution must be a crucial
part of any organization’s overall strategy.
17
1.4 Scope
• To create an open-source web vulnerability scanner to test for common
attack vulnerabilities.
2 Literature Review
2.1 Background
Before discussing the design of our tests, it is useful to briefly discuss the
vulnerabilities that web application scanners try to identify and to present
an abstract model of a typical scanner.
Then, for each input and for each vulnerability type for which the web
application vulnerability scanner tests, the attacker module generates values
that are likely to trigger a vulnerability. For example, the attacker module
would attempt to inject JavaScript code when testing for XSS vulnerabilities,
or strings that have a special meaning in the SQL language, such as ticks
and SQL operators, when testing for SQL injection vulnerabilities. Input
values are usually generated using heuristics or using predefined values, such
as those contained in one of the many available XSS and SQL injection cheat-
sheets.
The analysis module analyzes the pages returned by the web application in
response to the attacks launched by the attacker module to detect possible
vulnerabilities and to provide feedback to the other modules. For example,
if the page returned in response to input testing for SQL injection contains a
database error message, the analysis module may infer the existence of a SQL
20
• White box testing: consists of the analysis of the source code of the
web applications. This can be done manually or by using code analysis
tools. The problem is that the perfect source code analysis may be
difficult and cannot find all security flaws because of the complexity of
the code.
• Black box testing: includes the analyses of the execution of the appli-
cation to search for vulnerabilities. In this approach, also known as
penetration testing, the scanner does not know the internals of the web
application and it uses fuzzing techniques over the web HTTP requests.
This paper was presented by Yuma Makinov and Vitaly Klyuev. In this
work, they evaluated OWASP ZAP and Skipfish vulnerability scanners. They
found that OWASP ZAP is superior over Skipfish as far as in this experi-
mental situation. However, they note that both of them are not perfect yet
especially with detection of the RFI vulnerability. Furthermore, they realize
that we need a vulnerable web application without unintentional vulnerabil-
ities for more accurate evaluation [6].
This paper was presented by Jose Fonseca, Marco Vieira, and Henrique
Madeira. In this paper they propose an approach to evaluate and compare
web application vulnerability scanners. It is based on the injection of realistic
software faults in web applications in order to compare the efficiency of the
different tools in the detection of the possible vulnerabilities caused by the
22
injected bugs. The results of the evaluation of three leading web application
vulnerability scanners show that different scanners produce quite different
results and that all of them leave a considerable percentage of vulnerabili-
ties undetected. The percentage of false positives is very high, ranging from
20percent to 77percent in the experiments performed. The results obtained
also show that the proposed approach allows easy comparison of coverage
and false positives of the web vulnerability scanners. In addition to the
evaluation and comparison of vulnerability scanners, the proposed approach
also can be used to improve the quality of vulnerability scanners, as it easily
shows their limitations. For some critical web applications several scanners
should be used and a hand scan should not be discarded from the process [20].
point. This system is different from other systems in that we detect the
vulnerabilities based on the injection point.They get the information of each
injection point to find where the vulnerability is. The system consists of two
main components: Crawler and Injection point analyzer. Crawler will set up
the number of layers, then get the entire page of the web site and save as
a list. Injection point analyzer downloads the pages from the url list, then
analyzes the forms in each page to find injection points. Finally, save the list
of injection points to the database. This paper proves that this method can
achieve it and present the effectiveness of increasing detection accuracy. This
response analysis model was too impoverished, so we are keeping completing
more analysis rules [17].
Balume Mburano, Weisheng Si give us an idea about how we can test the
effectiveness of our scanner.The widespread adoption of web vulnerability
scanners and their differences in effectiveness make it necessary to bench-
mark these scanners. Moreover, the literature lacks the comparison of the
results of scanner effectiveness from different benchmarks. In this paper,
They first compared the performances of some open source web vulnerability
scanners of their choice by running them against the OWASP benchmark,
which is developed by the Open Web Application Security Project (OWASP),
a well-known non-profit web security organization. Furthermore, they have
compared results from the OWASP benchmark with the existing results from
the Web Application Vulnerability Security Evaluation Project (WAVSEP)
benchmark, another popular benchmark used to evaluate scanner effective-
ness. Thus evaluation of results allow us to make some valuable recommen-
dations for the practice of benchmarking web scanners [4].
2.2.2 Challenges
Crawling is arguably the most important part of a web application vulnera-
bility scanner; if the scanner’s attack engine is poor, it might miss a vulner-
ability, but if its crawling engine is poor and cannot reach the vulnerability,
then it will surely miss the vulnerability. Because of the critical nature of
crawling, we have included several types of crawling challenges in Vulnscan,
some of which hide vulnerabilities.
• Multi-Step Process Even though most web sites are built on top of
the stateless HTTP protocol, a variety of techniques are utilized to
introduce state into web applications. In order to properly analyze a
web site, web application vulnerability scanners must be able to under-
stand the state-based transactions that take place. In VulnScan, there
are several state-based interactions.
2.2.3 Limitations
• No matter how many vulnerabilities are checked for, there can always
be a vulnerability which was not considered because it was never dis-
covered. The vulnerability may manifest itself sometime in the future.
• The tool can only be tested on applications like DVWA, which limit its
possibilities to learn from a wider array of systems.
25
3 Proposed System
The proposed system for Vulnerability scanner is based on data protection
and encompasses all the traits of a basic security system as well as has the
characteristics of web scanners to enhance the security level. A proposed in
all the research papers that security of the website should be at the highest
priority. By using this system, one can add another layer of security in their
respective application.
Our proposed system involves the process of scanning the websites which
can have vulnerabilities that lead to the data brench and can lead to ir-
reparable brand damage. Our software is open source so that any small or-
ganization or individual can access and test their website before deployment.
They can also customize the software according to their own requirements.
The purpose is to detect the vulnerabilities in the websites and generate re-
ports which helps user to debug. Our system detects vulnerabilities like SQL
injection, XSS (Cross site system), Error based SQLi, Local file inclusion,
Sub-domain scanning, Open port detection. Our system tends to replace the
existing manual system for the scanning process which is time consuming,
less interactive and highly expensive. The main features of this system are
creating report and finding various types of vulnerabilities, storing scanning
data, process initiation and after that it generates a report of whole scanned
websites.
In order to keep the design open and flexible, we used a generic and mod-
ular architecture. The tool consists of a crawling and an attack part, which
can be invoked separately. Through this architectural decision, it is possible
to do a single crawling run (without attacking), to do a single attack run on a
26
During the crawling process, the tool uses a dedicated crawling queue.
This queue is filled with crawling tasks for each web page that is to be
analyzed for referring links and potential target forms. A queue controller
periodically checks the queue for new tasks and passes them on to a thread
controller. This thread controller then selects a free worker thread, which
then executes the analysis task. Each completed task notifies the workflow
controller about the discovered links and forms in the page. The workflow
controller then generates new crawling tasks as needed. After the attack
and analysis components complete their work, the task stores the detection
results into the report.txt file.
• Report creation
• Free of cost
This causes an SQL server exception. If the web application does not han-
dle exceptions or server errors, the result is a SQL error description being in-
cluded in the response page. Based on the previously described assumptions,
the SQL injection analysis module searches response pages for occurrences
of an a prioriconfigured list of weighted key phrases that indicate an SQL
error (see Figure 1). We derived this list by analyzing response pages of web
sites that are vulnerable to SQL injection. Depending on the database server
(e.g., MS SQL Server, Oracle, MySQL, PostgreSQL, etc.) is being used, a
wide range of error responses are generated. Table 1 shows the key phrase
table that we used in our SQL injection analysis module.
28
The simple XSS analysis module takes into account that some of the re-
quired characters for scripting (such as quotes or brackets) could be filtered
or escaped by the target web application. It also verifies that the script is
included at a location where it will indeed be executed by the client browser.
30
Apart from encoded characters, it also uses a mix of uppercase and lowercase
letters to further camouflage the keyword script.
way.
When a victim opens the affected web page in a browser, the XSS attack
payload is served to the victim’s browser as part of the HTML code (just like
a legitimate comment would). This means that victims will end up executing
the malicious script once the page is viewed in their browser.
applications often do not provide so much detail: a simple ‘500 Server Error’
or a custom error page might be issued, meaning that we need to use blind
injection techniques.
3.4.5 Coverage
The core strength and effectiveness of the scanning relies on the breadth and
depth of coverage.
With these insights, businesses can continuously work to reduce risk and
boost resource utilization.
3.5.2 Disadvantages
• It is a command-line program, hence it is not as user-friendly (Users
have to install python 3 and run our code).
4.1.3 Client
Client is responsible for requesting various resources services hosted by the
server.Client computer provides an interface to allow a user to request ser-
vices from the server and display the scanner report to the client. Client
computer will execute the scripts of attacks, it will scrap the webpages from
the client browser and perform different attacks as requested.
• URL: The user enters the URL of there website and the user_id and
password and type the captcha if required.
• Payload length: Select the payload length form the list of payloads
provided by the system.
39
Client :- the user will provide the URL on which attack has to be done.
access and retrieval and completes clients’ requests. Database server is con-
nected to the website.
Tool :- the tool accesses the website to test the functionality of the web-
site and check vulnerabilities of it. After attack the report will be generate
this help the client understand the bugs and vulnerability of the website.
At the client side, the user will enter the URL of target website. The
user is asked to provide username and password for authentication which
will helps crawler to automatically login to the website and access all the
pages. Crawl will find all the links from the target website and put it in a
crawl queue.
Crawl will takes the URL from crawl Queue and attempt to fetch URL
and index the document. After accessing the document crawl will look for
all the links, If any new link found it will add it into crawl queue.
For complete crawl the user needs to provide the username and password
which will activate user session. Crawl will attempt to fetch the document
from the URL provided by the user. For the newly discovered URL it checks
for robot.txt which contains the directories of that particular domain.
From robot.txt if the URL is prohibited then the URL will get rejected.
If the URL is not prohibited then, it will look for the pattern in the URL.
For example if URL contains www.xyz.com/page.php , so this will indicate
that the URL will redirect to the same page. So to avoid this we are checking
the patterns in the URL. If there is no match then the URL gets rejected.
After getting the URL, check it with the crawl queue and if the URL is
already present in the crawl queue then reject the URL.. Continue crawling
by attempting to fetch another URL
43
At the client side, the user will provide the target url, username and
password for the authentication. Once the system authenticates the user
and grants permission it will take the URL from the crawl queue.
For each URL the scanner will start searching for the forms in that par-
ticular web page. Forms contain the input tags, buttons, textbox etc where
the system can enter the data. For each form in that website our scanner
will execute SQL payloads and check whether it contains any vulnerabilities
or not. If it detects any vulnerability it will save it as a report.
44
At the client side, the user will provide the target url, username and password
for the authentication. Once the system authenticates the user and grants
permission it will take the URL from the crawl queue. After crawling the
scanner will search for the forum and input section where we can input the
script. If possible then store the result in the log fill. This log fill helps the
45
user to understand the vulnerability and debug the code.After that it’s just
checking that the input box creates an error or not. This all things are stored
into the file. If executed then submit it into the report or return to the crawl
queue.
The use case used in the use case diagram are as below:
• Crawl pages
• Provide username
• Provide password.
Postcondition: None
Overview: After crawling the target website, the links are stored in the
crawl queue. For each URL system will scan the page and check for the
forms and vulnerabilities in that particular page.
Precondition: Crawling of target website.
Main flow of event:
• Find Results
Overview: After scanning each and every page, the scanner result will be
stored in some log files or report files.
Precondition: Scan all the webpages of the target website.
Main flow of event:
Postcondition: None
47
5 System Implementation
5.1 System Requirements
5.1.1 Hardware requirements
• Any machine having minimum of 4 GB RAM
• Intel core processor
• Windows/Linux Operating System Monitor
5.2.1 Code
Port scanner checks the number of ports open on a given website. By check-
ing that port we can understand which attack can perform on this type of
port. For example if the FTP (21) if they know access information they
can access to the target machine terminal and transfer important data to
their local machine. So we have implemented multithreading in port scan-
ner which scan all the ports which are open by matching with the portsobject.
Port.py
” ’colorama” ’
import socket
import threading
import concurrent.futures
import colorama
from colorama import Fore
from urllib.parse import urlparse
print_lock = threading.Lock()
colorama.init()
portsobject =
21: ’FTP’,
22: ’SSH’,
23: ’Telnet’,
25: ’SMTP’,
43: ’Whois’,
53: ’DNS’,
68: ’DHCP’,
80: ’HTTP’,
110: ’POP3’,
115: ’SFTP’,
119: ’NNTP’,
123: ’NTP’,
139: ’NetBIOS’,
143: ’IMAP’,
161: ’SNMP’,
220: ’IMAP3’,
389: ’LDAP’,
443: ’SSL’,
49
def run_port(ip):
domain = urlparse(ip).netloc
def scan(ip,port):
scanner=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
scanner.settimeout(1)
try:
scanner.connect((ip,port))
scanner.close()
with print_lock:
print(Fore.WHITE+ f"[{port}]" +" " +
f"[portsobject[port]]" + Fore.GREEN+ " Opened")
except:
pass
with concurrent.futures.ThreadPoolExecutor(max_workers=100) as execu-
tor:
for port in range(1,1000):
executor.submit(scan,domain,port)
Subdirectory.py
import requests
import socket
import threading
import concurrent.futures
from concurrent.futures import ThreadPoolExecutor, as_completed
from time import time
50
def request(url):
try:
response = requests.get(url)
if response:
return url
except requests.exceptions.ConnectionError:
pass
except:
pass
def scan_subdirectory(target_url):
wordlist_file = open("common.txt").read().strip().split(”)
start = time()
print("for subdirectory...")
processes = []
with ThreadPoolExecutor(max_workers=100) as executor:
for word in wordlist_file:
processes.append(executor.submit(request, target_url + "/" + word))
for task in as_completed(processes):
if task.result()is None:
pass
else:
print("[+] Discovered Subdirectory –> "+task.result())
print(f’Time taken: time() - start’)
Subdomain.py
import requests
import socket
import threading
import concurrent.futures
from concurrent.futures import ThreadPoolExecutor, as_completed
from time import time
from os import system, name
def clear():
if name == ’nt’:
_ = system(’cls’)
else:
_ = system(’clear’)
def request(url):
try:
response = requests.get(url,timeout=1)
if response:
return url
except requests.exceptions.ConnectionError:
pass
except requests.exceptions.InvalidURL:
pass
except requests.exceptions.Timeout:
pass
except:
pass
def run_subdomain(target_url):
wordlistf ile =
open(”subdomains_list_1.txt”).read().strip().split(′′ )
start = time()
print(”Scanningf orsubdomains...”)
processes = []
withT hreadP oolExecutor(max_workers = 10)asexecutor :
f orurlinwordlist_f ile :
processes.append(executor.submit(request,
url + ”.” + target_url))
52
In the below script we have implemented the xss scanner and sql scanner.
To implement this scanner we have taken some payloads which are stored
in xss_payload.txt file and sql_payload.txt file. First the scanner will crawl
the complete website using target url and store all the links into the crawl
queue.From each link extract_forms() function will extract all the forms in
that particular web page .Then for each form it will find the input type and
actions where the form will submit automatically. So by giving payload as
input we will test whether the webpage is vulnerable or not.
Scanner.py
!/usr/bin/env python3
from operator import is_
import requests
import re
import urllib.parse as urlparse
from bs4 import BeautifulSoup
payload_xss = open("xss_payload.txt").read().strip().split(”)
payload_sql = open("sql_payload.txt").read().strip().split(”)
total_payload=len(payload_xss)
class Scanner:
def_i nit_( self, url, ignore_links) :
self.session = requests.Session()
self.target_url = url
self.target_links = []
self.linkst oi gnore = ignore_links
def extract_links_f rom(self, url) :
response = self.session.get(url)
html = response.text
returnre.f indall(r′ (? : href = ”)(.∗?)”′ , html)
53
if "" in link:
link = link.split("")[0]
self.target_links.append(link)
print(link)
self.crawl(link)
def extract_f roms(self, url) :
response = self.session.get(url)
parsed_html = Beautif ulSoup(response.text)
returnparsed_html.f indAll(”f orm”)
post_data[input_name] = input_value
if method == "post":
return self.session.post(post_url, data=post_data)
return self.session.get(post_url, params=post_data)
def run_scanner(self):
for link in self.target_links:
forms = self.extract_froms(link)
for form in forms:
print("[+] Testing form in " + link)
for payload in payload_xss:
isv ulnerable_to_xss = self.test_xss_in_f orm(f orm, link, payload)
if isv ulnerable_tox ss :
print(”[+ + +]XSSdiscoveredin” + link + ”inthef ollowingf orm”)
print(f orm.prettif y())
print(”P ayload : ” + payload)
break
if "=" in link:
print("[+] Testing " + link)
for payload in payload_xss:
isv ulnerable_to_xss = self.test_xss_in_link(link, payload)
if is_vulnerable_to_xss :
print(”[∗ ∗ ∗]DiscoveredXSSin” + link)
print(”P ayload : ” + payload)
break
def run_sql_scanner(self ) :
f orlinkinself.target_links :
f orms = self.extract_f roms(link)
f orf orminf orms :
print(”[+]T estingf ormin” + link)
f orpayloadinpayload_sql :
is_vulnerable_to_sql = self.test_sql_in_f orm(f orm, link, payload)
if isv ulnerable_to_sql :
print(”[+ + +]SQLdiscoveredin” + link + ”inthef ollowingf orm”)
print(f orm.prettif y())
print("Payload : "+payload)
break
55
if "=" in link:
print("[+] Testing " + link)
for payload in payload_sql:
is_vulnerable_to_sql = self.test_sql_in_link(link,payload) if isv ulnerable_to_sql :
ErrorSQLi.py
56
context.log_level = ’info’
def sqli nject(sqli_pt1, variable, sqli_pt2) :
BuildupU RLandexecuteSQLi
next_url = url + f ixed_query + sqli_pt1 + variable + sqli_pt2
debug(”T esting” + variable + ”onΞ ” + nextu rl + ”Ξ
”)
returnrequests.get(next_url, cookies = cookies)
def guess_len(guess_type, sqli_pt1, sqli_pt2) :
f oriinrange(1, 100) :
response = sql_inject(sqli_pt1, str(i), sqli_pt2)
error_message = re.search(r′ U ser.∗Ω′ , response.text).group(0)debug(errorm essage)if ”M ISSIN G
while(found_nextc har! = 2) :
comparison = next(comparison_types)
found_next_char += 1
name += chr(current_char)
info("Found char(" + str(i) + "): " + chr(current_char))
success(guess_type + name + ”)
return name
def error_func():
db_name_len = guess_len("DB Name Length: ", "’+and+length(database())+=",
"+%23") db_name = guessn ame(”DBN ame : ”, ”′ +and+ascii(substr(database(), ”, ”+
%23”, db_name_len, ord(′ a′ ), ord(′ z ′ ))
import requests
import scanner
import re
import subdirectory as subdir
import sys
import socket
import threading
import concurrent.futures
from concurrent.futures import ThreadPoolExecutor, as_completed
from time import time
from os import system, name
import errorsql as ErrorSQL
from operator import is_
import urllib.parse as urlparse
from bs4 import BeautifulSoup
import port as pt
import subdomaincrt as subdomain
import subdomain as subdm
59
def reset_database(target_url):
url = target_url+"setup.php"
response = requests.get(url)
parsed_html = BeautifulSoup(response.text)
forms = parsed_html.findAll("form")
for form in forms:
abc = submit_form(form,"",url)
print(".....Welcome to VulnScan.....")
# forms = vuln_scanner.extract_froms
("http://192.168.0.108/dvwa/vulnerabilities/xss_r/")
# # print(forms)
# reponse = vuln_scanner.test_xss_in_link
60
("http://192.168.0.108/dvwa/vulnerabilities/xss_r/?name=test")
# print(reponse)
pt.runp ort(target_url)
subdir.scan_subdirectory(target_url)
subdm.run_sub(target_url)
reset_dat(target_url)
vuln_scanner.crawl(target_url)
vuln_scanner.run_scanner()
resetd atabase(target_url)
vuln_scanner.run_sql_scanner()
reset_database(target_url)
ErrorSQL.error_f unc()
reset_database(target_url)
61
The user has to enter the IP/Name of the target website and the scanner
will check for the open ports. Here we have use our dvwa website for testing.
After scanning the port it will start crawling the complete website.
62
Fig.6.1
Scanning for subdirectory and subdomains and also display the time taken
by the scanner.
Fig.6.2
Testing for the XSS vulnerabilities in each link. It will also display the form
in which vulnerability detected and the payload which is used for attack.
63
Fig.6.3
Fig.6.4
Testing for the SQL vulnerabilities in each link. It will also display the form
in which vulnerability detected and the payload which is used for attack.
64
Fig.6.5
Fig.6.6
When the scanner tries to insert a malicious query in input fields and gets
some error which is regarding SQL syntax or database. Here it will return
the database length and the database name by finding the ascii character. It
will also return tablename, Field count,Field name and database version.
65
Fig.6.7
66
Fig.6.8
67
Fig.6.9
68
Fig.6.10
Fig.6.11
69
7 Summary
7.1 Conclusion
Security is an eternal concern for the software world. With increasing number
of tools and advancements in technology, cyber-attackers have a lot at their
disposal to disrupt a system. For this reason, research and investments in
web vulnerabilities should be encouraged and given a high priority. Most
commercial vulnerability testing tools are paid (only a few people can afford
to use them). Therefore, the open-source community can be major help to
achieve the goal of a safer and more robust internet for everyone.
Our scanner primarily focuses on two of the most common web vulner-
abilities: XSS and SQL-Injection. Our project is an open-source endeavour
and is also subject to improvement. Comparison with other systems reveals
both advantages and disadvantages.
While scanners can’t completely replace penetration testers, they can
certainly help people safeguard their systems before a disaster.
7.2.3 Speed
Some of our current algorithms might be improved to gain some speed. This
will help the system work with much more efficiency.
70
8 Bibliography
[1] Richard Amankwah, Jinfu Chen, Patrick Kwaku Kudjo, Beatrice Korkor
Agyemang Alfred Adutwum Amponsah. "An automated framework
for evaluating open-source web scanner vulnerability severity". Service
Oriented Computing and Applications volume 14, pages 297–307, 2020.
[2] Richard Amankwah, Jinfu Chen, Patrick Kwaku Kudjo, Dave Towey.
"An empirical comparison of commercial and open-source web vulnera-
bility scanners". National Natural Science Foundation of China, 2020.
[3] Rahul Maini, Rahul Pandey, Rajeev Kumar, Rajat Gupta. "Automatic
Web Vulnerability Scanner". International Journal of Engineering Ap-
plied Sciences and Technology, 2019.
[12] Y. Tung, S. Tseng, J. Shih and H. Shan. "W-VST: A Testbed for Eval-
uating Web Vulnerability Scanner" 14th International Conference on
Quality Software, 2014.
[14] Avinash Kumar Singh, Sangita Roy. "A network based vulnerability
scanner for detecting SQLI attacks in web applications". IEEE 2012 1st
International Conference On Recent Advances In Information Technol-
ogy, 2012.
[15] Katkar Anjali S., Kulkarni Raj B. "Web Vulnerability Detection and
Security Mechanism". International Journal of Soft Computing and
Engineering (IJSCE), 2012.
[16] Adam Doupe, Marco Cova, Giovanni Vigna. "Why Johnny Can’t Pen-
test: An Analysis of Black-Box Web Vulnerability Scanners". 7th Inter-
national Conference, DIMVA 2010, Bonn, Germany, July 8-9, 2010.
[20] Jose Fonseca, Marco Vieira, Henrique Madeira. "Testing and compar-
ing web vulnerability scanning tools for SQL injection and XSS attacks".
13th IEEE International Symposium on Pacific Rim Dependable Com-
puting, 2007.