Contents

Introduction.......................................................................................3
Default Procedure ..............................................................................5
Store this wordlist in a file..................................................................6
Generating wordlists of a certain length .............................................7
Retrieval of Emails from the website: .................................................7
To count the number of words repeated on the website ....................8
Increase Spider depth ........................................................................9
Verbose Mode ................................................................................. 10
Alphanumeric Wordlist .................................................................... 11
Cewl with Digest/Basic Authentication ............................................. 12
Lowercase all parsed words ............................................................. 13
Proxy Support .................................................................................. 14

Page 2 of 15
Introduction
CeWL – A custom wordlist generator is a ruby program that crawls a specific URL to a defined
depth and returns a list of keywords, which password crackers like John the Ripper, Medusa,
and WFuzz can use to crack the passwords. Cewl also has an associated command-line app FAB,
which uses the same metadata extraction techniques to generate author/producer lists from
already downloaded files using information extraction algorithms like CeWL.
CeWL comes preinstalled with Kali Linux. With this tool, we can easily collect words and phrases
from the target page. It is a robust program that can quickly scrape the webserver of any
website.
Open the terminal of Kali Linux and type “cewl -h” to see the lists of all the options it accepts,
with a complete description.
Syntax: cewl <url> [options]

Page 3 of 15
General Options :
-h, –help: Show help.
-k, –keep: Keep the downloaded file.
-d <x>, –depth <x>: Depth to spider to, default 2.
-m, –min_word_length: Minimum word length, default 3.
-o, –offsite: Let the spider visit other sites.
-w, –write: Write the output to the file.
-u, –ua <agent>: User agent to send.

Page 4 of 15
 -n, –no-words: Don’t output the wordlist.
–with-numbers: Accept words with numbers in as well as just letters
-a, –meta: include meta data.
–meta_file file: Output file for Meta data.
-e, –email: Include email addresses.
–email_file <file>: Output file for email addresses.
-c, –count: Show the count for each word found.
-v, –verbose: Verbose.
–debug: Extra debug information
Authentication
–auth_type: Digest or basic.
–auth_user: Authentication username.
–auth_pass: Authentication password.
Proxy Support
–proxy_host: Proxy host.
–proxy_port: Proxy port, default 8080.
–proxy_username: Username for proxy, if required.
–proxy_password: Password for proxy, if required.

Default Procedure
Use the following command to generate a list of words that will spider the given URL to a
specified depth and we can use it as a directory for cracking the passwords.
cewl http://www.vulnweb.com

Page 5 of 15
Store this wordlist in a file
Now to save this all wordlist in a file for record-keeping, efficiency and readability we will use
the -w option to save the output in a text file.
cewl http://www.vulnweb.com -w dict.txt

Here dict.txt is the file name where the wordlist will be stored. Once the file has been created
you can open it to see if the output is stored in the file.

Page 6 of 15
Generating wordlists of a certain length
If you want to create a wordlist of a specific length then you can choose to use option -m and
provide the minimum length for the keyword hence it will create wordlists for a certain length.
cewl http://vulnweb.com / -m 10 -w dict.txt

So basically, this will create a wordlist in which each word has a minimum of 10 letters and
store these keywords in the file dict.txt. Screenshot is attached for your reference.

Retrieval of Emails from the website:

In order to retrieve emails from the website, we can use the -e option, while the -n option will
hide the lists created while crawling the provided website. As you can see in the screenshot
attached it has found 1 email-id from the website.

cewl https://digi.ninja/contact.php -e -n

Page 7 of 15
To count the number of words repeated on the website
If you want to count the number of times a word is repeated on a website, then use the -c
option that will enable the count parameter.
cewl http://www.vulnweb.com -c

For your reference, a screenshot is added below which prints the count for every keyword
repeated on website.

Page 8 of 15
Increase Spider depth
You can use -d option with the depth number to activate depth parameter for more quick and
intense crawling so that a large list of words is created. The depth level is set to 2 as default.

cewl http://vulnweb.com -d 3

Page 9 of 15
Verbose Mode
We have a -v option for the verbose mode to extend the website crawling result and retrieve
complete detail of the website.
cewl http://vulnweb.com -v
So, this will display extended website crawling results. Below we have attached a screenshot so
that you will get a clear idea.

Page 10 of 15
Alphanumeric Wordlist
Sometimes it may happen that you may need an alpha-numeric wordlist that you can use –the
with-numbers option to get an alpha-numeric wordlist.
cewl http://testphp.vulnweb.com/artists.php --with-numbers

Page 11 of 15
Cewl with Digest/Basic Authentication
It may happen sometimes that some web applications may have an authentication page for
login and for that the above basic command will not give desired results. So for that, you need
to bypass the authentication page by using the command given below.
cewl http://testphp.vulnweb.com/login.php --auth_type Digest --auth_user test –auth_pass test -v

In this command we have used the following options:

–auth_type: Digest /Basic
–auth_user: Authentication Username

Page 12 of 15
–auth_pass: Authentication password

Lowercase all parsed words

When you need the keywords to be generated in lowercase for that you can use the –lowercase
option to generate the words in lowercase.

Page 13 of 15
Proxy Support
This default command for cewl will not work properly if you have attached a proxy server. We
tried to access the application through ip address but the proxy server is attached hence this
gave us a Forbidden Error page.

And here if we apply the default cewl command so it will generate the error page wordlist.
Hence to get the appropriate wordlist of the web application we have used commands as:
cewl http://192.168.1.141 --proxy_host 192.168.1.141 --proxy_port 3128
In this command we have used the following options:
–proxy_host: Your Host
–proxy_port: Port number of your proxy

Page 14 of 15
Page 15 of 15
 JOIN OUR
TRAINING PROGRAMS
H ERE
CLICK BEGINNER

Bug Bounty Network Security

Ethical Hacking Essentials

Network Pentest
Wireless Pentest

ADVANCED

Burp Suite Pro Web Pro Computer

Services-API Infrastructure VAPT Forensics

Advanced CTF
Android Pentest Metasploit

EXPERT

Red Team Operation

Privilege Escalation
APT’s - MITRE Attack Tactics
Windows
Active Directory Attack
Linux
MSSQL Security Assessment

www.ignitetechnologies.in