1

A penetration tester’s
guide to subdomain
enumeration By: Bharath

Note: This document is not created by a professional content writer so any mistake and
error is a part of great design

Social media:@viehgroup vww.viehgroup.com [email protected]

 2

Disclaimer

This document is generated by VIEH Group and if there is any contribution or or

credit, it’s mentioned on the first page. The information provided herein is for

educational purposes only and does not constitute legal or professional advice. While

we have made every effort to ensure the accuracy and reliability of the information

presented, VIEH Group disclaims any warranties or representations, express or

implied, regarding the completeness, accuracy, or usefulness of this document. Any

reliance you place on the information contained in this document is strictly at your

own risk. VIEH Group shall not be liable for any damages arising from the use of or

reliance on this document. also we highly appreciate the source person for this

document.

Happy reading !

Content Credit: Bharath

Social media:@viehgroup vww.viehgroup.com [email protected]

 3

Introduction
As a penetration tester or a bug bounty hunter, most of the times you

are given a single domain or a set of domains when you start a security

assessment. You’ll have to perform extensive reconnaissance to find

interesting assets like servers, web applications, domains that belong to

the target organisation so that you can increase your chances of finding

vulnerabilities.

Sub-domain enumeration is an essential part of the reconnaissance

phase. This blog post covers various sub-domain enumeration

techniques in a crisp and concise manner.

A gitbook will be released as a follow up for this blog post on the same

topic where we cover these techniques in-depth. We covered some of

these techniques in the “Esoteric sub-domain enumeration techniques”

talk given at Bugcrowd LevelUp conference 2017.

What is sub-domain enumeration?

Sub-domain enumeration is the process of finding sub-domains for one

or more domain(s). It is an essential part of the reconnaissance phase.

Social media:@viehgroup vww.viehgroup.com [email protected]

 4

Why sub-domain enumeration?

 Sub-domain enumeration can reveal a lot of domains/sub-

domains that are in scope of a security assessment which in turn

increases the chances of finding vulnerabilities

 Finding applications running on hidden, forgotten sub-domains

may lead to uncovering critical vulnerabilities

 Often times the same vulnerabilities tend to be present across

different domains/applications of the same organization

The famous Yahoo! Voices hack happened due to a vulnerable application deployed on a
yahoo.com sub-domain

Sub-domain enumeration techniques

1. Search engines like Google and Bing supports various advanced

search operators to refine search queries. These operators are

often referred to as “Google dorks”.

 We can use “site:” operator in Google search to find all the

sub-domains that Google has found for a domain. Google also

supports additional minus operator to exclude sub-domains

Social media:@viehgroup vww.viehgroup.com [email protected]

 5

that we are not interested in “site:*.wikimedia.org -www -

store -jobs -uk”

Using site operator in Google search to find sub-domains

 Bing search engine supports some advanced search operators

as well. Like Google, Bing also supports a “site:” operator that

you might want to check for any additional results apart from

the Google search

Social media:@viehgroup vww.viehgroup.com [email protected]

 6

Finding sub-domains using “site:” operator in Bing

2. There are a lot of the third party services that aggregate massive

DNS datasets and look through them to retrieve sub-domains for

a given domain.

Social media:@viehgroup vww.viehgroup.com [email protected]

 7

VirusTotal runs its own passive DNS replication service, built by

storing DNS resolutions performed when visiting URLs submitted

by users. In order to retrieve the information of a domain you just

have to put domain name in the search bar

Searching for sub-domains using virustotal

sub-domains found using VirusTotal

Social media:@viehgroup vww.viehgroup.com [email protected]

 8

 DNSdumpster is another interesting tools that can find

potentially large number of sub-domains for a given domain

Searching for sub-domains using DNSdumpster

The OWASP Amass tool suite obtains subdomain names by

scraping data sources, recursive brute forcing, crawling web
archives, permuting/altering names and reverse DNS
sweeping.

amass --passive -d appsecco.com # Amass 2.x

amass enum --passive -d appsecco.com # Amass 3.x

Social media:@viehgroup vww.viehgroup.com [email protected]

 9

Using OWASP Amass to discover subdomains for a given domain

3. Certificate Transparency(CT) is a project under which a Certificate

Authority(CA) has to publish every SSL/TLS certificate they issue

to a public log. An SSL/TLS certificate usually contains domain

names, sub-domain names and email addresses. This makes them

a treasure trove of information for attackers. I wrote a series of

technical blog posts on Certificate Transparency where I covered

this technique in-depth, you can read the series here.

The easiest way to lookup certificates issued for a domain is to use

search engines that collect the CT logs and let’s anyone search through

them. Few of the popular ones are listed below -

1. https://crt.sh/

2. https://censys.io/

Social media:@viehgroup vww.viehgroup.com [email protected]

 10

3. https://developers.facebook.com/tools/ct/

4. https://google.com/transparencyreport/https/ct/

Finding sub-domains of an organisation’s primary domain using crt.sh

Additional to the web interface, crt.sh also provides access to their CT

logs data using postgres interface. This makes it easy and flexible to run

some advanced queries. If you have the PostgreSQL client software

installed, you can login as follows:

psql -h crt.sh -p 5432 -U guest certwatch

We wrote few scripts to simplify the process of finding sub-domains

using CT log search engines. The scripts are available in our github repo

Social media:@viehgroup vww.viehgroup.com [email protected]

 11

Interesting sub-domain entry from CT logs for uber.com

The downside of using CT for sub-domain enumeration is that

the domain names found in the CT logs may not exist anymore
and thus they can’t be resolved to an IP address. You can use
tools like massdns in conjunction with CT logs to quickly
identify resolvable domain names.
# ct.py - extracts domain names from CT Logs(shipped with
massdns)
# massdns - will find resolvable domains & adds them to a file
./ct.py icann.org | ./bin/massdns -r resolvers.txt -t A -q -a -o
-w icann_resolvable_domains.txt -

Using massdns to find resolvable domain names

4. Dictionary based enumeration is another technique to find sub-

domains with generic names. DNSRecon is a powerful DNS

enumeration tool, one of it’s feature is to conduct dictionary based sub-

domain enumeration using a pre-defined wordlist.

Social media:@viehgroup vww.viehgroup.com [email protected]

 12

python dnsrecon.py -n ns1.insecuredns.com -d insecuredns.com -D

subdomains-top1mil-5000.txt -t brt

Dictionary based enumeration using DNSRecon

5. Permutation scanning is another interesting technique to identify

sub-domains. In this technique, we identify new sub-domains

using permutations, alterations and mutations of already known

domains/sub-domains.

 Altdns is a tool that allows for the discovery of sub-domains

that conform to patterns

python altdns.py -i icann.domains -o data_output -w icann.words -r -s

results_output.txt

Finding sub-domains that match certain permutations/alterations using AltDNS

Social media:@viehgroup vww.viehgroup.com [email protected]

 13

6. Finding Autonomous System (AS) Numbers will help us identify

netblocks belonging to an organization which in-turn may have

valid domains.

 Resolve the IP address of a given domain using dig or host

 There are tools to find ASN given an IP address —

https://asn.cymru.com/cgi-bin/whois.cgi

 There are tools to find ASN given a domain name —

http://bgp.he.net/

Finding AS Number using IP address

 The ASN numbers found can be used to find netblocks of the

domain. There are Nmap scripts to achieve that —

https://nmap.org/nsedoc/scripts/targets-asn.html

nmap --script targets-asn --script-args targets-asn.asn=17012 >

netblocks.txt

Social media:@viehgroup vww.viehgroup.com [email protected]

 14

Finding netblocks using AS numbers — NSE script

7. Zone transfer is a type of DNS transaction where a DNS server

passes a copy of full or part of it’s zone file to another DNS server.

If zone transfers are not securely configured, anyone can initiate a

zone transfer against a nameserver and get a copy of the zone

file. By design, zone file contains a lot of information about the

zone and the hosts that reside in the zone.

dig +multi AXFR @ns1.insecuredns.com insecuredns.com

Successful zone transfer using DIG tool against a nameserver for a domain

Social media:@viehgroup vww.viehgroup.com [email protected]

 15

8. Due to the way non-existent domains are handled in DNSSEC, it is

possible to “walk” the DNSSEC zones and enumerate all the

domains in that zone. You can learn more about this technique

from here.

 For DNSSEC zones that use NSEC records, zone walking can be

performed using tools like ldns-walk

ldns-walk @ns1.insecuredns.com insecuredns.com

Zone walking DNSSEC zone with NSEC records

 Some DNSSEC zones use NSEC3 records which uses hashed

domain names to prevent attackers from gathering the plain text

domain names. An attacker can collect all the sub-domain hashes

and crack the hashes offline

 Tools like nsec3walker, nsec3map help us automate the collecting

NSEC3 hashes and cracking the hashes. Once you install

nsec3walker, you can use the following commands to enumerate

sub-domains of NSEC3 protected zone

 # Collect NSEC3 hashes of a domain
$ ./collect icann.org > icann.org.collect# Undo the

Social media:@viehgroup vww.viehgroup.com [email protected]

 16

hashing, expose the sub-domain information.

$ ./unhash < icann.org.collect > icann.org.unhash# Listing
only the sub-domain part from the unhashed data
$ cat icann.org.unhash | grep "icann" | awk '{print $2;}'
del.icann.org.
access.icann.org.
charts.icann.org.
communications.icann.org.
fellowship.icann.org.
files.icann.org.
forms.icann.org.
mail.icann.org.
maintenance.icann.org.
new.icann.org.
public.icann.org.
research.icann.org.

9. There are projects that gather Internet wide scan data and make

it available to researchers and the security community. The

datasets published by this project are a treasure trove of sub-

domain information. Although finding sub-domains in this

massive dataset is like finding a needle in the haystack, it is worth

the effort.

 Forward DNS dataset is published as part of Project Sonar. This

data is created by extracting domain names from a number of

sources and then sending an ANY query for each domain. The

data format is a gzip-compressed JSON file. We can parse the

dataset to find sub-domains for a given domain. The dataset is

massive though (20+GB compressed, 300+GB uncompressed)

 # Command to parse & extract sub-domains for a given
domaincurl -silent

Social media:@viehgroup vww.viehgroup.com [email protected]

 17

https://scans.io/data/rapid7/sonar.fdns_v2/20170417-
fdns.json.gz | pigz -dc | grep “.icann.org” | jq

Enumerating domains/subdomains using FDNS dataset

10. Content Security Policy (CSP) defines the Content-Security-

Policy HTTP header, which allows you to create a whitelist of

sources of trusted content, and instructs the browser to only

execute or render resources from those sources. So basically,

Content-Security-Policy header will list a bunch of sources

(domains) that might be of interest to us as attackers. There are

deprecated forms of CSP headers -X-Content-Security-Policy and

X-Webkit-CSP. You can use following curl command to extract CSP

headers or a given URL —

curl -s -I -L “https://www.newyorker.com/" | grep -Ei ‘^Content-
Security-Policy:’ | sed “s/;/;\\n/g”

Sub-domain enumeration techniques — A comparison

We ran few of the discussed techniques against icann.org and

compared the results. The bar chart below shows the number of

unique, resolvable sub-domains each technique found for icann.org.

Social media:@viehgroup vww.viehgroup.com [email protected]

 18

Feel free to get in touch with us to know the methods we used to

gather this information.

Number of unique, resolvable sub-domains each technique found for icann.org

References

 https://github.com/appsecco/bugcrowd-levelup-subdomain-

enumeration

 https://blog.appsecco.com/open-source-intelligence-gathering-

101-d2861d4429e3

 https://www.databreaches.net/hackers-post-450k-credentials-

apparently-pilfered-from-yahoo/

Social media:@viehgroup vww.viehgroup.com [email protected]

 19

 http://info.menandmice.com/blog/bid/73645/Take-your-DNSSEC-

with-a-grain-of-salt

 https://www.peerlyst.com/posts/bsideslv-2017-breaking-ground-

with-underflow-bsides-las-vegas

-----x-----

Social media:@viehgroup vww.viehgroup.com [email protected]

 20

Thanks for reading

Social media:@viehgroup vww.viehgroup.com [email protected]