Experiment No.

02

AIM: Using OSINT tool such as (Harverster) you can gather information like emails,
subdomains, hosts, employee names, open ports and banners from different public sources
like search engines, PGP key server.
THEORY:
1.OSINT TOOLS:-
Open-source intelligence (OSINT) tools are software applications or services designed to
gather information from publicly available sources on the internet. OSINT tools are
commonly used by researchers, investigators, cybersecurity professionals, and law
enforcement agencies to collect data and gain insights for various purposes, such as threat
hunting, reconnaissance, competitive analysis, and more.
Here are some popular OSINT tools as of my last update in September 2021:
Maltego: Maltego is a powerful OSINT and data visualization tool that helps in the discovery
and analysis of relationships between entities on the internet.
The Harvester: This tool allows you to gather email addresses, subdomains, hosts, and other
useful information from public sources like search engines and PGP key servers.
Shodan: Shodan is a search engine that lets you find internet-connected devices, such as
servers, routers, webcams, and more. It provides detailed information about these devices,
including open ports, banners, and vulnerabilities.
Censys: Censys is another search engine that focuses on finding internetconnected devices,
such as servers and IoT devices. It provides valuable information like SSL certificates,
banners, and more.
FOCA (Fingerprinting Organizations with Collected Archives): FOCA is a tool for extracting
metadata, hidden information, and data from documents and files, helping to identify
potential security risks.
SpiderFoot: SpiderFoot is an open-source footprinting tool that automates the process of
gathering intelligence from various sources, including search engines, social media platforms,
DNS records, and more.
OSINT Framework: This is not a single tool but rather a collection of various OSINT tools
organized into categories. It's a great resource for OSINT enthusiasts looking for specific
tools for different tasks.
Ghidra: Developed by the NSA, Ghidra is a powerful reverse engineering tool used to
analyze and understand the inner workings of software and malware.
Snort: Snort is an open-source intrusion detection and prevention system (IDS/IPS) used to
monitor network traffic and identify potentially malicious activities.
Metasploit: Metasploit is a penetration testing framework that helps in identifying and
exploiting vulnerabilities in networks, applications, and systems.
2.What is theHarvester tool?
theHarvester is a tool for gathering email addresses, subdomains, hosts, employee names,
open ports, and banners from different public sources (search engines, PGP key servers). It is
designed to be used in the full reconnaissance and information-gathering phases of a
penetration test. This tool is useful for finding potential targets on a network and gathering
information about a company or organization. It can also be used to verify the security of
your own email server.
3.Installing in Kali Linux to install theHarvester tool in kali Linux follow the following steps:
Steps:- Step 1
download theharvester tool from theharvester GitHub page
git clone https://github.com/laramies/theHarvester
Output:

┌──(kali㉿kali)-[~]└─$ git

clone https://github.com/laramies/theHarvester Cloning into 'theHarvester'...remote: Enumerating

objects: 12250, done.remote:
Counting objects: 100% (69/69), done.remote: Compressing objects: 100 %
(50/50), done.remote: Total 12250 (delta 27), reused 51 (delta 19), packreused 12181Receiving
objects: 100% (12250/12250), 7.00 MiB | 213.00 KiB/s, done.Resolving deltas: 100% (7699/7699),
done.

Step 2
install python3-pip in theharvester directory
sudo apt install python3-pip
Output:

(kali㉿kali)-[~/theHarvester]└─$ sudo apt install python3-pipReading

package lists... DoneBuilding dependency tree... DoneReading state information... DoneThe

following additional packages will be installed: python3-pip-whlThe following packages will
be upgraded: python3python3-pip-whl2 upgraded, 0 newly installed, 0 to remove and 1433
upgraded.Need to get 3,034 kB of archives.After this operation, 61.4 additional disk space
will be used.Do you want to continue? [Y/n] yGet http://http.kali.org/kali kali-rolling/main
amd64 python3-pip all 22.3+ 1 [1,322 kB]Get:2 http://http.kali.org/kali kali-rolling/main
amd64 python3-pip-whl all 22.3+dfsg-1 [1,712 kB]Fetched 3,034 kB in 10s (
kB/s) (Reading database ... 348245 files and directories currently
installed.)Preparing to unpack .../python3pip_22.3+dfsg-1_all.deb ...Unpacking python3-pip
(22.3+dfsg-1) over (22.2+dfsg-1) ...Preparing to unpack
.../python3-pip-whl_22.3+dfsg1_all.deb ...Unpacking python3-pip-whl (22.3+dfsg-1) over
(22.2+dfsg
1) ...Setting up python3-pip-whl (22.3+dfsg-1) ...Setting up python3- pip
not
kB of
:1
dfsg-
296

pip
(22.3+dfsg-1) ...Processing triggers for man-db (2.10.2-1) ...Processing
triggers for kali-menu (2022.3.1) ...

Step 4
install basic requirements from dev.txt file in theharvester directory
python3 -m pip install -r requirements/dev.txt#else:python3 -m pip install -
r requirements/base.txt
Output:

(kali㉿kali)-[~/theHarvester]└─$ python3 -m pip install -r

requirements/dev.txt Defaulting to user installation because normal site

packages is not writeableRequirement already satisfied: aiodns
/usr/lib/python3/dist-packages (from -r requirements/ (3.0.0)Collecting
aiofiles==22.1.0 Downloading aiofilesnone-any.whl (14 kB)Collecting
aiohttp==3.8.3
3.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
(1.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
━━━━━━━
1.0/1.0 MB 101.7 kB/s eta 0:00:00Collecting aiomultiprocess Downloading -
aiomultiprocess-0.9.0-py3-none-any.whl (17 kB)Requirement already ==3.0.0 in
satisfied: aiosqlite==0.17.0 in /usr/lib/python3/dist-packages (from -r base.txt (line 1))
requirements/ 22.1.0-py3-
(0.17.0)Requirement already satisfied: beautifulsoup4== Downloading aiohttp-
/usr/lib/python3/dist-packages (from -r requirements/ (4.11.1)Collecting
censys==2.1.9 Downloading censys-2.1 any.whl (53 kB) ==0.9.0

base.txt (line 5))

4.11.1 in base.txt
(line 6))
.9-py3-none-
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
53.7/53.7 kB 233.7 kB/s eta 0:00:00

Step 5
Run theharvester tool by executing this command
python3 theHarvester.py -h
Output:
(
kali㉿kali)-[~/theHarvester]└─$ python3 theHarvester.py -h
************************************************************
******** _ _ _ ** | |_| |__ ___ /\
/\__ _ _ ____ _____ ___| |_ ___ _ __ ** | __| _ \ / _ \ / /_/ / _` | '__\ \ / /
_ \/ __| __/ _ \ '__| ** | |_| | | | __/ / __ / (_| | | \ V / __/\__ \ || __/ | **
\__|_| |_|\___| \/ /_/ \__,_|_| \_/ \___||___/\__\___|_| **
** theHarvester 4.3.0-dev ** Coded by Christian
Martorella ** Edge-Security Research **
[email protected] **
************************************************************
********usage: theHarvester.py [-h] -d DOMAIN [-l LIMIT] [-S START]
[-p] [-s] [--screenshot SCREENSHOT] [-v] [-e
DNS_SERVER] [-r] [-n] [-c] [-f FILENAME] [-b
SOURCE]theHarvester is used to gather open source intelligence (OSINT) on acompany or
domain.options: -h, --help show this help message and exit -d DOMAIN, --domain
DOMAIN Company name or domain to search. -l LIMIT, --limit LIMIT
Limit the
number of search results, default=500. -S START, --start START Start with
result number X, default=0. -p, --proxies Use proxies for requests, enter proxies in
proxies.yaml. -s, --shodan
Use Shodan to query discovered hosts. --screenshot SCREENSHOT
Take screenshots of resolved domains specify output -e to
directory: --screenshot output_directory -v, --virtual-host Verify host name via DNSlookup
resolution and search for virtual hosts.
DNS_SERVER, --dns-server DNS_SERVER DNS server use for
lookup. -r, --take-over Check for takeovers. -n, --dnsEnable DNS server lookup,
default False. -c, --dns-brute Perform a
DNS brute force on the domain. -f FILENAME, --filename FILENAME
Save the results to an XML and JSON file. -b SOURCE, --source
SOURCE anubis, baidu, bevigil, binaryedge, bing, bingapi,
bufferoverun, censys, certspotter, crtsh, dnsdumpster, duckduckgo,
fullhunt, github-code, hackertarget, hunter, intelx, otx, pentesttools,
projectdiscovery, qwant, rapiddns, rocketreach, securityTrails, sublist3r,
threatcrowd, threatminer, urlscan, virustotal, yahoo, zoomeye

Step 6 use theharvester tool feature by giving a domain name in the command
python3 theHarvester.py -d <DOMAIN NAME> -l 500 -b <SOURCE>
Output:

Step 7
On Kali Linux, run theHarvester in a terminal window to see if it's installed. If not, you'll see:

You can sometimes run apt-get theharvester and Kali will fetch this for you, but in my case, it
didn't work. So instead, clone it directly and confirm the installation by running the following
in terminal.the harvester: The objective of this program is to gather emails, subdomains,
hosts, employee names, open ports and banners from different public sources like search
engines, PGP key servers and SHODAN computer database.
Here we will use this tool to search for information about Microsoft organization
the harvester options
-d: Domain to search or company name
-b: data source: Google, GoogleCSE, Bing, Bing API, PGP, LinkedIn,
Google- profiles, people123, jigsaw, Twitter, Google+, all
-s: Start in result number X (default: 0)
-v: Verify hostname via DNS resolution and search for virtual hosts
-f: Save the results into an HTML and XML file
-n: Perform a DNS reverse query on all ranges discovered
-c: Perform a DNS brute force for the domain name
-t: Perform a DNS TLD expansion discovery
-e: Use this DNS server
-l: Limit the number of results to work with(bing goes from 50 to 50 results,
-h: use SHODAN database to query discovered hosts google 100 to 100, and PGP doesn’t use
this option)
Command: theharvester -d microsoft.com -l 500 -b google
these are the important information( host & email addresses) I got using this tool.
Another interesting feature is the capability to check for virtual hosts: through DNS
resolution, the tool verifies if a certain IP address is associated with multiple hostnames. This
is really important information because the Security for a given host on that IP depends not
only on its Security level but also from how securely are configured the others hosted on that
same IP. In fact, if an attacker compromises one of them and gains access to the underlying
server, then he can easily reach every other virtual host.
theHarves ter is also able to acquire names of persons related to the target domain by
crawling social networks such as LinkedIn; this can be done by simply using as data source
the argument “LinkedIn”
theHarvester is a valuable tool for OSINT which allows to quickly discover a good amount of
data, especially email addresses. Remember that you need to verify information: for example,
it could be that an employer is not working anymore on a certain company, but his email
address is still present on the web and so it will be returned in the results.
Conclusion:-Hence, we have used OSINT tools Theharvester to gather information like
emails, subdomains, hosts, employee names, open ports and banners from different public
sources like search engines, PGP key server