OSINT

OSINT
OSINT is Open-Source INTelligence

Information in the public domain or accessible

from public sources.

www.cybexer.com 2
OSINT
OSINT resources can be divided into the following
categories

• Internet resources which encompass most OSINT

resources. It includes discussion forums, blogs, social
media sites, all types of digital files (files, videos,
audio files, webpages, source code) available online
along with their associated metadata, IP addresses
and technical information

www.cybexer.com 3
OSINT
OSINT resources can be divided into the following
categories

• Traditional media sources, such as TV, Radio

broadcast, newspapers, magazines, or books
• Scientific and academic publications, such as
research papers, review articles, technical white
papers, dissertations, academic journals, theses,
books, or grey literature

www.cybexer.com 4
OSINT
OSINT resources can be divided into the following
categories

• Enterprise papers which include business profiles,

annual reports, conference proceedings, tax records,
and even immigration records
• Geospatial information, such as online maps,
commercial imagery satellites, and data generated
from GPS capable devices

www.cybexer.com 5
OSINT
There are 3 types of OSINT

• Passive Collection

• Semi-passive Collection

• Active Collection

www.cybexer.com 6
OSINT
Passive Collection

This is the most used type when collecting OSINT

intelligence, by default most OSINT gathering methods
should use passive collection because the main aim of
OSINT gathering is to collect information about the
target via publicly available resources.

www.cybexer.com 7
OSINT
Semi-passive Collection

Sends internet traffic to target servers in order to acquire

general information about them.
This traffic should resemble typical internet traffic to avoid
drawing any attention to your reconnaissance activities.
In this way, you are not implementing in-depth investigation
of the target’s online resources, but only investigating
lightly without launching any alarm within the group you
are investigating.

www.cybexer.com 8
OSINT
Active Collection

You interact directly with the system to gather intelligence about

it, but the target can become aware of the reconnaissance
process since the person/entity collecting information will use
advanced techniques to harvest technical data about the target
IT infrastructure such as accessing open ports, scanning
vulnerabilities (unpatched Windows systems), scanning web
server applications, and more.
This traffic will look like suspicious behavior and will more than
likely leave traces on the target’s intrusion detection system (IDS)
or intrusion prevention system (IPS).

www.cybexer.com 9
OSINT
The Wayback Machine is a digital archive of the World
Wide Web that stores snapshots of websites at various
points in time over the course of history.
Attackers may use it to gather compromising
intelligence about an organization through earlier
versions of its websites.

• https://web.archive.org/

www.cybexer.com 10
OSINT
Robtex uses various sources to gather public
information about IP numbers, domain names, host
names, Autonomous systems, routes etc. It then
indexes the data in a big database and provide free
access to the data

• https://www.robtex.com/

www.cybexer.com 11
OSINT
Shodan is a search engine that lets the user find
specific types of computers (webcams, routers, servers,
etc.) connected to the internet using a variety of filters
• https://www.shodan.io/

www.cybexer.com 12
OSINT
Shodan also uses search filter, here're some of them:
Search for results in given country(s)
• country:DE,CH,FR

www.cybexer.com 13
OSINT
Shodan also uses search filter, here're some of them:
Search for SSH on port 22 or 3333
• ssh port:22,3333

www.cybexer.com 14
OSINT
Shodan also uses search filter, here're some of them:
Search for name of software identified in banner
• product:Apache

www.cybexer.com 15
OSINT
Shodan also uses search filter, here're some of them:
Search for name of software identified in banner
• product:Apache

www.cybexer.com 16
OSINT
Shodan also uses search filter, here're some of them:
Search for Siemens Industrial Automation
• "Siemens, SIMATIC" port:161

www.cybexer.com 17
OSINT
Shodan also uses search filter, here're some of them:
Search Unprotected VNC services
• "authentication disabled" "RFB 003.008"

www.cybexer.com 18
OSINT
Full list of filters available on following link
• https://www.shodan.io/search/filters

www.cybexer.com 19
OSINT
Pastebin - is a website where you can store any text
online for easy sharing.
The website is mainly used by programmers to store
pieces of sources code or configuration information,
but there're also people who submit leaked password,
results of network scans etc.
• https://pastebin.com/

www.cybexer.com 20
OSINT
crt.sh - certificate transparency project, which uses
historical SSL certificate information.

• https://crt.sh/

www.cybexer.com 21
OSINT
Google has also certificate transparency site, which
uses historical SSL certificate information.

• https://transparencyreport.google.com/https/certif
icates?hl=en

www.cybexer.com 22
OSINT - google
Not a secret, that Google's search engine is very
powerful. It has billions of data in its search databases.

To get the best matching answers, you must know how

to write correct search query.

Google uses 'operator:search_term' approach.

www.cybexer.com 23
OSINT - google
To search for exact phrases, use double quotes
• "Hello world"

To exclude specific words from search results, use '-'

sign followed by excluded word
• linux distributions -ubuntu

www.cybexer.com 24
OSINT - google
Search for pages, which has searched text in its titles
• intitle:"login page"

Search for specific filenames in URL

• inurl:"login.php"

www.cybexer.com 25
OSINT - google
Limit search results to specific site
• site:cnn.com

List pages, which has links to search query

• link:cybexer.com

www.cybexer.com 26
OSINT - google
Search for specified file extension
• filetype:txt password

List pages, which has links to search query

• link:cybexer.com

www.cybexer.com 27
OSINT - google
Several examples of 'hacking' search queries

• allintext:username filetype:log site:id

• intitle:"index of" filetype:txt

• inurl:top.htm inurl:currenttime

• intitle:"live view" intitle:axis

www.cybexer.com 28
OSINT - whois
'Whois' is a widely used Internet record listing that
identifies who owns a domain/IP and how to get in
contact with them.

Whois - one of the main tools that IT people use on

Unix, Linux and Mac systems to look up the registration
information about a domain.

www.cybexer.com 29
OSINT - whois
To get 'whois' information for site, type in Linux
terminal 'whois' followed by domain name
• whois github.com

www.cybexer.com 30
OSINT - whois
To get detailed information about network owner of
the site, first you need get IP address from hostname.
In terminal windows type command 'host', 'nslookup'
or 'dig' followed by hostname.

All three commands will run DNS query against DNS

servers and give back IP address(s) of hostname.
Output will be different.

www.cybexer.com 31
OSINT - whois
• host github.com

• nslookup github.com

www.cybexer.com 32
OSINT - whois
• dig github.com

www.cybexer.com 33
OSINT - whois
To get authoritative DNS server(s) for specific domain
• dig github.com NS

To get mail server(s) for specific domain

• dig github.com MX

www.cybexer.com 34
OSINT - whois
Once you have identified IP address of hostname, run
again 'whois' command, but against IP address, not
hostname
• whois 13.234.210.38

www.cybexer.com 35
OSINT - whois
'Whois' information can be also viewed from several
web resources. Here is an example of such sites:

https://viewdns.info/whois/
https://ping.eu/ns-whois/
https://whois.domaintools.com/

Enter IP address of the site and view the 'whois'

information.

www.cybexer.com 36
OSINT - OSRFramework
Open Sources Research Framework - OSRFramework
Software has different applications related to username
checking, DNS lookups, information leaks research,
deep web search, regular expressions extraction and
many others.

www.cybexer.com 37
OSINT - OSRFramework
'OSRFramework' present in Kali's repository. Let's
install it, type in terminal window following command
• apt-get install osrframework

www.cybexer.com 38
OSINT - OSRFramework
You can also install 'OSRFramework' from Python
repository. To install it, type in terminal window
following command
• apt-get install python3-pip

www.cybexer.com 39
OSINT - OSRFramework
Let's install it using Python software installer 'pip3'
• pip3 install osrframework

www.cybexer.com 40
OSINT - OSRFramework
'OSRFramework' install following components
mailfy - find information about emails taken as a
reference nickname or email list
searchfy - find profiles linked to a full name
usufy - identify social media profiles using a given
nickname
checkfy - guesses possible emails based on a list of
candidate nicknames and a pattern

www.cybexer.com 41
OSINT - OSRFramework
Let's search for username profiles in different social
media networks. Since search can take very long time,
press 'Ctrl+c' after 10-15 seconds
• usufy -n 'billgates'

www.cybexer.com 42
OSINT - OSRFramework
Output of username 'billgates' search

www.cybexer.com 43
OSINT - OSRFramework
You can supply several usernames to search for
• usufy -n billgates jeffbezos

www.cybexer.com 44
OSINT - OSRFramework
By default, all social media platforms are searched. To
limit search to specific platform names, use '-p' option
followed by platform name
• usufy -n billgates jeffbezos -p facebook twitter

www.cybexer.com 45
OSINT - OSRFramework
To check for registered domains in different zones, use
command 'domainfy'
• domainfy -n google -t all

www.cybexer.com 46
OSINT - OSRFramework
To search for profiles by full names, use command
'searchfy'
• searchfy -q "Bill Gates"

www.cybexer.com 47
OSINT - UserRecon
'UserRecon' allows to find usernames across over 75
social networks. This program is not in the Kali
repository, so we need to clone it from GitHub
• cd /opt/
• git clone https://github.com/wishihab/userrecon

www.cybexer.com 48
OSINT - UserRecon
Let's move to newly created folder and make
'userrecon.sh' file executable
• cd userrecon/
• chmod a+x userrecon.sh

• ls -la userrecon.sh

www.cybexer.com 49
OSINT - UserRecon
Now we can start search for usernames from social
media networks. Type in following command and then
enter desired username at 'Input Username' prompt
• ./userrecon.sh

www.cybexer.com 50
OSINT - UserRecon
The results of findings will be appearing line by line on
your screen

www.cybexer.com 51
OSINT - DNS
DNS enumeration is the process of locating all the DNS
servers and their corresponding records for an
organization.

A company may have both internal and external DNS

servers that can yield information such as usernames,
computer names, and IP addresses of potential target
systems.

www.cybexer.com 52
OSINT - DNS
'fierce' - command line DNS enumeration tool. Default
dictionary contains 1594 records.
• fierce --domain csirt.crp

www.cybexer.com 53
OSINT - DNS
To use custom wordlist, append option '--subdomain-
file' followed by path to the file
• fierce --domain csirt.crp --subdomain-file ./dns.txt

www.cybexer.com 54
OSINT - DNS
'DNSenum' - a script, which enumerates sub-domains
records for specified domain with use of dictionary file.
Let's install 'DNSenum'. First, we move to '/opt' folder
and clone program from GitHub
• cd /opt
• git clone
https://github.com/theMiddleBlue/DNSenum

www.cybexer.com 55
OSINT - DNS
'DNSenum' already comes with 3 different dictionary
files, which can be used for DNS enumeration
• cd DNSenum/
• ls -la wordlist/

www.cybexer.com 56
OSINT - DNS
Now let's run DNS sub-domain enumeration
• ./dnsenum.sh -d csirt.crp -f ./wordlist/subdomains-
top1mil-5000.txt

www.cybexer.com 57
OSINT - DNS
Another type of DNS enumeration is reverse DNS
enumeration. This approach is resolving all IP address
from specific network range

• for i in {1..255} ; do host 13.234.210.$i | grep -v

NXDOMAIN ; done

www.cybexer.com 58
OSINT - wordlists
Very important for all types of enumerations (DNS,
users, HTTP, SMTP etc.) is to use proper dictionary file.
There're a lot of resources with different wordlists.
One good example is 'SecLists' on GitHub

• https://github.com/danielmiessler/SecLists

www.cybexer.com 59
OSINT - wordlists
For installation from Kali Linux repository, run following
command
• apt-get -y install seclists

www.cybexer.com 60
OSINT - wordlists
All 'seclists' dictionary files will be installed in
'/usr/share/seclists' folder
• ls -la /usr/share/seclists

www.cybexer.com 61
OSINT - metadata
Metadata is, information about other data.

Many files contain extra or even hidden data other than

the visual data you see at first glance. E-books,
photographs, movies, music and even documents can
contain data that you don’t see at first glance.

www.cybexer.com 62
OSINT - metadata
Metadata From Photos

Photos contain 'exif' data that can give you useful

information about the picture. Information such as
shutter speed and focal length are stored inside an
image. Likewise, you can find out where the photo was
taken by looking at the location information.

www.cybexer.com 63
OSINT - metadata
Video Metadata

Similar to photos, videos contain metadata info about

the location where the video was shot. Likewise,
container formats like AVI and MP4 contain meta
information about codecs, video and audio streams and
more.

www.cybexer.com 64
OSINT - metadata
Hidden Data In Documents

Documents can contain metadata too. They include

information such as file size and date of creation, but
also information about the author of a document and
the software used to create it.

www.cybexer.com 65
OSINT - exiftool
How to view 'exif' data in images?
Kali Linux has command line tool called 'exiftool'.
Let's install it from repository
• apt-get -y install exiftool

www.cybexer.com 66
OSINT - exiftool
Let's download sample image file
• wget http://files.csirt.crp/opsecfail.jpg -O
opsecfail.jpg

www.cybexer.com 67
OSINT - exiftool
To view 'exif' information from the image, run 'exiftool'
command followed by image name
• exiftool opsecfail.jpg

www.cybexer.com 68
OSINT - exiftool
'exiftool' shows a lot of information embedded into
image. Here's information about the device/camera,
which was used to take the photo

On some images, you may see geo location data

www.cybexer.com 69
OSINT - exiftool
By default, 'exiftool' show GPS coordinates in human
readable format, Google maps do not understand this
format.
Let's print Google-maps friendly GPS data from photo
• exiftool -gpslatitude -gpslongitude -n opsecfail.jpg

www.cybexer.com 70
OSINT - exiftool
Now, in your browser open Google maps
'maps.google.com' and enter retrieved GPS coordinates

www.cybexer.com 71
OSINT - exiftool
The '-c' option allows you to set the format of the
displayed coordinates. For example, if you have too
many (or too few) digits after the decimal point, then
their quantity can be changed with this option.
• exiftool -gpslatitude -gpslongitude -c '%+.6f'
opsecfail.jpg

www.cybexer.com 72
OSINT - exiftool
'exiftool' is also available for Windows and MAC
operating systems. Visit site 'https://exiftool.org/' and
download the "Windows Executable" archive or
'MacOS Package'.

www.cybexer.com 73
OSINT - exiftool
With 'exiftool' you can remove all GPS related data
from image. Let's first check all GPS metadata.
• exiftool -G opsecfail.jpg | grep -i gps

www.cybexer.com 74
OSINT - exiftool
With 'exiftool' you can remove all GPS related data
from image. Let's first check all GPS metadata.
• exiftool -gps:all= opsecfail.jpg -o
opsecfail_nogps.jpg

www.cybexer.com 75
OSINT - exiftool
Now, let's check GPS metadata from new image
• exiftool -G opsecfail_nogps.jpg | grep -i gps

As you can see, all GPS related data is removed from

new image.

www.cybexer.com 76
OSINT - exiftool
'exiftool' is very powerful program for displaying and
manipulating metadata of images. In additions to
showing meta information, it allows to modify existing
data. Let's check original's image metadata
• exiftool -G opsecfail.jpg | grep -i -E '(Make
)|(Model)|(Software)|(Device)|(Lens)'

www.cybexer.com 77
OSINT - exiftool
Now, lets modify camera's data and save to new file
• exiftool -Make='Samsung' -Model='Galaxy s40
Ultra' -Software='G988BXXU5CTKG' -
lensmake='Samsung Lens' -lensmodel='Samsung
Lens X40s' -makernotes:all= opsecfail.jpg -o
opsecfail_new_camera.jpg

www.cybexer.com 78
OSINT - exiftool
Now, let's check new file metadata
• exiftool -G opsecfail_new_camera.jpg | grep -i -E
'(Make )|(Model)|(Software)|(Device)|(Lens)'

www.cybexer.com 79
OSINT - exiftool
With 'exiftool' you can modify/spoof GPS coordinates in
the image. Let's change GPS location in image metadata
to 'Area 51'. According to Google maps, location of
'Area 51' is - 37.2431° N, 115.7930° W.
• exiftool -GPSLatitude='37.2431' -
GPSLongitude='115.7930' -GPSLatitudeRef='N' -
GPSLongitudeRef='W' opsecfail.jpg -o
opsecfail_new_gps.jpg

www.cybexer.com 80
OSINT - exiftool
Let's print Google-maps friendly GPS data from photo
with new GPS coordinates
• exiftool -gpslatitude -gpslongitude -n
opsecfail_new_gps.jpg

www.cybexer.com 81
OSINT - exiftool
Now, in your browser open Google maps
'maps.google.com' and enter retrieved GPS coordinates
from new image

www.cybexer.com 82
OSINT - exiftool
With 'exiftool' you can modify any meta information in the
image. Here's example of changing additional data
• exiftool -DateTime='2038:07:02 09:41:12' -
DateTimeOriginal='2038:07:02 09:41:12' -
DateTimeDigitized='2038:07:02 09:41:12' -
ModifyDate='2038:07:02 09:41:12' -
CreateDate='2038:07:02 09:41:12' -
GPSTimeStamp='09:41:12' -GPSDateStamp='2038:07:02'
-UserComment='Holidays' -LocationName='Bali' -Sub-
location='Beach' -City='Denpasar' -Province-State='BALI'
-Country-PrimaryLocationName='Indonesia'
opsecfail.jpg -o opsecfail_new_data.jpg

www.cybexer.com 83
OSINT - exiftool
It is often needed, to remove all meta information from
the image. Let's do so
• exiftool -all= opsecfail.jpg -o opsecfail_nometa.jpg

www.cybexer.com 84
OSINT - exiftool
Now let's view meta information in the new image
• exiftool opsecfail_nometa.jpg

www.cybexer.com 85
OSINT - mat2
Another software, which allows to remove meta
information from different files (jpg, docx, xls, pdf etc.) is
MAT2 - Metadata Anonymisation Toolkit v2.

MAT2 only removes metadata from your files, it does not

anonymise their content, nor can it handle watermarking,
steganography, or any too custom metadata field/system.
If you really want to be anonymous, use file formats that do
not contain any metadata, or better: use plain-text.

www.cybexer.com 86
OSINT - mat2
Let's install MAT2 on your Kali machine
• apt-get -y install mat2

www.cybexer.com 87
 OSINT - mat2
To view meta information from image, run 'mat2'
program, followed by '-s' option and file name at the
end
• mat2 -s opsecfail.jpg

www.cybexer.com 88
 OSINT - mat2
To remove all meta information from image, run 'mat2',
followed by image name. New file
'opsecfail.cleaned.jpg' will be created.
• mat2 opsecfail.jpg
Now check meta information in new file
• mat2 -s opsecfail.cleaned.jpg

www.cybexer.com 89
 OSINT - mat2
To remove all metadata without creating backup files
(will overwrite existing files), use '--inplace' option.
For cleaning meta data from all files in current folder,
use '*.jpg'
• mat2 --inplace *.jpg

Same approach is used for other files - *.pptx, *.pdf,

*.docx etc.

www.cybexer.com 90