Intrusion Detection

Dr. Farhana Khan

1
Links
• https://www.coursera.org/lecture/detecting-cyber-attacks/intrusion-detection-systems-UeDqJ
• https://www.coursera.org/learn/detecting-cyber-attacks/exam/pzXev/week-2-intrusion-detection-and-
prevention
• https://www.cse.wustl.edu/~jain/cse571-07/ftp/l_23ids/l_23ids.html
• https://www.coursera.org/learn/detecting-cyber-attacks/lecture/du5Bv/detection-methods
• http://people.duke.edu/~tkb13/courses/ece590-sec-2018fa/
• https://www.youtube.com/watch?v=izA04yceVaQ
• http://sharif.edu/~kharrazi/courses/40817-941/reading/Debar00a.pdf
q http://www.cse.psu.edu/~trj1/cse497b-s07/slides/cse497b-lecture-23-ids.pdf\S. Kumar, "Survey of Current Network Intrusion
Detection Techniques," http://www.cse.wustl.edu/~jain/cse571-07/p_nid.html
q NIST, Guide to Intrusion Detection and Prevention Systems (IDPS), Special Publication SP 800-94, Sep 2006,
http://csrc.nist.gove/publications/PubsSPs.html
q Open Directory Projects IDS Page, http://www.dmos.org/Computers/Security/Intrusion_Detection_Syste ms/ Has a list of 25
open source and 96 commercial tools, 79 security scanners, 25 security scanner services
q Architectural Issues of Intrusion Detection Infrastructure in Large Enterprises,
http://www.softpanorama.org/Security/intrusion_detection.shtml
q Gert DeLaet, Gert X. Schauwers, "Network Security Fundamentals," Cisco Press, Sep 2004, 400 pp., ISBN:
1587051672.
q Richard Bejtlich, "The Tao Of Network Security Monitoring : Beyond Intrusion Detection," Addison-Wesley, Jul 2004, 798 pp.,
ISBN:321246772.
q SANS Institute, "Intrusion Detection FAQ," http://www.sans.org/resources/idfaq/index.php?portal=46489b3fa83
24804cb8de1e1ff4ae9e7
q https://engineering.purdue.edu/kak/compsec/ß
q https://www.khanacademy.org/economics-finance-domain/core-finance/money-and-banking/bitcoin/v/bitcoin-digital-signatures

q https://www.markmonitor.com/download/webinar/2015/MarkMonitor-Webinar-150715-DeepWebDarknetBitcoin.pdf

2
Intrusion Detection and Prevention
• Intrusion
– Actions aimed at compromising the security of a target network
(confidentiality, integrity, availability of resources)

• Intrusion detection
– The identification of possible intrusion through intrusion
signatures and network activity analysis
– IDS: Intrusion Detection Systems

• Intrusion prevention
– The process of both detecting intrusion activities and managing
automatic responsive actions throughout the network
– IPS: Intrusion Prevention Systems
– IDPS: Intrusion Detection and Prevention Systems

3
Intrusion Detection Systems:

• IDS are automated systems that detect suspicious activity

• What can be detected:

– Attempted and successful misuse, both external and internal
agents
– Malware: Trojan programs, viruses and worms
– DOS (Denial Of Service) attacks

4
l Host-based IDS (HIDS)
l Monitors the characteristics of a single host for
suspicious activity
l monitor changes to host’s OS files and traffic
sent to the host
Comprises three logical components:
l Network-based IDS (NIDS)
• detect intrusions on one or more
network segments • Sensors - input to a sensor
l Monitors network traffic and analyzes network, includes network packets, log
transport, and application protocols to identify
suspicious activity
files, and system call traces.
• Analyzers - Analyzers receive
l Distributed or hybrid IDS input from one or more sensors
l Combines information from a number of sensors, or from other analyzers.
often both host and network based, in a central determine if intrusion has occurred
analyzer that is able to better identify and respond
to intrusion activity • User interface - view output or
control system behavior
5
 Network IDS Deployment
External Router /
Firewall
Internet
Internal
Router /
Firewall DB Production
Work
Server Server
DMZ Network Station

Internal Networks
DNS Web Email NIDS
Server Server Server

NIDS

6
Host-Based IDSs
• Using OS auditing mechanisms
• E.G., BSM on Solaris: logs all direct or indirect
events generated by a user
• trace for system calls made by a program
• Monitoring user activities
• e.g., Analyze shell commands
• Monitoring execution of system programs
• e.g., Analyze system calls made by sendmail

7
Basic Audit Modules (Hosts)
• eventLog - Uses the windows Event Logging system to track entries into all
three of the windows event logs: System, Security, Application
• netstat - Uses the information from the program netstat to provide information
about network usage on the machine
• health - Runs the program health to give current information about the system
(CPU usage, mem usage, swap usage)

8
HIPS
• Many industry observers see the enterprise endpoint, including desktop and laptop
systems, as now the main target for hackers and criminals

• Areas for which a HIPS typically offers desktop protection:

• System calls
• File system access
• System registry settings
• Host input/output

• Examples of the types of malicious behavior addressed by a HIDS

include:
• Modification of system resources
• Privilege-escalation exploits
• Buffer-overflow exploits
• Access to e-mail contact list
• Directory traversal
9
Ongoing activity to be provided as input to Analysis

• System call traces:

• Record of the sequence of systems calls by processes on a system, is widely
acknowledged. due to the extensive use of DLLs that unclear which processes
use specific system calls.

• Audit (log file) records:

accounting software that collects information on user activity. The advantage of using
this information is that no additional collection software is needed.
• The disadvantages are that
– the audit records may not contain the needed information or may not contain it in a convenient form,
– intruders may attempt to manipulate these records to hide their actions.

10
 Table 8.2 Linux System Calls and Windows DLLs Monitored

(a) Ubuntu Linux System Calls

accept, access, acct, adjtime, aiocancel, aioread, aiowait, aiowrite, alarm, async_daemon,
auditsys, bind, chdir, chmod, chown, chroot, close, connect, creat, dup, dup2, execv, execve,
exit, exportfs, fchdir, fchmod, fchown, fchroot, fcntl, flock, fork, fpathconf, fstat, fstat,
fstatfs, fsync, ftime, ftruncate, getdents, getdirentries, getdomainname, getdopt, getdtablesize,

Linux
getfh, getgid, getgroups, gethostid, gethostname, getitimer, getmsg, getpagesize,
getpeername, getpgrp, getpid, getpriority, getrlimit, getrusage, getsockname, getsockopt,
gettimeofday, getuid, gtty, ioctl, kill, killpg, link, listen, lseek, lstat, madvise, mctl, mincore,
mkdir, mknod, mmap, mount, mount, mprotect, mpxchan, msgsys, msync, munmap,
nfs_mount, nfssvc, nice, open, pathconf, pause, pcfs_mount, phys, pipe, poll, profil, ptrace,
System
putmsg, quota, quotactl, read, readlink, readv, reboot, recv, recvfrom, recvmsg, rename,
resuba, rfssys, rmdir, sbreak, sbrk, select, semsys, send, sendmsg, sendto, setdomainname,
Calls and
setdopt, setgid, setgroups, sethostid, sethostname, setitimer, setpgid, setpgrp, setpgrp,
setpriority, setquota, setregid, setreuid, setrlimit, setsid, setsockopt, settimeofday, setuid, Windows
shmsys, shutdown, sigblock, sigpause, sigpending, sigsetmask, sigstack, sigsys, sigvec,
socket, socketaddr, socketpair, sstk, stat, stat, statfs, stime, stty, swapon, symlink, sync, DLLs
Monitored
sysconf, time, times, truncate, umask, umount, uname, unlink, unmount, ustat, utime, utimes,
vadvise, vfork, vhangup, vlimit, vpixsys, vread, vtimes, vtrace, vwrite, wait, wait3, wait4,
write, writev

(b) Key Windows DLLs and Executables

comctl32
kernel32
msvcpp
msvcrt
mswsock
ntdll
ntoskrnl
(Table can be found on page
user32
280 in the textbook)
ws2_32
11
Ongoing activity to be provided as input to Analysis

• File integrity checksums:

– To periodically scan critical files for changes from the desired baseline, by
comparing a current cryptographic checksums for these files, with a record of
known good values.
– Disadvantages include the need to generate and protect the checksums using
known good files, and the difficulty monitoring changing files. Tripwire is a well-
known system using this approach.

Registry access:
– An access to the registry, given the amount of information and access to it used
by programs on these systems. However this source is very Windows specific,
and has recorded limited success.

12
Signature or Heuristic HIDS
• The alternative of signature or heuristic based HIDS is widely used,
particularly as seen in anti-virus (A/V), more correctly viewed as anti-
malware, products.
• These are very commonly used on Windows systems, and also
incorporated into mail and web application proxies on firewalls and in
network based IDSs.
• They use either a database of file signatures, which are patterns of data
found in known malicious software, or heuristic rules that characterize
known malicious behavior.
• These products are quite efficient at detecting known malware, however
they are not capable of detecting zero-day attacks that do not correspond to
the known signatures or heuristic rules.
• They are widely used, particularly on Windows systems, which continue to
be targeted by intruders

13
Network-based IDS (NIDS)
• A network-based IDS (NIDS) monitors traffic at selected points on a network or
interconnected set of networks.
• The NIDS examines the traffic packet by packet in real time, or close to real time, to
attempt to detect intrusion patterns. The NIDS may examine network-, transport-,
and/or application-level protocol activity.
• NIDS examines packet traffic directed toward potentially vulnerable computer systems
on a network. A host-based system examines user and software activity on a host.
• NIDS are typically included in the perimeter security infrastructure of an organization,
either incorporated in, or associated with, the firewall. They typically focus on
monitoring for external intrusion attempts, by analyzing both traffic pat- terns and
traffic content for malicious activity.

14
Classes of Intruders –Cyber Criminals

l Cyber criminals: Are either individuals or members of an

organized crime group with a goal of financial reward.
l To achieve this, their activities may include identity theft,
theft of financial credentials, corporate espionage, data
theft, or data ransoming.

l They meet in underground forums with names like

DarkMarket.org and theftservices.com to trade tips and data
and coordinate attacks.
l A darknet market or cryptomarket is a commercial website
on the web that operates via darknets such as Tor or I2P.
They function primarily as black markets, selling or brokering
transactions involving, cyber-arms, stolen credit card
details, forged documents
https://www.youtube.com/watch?v=6czcc1gZ7Ak 15
https://www.youtube.com/watch?v=wlP1JrfvUo0
May 2015 Tor 2

Privacy on Public Networks

• Internet is designed as a public network
• Wi-Fi access points, network routers see all traffic that passes
through them
• Routing information is public
• IP packet headers identify source and destination
• Even a passive observer can easily figure out who is talking to
whom
• Encryption does not hide identities
• Encryption hides payload, but not routing information
• Even IP-level encryption (tunnel-mode IPsec/ESP) reveals IP
addresses of IPsec gateways

16
 Web is not private by default
• The web is not private by default. Websites can use
cookies to track user actions on their site and even
across other sites. Browsers can track the browsing history
of a user, their search queries, and even their form
• Use cases for cookies

o There are many ways a website can use cookies to personalize an experience.
A search engine can use them to remember how many results a user prefers
seeing per page.
o A news site can use them to recommend headlines that are similar to the
articles you've already read.
o All sorts of websites can use cookies to track analytics, like how long you spent
on a page and which buttons you clicked.
o Any website with a log-in uses a cookie to keep you logged in on every page
of the site. When you log out of that site, it clears the cookie and doesn't set it
again until you login again.

17
 Web is not private by default
• A 2016 study found that the average website loaded in about
20 third-party cookies, and the average news site loaded
double that amount.
• What are they doing with all those cookies? Most third-party
cookies are used for advertising. Imagine a user that visits a
food blog with a recipe for gluten-free cookies. That blog
includes a Facebook ad with a cookie. The user then visits
facebook.com and notices a sudden uptick in ads about
gluten-free products. That's not a coincidence, that's cookies!
• Make an IP-based guess. Websites can't see a user's GPS or
WiFi network information. However, websites do see the IP
addresses of the HTTP request coming to their server. There are
databases that attempt to identify the approximate location
of IP addresses, and websites can lookup an IP address in
those databases.

18
 Web is not private by default
• Request access. The website can request the geolocation from the
browser, and the browser will ask the user for permission to share it.
The browser can typically make a good approximation of a user's
location using GPS (when available), WiFi/mobile network positioning
(most commonly), or as a last resort
• Ultimately, our ISP can see every HTTP request that goes over their
network. We can hide the contents of those requests by using HTTPS-
secured websites, but we can't hide the destination of the requests.
ISPs can use that information to find customers that are engaged in
illegal activities, like downloading movies illegally.
• Privacy-savvy users can obscure their activity by using a Virtual
Private Network (VPN), but most do not, since VPNs cost money and
slow down the online experience.based location.

19
 Understanding the Internet
Landscape

Surface Web
Searchable with standard
search engines

Deep Web
Un-indexed websites

Dark Web / DarkNet

Dark Web: web content that exists on
the DarkNet

DarkNet: Network that can only be

accessed with specific software,
configurations, or authorization

The Deep Web is hundreds of times larger than the ‘Surface Web'
20
What's in the Deep Web?
§ 96% of the content exists in the deep web
• 7500+ TB of content

§ Types of content that is in the deep web:

• Dynamic content
• Unlinked content
• Private Web
• Contextual Web
• Limited access content
• Scripted content
• Non-HTML/text content
• Software
• Web archives
• Un-indexed websites
• P2P networks 21
Deep Web Content of Concern

§ Any un-indexed web page

• Selling counterfeit or grey market goods
• Collecting user credentials
• Disseminating malware
• Engaged in false association
• Conducting consumer scam

§ P2P sites where piracy taking place

§ Marketplaces where counterfeit, grey market or unauthorized

goods are sold

§ Social media where impersonation is occurring

22
How Are Consumers Directed to the
Deep Web?

Social
Websites
Media

Email
Mobile
Apps

Paid
Search

23
So What Is the Darknet?

§ Within the Darknet both web surfers and website publishers are
entirely anonymous

§ Anonymity is usually achieved using Tor

§ There are a number of marketplaces (the online black market)

• Abraxas
• Agora Marketplace
• Middle Earth Marketplace
• Nucleus
• Silk Road 1, 2 and 3

24
What is Tor?

§ Acronym for The Onion Router

§ Free software for enabling anonymous communication

§ Originally developed on behalf of the U.S. intelligence

community

§ Today it is used by criminal enterprises, hacktivists, and law

enforcement agencies
• Users can remain anonymous
• Activities can remain untraceable
• Resources can remain hidden
25
How is Tor Accessed?

26
The Tor Browser

27
How Does Tor Work?

torproject.org

28
How Does Tor Work?

torproject.org

29
How Does Tor Work?

torproject.org

30
 May 2015 Tor 14

How Tor Works? --- Onion Routing

Alice Bob

M
√
M OR2
M C2 C3
M

OR1 OR3
C1 C2 C3 Port

• A circuit is built incrementally one hop by one hop

• Onion-like encryption
• Alice negotiates an AES key with each router
• Messages are divided into equal sized cells
• Each router knows only its predecessor and successor
• Only the Exit router (OR3) can see the message, however it does not
know where the message is from
31
 May 2015 Tor 15

How Onion Routing Works

1 2
u d
5 3
User u running client Internet destination d
4
Routers running servers

32 9
May 2015 Tor 16

How Onion Routing Works

1 2
u d
5 3
4
1. u creates l-hop circuit through routers

33 9
May 2015 Tor 17

How Onion Routing Works

1 2
u d
5 3
4
1. u creates l-hop circuit through routers

34 9
May 2015 Tor 18

How Onion Routing Works

1 2
u d
5 3
4
1. u creates l-hop circuit through routers

35 9
May 2015 Tor 19

How Onion Routing Works

1 2
u d
5 3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d

36 9
May 2015 Tor 20

How{{{m}Onion
}}
Routing Works
1 2
3 4 1

u d
5 3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged

37 9
May 2015 Tor 21

How Onion Routing Works

1 2
u d
5 3
{{m}3}4
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged

38 9
May 2015 Tor 22

How Onion Routing Works

1 2
u d
5 3
{m}3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged

39 9
May 2015 Tor 23

How Onion Routing Works

1 2
m
u d
5 3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged

40 9
May 2015 Tor 24

How Onion Routing Works

1 2
u m’ d
5 3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged

41 9
May 2015 Tor 25

How Onion Routing Works

1 2
u d
5 3
{m’}3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged

42 9
May 2015 Tor 26

How Onion Routing Works

1 2
{{m’}3}4
u d
5 3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged

43 9
May 2015 Tor 27

How Onion Routing Works

{{{m’}3}4}1 1 2
u d
5 3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged

44 9
May 2015 Tor 28

How Onion Routing Works

1 2
u d
5 3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged.
4. Stream is closed.
45
May 2015 Tor 29

How Onion Routing Works

1 2
u d
5 3
4
1. u creates l-hop circuit through routers
2. u opens a stream in the circuit to d
3. Data are exchanged.
4. Stream is closed.
5. Circuit is changed every few minutes. 46 9
Accessing Underground
Marketplaces

47
What’s Sold on a Typical Tor Marketplace?

Drugs

Information

eBooks

Erotica

Counterfeits

Services

Data

Electronics

Misc

Software

Hacking

Fraud

Security

0 1000 2000 3000 4000 5000 6000 7000

Source: Abraxas marketplace

48
 Underground Marketplaces on
Tor

§ Typically require registration

§ Some require invitation to join

• Link on another site – not difficult to obtain

§ Some marketplaces are sophisticated

• Seller ratings
• Seller profiles
• Order history
• Online discussions groups

49
Typical Tor Marketplace

50
Typical Tor Marketplace - Pirated Software

51
Typical Tor Marketplace - Luxury
Goods

52
Typical Tor Marketplace - Pharmaceuticals

53
Typical Tor Marketplace -
Counterfeits

54
Typical Tor Marketplace – Account
Info

55
So What Role Does Bitcoin Play?

§ Sites utilize Bitcoin to conduct transactions

§ Other types of cryptocurrency are sometimes accepted

• Dash (formerly known as Darkcoin)

56
 How Does Bitcoin Work?

§ Anonymous payment system

• Utilizes peer-to-peer technology to operate with no central authority
• Relies upon “Miners” rewarded with Bitcoin to conduct network
transactions
• Transactions are conducted electronically using URIs which can be
imbedded in QR codes for use with mobile devices

§ Bitcoin can be bought and sold through online exchanges

57
Classes of Intruders –Activists

l Are either individuals, usually working as insiders, or members

of a larger group of outsider attackers, who are motivated by
social or political causes
l Also know as hacktivists
l Skill level is often quite low
l Aim of their attacks is often to promote and publicize their
cause typically through:
l Website defacement
l Denial of service attacks
l Theft and distribution of data that results in negative
publicity or compromise of their targets

58
 Classes of Intruders –
State-Sponsored Organizations

l Groups of hackers sponsored by governments to

conduct spying or disruption activities
l Also known as Advanced Persistent Threats (APTs) due to
the concealed nature
l Widespread nature and scope of these
activities by a wide range of countries
from China to the USA, UK, and their
intelligence allies

59
Classes of Intruders –Others

• Include classic hackers or crackers who are

motivated by technical challenge or by peer-group
esteem and reputation

• Given the wide availability of attack toolkits, there is

a pool of “hobby hackers” using them to explore
system and network security

60
Efficiency of intrusion-detection
systems
• Accuracy.
o Accuracy deals with the proper detection of attacks and the absence of
false alarms.

• Performance.
o The performance of an intrusion-detection system is the rate at which
audit events are processed.
o If the performance of the intrusion-detection system is poor, then real-
time detection is not possible.

• Completeness.
o Completeness is the property of an intrusion-detection system to detect all
attacks.
o Incompleteness occurs when the intrusion-detection system fails to detect
an attack. This measure is much more difficult to evaluate than the others
because it is impossible to have a global knowledge about attacks or
abuses of privileges.

61
 Efficiency of intrusion-detection
systems
• Fault tolerance
o An intrusion-detection system should itself be resistant to attacks,
especially denial-of- service-type attacks, and should be designed with
this goal in mind.

• Timeliness
o An intrusion-detection system has to perform and propagate its analysis
as quickly as possible to enable the security officer to react before much
damage has been done, and also to prevent the attacker from
subverting the audit source or the intrusion-detection system itself.

62
 IDS: Time aspect
• Real-time IDS
• Analyzes the data while the sessions are in progress
• Raises an alarm immediately when the attack is detected
• Off-line IDS
• Analyzes the data after the information has been already collected
• Useful for understanding the attackers’ behavior

63
Intrusion Detection Systems
• IDS’s really refer to two kinds of
detection technologies
‣ Behavior-based Detection
‣ Misuse Detection

6
4
Intrusion Detection Techniques
• Misuse detection
– Use attack “signatures” (need a model of the attack)
• Sequences of system calls, patterns of network traffic, etc.
– Must know in advance what attacker will do (how?)
– Can only detect known attacks
– Relatively few false positives
• Anomaly detection
– Using a model of normal system behavior, try to detect
deviations and abnormalities
• E.g., raise an alarm when a statistically rare event(s) occurs
– Can potentially detect unknown attacks
– Many false positives

slide 38
 Intrusion Detection Systems
Misuse Detection
• The system is equipped with a number of attack
descriptions (“signature”). Then matched against the audit
data to detect attacks.
• Pro: less false positives (But there still some!)
• Con: cannot detect novel attacks, need to update the
signatures often.
• Approaches: pattern matching, security rule specification.

66
 Intrusion Detection Systems
Behavior-based IDS
• Good completeness, bad accuracy
• Involves the collection of data relating to the behavior of legitimate users over
a period of time
• Current observed behavior is analyzed to determine whether this behavior is
that of a legitimate user or that of an intruder
• Detect intrusion by observing a deviation from the normal or expected
behavior of the system or the users
• Can detect attempts to exploit new and unforeseen vulnerabilities
• Behavior-based IDS
• Statistics
• Expert systems
• Neural networks
67
• User intention identification
 Behavior-based IDS
• In essence, anomaly approaches aim to define normal, or expected,
behavior, in order to identify malicious or unauthorized behavior.

• However only anomaly detection is able to detect unknown, zero-

day attacks, as it starts with known good behavior and identifies
anomalies to it.
o Given this advantage, clearly anomaly detection would be the
preferred approach, were it not for the difficulty in collecting and
analyzing the data required, and the high level of false alarms

68
 Anomaly Detection
A variety of classification approaches are
used:

Statistical Knowledge based Machine-learning

• Analysis of the • Approaches use • Approaches

observed an expert system automatically
behavior using that classifies determine a
univariate, observed suitable
multivariate, or behavior classification
time-series according to a model from the
models of set of rules that training data
observed metrics model legitimate using data
behavior mining
techniques
69