Intrusion Detection

Network Security
Agenda
What is IDS?

Types of IDS

How IDS work

Introduction to Snort

2
The Need for Firewalls

Workstation Workstation
A B

Internal External
Network Network
(LAN) Firewall router/gatew (WAN/Internet)
ay

Database Application
3
Server Server Clients & Customers
The Need for Firewalls
Ports status

80, 443, 22 open

21, 23, 3389 closed

Workstation Workstation
A B

Internal External
Network Network
(LAN) Firewall router/gatew (WAN/Internet)
ay

Database Application
4
Server Server Clients & Customers
Interview Questions

1. If we installed a firewall to protect our network against

malicious traffic, why would we install an intrusion
detection system?

2. What is the difference between and Application Firewall and

an IDS?
 The Need for IDS
• Firewalls currently represent the most widely used security mechanisms
in computer networks
• Firewalls, however, can be compromised or bypassed, and do precisely
little to protect against insider attacks.
• Firewalls cannot defend computer systems against certain classes of
malicious behavior. When a machine is connected to the Internet and has
to provide publicly accessible services
• So, it is very important to have additional protection mechanisms on the
internal hosts and network.
• Intrusion detection systems fulfill such purpose by monitoring.
computing systems and reporting intrusive behaviors.
6
 What is an Intrusion
• A sequence of related actions with malicious intention executed by an
external entity that results in the compromise of a target system.
• Systems that attempt to detect malicious behavior that is targeted against
a network, and its resources are called Intrusion detection systems (IDSs)
• The term intrusion usually refers to external malicious activities or
malicious payload exist in incoming network traffic (e.g. SYN flooding
attack)

7
 Intrusion Detection System
• Intrusion detection can be defined as the process of identifying unauthorized activities by
inspecting individual machines and inbound network traffic.

8
 What is an Extrusion
• The term extrusion usually refer to internal malicious activities or
malicious payload exist in outgoing traffic (e.g. Botnet activities,
espionage|sabotage)
• Extrusion detection or prevention system are special IDS system that is
designed to detect and prevent data leaks attempts. By avoiding
unauthorized data (e.g. packets, emails attachments, file sharing) from
leaving the network
• Extrusion detection systems focus on insider threat detection.

9
 Extrusion Detection System
• Extrusion detection can be defined as the process of identifying unauthorized
activity by inspecting outbound network traffic

If the IDS sensor pays attention to

outbound traffic, it has greater chance
of detecting internal intrusions

10
 IDS Taxonomy
Intrusion detection systems may be classified according to different characteristics
1. Audit Source Location
2. Behavior on Detection
3. Usage Frequency
4. Detection Method

11
 IDS Taxonomy: Audit Source Location
Intrusion Detection Systems can be characterized according to the source of the events they analyze:
Host -Based IDS (HIDS):
• Detect attacks against a specific host by analyzing audit data produced by the host operating systems.
• HIDS monitor running process to identify malicious processes or abnormal system behaviors. For example
CPU usage, open sockets and ports, File I/O operations and operating system logs
Network - Based IDS (NIDS):
• Detect attacks against a specific host by analyzing incoming and outgoing network traffic.
• Must implement a network traffic sniffer or has the mean to capture network traffic.

12
 IDS Taxonomy: Behaviors on Detection
• When an IDS detects a potential attack or an action that appears to be
malicious, it can react with a passive response or an active response
• The passive repose the most common behaviors of intrusion detection
systems. In the case of passive response, it will generate an alert
message to report the potential attack and notify security engineers.
• The active response means the IDS will act with the intention to
prevent the attack. For example drop network packets, close
connection, add a new firewall rule, kill system process, etc. It is
common to refer to IDS in this case as intrusion prevention system IPS

13
 IDS Taxonomy: Usage Frequency
• Dynamic IDS act in an online and real-time mode and analyze event,
actions and audit data as soon as the data become available.
• Static IDS run offline at specific intervals. They analyze a snapshot of
the system state and produce an evaluation of the security of that state.
They do not provide any security in between two consecutive runs.
• Static IDS has less effect on the system performance comparing to
dynamic IDS. They are are usually triggered by critical system events
(e.g. system load, system update, writing in-memory data to file
system, etc)

14
 IDS Taxonomy: Detection Mode
IDS could be categorized into three main categories based on the applied detection mode
• Anomaly detection: analyzes a set of characteristics of the system and compares their
behavior with a set of expected values. It reports an intrusion when there is a deviation
from the expected behavior.

• Signature-based Detection: determines whether a sequence of instructions being

executed is known to violate the site security policy and if so it reports a potential
intrusion.

• Misuse or Specification-based Detection: determines whether or not a sequence of

instructions violates a specification of how a program or system should execute, and
then reports an intrusion in case of violation

15
 Detection Mode: Anomaly Detection
• Assumes that there is a normal behavior of the system
• Consider a deviation from the normal behavior, i.e., an anomaly, as an intrusion

16
 Detection Mode: Anomaly Detection
• The description of the system (machine, network) behavior is based on extracting information
from

a. Users accounts, Process, Files

b. Network traffic

c. System logs

• Anomaly detection IDS build models of “normal” behavior of a system using machine learning
or data mining.

• A deviation from the constructed model is considered & anomaly.

• Anomaly detection IDS can detect previous unseen attacks but in general has a higher false
positives from time to time, and hard to train for a very dynamic environment.

17
 Detection Mode: Anomaly Detection
Example Host-based Anomalies
• Consider a host-based IDS in which user activity records are
represented using the following format:
<login_time, average_cpu_usage, logout_time>
Given the following log which activities you think abnormal, and why?
01: <08:00, 23%, 17:00>
02: <07:50, 24%, 17:10>
03: <07:56, 23%, 16:45>
04: <20:25, 25%, 22:34>
06: <08:13, 27%, 17:03>
07: <08:30, 22%, 17:00>
08: <08:00, 60%, 16:30>
18
 Detection Mode: Anomaly Detection
Example Host-based Anomalies
• Consider a host-based IDS in which user activity records are represented
using the following format:
<login_time, average_cpu_usage, logout_time>
Given the following log which activities you think abnormal, and why?
Assume the user profile is : <7:30-8:30, 20-30%, 16:30-17:30>.

01: <08:00, 23%, 17:00>

02: <07:50, 24%, 17:10>
03: <07:56, 23%, 16:45>
04: <20:25, 25%, 22:34>
06: <08:13, 27%, 17:03>
07: <08:30, 22%, 17:00>
08: <08:00, 60%, 16:30> 19
 Detection Mode: Anomaly Detection
Example Network-based Anomalies

No. Date time prot sPort dPort sIP dIP

20
 Detection Mode: Anomaly Detection
Example Network-based Anomalies

No. Date time prot sPort dPort sIP dIP

21
 Detection Mode: Anomaly Detection
• An anomaly detection NIDS will extract different metrics or features from
the network traffic to describe the behaviors of the network.
• Here are few examples of these metrics:
a. The average number of incoming packets per unit of time (e.g. per
second)
b. The average number of outgoing packets per unit of time (e.g. per
second)
c. The number of destination ports to the same destination IP.
d. The average size of the message payload
e. The number of similar network flows
• The information extracted from these metrics are used to build the
normal profile of a given network, which is used as a reference to
distinguish between normal and abnormal behaviors 22
 Detection Mode: Anomaly Detection
Limitations
– Require training and learning
– Require data scientists and intrusion analysts
– Expensive to update (require retraining)
– Decision hard to explain.
Advantages
– Can detect zero-day exploit or vulnerability
– Does not require human expert to continuously update the detection
model.

23
 Detection Mode: Signature-Based
• Signature-based IDS requires knowledge of system vulnerabilities or
potential vulnerabilities that attackers attempt to exploit.
• The IDS uses a knowledge-base that contains known attack patterns
(“signature”).

24
 Detection Mode: Signature-Based
• Examples: access control attack with dictionary attack.
– If the number of unsuccessful login attempts for the same user account is more
than 4 per minute

25
 Detection Mode: Signature-Based
• Normally, a signature-based IDS will use a set of rules with the if-then-else
structure to perform pattern matching or signature matching.

Example:

If src.IP == dst.IP then report "land-attack" Else next-rule

• The rules are executed in the same way as firewall rules from top to button but the
rules are not chained as firewall rules.

• The rules and the patterns are written by security expert known as intrusion
analyst.

• The detection accuracy are limited by the quality of the patterns and the rules.

26
 Detection Mode: Signature-Based
Signature-based ID could use security metrics similar to anomaly detection, but
the metrics are used to describe the rules not to build a learning model.

if protocol_type=tcp and Parameters Meaning

same_srv_rate=1.00 then intrusion protocol_type protocol type

same_srv_rate percentage of connections

if protocol_type=icmp and to the same services
dst_host_srv_count>160 and srv_count number of connections from
the same source service to the same destination
syn_err=0 then intrusion service
dst_host_srv_count number of connections from the same
if protocol_type=udp and srv_count>367 destination service to the same source service
and synflood=0 then intrusion synflood connections that have “SYN” errors

27
 Detection Mode: Signature-Based
• Limitations:
– Can not detect zero-day exploit or vulnerability.
– The deployed rules require continuous maintenance and tracking
– Require a security expert to describe the attack pattern
• Advantages:
– Easy to deploy and run
– Does not require learning-time
– Easy to update (simply update the attack pattern and rules)
– Any reported attack can be traced to one or more rule.

28
 Detection Mode: Specification-Based
• Specification-based IDS or context-aware IDS are IDS systems that
require the expected behaviors of the environment to be manually
described in a machine-readable format.
• Specification-based IDS use positive detection approach to detect
intrusion and attacks.
• It combines the strengths of signature detection (accurate detection of
known attacks) and anomaly detection (ability to detect novel attacks)

What are the challenges of specification-based IDS?

29
 IDS Performance Evaluation
• There are two metrics that we use to evaluate the performance of intrusion
detection systems. These metrics are the false positive rate and the false
negative rate
• The false positive rate refers to the ratio of normal activities (e.g., processes,
TCP/IP traffic) that were wrongly considered malicious by the IDS.

• The false negative rate refers to the ratio of malicious activities (e.g.,
processes, TCP/IP traffic) that were wrongly considered normal by the IDS.

30
 IDS Design Goals
What are the characteristics of a proper IDS?
1. Detect a wide variety of attacks, both known and unknown.
2. Detect intrusions in a timely fashion (not necessarily
real-time).
3. Present the analysis in a simple, easy-to-understand format.
4. Achieve high detection accuracy; accuracy is measured
using metrics such as false positive rate (FPR) and false
negative rate (FNR).

31
 IDS Alerts Analysis
The process of investigating the reported IDS alerts by one
or more IDS sensor to:
1. Extract attack intelligence
2. Isolate the compromised host
3. Recover from a security breach
4. Implement defensive/prevention strategies

32
 IDS Alerts Analysis Techniques
1. Alerts normalization: alerts generated by different IDS sensors are usually stored
in different data stores and different format which create an interoperability
problem.

2. Alerts filtering or verification: Not all the generated IDS alerts report actual attack
attempts. False positive alerts is a known issue in IDS. Therefore a security analyst
needs to inspect the reported alerts to check if the attack is real or not

3. Alerts fusion or aggregation: Eliminate duplicated, near duplicate alerts and

similar alerts to reduce the number of alerts needs investigation

4. Alerts correlation : Group related alerts that belong to the same attack pattern or
scenario to reconstruct the attack scenario. Sophisticated attacks are usually a
multistage attack scenario.

33
 Challenges in IDS Alerts Analysis
The challenges in IDS alerts analysis could be grouped into two main
categories:
• Correctness Challenges:
– Alert Flooding
– False Positive
– False Negative
– Isolation
• Automation Challenges:
– Interoperability
– Machine Readability
– Complex and Symbolic Data

34
Working with Snort
What is SNORT?
Snort is a free open source network traffic analysis tool, written by Martin Roesch at
Sourcefire (Cisco acquired for $2.7 billion in July 2013.)

Snort support the following network analysis functionalities:

1. Sniffer
2. Packet Logger
3. Intrusion Detection
4. Intrusion Prevention

Snort could be used for real-time network traffic analysis or for after the fact network
forensics analysis.
36
SNORT Design & Architecture
SNORT has a lightweight design
based on a plug-in architecture.
This plug-in architecture give
SNORT endless flexibility to
extend and customize SNORT
features and capabilities.

37
SNORT Plugins Types
You can extend snort by implementing your own components and integrating them
with snort, rather than (or in addition to) using the default components.
● Preprocessor
○ Packets are examined/manipulated before being handed to the detection
engine
● Detection
○ Perform single or multiple tests on one or more aspect/field of the packet
● Output
○ Report results from the other plug-ins

38
SNORT Architecture
Snort has also a layer architecture where each layer receive an input form the top
layer, perform some actions and pass the output to the bottom layer

39
SNORT Architecture and Components
The key components in SNORT are:
1. Network Sniffer
2. Preprocessor
3. Detection Engine
4. Knowledge-base
5. Notifier (Logging and Alerts)

40
SNORT Components: Network Sniffer
This a basic packet sniffer has similar features other network traffic sniffer (e.g.
tcpdump, wireshark) and it is based on libpcap

The network sniffer also known as packets decoder takes the packets from different
types of network interfaces

Send the packets to the preprocessor if the packets require preprocessing or send
the packets to the detection engine if preprocessing is not required.

41
SNORT Components: Preprocessor
The preprocessing allowing users and programmers to drop modular plugins into
Snort
The preprocessing code runs before the detection engine is called, but after the
packet has been decoded.
The preprocessors can modify and edit the packets data to prepare them for the
detection engine if require.
A preprocess could rearrange packets contents that have been crafted by the
hacker to avoid detection during deep packet inspection. Or reassemble packets
fragments and send the whole packet to the detection engine for signature testing.

42
SNORT Components: Detection Engine
The detection engine uses a set of rules to catch any intrusion activity exists in a
packet.
It can dissert a packet and apply rules on different parts of the packet. This includes
the:
1. The IP header of the packet
2. The Transport layer header: e.g. TCP, UDP.
3. The application layer level header: e.g. SSH, FTP, HTTP, SNMP, SMTP, IMAP, etc
4. Packet payload: you can create a rule to find a string inside the data.

43
SNORT Components: Knowledge-base
Snort support a large rule-based that is updated regularly by the snort community
and CISCO/Sourcefire.

Snort users can design and write their own custom rules. The rules have
if-then-else structure. Rules are usually grouped or categorized by protocols or
attacks
 Multimedia  Scan
. Backdoors
. MySQL . Shellcode
. Chat
. NETBIOS . SMTP
. DDoS
. NNTP . SNMP
. Finger
. Oracle . SQL
. FTP
. P2P . Telnet
. ICMP
. POP . TFTP
. IMAP
. RPC . Virus
Security Spring 2013
. Web…
. X11 44
SNORT Components: Notifier|Logging and Output
The captured packet may be used to log the activity or generate an alert.

Logs are kept in simple text files, tcpdump-style files, or some other formats

Log files are stored under /var/log/snort folder by default on Debian like OS

Snort support many output plugins such as: text output, syslog server, XML, IDMEF (
Intrusion Detection Message Exchange Format), MySQL, Oracle, SPLANK, SMB, etc

45
Installing and Configuring SNORT
Snort can run on Windows, Linux, and Mac OS. However, the recommend platform
for deploying and running SNORT is Linux-like OS

Snort essentially runs via command line interface. There are some third parties GUIs
for snort, although the most common use of snort is through a command line.

The basic Snort configuration on different platforms is almost the same with some
minor changes.

Snort comes with a default configuration that can be modified to execute specific
functionality.

46
Installing SNORT
You can download snort from www.snort.org

47
Installing SNORT
There are different options to download and install SNORT, depend on you need and
setting.

1. You can download SNORT binaries for your platform (e.g. ubuntu, mac, windows,
redhat, etc)

2. You can download and install snort using your Linux distro software and package
management tool.

3. You can download the source code and build and compile snort from the source
code.

48
Installing SNORT on Ubuntu
To install SNORT on Ubuntu using the software and package manager, use the
following commands:
sudo apt-get install snort
You will need to configure the network interface and the home network during
installation. You can edit this information at anytime
You can check snort version after the installation complete using the command
snort --version

49
Installing SNORT on Ubuntu
SNORT sniffer configuration

50
Installing SNORT on Ubuntu
SNORT sniffer configuration and home network

51
Installing SNORT on Ubuntu
Check snort installation and version by typing:

snort --version

52
Configuring SNORT
Snort comes with a default configuration file that you can use as a template to
configure your snort deployment or to create different configuration files for different
analysis tasks. To open and edit the configuration file use the following command

sudo gedit /etc/snort/snort.conf

53
Configuring SNORT
To open and edit the configuration file use the following command

sudo gedit /etc/snort/snort.conf

54
Configuration Options

55
Configuring Home Network
You can define your home network or use the default

56
Configuring Home Network
To setup the network addresses you want to protect, you need to update the
$HOME_NET accordingly. The default setting in the snort configuration snort.conf is
as follows:
ipvar HOME_NET any

For instance, to protect subnet 142.104.64.0/24, change the above line in “snort.conf”
to the following:
ipvar HOME_NET 142.104.64.0/24

57
Configuring Network Services
By default, snort will monitor all the servers running on your network. The following
default line indicates that snort will monitor all the web servers on your network:
ipvar HTTP_SERVERS $HOME_NET

To monitor specific web servers on your network, for instance, running at

142.104.64.199 and 142.104.64.201, change the above line in “snort.conf” to the
following:
ipvar HTTP_SERVERS [142.104.64.199, 142.104.64.201]

58
Configuring Network Services
You can also configure snort to monitor specific ports. For instance, by updating the
HTTP_PORTS variable, you can monitor specific ports running on the web server. For
example, the default setting for HTTP is defined in the snort.conf file as follows:
ipvar HTTP_PORTS 80
You can add additional ports by changing the above setting. For instance, the
following line will allow monitoring ports 80, 81, and 8080 on the web server:
ipvar HTTP_PORTS [80,81, 8080, 443]

59
Configuring Output Module
This consists of selecting the output plugins and format for Snort. The output plugins entry
specifies how snort alerts messages will be logged.

60
Configuring Output Module
There are other options to configure the output module. For example, the following
line instructs snort to use the CSV format to log the alerts:

output alert_csv: alert.csv default

You can also log to a database like MySQL, in this case we need first to create and add
SNORT database to our MySQL engine. Snort come with a file called create_mysql,
which has the schema for the database.

output database: log, mysql, user=snort

password=snortpass dbname=snort host=mysql.host

61
Configure SNORT rules
Using the snort configuration file we can enable or disable any rules-set. Every group of
rules-set is usually stored in one rules file

62
Configure SNORT rules
Using the snort configuration file we can enable or disable any rules-set. Every group of
rules-set is usually stored in one rules file

63
Working with SNORT Rules
When installing snort using your Linux distro it usually come with many rules set
installed on your machine. For Debian-like OS the rules are under
/etc/snort/rules/

64
Working with SNORT Rules
You may use any text editor to open, view, and edit the rules files. For example we can
open the ddos.rules file using the following command

gedit /etc/snort/rules/ddos.rules

65
Understanding SNORT Rules

Action: informs Snort what kind of action to be performed when it detects a packet
that matches the rule description. The default action is alert, the other actions are:
log, pass, drop, reject and sdrop.

66
SNORT Question

67
Understanding SNORT Rules

Protocol: this option tell snort to apply this rule for a specific protocol e.g (ip, tcp, udp,
icmp, any)
Source IP: this option tell snort to apply this rule for a specific source IP address, subnet or
any ip address.
Source Port: This part of header describes the source Port from which traffic is coming.

68
Understanding SNORT Rules

Direction operator (“->”, “<>”): It denotes the direction of traffic flow between sender and receiver networks.

Destination IP: This part of header describes the destination network interface in which traffic is coming for
establishing a connection.

Destination Port: This part of header describes the destination Port on which traffic is coming for
establishing a connection.

69
Understanding SNORT Rules
Rule Options: The body for rule option is usually written between circular
brackets “( )” that contains keywords with their argument and the keyword are
separated by semicolon “;”
There are general options and the keywords, and there are options and keywords that
are protocol specific.
In the options we may specify or search for a unique pattern in the packet payload
using the keyword content
You may use regex or regular expression to match payload contents

70
Writing Snort Rules
We can write a SNORT rule to generate an alert when we detect incoming or outgoing
ICMP ping request
alert icmp any any < > any any (itype:8;msg: "ping detected"; sid:1000001;)
The above rule will generate an alert message on every ping packet detect by the IDS
regardless the ping message direction.

The above rule is an example of a valid SNORT rule but a bad one.

71
Writing Snort Rules
Let us write a rule to detect a DOS Jolt attack.
Jolt attack is a denial of service (DoS) attack caused by a very large ICMP packet that
is fragmented in such a way that the targeted machine is unable to reassemble it
alert ip $EXTERNAL_NET any -> $HOME_NET any (msg:"DOS Jolt attack";
dsize:408; fragbits:M; reference:cve,1999-0345; classtype:attempted-dos;
sid:268; rev:4;)

72
Writing Snort Rules
Let us write a rule to detect potential SQL injection attacks that use SQL keywords like [and,
or] or attacks that use special characters

alert tcp any any -> any 80 (msg: “AND SQL Injection”; content:
“and” ; nocase; sid:100000008; )
alert tcp any any -> any 80 (msg: “OR SQL Injection”; content:
“or” ; nocase; sid:100000009; )

alert tcp any any -> any 80 (msg: “Form Based SQL Injection”;
content: “%27” ; sid:1000003; )

73
Writing Snort Rules
Let us write a rule to malicious attachments (e.g. virus, botnet, etc) in TCP traffic.
alert tcp $HOME_NET any -> $EXTERNAL_NET 25 (msg:"VIRUS OUTBOUND bad file
attachment"; flow:to_server,established; content: "Content-Disposition|3A|";nocase;
pcre:"/filename\s*=\s*.*?\.(?=[abcdehijlmnoprsvwx])(a(d[ep]|s[dfx])|c([ho]m|li|md|pp)
|d(iz|ll|ot)|e(m[fl]|xe)|h(lp|sq|ta)|jse?|m(d[abew]|s[ip])|p(p[st]|if|[lm]|ot)|r(eg|tf)|s(cr|[
hy]s|wf)|v(b[es]?|cf|xd)|w(m[dfsz]|p[dmsz]|s[cfh])|xl[tw]|bat|ini|lnk|nws|ocx)[\x27\x22
\n\r\s]/iR"; classtype:suspicious-filename-detect; sid:721; rev:8;)

74
SNORT Question

75
Write and Deploy your own Snort Rules
As a new SNORT user it is recommend that you write your new snort rules into the
file local.rules. Which is commonly used to store custom SNORT rules for a given site.

To open this file use any text editor using the following command

sudo gedit /etc/snort/rules/local.rules

76
Write and Deploy your own Snort Rules
As you can see the local.rules is by default an empty file (no predefined rules)

77
Write and Deploy your own Snort Rules
Let us add the following rule and save the file:

alert tcp any any -> any 443 (msg: "detect HTTPs traffic"; sid:80000000001; rev:1;)

78
SNORT Log files
SNORT logs are by default stored /var/log/snort/

79
SNORT Alerts in CSV file

80
IDS Rules Best Practice
1. Disable all rules and only enable the rules that match your organization security
policy.

2. Use passive actions like log and alert until you are confident that the rules are
correct. Then, you may consider more active actions.

3. Use simple rules and avoid complex rules that try to detect multiple attacks
patterns.

81
IDS Rules Best Practice
4. Avoid overgeneralized rules; overgeneralized rules will result in false positives

82
IDS Rules Best Practice
5. Avoid overfitting rules that use single attribute for the matching like port
number or content. This is usually will increase the false negative.

83
IDS Rules Best Practice
6. Group your custom rules by application, services or attacks.

7. Only log packages that you plan to inspect or use for forensic analysis. Example
log packets about a virus and malware but do not log packets with spoofed IP

8. Use informative log and alert messages.

9. Check the available ruleset to make sure you are not creating duplicate rules

84
IDS Rules Best Practice
10. Add as many as possible information about the attack pattern or signature

85
Certified Intrusion Analyst
https://www.giac.org/certification/certified-intrusion-analyst-gcia

86
The End
Questions??

87