Module 02 - IDS
Module 02 - IDS
• Analysis - people
• Escalation - processes
www.cybexer.com 2
Intrusion Detection Systems
• NIDS - network-based intrusion detection system
www.cybexer.com 3
Intrusion Detection Systems
Types of IDSs
www.cybexer.com 4
NIDS
• Monitors entire network (or network segment)
• Operates in promiscuous or non-promiscuous mode
• Inline or offline mode
www.cybexer.com 5
HIDS
• Granular configuration of end-points
• Centralized management
www.cybexer.com 6
DIDS
• Centralized collector
www.cybexer.com 7
Be aware of
• False positives - alert is triggered on legitimate traffic
where no intrusion or attack is occurred
www.cybexer.com 8
IDS will do
• continuously watch packets
• understand them
• almost immediate alerting
• identify attack trends and patterns
• block the suspicious traffic
www.cybexer.com 9
SNORT
• "sniffer and more"
• almost 20 y.o
• first version had 2 files
• rule-based analyzer
www.cybexer.com 10
Snort's architecture
• sniffer
• preprocessor
• detection engine
• output
www.cybexer.com 11
Installing Snort
Most of the Linux software can be installed in 2 ways
www.cybexer.com 12
Installing Snort
Login to 10.XX.32.4 with root user
www.cybexer.com 13
Installing Snort
Verify your system is up-to-date
www.cybexer.com 14
Installing Snort
Install Snort from repository
• apt-get install snort
www.cybexer.com 15
Installing Snort
By default, Snort offers 'eth0' network interface. Be
sure you set correct one. For Ubuntu server it should
be 'ens160'
www.cybexer.com 16
Installing Snort
Set proper local network range during installation.
Set address range to your network: 10.XX.32.0/24
Multiple networks must be comma-separated.
www.cybexer.com 17
Installing Snort
Verify you have correct network interface - ens160
www.cybexer.com 18
Installing Snort
If Snort installed successfully, then you should not see
any error messages after installation
www.cybexer.com 19
Installing Snort
Before continuing with Snort setup, we have to verify
its configuration.
Snort's configuration file located in
'/etc/snort/snort.conf'
Open Snort configuration file in text editor and review
variables 'HOME_NET' and 'EXTERNAL_NET'
www.cybexer.com 20
Installing Snort
In case 'HOME_NET' variable set to 'any', change it to
your network range
www.cybexer.com 21
Installing Snort
Best way to understand how Snort rules work, is to
disable all rules
• sed -i "s/include \$RULE\_PATH/#include
\$RULE\_PATH/" /etc/snort/snort.conf
www.cybexer.com 22
Installing Snort
Verify Snort installation
• snort -V
www.cybexer.com 23
Installing Snort
Check paths for Snort rules in '/etc/snort/snort.conf'
Following paths are correct ones
▪ var RULE_PATH /etc/snort/rules
▪ var SO_RULE_PATH /etc/snort/so_rules
▪ var PREPROC_RULE_PATH /etc/snort/preproc_rules
• grep 'RULE_PATH ' /etc/snort/snort.conf
www.cybexer.com 24
Installing Snort
Adjust Snort logging settings in '/etc/snort/snort.conf'
www.cybexer.com 25
Installing Snort
Use sed one-liner to make changes
• sed -i "/^output unified2/coutput unified2:
filename snort.u2, limit 128" /etc/snort/snort.conf
www.cybexer.com 26
Installing Snort
Since we disabled all Snort rules, we have to enable
custom rule's location in '/etc/snort/snort.conf'
Uncomment line '#include $RULE_PATH/local.rules'
www.cybexer.com 27
Installing Snort
Or in your terminal use sed one-liner
www.cybexer.com 28
Installing Snort
It is important to archive Snort's logs
Create new folder for archived logs
• mkdir /var/log/snort/archived_logs
www.cybexer.com 29
Installing Snort
Create additional Snort folders
• mkdir /etc/snort/preproc_rules
• mkdir /etc/snort/so_rule
www.cybexer.com 30
Verify Snort installation
After Snort configuration changes, we have to verify,
that configuration file does not have any errors.
Be sure to set correct network interface name
• snort -T -c /etc/snort/snort.conf -i ens160
www.cybexer.com 31
Verify Snort installation
If Snort's configuration file has some issues, you will
see an error
www.cybexer.com 32
Verify Snort installation
Fix your Snorts configuration file
'/etc/snort/snort.conf' and re-run configuration tester
• snort -T -c /etc/snort/snort.conf -i ens160
www.cybexer.com 33
Snort - rules
Snort configuration tester command also shows rule
statistics.
Since we disabled all the rules, you will see all zeros in
'Rule Port Counts' block
www.cybexer.com 34
Snort - rules
Now, let's add new custom Snort rule.
Open in text editor custom rule configuration file
• vi /etc/snort/rules/local.rules
www.cybexer.com 35
Snort - rules
Or you can use your Linux terminal shell
www.cybexer.com 36
Snort - rules
Next, we have to create Snort ID mapping.
www.cybexer.com 37
Snort - rules
Or you can use your Linux terminal shell
www.cybexer.com 38
Snort - rules
To verify, that Snort accepted new rule we have to run
configuration checker:
• snort -T -c /etc/snort/snort.conf -i ens160
www.cybexer.com 39
Snort - rules
Let's test our first Snort rule
Note! At this stage you will not see any output on the
screen
www.cybexer.com 40
Snort - rules
Now ping Snort server from remote machine.
If Snort configured properly and rule is correct, then
you should see on console 'ICMP test detected'
messages
www.cybexer.com 41
Snort - rules
Congratulations!
www.cybexer.com 42
Reading Snort logs
www.cybexer.com 43
Barnyard2 installation
Open-source interpreter for Snort unified binary output
files.
www.cybexer.com 44
Barnyard2 installation
Firstly, we must install required software
• apt-get install mysql-server libmysqlclient-dev
mysql-client autoconf libtool libdumbnet-dev
libpcap-dev make libdaq-dev libdnet-dev libphp-
adodb libdnet
www.cybexer.com 45
Barnyard2 installation
During MySQL database installation, installer script might
prompt for new password (3 times). For test environment,
we can leave empty password
www.cybexer.com 46
Barnyard2 installation
Before downloading and installing Barnyard2, following
steps must be done
• ln -s /usr/include/dumbnet.h /usr/include/dnet.h
• mkdir -p /opt/snort
Download Barnyard2 from Github
• cd /opt/snort
• git clone https://github.com/firnsy/barnyard2.git
www.cybexer.com 47
Barnyard2 installation
Change to Barnyard2 source code folder
• cd barnyard2/
www.cybexer.com 48
Barnyard2 installation
Prepare Barnyard2 for compilation
• ./configure --with-mysql --with-mysql-
libraries=/usr/lib/x86_64-linux-gnu
www.cybexer.com 49
Barnyard2 installation
Compile Barnyard2
• make
www.cybexer.com 50
Barnyard2 installation
Install Barnyard2 (you must be root user)
• make install
www.cybexer.com 51
Barnyard2 installation
After compilation and installation of Barnyard2, we
have to verify installation
• /usr/local/bin/barnyard2 -V
www.cybexer.com 52
Barnyard2 installation
Since Barnyard2 is used with Snort, we have copy
configuration file to Snort's directory
• cp /usr/local/etc/barnyard2.conf /etc/snort/
www.cybexer.com 53
Barnyard2 installation
Barnyard2 must have some files and folder in place
• mkdir /var/log/barnyard2
• chown snort:snort /var/log/barnyard2
• touch /var/log/snort/barnyard2.waldo
• chown snort:snort /var/log/snort/barnyard2.waldo
www.cybexer.com 54
Barnyard2 configuration
By default, Barnyard2 logging is disabled.
Let enable logging in main configuration file
'/etc/snort/barnyard2.conf'
Open configuration file in text editor, uncomment
'config logdir' option and set correct path to it
• config logdir: /var/log/barnyard2
www.cybexer.com 55
Barnyard2 configuration
Or use 'sed' one-liner
www.cybexer.com 56
Barnyard2 configuration
Barnyard2 provides logging ability to different database
types.
www.cybexer.com 57
Barnyard2 configuration
Or use sed one-liner
• echo 'output database: log, mysql, user=snort
password=sn0rt dbname=snort host=localhost' >>
/etc/snort/barnyard2.conf
www.cybexer.com 58
Barnyard2 configuration
Since Barnyard2 saves alerts to MySQL database, we
need to create that database, as well as a ‘snort’
MySQL user to access that database
Login to MySQL database with root user
• mysql -u root -p
www.cybexer.com 59
Barnyard2 configuration
Create new database 'snort' in MySQL
• create database snort;
www.cybexer.com 60
Barnyard2 configuration
Create required tables by running SQL script
• source /opt/snort/barnyard2/schemas/create_mysql
Note! You should not see any errors at the end script output
www.cybexer.com 61
Barnyard2 configuration
Create MySQL user for Barnyard2
www.cybexer.com 62
Barnyard2 configuration
Allow MySQL 'snort' user only to access 'snort'
database with limited permissions
• grant create, insert, select, delete, update on
snort.* to 'snort'@'localhost';
www.cybexer.com 63
Running Barnyard2
First run of Barnyard2
• barnyard2 -c /etc/snort/barnyard2.conf -d
/var/log/snort -f snort.u2 -w
/var/log/snort/barnyard2.waldo -u snort -g snort
www.cybexer.com 64
Finalizing Barnyard2
By default, Barnyard2 will not start automatically on boot time. Let's
create auto-startup script. Save following content to
'/lib/systemd/system/barnyard2.service' file
[Unit]
Description=Barnyard2 Daemon
After=syslog.target network.target
[Service]
Type=simple
ExecStart=/usr/local/bin/barnyard2 -c /etc/snort/barnyard2.conf -d
/var/log/snort -f snort.u2 -q -w /var/log/snort/barnyard2.waldo -u
snort -g snort -D -a /var/log/snort/archived_logs
[Install]
WantedBy=multi-user.target
www.cybexer.com 65
Finalizing Barnyard2
Let's fetch startup script from the server and save it to
correct location
• wget http://files.csirt.crp/barnyard2.service.txt -O
/lib/systemd/system/barnyard2.service
www.cybexer.com 66
Finalizing Barnyard2
Reload
• systemctl daemon-reload
Enable autostart of Barnyard2
• systemctl enable barnyard2
Start Barnyard2
• systemctl start barnyard2
www.cybexer.com 67
Finalizing Barnyard2
Verify Barnyard2 is running
• systemctl status barnyard2
www.cybexer.com 68
Finalizing Barnyard2
Now let's verify that Snort is generating alerts based on
current rules and Barnyard2 is importing alerts into
MySQL database.
Restart Snort and Barnyard2 services
• service snort restart
• service barnyard2 restart
www.cybexer.com 69
Finalizing Barnyard2
Check that MySQL database has events in 'event' table.
Type in Linux terminal following command
• mysql -u snort -p -D snort -e "select count(*) from
event"
www.cybexer.com 70
BASE - Snort Web GUI
BASE is a simple web GUI for Snort.
BASE stands for Basic Analysis and Security Engine
www.cybexer.com 71
BASE - ADOdb installation
BASE relies on ADOdb (database abstraction library for
PHP).
Let's install ADOdb
• cd /opt/snort
• wget
'https://sourceforge.net/projects/adodb/files/adod
b-php5-only/adodb-520-for-php5/adodb-
5.20.9.tar.gz'
www.cybexer.com 72
BASE - ADOdb installation
Extract ADOdb source
• tar -xzf adodb-5.20.9.tar.gz
www.cybexer.com 73
BASE - install Apache
BASE requires a web server with PHP support.
We will install Apache web server.
Add Apache2 and PHP version 5.6 repository
• add-apt-repository ppa:ondrej/php
www.cybexer.com 74
BASE - install Apache
After adding new repositories, you must update them
• apt-get update
www.cybexer.com 75
BASE - install Apache
Install Apache web server and PHP. Additional PHP
modules also must be installed
• apt-get install apache2 libapache2-mod-php5.6
php5.6-mysql php5.6-cli php5.6 php5.6-common
php5.6-gd php5.6-cli php-pear php5.6-xml
www.cybexer.com 76
BASE - install Apache
To verify Apache web server is running and PHP
scripting language is working, we have to create a
simple PHP file
www.cybexer.com 77
BASE - install Apache
Open in your browser IP address of Snort BASE system
'http://10.XX.32.4/phpinfo.php'
www.cybexer.com 78
BASE - Installation
Navigate to '/opt/snort' folder
• cd /opt/snort
Download latest version of BASE source code
• wget
http://sourceforge.net/projects/secureideas/files/B
ASE/base-1.4.5/base-1.4.5.tar.gz
www.cybexer.com 79
BASE - Installation
Extract downloaded archive
• tar xzf base-1.4.5.tar.gz
www.cybexer.com 80
BASE - Installation
Copy sample configuration file 'base_conf.php.dist' to
'base_conf.php'
• cp base_conf.php.dist base_conf.php
www.cybexer.com 81
BASE - Configuration
Before using BASE, we need to adjust configuration
option in '/var/www/html/base/base_conf.php'
Remove ' DejaVuSans' font from variable
• $graph_font_name = "";
www.cybexer.com 82
BASE - Configuration
Next steps are to define URL for BASE web application,
set correct location for ADOdb and adjust MySQL
options in BASE configuration file
'/var/www/html/base/base_conf.php'
$BASE_urlpath = '/base';
$DBlib_path = '/var/adodb';
$alert_dbname = 'snort';
$alert_host = 'localhost';
$alert_port = '';
$alert_user = 'snort';
$alert_password = 'sn0rt';
www.cybexer.com 83
BASE - Configuration
Fetch file from server and save it to correct location
• wget http://files.csirt.crp/base_conf.php.txt -O
/var/www/html/base/base_conf.php
www.cybexer.com 84
BASE - Configuration
To finalize BASE installation, restart Apache web server
• service apache2 restart
www.cybexer.com 85
First run BASE
Before using BASE system, MySQL database must be
prepared. Click 'Setup page' link on main page
www.cybexer.com 86
First run BASE
Click on the "Create BASE AG" button on the upper
right part of the page.
www.cybexer.com 87
First run BASE
Script will automatically create all required tables and
populate them with required data
www.cybexer.com 88
Putting all together
Main view of BASE system
www.cybexer.com 89
Putting all together
Click on the number next to 'Total Number of Alerts'
www.cybexer.com 90
Putting all together
Click on the number next to 'Total Number of Alerts'
www.cybexer.com 91
Snort rules
Simple Snort rule structure is very straightforward:
alert ip any any -> any any (msg: "IP Packet
detected";)
• alert - generate an alert if criteria met for captured
packet
• ip - protocol type (ip, tcp, udp, icmp)
• any - source IP address
• any - source port (IP layer do not consider ports)
www.cybexer.com 92
Snort rules
Simple Snort rule structure is very straightforward:
alert ip any any -> any any (msg: "IP Packet
detected";)
• -> - direction of packet
• any - destination IP address
• any - destination port
• msg - message that will be logged
www.cybexer.com 93
Snort rules
Sources and destinations:
• 192.168.1.3/32
• 172.28.28.0/24
• $HOME_NET, $EXTERNAL_NET (from snort.conf)
• [10.10.13.0/24,192.168.8.0/24]
• ![172.28.24.0/24,192.168.44.0/23]
www.cybexer.com 94
Snort rules
Ports:
• 80
• 1024:2000
• !53
www.cybexer.com 95
Snort rules - classtypes
Rules can have classifications and priorities
Check /etc/snort/classification.config
config classification: http-event,Generic HTTP event,3
• http-event - name for classification
• Generic HTTP event - description of classification
• 3 - priority (lower number - higher priority)
www.cybexer.com 96
Snort rules - classtypes
Setting classtype to the rule:
www.cybexer.com 97
Snort rules - SID
Snort ID - rule identificator for output modules
www.cybexer.com 98
Writing Snort rules
Let's create new rule in /etc/snort/rules/local.rules
www.cybexer.com 99
Writing Snort rules
Now we need to add new classification to
'/etc/snort/classification.config' file
www.cybexer.com 100
Writing Snort rules
Create SID for new Snort rule in '/etc/snort/sid-
msg.map'
www.cybexer.com 101
Writing Snort rules
To apply new rule, classification and SID mapping data
we have to restart 'Snort' and 'Barnyard2'
www.cybexer.com 102
Writing Snort rules
Verify that new rule is added to Snort
• snort -T -c /etc/snort/snort.conf -i ens160
www.cybexer.com 103
Writing Snort rules
Now let's test the rule. Open in browser your Snort
machine's IP address and type in any where in URL
'ABCD'
http://10.XX.32.4/test1/ABCD/test2
www.cybexer.com 104
Writing Snort rules
Now open BASE system in your browser and check new
events by clicking number next to 'Total Number of
Alerts'
www.cybexer.com 105
Writing Snort rules
Verify that new rule is added to Snort
• snort -T -c /etc/snort/snort.conf -i ens160
www.cybexer.com 106
Writing Snort rules
Snort has hundreds of predefined rules.
Check '/etc/snort/rules' folder for examples
• ls -la /etc/snort/rules
www.cybexer.com 107
Filebeat - Snort logs
By default, Snort logs in binary format.
Let's configure Snort to write alerts in plain text files.
Open /etc/snort/snort.conf file and add following line
• output alert_fast: alert.fast
www.cybexer.com 108
Filebeat - Snort logs
By default, many Linux servers have so-called 'message
reduction' option enabled in Rsyslog service.
This parameter specifies whether or not repeated
messages should be reduced.
Let's turn off message reduction and then verify results
• sed -i -e
"s/\$RepeatedMsgReduction.*/\$RepeatedMsgRed
uction off/" /etc/rsyslog.conf
• grep -i reduction /etc/rsyslog.conf
www.cybexer.com 109
Filebeat - Snort logs
After 2 last modifications, we have to restart 2 services:
rsyslogd and snort
www.cybexer.com 110
Filebeat - Snort logs
Enable Snort Filebeat module
• filebeat modules enable snort
www.cybexer.com 111
Filebeat - Snort logs
Ping Snort machine from your laptop
• ping 10.XX.32.4
www.cybexer.com 112
Filebeat - Snort logs
Now access Graylog's web interface and check the logs
www.cybexer.com 113
Filebeat - Snort logs
If you have complex Filebeat setup, then it's advised to
set tags to different logs
www.cybexer.com 114
Filebeat - Snort logs
Then in Graylog's search query will be following:
• filebeat_tags:snort
www.cybexer.com 115
Filebeat - Snort logs
Then in Graylog's search query will be following:
• filebeat_tags:snort
www.cybexer.com 116
Filebeat - Snort logs
Snort alerts can differ, based on the protocol used.
For ICMP alerts, there're no source or destination
ports.
But for TCP and/or UDP alerts - there're source and
destination ports.
www.cybexer.com 117
Filebeat - Snort logs
Compare two Snort alerts:
03/02-16:16:02.974874 [**] [1:10000011:1]
Custom content detected [**] [Classification:
Custom content detected] [Priority: 1] {TCP}
172.16.63.243:1284 -> 10.30.32.4:80
www.cybexer.com 118
Filebeat - Snort logs
To create correct GROK pattern, you have to use 'OR'
conditions:
• (?:%{IP:src_IP}|%{IP:src_ip}\:%{INT:src_port})\s+\-
\>\s+(?:%{IP:ipdst}|%{IP:ipdst}:%{INT:ipdport})
www.cybexer.com 119
Filebeat - Snort logs
Let's create new pattern for Snort logs.
Search for Snort events in Graylog:
• filebeat_tags:snort
www.cybexer.com 120
Filebeat - Snort logs
Click on any Snort event and scroll down to 'message'
part
www.cybexer.com 121
Filebeat - Snort logs
At the end of 'message' field click on arrow button and
pick 'Create extractor' on the popup
www.cybexer.com 122
Filebeat - Snort logs
From extractor type drop-down, select 'Grok pattern'
and then click 'Submit'
www.cybexer.com 123
Filebeat - Snort logs
Select 'Named captures only' and enter GROK pattern.
Be sure to remove new lines in Grok pattern.
www.cybexer.com 124
Filebeat - Snort logs
GROK pattern for parsing Snort logs will be following:
• %{MONTHNUM:month}\/%{MONTHDAY:day}-
%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:seco
nd}\s+\[\*\*\]\s+\[%{INT:ids_gid}\:%{INT:ids_sid}\:
%{INT:ids_rev}\]\s+%{DATA:ids_proto}\s+\[\*\*\]\s
+\[Classification:\s+%{DATA:ids_classification}\]\s+
\[Priority:\s+%{INT:priority}\]\s+\{%{WORD:ids_pr
oto2}\}\s+(?:%{IP:src_IP}|%{IP:src_ip}\:%{INT:src_p
ort})\s+\-
\>\s+(?:%{IP:ipdst}|%{IP:ipdst}:%{INT:ipdport})
www.cybexer.com 125
Filebeat - Snort logs
Before saving the extractor, it's a good idea to test
current GROK pattern. Click on 'Try against example'.
If no errors show, you can proceed with saving
extractor.
www.cybexer.com 126
Filebeat - Snort logs
Set extractor title and click 'Create extractor'
www.cybexer.com 127
Filebeat - Snort logs
In some complex GROK extractors you can add
additional keyword/pattern, which trigger current
extractor.
Set desired keyword and hit 'Try'. If you see message
'Matches', then you GROK extractor will have
conditional trigger
www.cybexer.com 128