Walkthrough: Buffer Usage Monitor

(Additional Information: appendices_help_book, self_monitoring_help_book manuals)

Requirements

1) IBM InfoSphere™ Guardium® Collector Hardware or Software Appliance

2) IBM InfoSphere™ Guardium® Version: 8+
3) IBM InfoSphere™ Guardium® Account: admin

Assumptions

1) We will assume that the presentation: 4.7 – Advanced Monitoring has been reviewed
2) We will assume that the laboratory: 4.8 – Advanced Monitoring has been conducted
3) We will assume that the Guardium® Web Interface has been started and is working correctly
4) Calculations are approximate, based on black-box testing

Definition

The Buffer Usage Monitor is a built-in report stored within the Guardium® Collector and is
utilized for displaying a wealth of self-monitoring statistical information about the appliance.

Additionally, it is important to note that there are actually two versions of the report:

■ Buffer Usage Monitor: designed for individual appliance self-monitoring

(i.e. un-managed nodes).

■ Enterprise Buffer Usage Monitor: designed to display the aggregate values of all
managed appliance self-monitoring data that has been uploaded.

Also, realize that if a collector is under central management (i.e. it is a managed node), then
the individual Buffer Usage Monitor report may not display values, or vice-versa.

Tip: Use the Buffer Usage Monitor for stand-alone collectors (un-managed), and the
Enterprise Buffer Usage Monitor for managed collectors.
Report Location

The Buffer Usage Monitor and Enterprise Buffer Usage Monitor can be found by logging
into the collector appliance Web interface using the admin account and respectively
navigating to:

Guardium Monitor → Buffer Usage Monitor

Guardium Monitor → Enterprise Buffer Monitor Usage

Background

In order to gain a solid understanding of the Buffer Usage Monitor, it is first necessary to
examine a few background concepts:

■ Query Domains and Entities: Where are the data metrics stored?
■ Software Foundations: How are metrics determined?
■ Data Refresh Interval: How often are metrics re-calculated (sampled)?
■ Audit Process Fundamentals: What is the overall process to calculate metrics?

Query Domains

As per the presentation, 4.7 – Advanced Monitoring, it is important to realize that the
information for the Buffer Usage Monitor and Enterprise Buffer Usage Monitor are stored in
their own entities, and own domains as follows:

Report Domain Entity

Sniffer Buffer Usage Sniffer Buffer Usage
Buffer Usage Monitor
Tracking
Enterprise Buffer Usage Monitor Custom Domain Builder CM Buffer Usage Monitor

Tip: If you ever want to build your own custom queries or reports using the self-monitoring
information found in either report, you can always use the Sniffer Buffer Usage or CM Buffer
Usage Monitor entities directly.
Software Foundations

It is important to recall that the Guardium Collector uses several existing software components
under the hood as part of its overall solution:

■ Operating System: Hardened Red Hat Enterprise Linux (RHEL)

■ Web Server: Apache Tomcat
■ Database: MySQL

As such, it is interesting to realize that due to its use of Linux as its operating system, many of
the metrics shown on the Buffer Usage Monitor report are actually the results/output of several
fundamental Linux utilities (e.g. top, ps, df, ifconfig, etc.).

Similarly, several of the report metrics that relate to MySQL are simply calculated using
MySQL configuration parameters, and run-time parameters within the operating system.

Hence, it is the output of these Linux operating system commands, that are used to populate
the Sniffer Buffer Usage entity (stored in the internal TURBINE MySQL database), and
consequently, the Buffer Usage Monitor report, the with data/values.

Tip: You can always demonstrate how metrics are calculated if you are aware of the core linux
commands that were used to obtain the information.
Data Refresh Interval

It is interesting to note that the time period/interval at which the metrics are calculated
(sampled) is controlled by a configurable property within the collector appliance. This
property can be viewed or set by using the following CLI commands (respectively):

■ CLI> show monitor buffer usage interval

■ CLI> store monitor buffer usage interval <N> (N = Number of Minutes)

Values for this interval are displayed and/or set in: Minutes
Minimum Value: 1 Minute

As an example, if we set this interval to 1 minute, all of the metrics/values on the buffer usage
monitor report will be re-calculated (sampled) every 1 minute. As such, in the report, you
should see a new row entry every 1 minute. Consequently, the time difference between
consecutive rows will also be approximately 1 minute.
 Tip: Some values on the Buffer Usage Monitor report are 'delta' values; that is, they are
calculated using: [new value – old value] / [Data Rate Interval in Seconds]
Audit Process Fundamentals

While most of the metrics on the Buffer Usage Monitor report are operating system-related
(e.g. CPU usage, memory usage, disk usage), there are many that relate to specific components
(steps) within the overall Guardium® collection/inspection process. Let us review this overall
process and tie all of these concepts together.

The overall Guardium® collection/inspection process consists of four main components:

■ Sniffer (snif): captures traffic from the network interfaces and applies filters and
policies at the TCP/IP level.

■ Analyzer1: verifies packet sequencing, network protocol, session state, db-type, etc.

■ Parser2: parses database SQL into highly normalized constructs (down to the field and
value level), and applies real-time alerts and policies

■ Logger: inserts parsed (normalized) activity into the internal MySQL database

As a result of this process, several files are created within the collector appliance specifically
with regards to the Buffer Usage Monitor:

■ snif_buf_usage.txt: created as a result of the overall collection/inspection process.

1 There is one Analyzer per database type.

2 Parsers are database-specific
 This file is updated by several Guardium® internal components at the frequency
specified by the Data Refresh Interval (i.e. monitor buffer usage interval). All of the
fields on the Buffer Usage Monitor report related to the 'sniffer|analyzer|parser|logger'
are actually parsed from this file.

■ sniff_buffer_usage.log: serves as an error log for the metrics calculation process.

Specifically, if a metric cannot be parsed from the snif_buf_usage.txt file, or
subsequently, cannot be inserted into the Sniffer Buffer Usage entity, then an error will
be written to this log.

■ snif_ouput.txt3: serves as a STDOUT file for the overall collection/inspection process.

Output entries related to the analyzer and application of policies will be written here.

■ snif_stderr.txt4: serves as a STDERR file for the overall collection/inspection process.

Output entries related to heartbeat, tap-starts, etc. will be written here.

Tip: The snif_buf_usage.txt file is accessible for viewing in a variety of ways. You can always
demonstrate or view the updates to this file (every interval) for debugging.

Analysis

Now that we have a good idea on how the Buffer Usage Monitor report metrics are updated
3 Filenames may appear as snif_output.txt.# (these are rollover log files)
4 Filenames may appear as snif_stderr.txt.# (these are rollover log files)
 and generated, it is time to examine how each field on the report is actually calculated.

Note: Due to the sheer size and complexity of the report, the explanations for each field have
been placed in a separate document:

■ Buffer Usage Monitor – Detailed Analysis: an external spreadsheet file describing

each of the report fields, along with units, formatting, and extra explanations relating to
how the field value was calculated (i.e. Linux commands or snif_buf_usage.txt parsing)

Buffer Usage Monitor Report

Please see the external spreadsheet document entitled:

“Buffer Usage Monitor – Detailed Field Analysis.”

Enterprise Buffer Usage Monitor Report

As previously noted, this report represents the same values as the Buffer Usage Monitor, except
values are the aggregate of all metrics from the individual collectors. However, when
viewing the report, you will notice that the field names (column headers) are different. As
such, please see the mapping table in the external spreadsheet, “Buffer Usage Monitor –
Detailed Field Analysis” to see which fields on the Enterprise Buffer Usage Monitor report are
equivalent to those on the Buffer Usage Monitor report.
Additional Debugging

In this section we will explore additional methods of obtaining Buffer Usage Monitor metrics,
and collection/inspection process information.

Changing Sniffer State

To change the state of the guardium sniffer process (i.e. enable/disable), we can utilize the
following CLI command:

■ CLI> start inspection-core

■ CLI> stop inspection-core

Debugging the Sniffer Process

To view additional messages regarding the sniffer, we can utilize the remote syslog forwarding
capabilities as described in presentation 4.7, Advanced Monitoring:
Tip: In the forwarded (remote) syslog messages, search for the 'guard_sniffer' string in each
log entry.

Viewing Sniffer Buffer Usage

There are actually multiple ways to view the snif_buf_usage.txt file on which many of the
Buffer Usage Monitor report values are based:

■ CLI> show buffer

■ CLI> fileserver

After issuing this command, navigate to the URL specified on the CLI using your Web
browser. Next, click on the SQLguardlogs link

Find and click on the snif_buf_usage.txt file:

A sample screen-shot of the file has been shown below:
 Tip: You can view the updates to the file simply by clicking the refresh button in your
Web browser.
Diag Utility

We can view a live output of the buffer (snif_buf_usage.txt) via:

diag → 3 System Interactive Queries → 11 Watch Buffer
Tip: We can also utilize the SLON utility via the CLI diag command, to debug the sniffer and
analyzer processes.

System View

We can also get self-monitoring information about the collector by navigating to the following
page in the Guardium® Web interface: System View → Current Status Monitor
The metrics at the top of the diagram are basically an output of the Linux 'vmstat' command,
while individual inspection engine metrics are recorded according to database type.

Format: Queued-Dropped
Processed