Critical Logs to

Monitor: A
Guide for SOC
Analysts

Wojciech Ciemski
https://www.linkedin.com/in/wojciech-ciemski/
Table of Contents
1. Introduction ......................................................................... 3
1.1. Importance of Log Monitoring in SOC ............................................................................ 3
1.2. Scope and Purpose of the Guide ................................................................................... 4

2. Key Types of Logs .................................................................. 6

2.1. System Logs (Windows, Linux, macOS .......................................................................... 6
2.2. Network Logs (Firewall, Router, IDS/IPS) ....................................................................... 9
2.3. Application and Database Logs .................................................................................. 15
2.4. Security Logs (AV, EDR, XDR) ...................................................................................... 19
2.5. Cloud Logs (AWS, Azure, GCP) and Container Logs (Docker, Kubernetes) ..................... 23
2.6. IoT/SCADA/OT Logs .................................................................................................... 28

3. Key Monitoring Practice ....................................................... 33

3.1. Detecting Anomalies and Incidents (Alerts, Correlation) .............................................. 33
3.2. Log Retention and Security ......................................................................................... 34
3.3. Supporting Tools (SIEM, SOAR) ................................................................................... 35

P a g e 2 | 36
1. Introduction
Logs are the footprints of every digital activity, serving as a chronological record of events within
systems, networks, and applications. In a modern Security Operations Center (SOC), analysts
rely on these logs to detect threats, investigate security incidents, and maintain an
organization’s overall security posture. Without structured and well-monitored logs, even the
most advanced security solutions can miss critical indicators of compromise (IoCs) or fail to
correlate suspicious behaviors across multiple systems. The goal of this guide is to highlight
which logs matter most, why they are essential, and how to approach their monitoring in a way
that benefits both junior and mid-level SOC analysts.

1.1. Importance of Log Monitoring in SOC

In any sizable IT infrastructure, raw data volumes can be massive—firewalls alone can generate
thousands of log entries per second. While these logs may sometimes appear as unremarkable
lines of text, they hold valuable insights that help detect and counteract security threats. Proper
log monitoring is crucial for several reasons:

1. Visibility and Context

Logs provide context by showing what happened, when it happened, and how it was
executed. This visibility is essential in distinguishing normal behaviors from anomalies.
For example, an unexpected privilege escalation in a Windows system log can point to
lateral movement by an attacker. Similarly, repeated authentication failures in a Linux
environment might indicate a brute-force attack.

2. Incident Detection and Response

Automated alerts from a SIEM (Security Information and Event Management) platform
often originate from suspicious patterns in logs. These alerts enable SOC analysts to
quickly identify and respond to potential incidents. For instance, correlation rules might
flag a user logging in from two geographically distant locations within a short timeframe,
suggesting a stolen credential.

3. Audit and Compliance

Many regulatory frameworks, such as PCI DSS, HIPAA, or ISO 27001, mandate log
retention and regular review. By monitoring logs, organizations ensure they meet
compliance requirements and can produce a clear audit trail during investigations or
audits. Logs are often the first place auditors check to confirm security controls are in
place and functioning as intended.

4. Threat Hunting
Beyond detection, logs form the basis for proactive threat hunting. Analysts look for
unusual patterns—like the execution of a PowerShell script in an environment where
PowerShell usage is rare—to uncover stealthy attacks. By analyzing logs over time,
threat hunters can identify trends and adversary tactics that might be missed by
automated systems alone.

5. Forensic Investigations
When an incident does occur, well-structured logs are the key to forensic investigations.
They help recreate the timeline of an attack, show which systems were accessed, and
highlight the data that was exfiltrated. Detailed logs of user actions, network

P a g e 3 | 36
 connections, and system calls can be the di erence between accurately attributing an
incident and letting attackers remain undetected.

Real-World Example
Consider a scenario where a SOC analyst notices unusual outbound tra ic from a critical
server. By reviewing firewall logs correlated with Windows Event Logs, the analyst uncovers a
malicious process communicating with an external IP address. Quick analysis shows that the
communication began right after a suspicious privilege escalation event. This correlation can
guide incident response teams to isolate the server, contain the threat, and remediate the
vulnerability before data is compromised.

Practical Tip
On Linux systems, commands like journalctl -p warning -r can help you quickly locate higher-
priority events in reverse chronological order, allowing faster triage of potential security issues.
On Windows, tools such as wevtutil qe Security /rd:true /f:text /q:"*" | findstr /i "4624 4625 4634
4672" can filter the security event log for specific Event IDs related to logons.

1.2. Scope and Purpose of the Guide

This guide targets both new and mid-level SOC analysts who want to enhance their skills in log
monitoring. It covers common sources of logs—like operating systems, networks, applications,
and security tools—and highlights what to look for in each. By focusing on the most critical logs
and describing how they fit into the overall security strategy, this guide aims to streamline the
day-to-day work of SOC professionals. Specifically, it aims to:

 Identify Key Log Sources

We will look at system logs (Windows, Linux, macOS), network logs (firewall, router,
IDS/IPS), application and database logs, security logs (AV, EDR, XDR), cloud logs
(AWS, Azure, GCP), container logs (Docker, Kubernetes), and IoT/SCADA/OT logs. The
primary focus is on what makes each category critical, how to collect them, and which
events are most indicative of a security issue.

 Present Practical Monitoring Techniques

From correlation rules to anomaly-based detection, we will discuss the practices that
translate raw logs into actionable insights. We will also cover topics like log retention,
log security, and best practices around data classification to ensure that sensitive logs
remain protected.

 Showcase Real-Life Use Cases

Each log type comes with its unique set of challenges and attack vectors. We will walk
through realistic scenarios—such as detecting lateral movement, privilege escalation,
or malicious uploads—and demonstrate how the logs serve as vital evidence.

 Guide on Supporting Tools

SIEM (Security Information and Event Management) and SOAR (Security Orchestration,
Automation, and Response) tools are at the heart of modern SOCs. The guide explains
how these platforms integrate with di erent log sources, automate alerting, and help
orchestrate response actions.

P a g e 4 | 36
  Encourage Continuous Learning
Cyber threats evolve rapidly, and so do best practices in logging and monitoring. With
references to resources like NIST SP 800-92 (Guide to Computer Security Log
Management) and o icial vendor documentation (e.g., Microsoft’s Windows Event Log
documentation), this guide points readers to reliable sources for ongoing education.

By focusing on these areas, the guide aims to equip analysts with the knowledge and skills to
prioritize logs e ectively and detect potential breaches before they escalate. Through a mix of
theoretical explanation and practical examples, readers will gain confidence in setting up
logging strategies, tuning alerts, and conducting thorough investigations.

P a g e 5 | 36
2. Key Types of Logs
2.1. System Logs (Windows, Linux, macOS
System logs form the backbone of incident detection and response e orts, providing analysts
with the essential baseline data needed to investigate abnormal events, track user activities,
and diagnose security threats. Across Windows, Linux, and macOS, these logs share the
common goal of recording key operating system (OS) events, though each platform organizes
and structures logs in its own way. Understanding how they work, what they log, and how to
interpret them is crucial for SOC analysts.

Windows Logs
Common Log Sources

 System Log: Captures events generated by the Windows operating system and its built-
in services. It records driver issues, service startups and shutdowns, and kernel-level
messages.

 Application Log: Stores application-specific events, such as errors, warnings, or

informational messages from software installed on the system (e.g., database clients,
productivity tools).

 Security Log: Focuses on security-related events: login attempts, account lockouts,

and user right assignments. Often used for auditing and forensic investigations.

 Other Logs: Windows also creates dedicated logs for specialized services, like DFS
Replication and PowerShell, which can be viewed under the Applications and
Services Logs in the Event Viewer.

Practical Monitoring Tips

1. Event Viewer: Built into Windows, Event Viewer o ers a quick way to view and filter
events. Analysts can group events by severity (Critical, Error, Warning, Information) or by
Event ID.

2. Filtering and Searching: Use XML filtering in Event Viewer or PowerShell commands to
hunt for specific event IDs (e.g., 4624 for successful logins, 4625 for failed logins).

3. Security Baselines: Monitor high-value Event IDs. For example:

o 4624 (Successful account login)

o 4625 (Failed login)

o 4672 (Special privileges assigned to a user)

o 4688 (A new process has been created)

o 4648 (A logon was attempted using explicit credentials)

4. PowerShell Logging: By enabling Module Logging and Script Block Logging, analysts
can track suspicious or obfuscated commands. Refer to Microsoft Docs (PowerShell
Logging) for guidelines.

P a g e 6 | 36
Example: Filtering Security Events via PowerShell
Get-WinEvent -LogName Security | Where-Object {
$_.Id -in 4624, 4625
}

This command pulls Security Log events for successful and failed logins, enabling quick
detection of abnormal activity.

Linux Logs
Syslog and Journald

Most Linux distributions rely on syslog or systemd-journald to collect and manage log
messages:

 /var/log/syslog or /var/log/messages: Contains informational and non-critical system

events.

 /var/log/auth.log or /var/log/secure: Focuses on authentication-related messages.

Essential for detecting brute-force login attempts, sudo activity, or SSH logins.

 /var/log/kern.log: Stores kernel-level messages, useful for diagnosing driver issues or

unusual kernel events.

 Journal Logs (systemd-based distros): Consolidates logs in a binary format, accessible

via journalctl.

Key Areas to Monitor

1. Authentication: Watch for repeated failed login attempts, new user additions in
/etc/passwd, or sudden changes in sudo usage.

2. Cron Jobs: Check /var/log/cron or associated logs for unauthorized scheduled tasks.
Cron jobs can be used by adversaries for persistence.

3. Kernel Messages: Investigate repeated kernel warnings or errors that could indicate
hardware issues or potential rootkit activity.

4. Service Logs: For services like Apache, Nginx, or SSH, monitor dedicated logs (e.g.,
/var/log/apache2/access.log, /var/log/nginx/access.log, /var/log/secure) for unexpected
tra ic or repeated authentication failures.

Example: Using Journalctl

# View all logs related to SSH
journalctl -u sshd
# Filter logs for a specific time range
journalctl --since "2023-01-01" --until "2023-01-31"

This approach helps analysts quickly search for anomalies within a particular service or
timeframe.

P a g e 7 | 36
macOS Logs
Unified Logging System

Since macOS Sierra (10.12), Apple introduced a unified logging system that stores log messages
in a structured format:

 Console App: The built-in Console allows viewing of system logs, diagnostic reports,
and crash logs.

 Log Commands: The log utility in the terminal o ers extensive filtering, streaming, and
searching capabilities. For example:

# View live log messages (system-wide)

log stream --level=info

# Search for specific processes or errors

log show --predicate 'process == "sshd" AND eventMessage
CONTAINS "Failed password"'

 Subsystems and Categories: macOS logs categorize messages by subsystem (e.g.,

com.apple.networking) and category (e.g., connection). This helps analysts narrow
down events.

Security-Specific Logs

 /var/log/system.log: Retains many core system messages and is often the first stop
when troubleshooting.

 Apple System Log (ASL): Legacy logging that coexists with the unified logging system,
accessible via command-line tools for older macOS versions.

 Authentication Logs: Attempts to log in via SSH or local accounts can appear in
/var/log/asl/ or through the unified logging interface.

Monitoring and Detection

1. Focus on Repeated Failures: Like Linux, repeated failed SSH attempts or unexpected
process launches warrant attention.

2. Check Crash Reports: Attackers sometimes induce crashes of security tooling. Crash
logs in macOS can provide early indicators of tampering.

3. Leverage Built-in Tools: Use the Console to filter logs by Process or Message Type.
Apple’s Developer documentation on Unified Logging provides details on advanced
usage.

Cross-Platform Considerations
Aspect Windows Linux macOS

Event Viewer (System, /var/log/syslog, Unified Logging System

Log Files
Security, etc.) /var/log/auth.log, etc. (log show, log stream)

P a g e 8 | 36
Aspect Windows Linux macOS

Common PowerShell, Event

tail, grep, awk, journalctl Console App, log CLI
Tools Viewer, WMI

SSH failures, privilege SSH failures, system

Event IDs (4624, 4625,
Alert Focus escalations, systemd crashes, unexpected
etc.), policy changes
service errors subsystem messages

Windows Event Export logs via the log

Rsyslog, Syslog-ng,
Centralization Forwarding (WEF), collect feature or
systemd-journald to SIEM
Sysmon logs to SIEM streaming to a SIEM

Logging Agents and Centralization

Many organizations opt to forward Windows, Linux, and macOS logs to a central SIEM or log
management platform:

 Windows: Windows Event Forwarding (WEF), Sysmon for detailed process-level logging,
or third-party agents like NXLog or Splunk Universal Forwarder.

 Linux: Rsyslog, Syslog-ng, or systemd-journald can forward logs to remote servers.

Beats (Filebeat, Metricbeat) from Elastic can also collect and ship logs.

 macOS: Use third-party agents (e.g., Osquery for query-based logging, or Splunk,
Datadog agents) to unify logs under a single pane.

2.2. Network Logs (Firewall, Router, IDS/IPS)

Logs from network devices and security systems are among the most critical data sources in a
Security Operations Center (SOC). By analyzing firewall, router, and IDS/IPS logs, SOC analysts
gain visibility into tra ic patterns, security events, and potential anomalies. This visibility is
crucial for identifying malicious behavior early and for responding to incidents before they can
propagate within the environment. Below are the key concepts, best practices, and real-world
scenarios that illustrate how to work e ectively with network logs.

1. Firewall Logs
Firewalls are often the first line of defense, filtering tra ic based on predefined rules. Monitoring
firewall logs provides insights into both permitted and denied network connections.

1.1. Common Fields in Firewall Logs

Typical firewall logs will include fields such as:

 Timestamp: The date and time the event was recorded.

 Source IP / Destination IP: IP addresses of the client and server.

 Source Port / Destination Port: Ports used by the services or applications

communicating.

 Protocol: Network protocol in use (e.g., TCP, UDP, ICMP).

 Action: Indicates whether the tra ic was allowed, denied, dropped, or rejected.
P a g e 9 | 36
  Rule or Policy Name: Identifies which firewall rule triggered the log entry.

Firewalls may also log additional details like interface names (e.g., eth0, WAN, LAN), packet
size, or the reason for a deny or reject action. Modern firewalls, especially Next-Generation
Firewalls (NGFWs), can log application-level data and user information if integrated with identity
management systems.

Field Description Example Value

Timestamp Date and time of event 2025-01-25 10:15:32

Source IP Originating IP address 192.168.10.5

Destination IP Target IP address 10.0.5.20

Source Port Originating TCP/UDP port 53452

Destination Port Target TCP/UDP port 443

Protocol Network protocol (TCP, UDP, ICMP) TCP

Action Allowed, denied, dropped, etc. Allowed

Rule Name Firewall policy or rule name Block_Telnet

1.2. Practical Use Cases

 Blocked Connection Attempts: Monitoring repeated connection attempts on sensitive
ports (e.g., 22 for SSH or 3389 for RDP) can reveal brute-force attempts or port scans.

 Unusual Tra ic Volumes: A sudden spike in tra ic from a single IP or subnet might
indicate a DoS or DDoS attempt.

 Inbound vs. Outbound Monitoring: Outbound connections to suspicious IP addresses

or countries where the organization does not conduct business can be early indicators
of compromised hosts (e.g., malware calling home).

1.3. Example Firewall Log Analysis in SIEM

Below is an example Splunk query that filters for denied connections with a focus on TCP port
3389 (RDP):
index=firewall_logs action=DENY dest_port=3389
| stats count by src_ip, dest_ip, action, rule_name

This query helps highlight any source IPs that are repeatedly trying to reach RDP services but are
being denied, which might indicate an attempted intrusion.

2. Router Logs
Routers primarily forward packets between networks and maintain routing tables. Logs
generated by routers often focus on system messages, routing updates, and interface errors
rather than application-specific data. However, they are still critical for overall visibility,
especially in environments with distributed architectures.

P a g e 10 | 36
2.1. Types of Router Logs
 System or Event Logs: Includes messages about device reboots, software crashes, or
configuration changes.

 Routing Protocol Logs: Information related to BGP, OSPF, EIGRP, or other routing
protocols.

 Interface Logs: Status changes on interfaces (up/down), packet errors (CRC errors,
collisions), and bandwidth usage.

 Authentication Logs: Successful or failed logins via SSH, Telnet, or console access to
the router.

Router logs often follow the standard Syslog format (e.g., Cisco routers with severity levels 0–7).
Integrating these logs into a SIEM allows analysts to correlate network topology changes with
security events (for instance, if a router interface goes down just before a security incident on
that segment).

2.2. Example: Cisco Router Syslog Messages

Cisco routers send Syslog messages with various severity levels. An example message might
look like this:
<189>Jan 25 10:25:10 MY-ROUTER: %LINK-3-UPDOWN: Interface
GigabitEthernet0/1, changed state to up

 189 corresponds to the Syslog priority.

 %LINK-3-UPDOWN indicates a link status change with severity level 3 (Error).

 The message indicates which interface changed state.

In a SIEM, you might filter for %LINK-3-UPDOWN events to track unexpected interface state
changes. If an interface goes down suddenly, it may indicate a physical issue, misconfiguration,
or malicious activity aiming to disrupt network segments.

3. IDS/IPS Logs
Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS) monitor network
tra ic for signs of malicious behavior, policy violations, or known attack signatures. While
firewalls typically operate at the transport or network layer, IDS/IPS solutions can inspect
packets in more depth (Layer 7), providing richer context about application-level threats.

3.1. IDS vs. IPS

 IDS (Intrusion Detection System): Detects potential threats and generates alerts. It
does not automatically block the tra ic.

 IPS (Intrusion Prevention System): Detects threats and can take preventive actions,
such as dropping malicious packets or blocking IP addresses in real-time.

3.2. Common Fields in IDS/IPS Logs

 Signature ID: A unique identifier for the rule or signature triggered (e.g., Snort rules have
SID values).

P a g e 11 | 36
  Event or Alert Message: The name or description of the suspicious activity (e.g., “ET
TROJAN Zeus Tracker”).

 Severity or Priority: Indicates the criticality of the alert.

 Source IP / Destination IP / Ports: Information about the tra ic flow.

 Action: Whether the tra ic was detected, dropped, or allowed.

3.3. Practical Example with Suricata

Suricata is a popular open-source IDS/IPS engine. Suricata outputs JSON logs that can be
ingested by SIEM tools like Elasticsearch or Splunk. A typical Suricata alert entry in JSON format
might include:
{
"timestamp": "2025-01-25T10:30:45.123456+0000",
"flow_id": 1234567890,
"event_type": "alert",
"src_ip": "192.168.1.100",
"src_port": 53452,
"dest_ip": "10.0.5.20",
"dest_port": 80,
"proto": "TCP",
"alert": {
"action": "blocked",
"gid": 1,
"signature_id": 2010935,
"rev": 3,
"signature": "ET TROJAN Known Malicious Domain",
"category": "Trojan Activity",
"severity": 2
}
}

In the above example:

 signature_id: 2010935 corresponds to a Suricata rule ID that references a specific

Trojan signature.

 action: The tra ic was blocked by Suricata (IPS mode).

 category: “Trojan Activity” indicates the type of threat.

P a g e 12 | 36
3.4. Real-Life Attack Scenarios
 SQL Injection Attempts: IDS/IPS solutions look for patterns in HTTP requests that
match known SQL injection techniques.

 Exploit Kits: If a host attempts to download or connect to an exploit kit domain, IDS/IPS
logs can reveal the suspicious domain name and signature match.

 Lateral Movement: Attackers may try to move horizontally within a network. IDS/IPS
can detect unusual SMB or RDP tra ic patterns.

4. Parsing and Analyzing Network Logs in Practice

4.1. Log Management and SIEM Integration
SOC analysts typically centralize firewall, router, and IDS/IPS logs into a SIEM for correlation and
analysis. This allows cross-referencing of events from multiple sources. For instance, if an IDS
alert indicates a Trojan signature and the firewall logs show outbound tra ic to a suspicious IP,
the SIEM can generate a higher-priority alert.

Example Logstash configuration snippet to parse Suricata JSON logs:

input {
file {
path => "/var/log/suricata/eve.json"
type => "suricata"
codec => "json"
}
}
filter {
if [event_type] == "alert" {
mutate {
add_tag => ["suricata_alert"]
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "suricata-alerts-%{+YYYY.MM.dd}"
}
}

P a g e 13 | 36
This configuration reads Suricata’s eve.json file, filters for alert events, and then tags them as
suricata_alert before sending them to Elasticsearch.

4.2. Correlation Rules

In a SIEM, correlation rules can look for conditions such as:

1. High Volume of Denied Connections: If more than 100 firewall denies occur from the
same source IP in 5 minutes, generate an alert.

2. Multiple IDS Alerts for the Same Host: If a host triggers more than 3 di erent IDS
signatures within a short period, raise the priority of the incident.

3. Router Interface Down + IDS Alerts: If a critical interface goes down and multiple IDS
alerts are detected on adjacent network segments, investigate potential sabotage or
widespread compromise.

By creating correlation rules that combine di erent log types, SOC analysts can detect
coordinated attacks and reduce the volume of false positives.

5. Challenges and Best Practices

 Log Volume: Network devices can generate a massive amount of data. Using filters or
sampling may be necessary, but be cautious not to discard important information.

 Normalization: Di erent vendors (Cisco, Palo Alto, Fortinet, etc.) often have unique log
formats. Normalizing fields (e.g., ensuring consistent naming of src_ip, dest_ip) is
crucial for e ective correlation.

 Encryption and Secure Transport: Ensure that log data is transmitted securely, for
instance using TLS for Syslog (Syslog over TLS). Unencrypted logs can be intercepted
and manipulated by adversaries.

 Regular Tuning: IDS/IPS rulesets need regular updates to reflect new threats. Similarly,
firewall policies should be reviewed to ensure they align with the evolving network
environment.

 Time Synchronization: NTP (Network Time Protocol) should be enabled and correctly
configured on all devices to maintain consistent timestamps. Accurate timestamps are
critical for event correlation.

6. References and Further Reading

 O icial Suricata Documentation: https://suricata-ids.org/docs/

 Snort (IDS/IPS) Documentation: https://www.snort.org/

 Cisco Syslog Guide: https://www.cisco.com/c/en/us/support/docs/security-

vpn/syslog/

 Firewall Best Practices (NIST):

https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-41r1.pdf

These sources provide authoritative insights into configuration, logging standards, and threat
detection patterns for network devices and security solutions.

P a g e 14 | 36
2.3. Application and Database Logs
Application logs capture events and behaviors tied directly to an application’s functionality.
They may include user interactions, system operations, exceptions, debug details, and custom
events defined by developers. Database logs, on the other hand, record all actions related to
data transactions, schema changes, authentication, and potential errors or performance
bottlenecks in a database system. Together, these logs provide a holistic view of how software is
functioning and how data is being accessed or manipulated. This is crucial for detecting
unauthorized activities, performance issues, and other anomalies. Below are the key
considerations, best practices, and real-world examples to help you e ectively monitor and
analyze both application and database logs.

Understanding Application Logs

Common Types of Application Logs

1. Error and Exception Logs

These capture unexpected behaviors and stack traces. They’re typically generated by
frameworks such as Java’s Log4j or Logback, Python’s logging library, or .NET’s built-in
logging features.

2. Debug and Diagnostic Logs

These logs detail internal operations, often containing fine-grained information used
during development or troubleshooting. Debug logs can be extremely verbose, so
they’re typically enabled only in test or development environments unless a production
issue requires deeper insights.

3. Transaction or Event Logs

Applications that handle user transactions—such as e-commerce checkouts—often
generate transaction logs. These logs detail each step in the user flow (e.g., adding items
to a cart, checking out, payment processing).

4. Audit Logs
Some applications produce audit logs for compliance or security reasons. These logs
track user access, role changes, or critical configuration updates, helping you see who
did what and when.

Logging Frameworks and Formats

Modern applications often rely on standardized logging frameworks:

 Java (Log4j, Logback)

 .NET (Serilog, NLog)

 Python (logging)

 Node.js (winston, pino)

These frameworks allow developers to configure log levels, structure log messages (for
example, in JSON), and specify output targets like console, files, or external aggregation
services. Consistency in format is important for parsing and correlation within a SIEM.

P a g e 15 | 36
Example Configuration (Log4j2 in Java)
<Configuration status="warn">
<Appenders>
<File name="FileLogger" fileName="logs/application.log">
<PatternLayout pattern="%d{ISO8601} [%t] %-5level %logger{36}
- %msg%n" />
</File>
</Appenders>
<Loggers>
<Logger name="com.example.app" level="info" additivity="false">
<AppenderRef ref="FileLogger"/>
</Logger>
<Root level="error">
<AppenderRef ref="FileLogger"/>
</Root>
</Loggers>
</Configuration>

This snippet specifies an application.log file, using a pattern to log timestamps, thread names,
log levels, and the actual message. By setting the root level to error and the application logger to
info, you can avoid unnecessary noise.

Monitoring Application Logs in Practice

1. Establish Baseline Behavior
Knowing the normal application behavior (e.g., average response times, typical error
rates) helps detect anomalies, such as a sudden influx of specific exceptions that might
indicate an attack or misconfiguration.

2. Look for Common Attack Patterns

Unauthorized access attempts often show up as repeated login failures, suspicious
parameters in URLs (e.g., SQL injection probes), or unusual behaviors in session
management. Web applications might log 404 errors or suspicious HTTP methods in
higher frequency under attack.

3. Integration with SIEM

To correlate application logs with system or network events, use a SIEM tool like Splunk,
IBM QRadar, or Elastic Security. For example, by correlating an application’s “multiple
failed logins” event with a firewall log showing suspicious IP scanning, you can quickly
confirm or rule out an intrusion attempt.

4. Alerting and Thresholds

Threshold-based alerts on error rates, transaction volume drops, or spikes in exceptions

P a g e 16 | 36
 can catch incidents early. Machine learning-driven anomaly detection in tools such as
Azure Sentinel or AWS Security Hub can further refine alerts by identifying patterns not
captured by static rules.

5. Retention Policies
Due to volume, application logs can grow quickly. You need to define retention policies
balancing security requirements and storage costs. Compliance frameworks (e.g., PCI
DSS, HIPAA) sometimes dictate minimum retention periods for specific log types.

Understanding Database Logs

Key Types of Database Logs

1. Transaction Logs
Capture every change to the database’s data. They’re critical for recovery and forensic
analysis. For instance, in Microsoft SQL Server, the transaction log tracks every
modification in the order they occur, enabling point-in-time recovery.

2. Error Logs
These highlight critical events, such as server startup issues or serious errors that a ect
database availability. Examples include MySQL’s error.log or Oracle’s alert logs.

3. General Query Logs (MySQL) / Audit Logs (Various Vendors)

Record all queries received by the server or track account activity. They are invaluable in
detecting SQL injection attacks, suspicious data extraction, or attempts at privilege
escalation.

4. Slow Query Logs

Found in MySQL or PostgreSQL, these capture queries that exceed a certain execution
time threshold. Slow queries might indicate performance issues or potential denial-of-
service attempts if queries are being manipulated by an attacker.

Practical Monitoring Strategies

1. Regular Review for Suspicious Queries
Monitor queries that drop or alter critical tables without expected change-control
tickets. Also look for wildcard searches or large data extracts happening at unusual
hours.

2. Privilege Abuse Detection

If a user with minimal privileges starts running queries typical of an administrator, it’s a
strong indicator of compromised credentials or privilege escalation. Enforcing least
privilege and then reviewing logs for anomalies is a potent strategy.

3. Error Pattern Analysis

Repeated database error messages, such as “invalid column name” or “syntax error,”
can indicate attempts at SQL injection. SOC analysts can configure SIEM correlation
rules to flag repetitive errors from a single source IP.

4. Performance Data
Logs that point to high resource usage or timeouts can be an early warning of a brute-
force or denial-of-service attack at the database layer.

P a g e 17 | 36
Example Configurations and Queries
MySQL
To enable the general query log:
SET GLOBAL general_log = 'ON';
SET GLOBAL general_log_file = '/var/log/mysql/general.log';
To enable the slow query log:
SET GLOBAL slow_query_log = 'ON';
SET GLOBAL long_query_time = 2; -- Queries taking longer than 2
seconds will be logged

Note: Logging all queries can significantly impact performance, so only enable it temporarily for
diagnostics or funnel logs to a centralized system where you can parse and analyze them
e iciently.

PostgreSQL
PostgreSQL has extensive logging configurations in postgresql.conf. For instance:
logging_collector = on
log_directory = 'pg_log'
log_filename = 'postgresql-%a.log'
log_statement = 'all'
log_min_duration_statement = 2000 # logs queries over 2ms

By setting log_statement to all, you can see every statement, though this is typically too verbose
for production.

Real-World Scenarios
 Detection of Data Exfiltration
A SOC analyst notices an application log showing unusual parameter values in a REST
API call. By cross-referencing the database logs, the analyst confirms multiple large
SELECT statements retrieving sensitive customer data. Additional correlation with
network logs shows a large data transfer to an external IP. The logs collectively point to
an ongoing data exfiltration attempt.

 Auditing for Compliance

In a financial services application, compliance requirements mandate auditing every
transaction. By reviewing application logs (which capture the logic layer) and the
database’s transaction logs (which capture the final record changes), auditors can
confirm that every deposit or withdrawal is authorized and properly executed.

 Identifying Performance Attacks

A series of slow queries might initially look like a performance bottleneck. However,
investigating further reveals that attackers are intentionally crafting resource-intensive
queries to degrade the application’s responsiveness. Alerts in the SIEM correlate these
slow queries with repeated 503 errors on the web server, confirming a denial-of-service
attempt.
P a g e 18 | 36
Additional Resources
 OWASP Cheat Sheet Series: https://cheatsheetseries.owasp.org/
O ers guidelines on secure logging practices, specifically around sanitizing logs and
preventing log forging.

 MySQL O icial Docs: https://dev.mysql.com/doc/

Detailed instructions on configuring error logs, general query logs, and slow query logs.

 PostgreSQL Documentation: https://www.postgresql.org/docs/

Contains comprehensive configuration guides for logging and auditing features.

 Microsoft SQL Server Docs: https://docs.microsoft.com/en-us/sql/

Explains how to manage and interpret transaction logs, error logs, and other diagnostic
data.

 Oracle Database Docs: https://docs.oracle.com/en/database/

Provides details on the alert log, trace files, and advanced auditing configurations.

Comparisons and Data

Log Type Examples Typical Use Case Potential Security Insight

Frequent exceptions might

Debugging application
Stack traces, code hint at malicious input or
Error/Exception crashes, identifying
line references attempts to exploit
faulty modules
vulnerabilities

Real-time monitoring helps

E-commerce logs, Auditing success/failure
Transaction detect fraudulent
banking transactions of critical actions
transactions

User actions, role Compliance with Pinpointing unauthorized

Audit (App &
changes, schema regulations, admin actions or privilege
DB)
modifications accountability escalations

Identifying possible DoS

Queries exceeding a Performance tuning or
Slow Query attempts or resource
time threshold bottleneck analysis
exhaustion attacks

When collecting and analyzing these logs, consider normalizing fields (timestamps, user IDs,
hostnames) so di erent log sources can be correlated e ectively. Some SIEM platforms or
centralized logging solutions (e.g., the ELK Stack) allow you to define common field mappings
and dashboards that unify application and database insights.

2.4. Security Logs (AV, EDR, XDR)

Security logs generated by Antivirus (AV), Endpoint Detection and Response (EDR), and
Extended Detection and Response (XDR) solutions are crucial for modern SOC operations. They
o er granular visibility into potential threats a ecting endpoints and the broader environment.
Below is an exploration of the fundamentals, along with real-world examples and guidance for
e ective log monitoring.

P a g e 19 | 36
Understanding the Components
Antivirus (AV) Logs
Antivirus solutions focus primarily on detecting known malware signatures and blocking
suspicious files. Their logs typically include:

 Malware Detections: Alerts triggered when a file matches a known signature or exhibits
malicious behavior.

 Quarantine and Remediation Actions: Logs showing which files were quarantined,
deleted, or otherwise neutralized.

 Update and Scan Events: Records of signature updates, scheduled scans, and on-
demand scan results.

Real-World Example
A traditional AV tool like Microsoft Defender Antivirus (part of Windows Security) generates
logs under the Windows Event Log:

 Event ID 1116 indicates malware detection.

 Event ID 5001 logs the scanning engine starting up.

By aggregating these event IDs in a SIEM, analysts can quickly see patterns of infection
attempts and confirm that updates have been applied.

Endpoint Detection and Response (EDR) Logs

EDR solutions expand on basic antivirus features by providing in-depth endpoint telemetry, real-
time threat detection, and response capabilities. Common log data includes:

 Process Creation and Termination: Detailed tracking of command-line parameters,

user context, and file paths.

 Behavioral Indicators: Observations related to suspicious activities such as code

injection, privilege escalations, or unusual registry modifications.

 Isolation and Response Actions: Logs showing when and why an endpoint was
isolated, network connections were blocked, or an automated script was run for
containment.

EDR logs often present a sequence of correlated events, making it easier for SOC analysts to
reconstruct the timeline of an attack. Tools like CrowdStrike Falcon, SentinelOne, or Carbon
Black o er dashboards that display triggered detection rules (e.g., MITRE ATT&CK techniques)
alongside automated remediation actions.

Sample EDR Log Analysis (Splunk)

Below is an example of how you might parse EDR logs in Splunk to identify suspicious child
processes of PowerShell:
index=edr_logs parent_process=PowerShell.exe
| stats count by child_process, user, host
| where count > 3

P a g e 20 | 36
This query looks for any child process spawned by PowerShell and flags any repeated
occurrences, which might indicate malicious scripts or living-o -the-land techniques.

Extended Detection and Response (XDR) Logs

XDR solutions take the endpoint-centric approach of EDR and extend it to incorporate data from
network appliances, cloud workloads, and applications. The goal is to unify detection,
investigation, and response across multiple layers of the IT environment.

 Cross-Source Correlation: XDR aggregates logs from endpoints, email gateways,

identity providers, and more, applying analytics to uncover hidden threats.

 Cloud and Hybrid Integrations: Telemetry from cloud platforms and containerized
workloads often merges with endpoint data, o ering a complete view of complex
attacks.

 Adaptive Response: Based on machine learning and correlation rules, XDR can trigger
automated playbooks that respond to threats in real time (e.g., disabling compromised
user accounts, isolating infected hosts, or blocking suspicious domains at the firewall).

Reference Architectures
 Microsoft 365 Defender integrates data from endpoints (Defender for Endpoint), email
(Defender for O ice 365), identities (Azure Active Directory), and cloud apps (Defender
for Cloud Apps).

 Palo Alto Cortex XDR processes data from endpoints and integrates with network
sensors or firewalls to provide enhanced correlation.

Log Collection Best Practices

1. Centralize Logs in a SIEM: Consolidate all AV, EDR, and XDR logs into a SIEM platform
like Splunk, Elastic Stack, or IBM QRadar. This ensures a single view for threat hunting
and alert triage.

2. Use Consistent Log Formatting: Where possible, standardize the format (e.g., JSON,
Syslog) to streamline parsing, correlation, and long-term storage.

3. Retain Su icient History: Depending on regulatory requirements and threat modeling,

keep historical logs long enough to investigate slow-moving attacks or advanced
persistent threats.

4. Correlate Across Multiple Sources: Antivirus alerts alone may provide minimal
context. When cross-referenced with endpoint telemetry and user login patterns, they
reveal the bigger picture—especially relevant for advanced or multi-stage attacks.

5. Implement Automated Detection Rules: Leverage built-in detection capabilities of

your EDR/XDR solution and supplement them with custom rules tailored to your
environment. For example, create an alert when a known safe process spawns an
unusual child process (e.g., outlook.exe launching cmd.exe).

6. Leverage Threat Intelligence: Enrich detection events with threat intelligence feeds
(e.g., VirusTotal, AlienVault OTX). This helps validate suspicious activity, especially when
an alert references a known malicious domain or file hash.

P a g e 21 | 36
Common Security Events to Watch For
Event Type Key Indicators Example Tools

Malware File hashes, known signatures, suspicious Microsoft Defender,

Detections file behaviors McAfee, Symantec

Behavioral Unusual registry modifications, abnormal CrowdStrike Falcon,

Anomalies process trees SentinelOne

Privilege Attempts to change user privilege or run

Sysmon + EDR correlation
Escalations processes as admin

Exfiltration Network connections to suspicious Palo Alto Cortex XDR,

Attempts domains or large data transfers Splunk logs

Persistence New services, startup items, scheduled Sysmon + EDR detection

Mechanisms tasks rules

Monitoring these events in near-real time allows SOC analysts to prioritize the highest-risk alerts
and initiate containment actions quickly.

Practical Monitoring Tips

 Track Failed and Successful Remediation Attempts: If an AV tries to quarantine a file
repeatedly but fails, it could be a sign of advanced malware or user tampering.

 Monitor EDR Agent Health: Regularly ensure that EDR agents are running on all
endpoints. Unexpected agent downtime may be an early indicator of an attacker’s
attempt to disable security controls.

 Review Automated Playbook Outcomes: XDR platforms often run automated

responses. Confirm that these responses are both e ective and aligned with your
organization’s incident response procedures.

 Engage With Vendor Documentation: Each AV, EDR, or XDR vendor has specific best
practices for log collection and interpretation. For instance, Microsoft Defender for
Endpoint publishes detailed logging guidelines at Microsoft Docs.

Example Incident Flow Using AV, EDR, and XDR

1. AV Alert: Triggers on a suspicious executable with a known malicious hash.

2. EDR Correlation: Maps the suspicious executable back to a process tree, showing it
was launched by an unusual script.

3. XDR Visibility: Confirms the script was downloaded from an unrecognized domain and
ties this domain to a known threat actor via threat intelligence feeds.

4. Automated Response: XDR or a SOAR platform quarantines the endpoint, blocks the
domain at the firewall, and opens a ticket in the incident management system.

5. SOC Analyst Action: Investigates the entire chain of events, verifies threat removal, and
updates detection rules to prevent similar attacks.
P a g e 22 | 36
By combining the strengths of AV, EDR, and XDR logs in a well-structured monitoring strategy,
SOC analysts can respond swiftly to a wide range of threats—from commodity malware to
sophisticated, persistent attacks.

2.5. Cloud Logs (AWS, Azure, GCP) and Container Logs (Docker,
Kubernetes)
Cloud platforms and container orchestration systems have become an essential part of many
organizations. In a SOC environment, monitoring logs from these platforms is critical for threat
detection, compliance, and troubleshooting. Below is an overview of the most important log
sources and practical considerations for AWS, Azure, GCP, Docker, and Kubernetes.

AWS Logs
Common Log Types
1. CloudTrail Logs

o Purpose: Track API calls and account activity across AWS services.

o Key Fields: eventName, eventSource, awsRegion, sourceIPAddress, userAgent,

requestParameters, responseElements.

o Use in Security: Identifies suspicious or unauthorized actions, such as

unexpected changes to IAM policies, creation or deletion of critical resources, or
unusual console logins.

2. CloudWatch Logs

o Purpose: Centralized logging for AWS services (EC2 system logs, Lambda
function logs, etc.).

o Key Fields: Vary depending on service-specific events; typically include

timestamps, log level (ERROR, WARNING, INFO), and custom application
messages.

o Use in Security: Helps correlate system-level events with higher-level activities.

Example: correlating an EC2 instance’s system error logs with an unauthorized
access attempt shown in CloudTrail.

3. VPC Flow Logs

o Purpose: Capture network flow information (source/destination IP, ports, tra ic

acceptance/rejection).

o Key Fields: version, account-id, interface-id, srcaddr, dstaddr, srcport, dstport,

protocol, action, log-status.

o Use in Security: Identifies unusual tra ic patterns or data exfiltration attempts,

such as large outbound data transfers or tra ic from unknown IP ranges.

Practical Example
A typical workflow for ingestion involves forwarding CloudTrail and VPC Flow Logs to an S3
bucket, then using Amazon Kinesis or a third-party tool (e.g., Logstash) to parse and send events
to a SIEM. For example, with AWS CLI you can enable CloudTrail logging:
P a g e 23 | 36
aws cloudtrail create-trail \
--name MySecurityTrail \
--s3-bucket-name my-security-logs \
--include-global-service-events

For more details, see the AWS CloudTrail Documentation.

Azure Logs
Common Log Types
1. Azure Activity Logs

o Purpose: Provide insights into management operations (e.g., resource creation,

modification, or deletion).

o Key Fields: authorization, caller, category, operationName, resourceId, status.

o Use in Security: Detect unauthorized resource creation, changes to security

groups, or attempts to elevate privileges.

2. Azure Monitor Logs (Log Analytics)

o Purpose: Collect logs from Azure resources, containers, VMs, and applications.

o Key Fields: Vary based on the resource type; commonly include timestamps,
operation IDs, user details, and other contextual data.

o Use in Security: O ers extensive querying and correlation capabilities. SOC

teams can detect anomalies by combining signals from multiple sources
(Activity Logs, VM logs, etc.).

3. Diagnostics Logs

o Purpose: Detailed insights from specific Azure services, such as Key Vault
access logs, Azure App Service logs, or Azure Storage logs.

o Key Fields: Depend on the service but often include request endpoints,
authentication details, and result codes.

o Use in Security: Detects potential credential misuse, suspicious activity in data

storage, or unusual application behavior.

Practical Example
Sending logs to Azure Monitor can be done by configuring a Diagnostic Setting for each
resource. For instance, to route Activity Logs to Azure Monitor and a storage account:
Set-AzDiagnosticSetting -ResourceId
/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/pro
viders/Microsoft.Web/sites/<APP_NAME> `
-WorkspaceId <AZURE_MONITOR_WORKSPACE_ID> `
-StorageAccountId
/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/pro
viders/Microsoft.Storage/storageAccounts/<STORAGE_ACCOUNT_NAME> `
P a g e 24 | 36
 -Enabled $true

Refer to Azure Monitor Documentation for more details.

GCP Logs
Common Log Types
1. Cloud Audit Logs

o Purpose: Record admin and data access events for GCP services (similar to
AWS CloudTrail).

o Key Fields: protoPayload.serviceName, protoPayload.methodName,

resourceName, authenticationInfo, requestMetadata.

o Use in Security: Surface privilege escalation attempts or suspicious

modifications to GCP resources (e.g., enabling/disabling critical services).

2. VPC Flow Logs

o Purpose: Collect network flow information for Google Cloud VPCs.

o Key Fields: srcIP, destIP, srcPort, destPort, protocol, connectionEstablished,

bytesSent, bytesReceived.

o Use in Security: Spot reconnaissance or exfiltration activity by analyzing

inbound and outbound tra ic patterns.

3. Cloud Logging

o Purpose: Central logging service for events from GCP services, containers,
custom applications.

o Key Fields: Service-specific data, timestamps, severity levels, resource labels

(e.g., k8s_container, gce_instance).

o Use in Security: Enables correlation of application-level logs with

infrastructure-level events.

Practical Example
To export GCP logs to a SIEM, you can create a sink that routes logs to a Pub/Sub topic, which a
custom or third-party collector can then forward. An example using the gcloud CLI:
gcloud logging sinks create my-security-sink \
storage.googleapis.com/<BUCKET_NAME> \
--log-filter="resource.type=gce_instance AND severity>=WARNING"

For detailed guidance, see Google Cloud Logging Documentation.

Container Logs (Docker, Kubernetes)

Containers package applications and their dependencies into a single lightweight unit. Because
containers often run ephemeral workloads, continuous and standardized logging is key to
security monitoring.

P a g e 25 | 36
Docker Logs
1. Docker Engine Logs

o Location: Typically stored in /var/log/docker.log on Linux hosts.

o Key Fields: Daemon-level events, such as container starts/stops, image pulls,

errors from container runtime.

o Use in Security: Identify unauthorized container creation or malicious images

being pulled from untrusted registries.

2. Container STDOUT/STDERR Logs

o Location: By default, stored in

/var/lib/docker/containers/<container_id>/<container_id>-json.log.

o Use in Security: Detect anomalies within running applications (e.g., repeated

error messages indicating a brute-force attempt or application misuse).

3. Docker Logging Drivers

o Types: json-file, syslog, fluentd, gelf, awslogs, and others.

o Use in Security: Can integrate with centralized logging solutions, reducing the
chance of log tampering if the container is compromised.

Docker Example
Using the syslog logging driver, you can direct container logs to a remote syslog server:
docker run --log-driver=syslog --log-opt syslog-
address=tcp://192.168.1.10:514 \
--log-opt tag="{{.ImageName}}/{{.Name}}/{{.ID}}" \
my_secure_image

Refer to Docker Logging Documentation for configuration details.

Kubernetes Logs
1. Container Logs

o Collection: Typically gathered via kubectl logs <pod_name> or via a logging

agent (Fluentd, Logstash, or a sidecar pattern).

o Use in Security: Detect suspicious application errors or specific triggers such

as repeated 401/403 responses indicating an authentication brute-force
attempt.

2. Kubelet Logs

o Location: Paths di er depending on the OS distribution; can include

/var/log/kubelet.log.

o Use in Security: Track container scheduling issues, unauthorized attempts to

schedule privileged pods, or interactions that could indicate a compromised
node.

P a g e 26 | 36
 3. Control Plane Logs (API Server, Scheduler, Controller Manager)

o Location: Often under /var/log/ on the control plane node or aggregated using a
centralized logging solution.

o Use in Security: Identify unauthorized API calls, suspicious pod creations, or

attempts to escalate privileges via Kubernetes role bindings.

4. Audit Logs

o Purpose: Record every request to the Kubernetes API server.

o Configuration: Enable auditing by modifying the --audit-log-path and --audit-

policy-file flags on the API Server.

o Use in Security: Fundamental for investigating incidents. You can track

everything from RBAC changes to privileged container spawns.

Kubernetes Example
A simple audit policy file (audit-policy.yaml) might look like:
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
resources:
- group: ""
resources: ["secrets"]
- level: RequestResponse
resources:
- group: ""
resources: ["pods/exec"]

You can reference the Kubernetes Auditing Documentation for more advanced configurations.

Practical Considerations and Real-World Tips

 Centralization: Whether using AWS CloudWatch, Azure Monitor, Google Cloud Logging,
or self-hosted ELK stacks, centralizing logs from multiple cloud providers and container
platforms is essential for correlation.

 Access Controls: Make sure that logs, especially those containing sensitive information
(credentials, personal data), are stored in restricted areas. Configure IAM roles or
equivalent to control who can read or export logs.

 Alerting and Dashboards: Build targeted alerts. For example, create an alert if a new
Kubernetes cluster role binding is created that grants cluster admin privileges.

P a g e 27 | 36
  Retention Policies: Align with regulatory requirements. Some industries require keeping
logs for extended periods, whereas others may prioritize cost optimization.

 Log Volume vs. Relevance: Filtering out excessive “noise” helps avoid data overload.
Set up granular logging only where necessary, or implement log sampling for high-
volume events like container debug logs.

 Cross-Platform Correlation: When investigating an incident, cross-reference container

logs with the underlying cloud infrastructure logs. For instance, if a malicious container
is identified, reviewing AWS CloudTrail or GCP Cloud Audit logs can show who deployed
it and from where.

By covering these areas, SOC analysts gain better visibility into cloud-based and containerized
environments. Each platform o ers di erent types of logs and various ways to configure them,
but the overarching principle remains the same: you need complete, centralized, and reliable
logging to e ectively detect and respond to security incidents.

2.6. IoT/SCADA/OT Logs

IoT (Internet of Things), SCADA (Supervisory Control and Data Acquisition), and OT (Operational
Technology) devices play a critical role in modern industries, from manufacturing floors to
energy grids. Logs generated by these systems provide valuable insights into operational status,
performance metrics, and potential security threats. Monitoring these logs e ectively can be
challenging due to the diversity of protocols, the variety of operating systems and firmware
involved, and the high availability requirements that often characterize industrial environments.
Below is an overview of the main considerations, examples of log sources, and best practices to
ensure comprehensive monitoring.

Understanding IoT, SCADA, and OT Environments

IoT Overview
IoT devices are typically embedded systems used in various contexts—smart homes, industrial
sensors, healthcare devices, and more. They often have:

 Limited resources (CPU, memory) making local storage of logs di icult.

 Custom firmware that may or may not produce standardized logs.

 Network constraints such as low-bandwidth or intermittent connectivity.

SCADA and OT Systems

SCADA and other OT systems control and monitor industrial processes in energy,
manufacturing, transportation, and critical infrastructure. Key distinctions include:

 Real-time or near-real-time processing with strict performance and availability

requirements.

 Use of specialized protocols (e.g., Modbus, DNP3, OPC-UA) where logging capabilities
can di er from those found in IT environments.

 Legacy components that might not support current cybersecurity standards or modern
logging frameworks.

P a g e 28 | 36
Types of Logs to Monitor
1. System Logs: Embedded operating systems or specialized OS variations used by
industrial controllers (PLCs, RTUs, HMIs) might produce kernel messages or standard
syslog entries when available.

2. Network Tra ic Logs: Many industrial protocols can be captured by network sensors or
specialized gateways. Anomalies in tra ic—like unexpected Modbus function codes—
may indicate malicious activity or misconfiguration.

3. Application Logs: SCADA software logs events such as operator actions, process
thresholds, alarm states, and device connectivity issues. These logs can reveal
unauthorized changes to critical setpoints.

4. Firmware/Device Logs: IoT and OT devices often generate messages related to

firmware integrity checks or patch updates. Monitoring these can help detect suspicious
attempts to install unauthorized firmware.

5. Security Appliance Logs: When perimeter security is present in industrial networks,

firewalls, IDS/IPS, and dedicated OT security appliances produce logs regarding
intrusion attempts, blocked tra ic, and detected threats.

Practical Challenges in Log Collection

1. Protocol Complexity
Many IoT and industrial protocols are proprietary or only partially documented.
Interpreting logs requires an understanding of the specific protocol version and vendor
implementation.

2. Limited Storage and Processing Power

Some devices rotate logs quickly due to limited disk space. SOC analysts may need to
forward these logs to a central server in real time to avoid data loss.

3. High Availability Requirements

Stopping or reconfiguring a production system to enable certain logs might be infeasible
if it disrupts critical operations. Analysts must plan logging configurations carefully,
often during scheduled downtimes.

4. Segmentation and Air Gaps

Industrial networks are sometimes isolated (“air-gapped”) from corporate networks.
Secure mechanisms (e.g., data diodes, jump hosts) are needed to relay logs without
introducing new vulnerabilities.

Best Practices for Monitoring

1. Standardize and Normalize Logs
Whenever possible, configure devices to output logs in a standardized format, such as syslog or
JSON. This step simplifies ingestion into a SIEM or log management solution.

Example: Configuring syslog on a Linux-based industrial controller

sudo apt-get install rsyslog
sudo systemctl enable rsyslog

P a g e 29 | 36
# Configure /etc/rsyslog.conf to forward logs
*.* @192.168.100.10:514

In an OT environment, you may need vendor-specific documentation to enable syslog

forwarding. If syslog is not an option, use an industrial gateway or a protocol converter that can
parse native logs and send them in a common format.

2. Correlate with Physical Process Data

SCADA systems often track process variables (e.g., temperature, pressure, flow). Cross-
referencing these metrics with login attempts or configuration changes can reveal malicious or
erroneous actions. Correlation rules in your SIEM might look for:

 Sudden setpoint changes followed by alarm acknowledgments.

 Unauthorized user accounts created just before critical equipment is taken o line.

 Repeated network scans coinciding with elevated temperature readings on IoT

sensors.

3. Implement Least Privilege and Access Controls

Modern industrial solutions often include role-based access controls. Logs from identity and
access management tools show who accessed the system and what changes were made:

 Ensure all logins and role escalations are recorded.

 Enable multi-factor authentication (MFA) for remote connections to SCADA and OT

consoles.

4. Monitor for Firmware Updates and Integrity

Unscheduled or unauthorized firmware updates can be an early sign of compromise. Monitor
logs for:

 Firmware version mismatch or unexpected reboots.

 Device reimaging events occurring outside normal maintenance windows.

Many industrial device vendors provide integrity-check features. Leverage these and forward
related events to the SOC for review.

5. Leverage Specialized Threat Intelligence

Threat intelligence feeds focusing on ICS/SCADA vulnerabilities can help enrich your log
analysis. For instance, MITRE ATT&CK for ICS
(https://collaborate.mitre.org/attackics/index.php/Main_Page) lists techniques and tactics used
by adversaries targeting operational technology. Incorporating these indicators into your
monitoring rules can enhance detection capabilities.

Real-World Examples
Example 1: Power Grid Manipulation Attempt
An attacker gains access to a SCADA workstation used to manage a regional power grid. Review
of the logs shows:

1. Multiple failed RDP logins from an external IP.

P a g e 30 | 36
 2. Successful login under an admin account (possibly via stolen credentials).

3. Sudden changes in circuit breaker open/close commands issued at unusual times.

Cross-referencing logs from the SCADA software with firewall logs reveals inbound connections
bypassed normal VPN channels, indicating a compromise on the perimeter. The timely
correlation of these logs prevented a large-scale outage.

Example 2: Compromised IoT Sensor Network

A manufacturing facility experiences irregular temperature readings from a cluster of IoT
sensors. Logs collected from the device management platform show:

1. Unusual spike in network tra ic directed at the sensors.

2. Firmware tampering attempts logged by the device’s built-in integrity checks.

3. Outbound connections from the sensors to unauthorized IP addresses.

Investigation finds that the sensors had outdated firmware with a known vulnerability. Patching
them quickly and blocking the malicious IP addresses at the firewall mitigated further data
exfiltration and potential system damage.

Comparative Overview
Aspect IoT SCADA / OT

Primary Focus Smart devices & sensors Industrial process control

Logging Formats Often proprietary or minimal Syslog, proprietary (e.g., PLC logs)

Protocols MQTT, CoAP, HTTP(S) Modbus, DNP3, OPC-UA

Security Varies widely; often unpatched Safety, availability, real-time ops

Challenges Resource constraints Legacy systems, air-gapped networks

Actionable Steps for SOC Analysts

1. Identify Key Log Sources: Prioritize critical controllers (PLCs, RTUs) and high-impact
IoT devices.

2. Establish Secure Log Forwarding: Use encrypted channels (e.g., TLS, SSH tunnels)
when sending logs across network boundaries.

3. Create Baseline Profiles: Understand normal operation of devices and detect

deviations. For instance, if a PLC typically receives commands only during business
hours, an alert can trigger on after-hours changes.

4. Combine Network and Host-Based Monitoring: Many attacks against OT systems

involve lateral movement or pivot from the IT side. Include NetFlow, firewall, and
endpoint logs in your analysis.

P a g e 31 | 36
5. Review Vendor Guidance: Major industrial vendors like Siemens, Rockwell Automation,
and Schneider Electric publish documentation on best practices for logging and
security. Stay up to date with vendor patches and advisories.

P a g e 32 | 36
3. Key Monitoring Practice
E ective log monitoring hinges on sound processes, properly configured tools, and clear
objectives. SOC analysts should focus on strategies that help distinguish normal from
suspicious behaviors, preserve and protect relevant data, and leverage automation where
possible. The following practices outline core considerations for detecting anomalies, retaining
logs securely, and employing supporting technologies.

3.1. Detecting Anomalies and Incidents (Alerts, Correlation)

Anomaly Detection vs. Signature-Based Detection
Anomaly detection involves establishing a baseline of normal operations and flagging
deviations. This is useful for identifying zero-day threats or unusual user behavior. In contrast,
signature-based detection relies on known indicators of compromise (IoCs), such as specific IP
addresses, hash values, or attack patterns. Most modern SOCs use a hybrid approach to
capture both unknown threats (anomalies) and known malicious activity (signatures).

Contextual Analysis of Logs

When investigating events, it is rarely enough to look at a single log source. Correlating data
across multiple sources (e.g., firewall, endpoint detection and response [EDR], and Active
Directory logs) can reveal sophisticated attacks. For instance, seeing repeated user
authentication failures in an Active Directory log and simultaneous unusual outbound
connections in a firewall log could point to a brute-force attempt followed by data exfiltration.

Alert Thresholds and Fine-Tuning

SOC teams often deploy alert rules on SIEM or IDS/IPS systems to notify them of suspicious
activities. Balancing these thresholds is critical:

 Too strict: Risk flooding the SOC with false positives, causing alert fatigue and missed
real threats.

 Too relaxed: Allows significant security incidents to go unnoticed, delaying response

and remediation.

Finding the right balance often requires iterative tuning based on historical data, environment
specifics, and known business processes.

Real-World Example
Consider a situation where a user account is suddenly accessing hundreds of files on a file
server at unusual hours. Anomaly detection rules might flag this behavior if it deviates from the
normal usage pattern of that user. Meanwhile, a signature-based rule might detect that some of
these files match known malicious toolkits (e.g., Mimikatz or similar). Correlating both alerts
enables SOC analysts to identify a potential account compromise and data theft incident much
faster.

Sample SIEM Query (Splunk)

index=windows_logs sourcetype=WinEventLog:Security
EventCode=4625 OR EventCode=4624

P a g e 33 | 36
| stats count by Account_Name, EventCode
| where count > 20

In this example, the query checks for successful logons (4624) or failed logons (4625) and looks
for any account logging multiple times beyond a threshold, which could indicate brute force or
lateral movement attempts.

3.2. Log Retention and Security

Retention Policies
Logs must be kept for a specified duration, often defined by organizational policies, regulations
(e.g., PCI DSS, HIPAA), and compliance standards. Typical retention periods range from 90 days
to multiple years, depending on the data sensitivity and industry requirements. SOC analysts
should verify that retention policies align with both threat-hunting needs and legal obligations.

Log Storage Considerations

 Centralized Storage: Storing logs in a single repository (e.g., a SIEM or log management
platform) simplifies searching, correlation, and backup.

 Redundancy: Using multiple storage locations or clustering ensures logs remain

available even if hardware fails.

 Encryption: Encrypting logs at rest (e.g., using disk-level encryption) and in transit (e.g.,
TLS for log forwarding) prevents unauthorized access.

 Access Controls: Implement role-based access controls (RBAC) so that only

authorized personnel can view or manipulate sensitive logs.

Handling Log Integrity

To preserve evidentiary value, organizations should ensure logs cannot be easily tampered with:

1. Hashing: Generating hashes (e.g., using SHA-256) for log files and storing them
separately helps detect unauthorized modifications.

2. Write-Once-Read-Many (WORM) Storage: Some platforms support WORM-like

functionality where logs can be written but not altered afterward.

3. Audit Trails: Keep track of who accessed the log repository, when they accessed it, and
what changes (if any) were made.

Example of Secure Log Storage

A company might use an Amazon S3 bucket with versioning and server-side encryption enabled
for archiving logs from on-premises systems. The AWS Key Management Service (KMS) provides
secure key storage, while AWS Identity and Access Management (IAM) enforces strict
permissions. O icial documentation on this setup can be found on the AWS Documentation
pages.

P a g e 34 | 36
3.3. Supporting Tools (SIEM, SOAR)
SIEM (Security Information and Event Management)
SIEM solutions collect, parse, and normalize logs from various sources, allowing analysts to
search, correlate, and generate alerts in near real time. Common SIEM platforms include
Splunk Enterprise Security, IBM QRadar, and Microsoft Sentinel. Key features:

 Log Aggregation and Normalization: Standardizes events to a common format.

 Correlation Rules: Creates alerts when multiple indicators occur in a defined

sequence.

 Dashboarding and Reporting: O ers visual interfaces for monitoring security posture
and presenting metrics to management.

SOAR (Security Orchestration, Automation, and Response)

SOAR platforms automate tasks that analysts would otherwise perform manually. Examples
include Palo Alto Networks Cortex XSOAR (formerly Demisto) and Splunk Phantom. These tools
can be configured to:

1. Enrich Alerts: Automatically gather host or network information from threat intelligence
sources.

2. Contain Incidents: For instance, disable a compromised user account or isolate a

malicious endpoint.

3. Orchestrate Responses: Trigger workflows that involve multiple security and IT

systems.

Automation and Playbooks

A standard approach in SOAR is to develop playbooks—automated workflows that define how
to respond to specific incidents. For example, if an alert indicates a suspicious PowerShell
script ran on a server, a playbook might:

1. Retrieve relevant endpoint logs.

2. Compare the script hash with a threat intelligence database.

3. Quarantine the host if the hash is malicious.

4. Create a ticket in the incident management system.

High-Level Comparison of SIEM vs. SOAR

Feature SIEM SOAR

Centralizing logs, correlation,

Primary Focus Automating and orchestrating responses
alerts

Data Aggregation and analysis of large Integration with multiple security/IT tools
Processing volumes of log data to enrich and act on alerts

P a g e 35 | 36
Feature SIEM SOAR

Security alerts, dashboards, Workflow automation, playbooks, and

Typical Output
reports containment actions

Usage Medium to High (depends on desired

Medium to High
Complexity automation)

In many cases, SOCs integrate both SIEM and SOAR for comprehensive coverage. The SIEM
handles large-scale ingestion and correlation, while the SOAR platform automates investigation
and response steps. This integration reduces mean time to detect (MTTD) and mean time to
respond (MTTR), ultimately strengthening the organization’s security posture.

P a g e 36 | 36