Tech Note - Packet Based Debugging
Tech Note - Packet Based Debugging
Using Packet Filtering, Capture and Debug Logs in PAN-OS 3.1 and later
October 2011
Packet filter is enabled using the command “debug dataplane packet-diag set filter on”
-------------------------------------------------------------------------
Packet diagnosis setting:
-------------------------------------------------------------------------
Packet filter
Enabled: yes
Match pre-parsed packet: no
Index 1: 1.1.1.50[0]->0.0.0.0[0], proto 0
ingress-interface any, egress-interface any, exclude non-IP
Index 2: 1.1.1.5[0]->2.1.1.5[80], proto 6
ingress-interface any, egress-interface any, exclude non-IP
-------------------------------------------------------------------------
Logging
Enabled: no
Log-throttle: no
Aggregate-to-single-file: yes
Features:
-------------------------------------------------------------------------
Packet capture
Enabled: no
-------------------------------------------------------------------------
In the above example, there are two packet filters configured. The packets matching either one of
the filters are captured.
Clearing a filter
PAN-OS device supports four concurrent filters to be configured. Existing filters must be removed
in order to configure new filters. Filters are referenced by index numbers. To clear a filter use the
command debug dataplane packet-diag clear filter
Pre-parse-match
The pre-parse-match option is added for advanced troubleshooting purposes. From the moment
packet enters ingress port it has to go through number of processing steps before it gets parsed for
match against pre-configured filters. Therefore, it is entirely possible that packet, due to some
failure, never gets to filtering stage. Typical example would be route lookup – if this fails, a packet
will never reach the filter (although interface counters can be used to quickly identify this error
condition). Setting pre-parse-match emulates positive match for every packet entering the system
so we can capture even those packets that don’t make it to filtering process. If packet does
manage to get through to the filtering stage, it is then processed according to filter configuration
and match is discarded if packet fails to meet filtering criteria. By default pre-parsed packets are
not matched. To enable pre-parse match use the command
admin@PA-4050(active)> debug dataplane packet-diag set filter pre-parse-
match yes
To enable packet capture use the command debug dataplane packet-diag set capture on
• drop: when packet processing encounters an error and this packet will be discarded
• firewall: packet has a session match, or a first packet with session successfully created
• receive: packet received on dataplane CPU
• transmit: packet is to be transmitted on dataplane CPU
The receive and transmit stage capture can be viewed as shown in the figure shown below
You can also define the maximum number of bytes to capture and also the maximum packets to be
captured for each stage.
Trigger captures
Packet capture is conditional for a given session, it starts when triggering event occurs. Currently
the only trigger event is “application”, which means to trigger when the application changes from
one to another.. For example when user access gmail, the session starts off as web-browsing and
then switches to gmail. It is possible to define the max byte count before the filter stops capturing,
and also the max packets to be captured. A packet match filter is required for application trigger to
capture packets.
Note: If you are capturing packets at different stages, it recommended having a separate PCAP file
for each stage of the capture
You can view all the file names using “?” in place of the filename.
The view-pcap command provides user with lots of different options to view the PCAP. Some of the
options include, hex, hex-ascii, no-dns-lookup, no-port-lookup. To view the complete option list, use
“?” as the first argument after “view-pcap”. To view the PCAP with options use the following
command:
The pcap file can also be exported to an external host to be viewed using any PCAP viewing utility.
TFTP and SCP are the supported methods.
Note: The PCAP file is created only when the traffic traverses the firewall.
Note:
Clearing capture
To clear capture use the debug dataplane packet-diag clear command. More specific options to
clear a particular stage and trigger are also available
PCAP examples
-------------------------------------------------------------------------
Packet diagnosis setting:
-------------------------------------------------------------------------
Packet filter
Enabled: yes
Match pre-parsed packet: no
In the example the IP address and the port numbers of the packet are as shown
Original packet
172.16.100.87/32919 172.16.101.100/21
Translated packet
172.16.101.1/43828 172.16.101.100/21
When NAT is configured, it is important to note the source and destination IP addresses of the
packet at different capture points.
Transmit stage:
Receive and firewall stage always captures post NAT addresses
The first packet transmitted by the firewall will have source IP/port=172.16.101.1/43828 and the
destination IP/port=172.16.101.100/21, i.e. the translated packet
The response packet transmitted by the firewall will have source IP/port=172.16.101.100/21 and
destination IP/port=172.16.100.87/32919. The destination IP is the IP that is translated to the
original address from the response packet
To enable packet capture use the command debug dataplane packet-diag set log on
PAN-OS offers multiple features to log packets. Each feature can have sub features where packets
can be logged.
tail command by default displays the last 10 lines of the debug file. To output the last N lines,
instead of the last 10 use the command
tail lines <value> dp-log pan_packet_diag.log
where value is 1-65535
Note:
1. For the PA-5000 series of firewall the command to view debug log is less dp0-log
pan_packet_diag.log
2. For the PA-200 use the command less mp-log pan_packet_diag.log
In this example, we capture packets for all FTP traffic from source 172.16.100.87 to destination
172.16.101.100
== Jun 21 19:14:39 ==
Packet received at np stage
Packet info: len 74 port 16 interface 16
wqe index 229308 packet 0x0x8000000416fdc0e6
Packet decoded dump:
L2: 00:14:c1:4b:d8:f9->00:1b:17:14:4f:10, type 0x0800
IP: 172.16.100.87->172.16.101.100, protocol 6
version 4, ihl 5, tos 0x00, len 60,
id 30220, frag_off 0x4000, ttl 64, checksum 41683
TCP: sport 33987, dport 21, seq 1819534672, ack 0,
reserved 0, offset 10, window 5840, checksum 27398,
flags 0x0002 ( SYN), urgent data 0
TCP option:
00000000: 02 04 05 b4 04 02 08 0a 3a 54 93 5c 00 00 00 00 ........
:T.\....
00000010: 01 03 03 07 ....
== Jun 21 19:14:39 ==
Packet received at ingress stage
Packet info: len 74 port 16 interface 16
wqe index 229308 packet 0x0x8000000416fdc0e6
Packet decoded dump:
L2: 00:14:c1:4b:d8:f9->00:1b:17:14:4f:10, type 0x0800
IP: 172.16.100.87->172.16.101.100, protocol 6
version 4, ihl 5, tos 0x00, len 60,
id 30220, frag_off 0x4000, ttl 64, checksum 41683
TCP: sport 33987, dport 21, seq 1819534672, ack 0,
reserved 0, offset 10, window 5840, checksum 27398,
flags 0x0002 ( SYN), urgent data 0
TCP option:
00000000: 02 04 05 b4 04 02 08 0a 3a 54 93 5c 00 00 00 00 ........
== Jun 21 19:14:39 ==
PAN-OS allows for searching specific keywords within the log by typing /<pattern>. These are case
sensitive. For example to see the route lookup in the above example you can type /Route
Summary
PAN-OS 3.1 offers restructured packet-related diagnosis facilities. The improvements with global
counters, filtering; debug logs and dataplane packet capture empowers firewall administrators to
troubleshoot issues with device or network.
debug dataplane <on debug | off> debug dataplane packet-diag set log <on | off>
debug dataplane set <module> <feature> debug dataplane packet-diag set log feature
<module> <feature>
debug dataplane mode <sync | no-sync> debug dataplane packet-diag set log log-option
aggregate-to-single-file <yes|no>
debug dataplane filter set debug dataplane packet-diag set capture stage
firewall
debug dataplane drop-filter set debug dataplane packet-diag set capture stage
drop
debug dataplane packet-diag clear capture <all |
stage … | trigger …>