Fortigate Debug Guide
Fortigate Debug Guide
To enable debug set by any of the commands below, you need to run diagnose
NOTE debug enable. This is assumed and not reminded any further. Use dia debug info
1
to know what debug is enabled, and at what level.
To disable and stop immediately any debug, run dia deb res which is short for
NOTE
diagnose debug reset.
All debug will run for 30 minutes by default, to increase use diagnose debug
NOTE duration <minutes>, setting to 0 means unlimited by time. Reboot will reset this
setting.
Command Description
diagnose firewall iprope lookup <src IP> <src Policy lookup for any combination of IPs and
port> <dst IP> <dst port> <IANA protocol ports - use to see what policy (if any) matches
number> <src interface> traffic between specific IP addresses and ports.
E.g. dia firewall iprope lookup 10.10.10.1
34567 8.8.8.8 443 6 LAN1
diagnose debug flow filter Show the active filter for the flow debug
diagnose debug filter clear Remove any filtering of the debug output set
diagnose debug flow filter <filtering param> / Set filter for security rulebase processing
dia debug flow filter6 <param> packets output. You can set multiple filters - act
as AND, by issuing this command multiple times.
Parameters:
diagnose debug filter6 <parameter> Same as diagnose debug filter but for IPv6
packets. The rest of matching and conditions
remain of the same syntax.
diagnose debug flow show function-name Show function names responsible for each step
enable in processing.
diagnose debug flow trace start [number] Actually start the debug with optional number to
limit number of packets traced.
diagnose debug flow trace start6 [number] Start the debug trace for IPv6 traffic, with
optional number to limit number of packets
traced.
3
Packet Sniffer (diagnose sniffer packet)
Command Description
dia sni pa if-name/any 'tcpdump syntax filter' Network level packet sniffer like
verbosity count time-format tcpdump/tshark/wireshark, presenting captured
packets on CLI. It gives definite answers
whether a packet reached the Fortigate, whether
it was dropped by firewall rules, what was
incoming/outgoing interface, and contents of the
packet if needed.
time-format:
• l - local time
Command Description
get sys stat Get statistics about the Fortigate device: FortiOS
used, license status, Operation mode, VDOMs
configured, last update dates for AntiVirus, IPS,
Application Control databases.
get sys performance stat Show real-time operational statistics: CPU load
per CPU, memory usage, average
network/session, uptime.
diagnose sys top [refresh] [num-of-processes] Print list of running processes updated every
[iterations] refresh seconds (default 5), for iterations times,
sorted in descending order by the CPU load. This
top command does not display all processes by
default, to show them all, set num-of-processes to
high number, for example 100. Press "m" to sort
the processes by memory consumption. The
displayed table is in this order: Process id,
process state: ®unning, (S)leep, (Z)ombie,
(D)isk Sleep, < Means higher priority, CPU used,
Memory used.
dia sys kill signal-id process-id Forcefully kill the process with the id of process-
id, sending it the given signal-id (Linux signals,
e.g. 9, 11).
diagnose debug crashlog read Display crash log. Records all daemons crashes
and restarts. Some daemons are more critical
than others.
5
Command Description
dia sys top-mem [num-processes] [detail] Show top (default 5) processes by memory
usage, optionally set number of processes to
show with num-processes, and use detail to get
verbose output (a lot).
dia hardware sysinfo conserve Info whether the conserve mode on or off, total
memory available, conserve mode thresholds
red and green
dia sys flash list Show contents of the flash memory holding
FortiOS firmware images. One of the images will
have Active set to yes, which means it is the used
one.
diagnose hardware deviceinfo disk Show all storage attached to the firewall,
including disk type, volume, free space.
get system session status / get system session6 Show current number of sessions passing the
status Fortigate (IPv4/IPv6). Run inside the VDOM in
multi-vdom environment to get number of
connections/sessions for this specific VDOM.
get sys session-info ttl Show the default TTL setting for the connections
in the table, default being 3600 seconds.
diagnose sys session filter <filter parameter> Set filter to show/manipulate only specific
<filter value> / diagnose sys session6 filter connections in the stateful table. Run without
<filter parameter> <filter value> any filter parameters this command displays the
current filter applied if any. Parameters:
7
Command Description
diagnose sys session clear / dia sys session6 Clear/delete connections from the session table.
clear IMPORTANT: If no session filter is set (see above)
before running this command, ALL connections
passing the Fortigate will be deleted! Which
means they will be disconnected. So use
carefully.
diagnose sys session list / dia sys session6 list List connections limited to the filter set if any, or
all session table if not.
Command Description
diagnose sys ha dump-by group Print detailed info per cluster group, shows
actual uptime of each member in start_time, as
well monitored links failures, status.
diagnose sys ha checksum cluster Shows configuration checksum for each cluster
member separated in individual VDOMs and
global. In properly synchronized cluster all
member checksums should be identical, look at
all value.
diagnose sys ha checksum show < Print detailed synchronization status for each
VDOM/global> configuration part. Use after seeing out-of-sync
in diagnose sys ha checksum cluster to know
which part of configuration causes members to
be out-of-sync. Need to run on each cluster
member and compare, long output - use diff
/vimdiff/Notepad++ Compare plugin to spot the
differences.
diagnose sys ha checksum show < Show exact setting inside the settings tree that
VDOM/global> <settings part name> causes out-of-sync. Use output from diagnose
sys ha checksum show (see above) for settings
part name. E.g. if diagnose sys ha checksum show
root indicates that firewall.vip is out-of-sync,
running diagnose sys ha checksum show root
firewall.vip will give checksums of each VIP in
the root domain to compare with those of
secondary member.
config sys ha
set ha override disable
9
IPSEC VPN debug
Table 4. IPSEC VPN Debug
Command Description
diagnose vpn ike log-filter <parameter> Filter VPN debug messages using various
parameters:
diagnose debug application ike -1 Enable IPSec VPN debug, shows phase 1 and
phase 2 negotiations (for IKEv1) and everything
for IKEv2. "-1" sets the verbosity level to
maximum, any other number will show less
output.
diagnose vpn ike gateway flush name Flush (delete) all SAs of the given VPN peer only.
<vpn_name> Identify the peer by its Phase 1 name.
diagnose vpn tunnel list [name <Phase1 Show operational parameters for all or just
name>] specific tunnels: Type (dynamic dial up or
static), packets/bytes passed, NAT traversal state,
Quick Mode selectors/Proxy Ids, mtu, algorithms
used, whether NPU-offloaded or not, lifetime,
DPD state.
diagnose vpn ike gateway list Show each tunnel details, including user for
XAuth dial-up connection.
dia vpn tunnel shut <Phase2 name> <Phase1 Bring the named tunnel down by its Phase2 and
name> Phase2 name.
dia vpn tunnel up <Phase2 name> <Phase1 Bring the tunnel up by its name.
name>
get vpn ipsec tunnel details Detailed info about the tunnels: Rx/Tx
packets/bytes, IP addresses of the peers,
algorithms used, detailed selectors info, lifetime,
whether NAT Traversal is enabled or not.
get vpn ipsec stats tunnel Short general statistics about tunnels: number,
kind, number of selectors, state
get vpn ipsec tunnel summary Short statistics per each tunnel: number of
selectors up/down, number of packets Rx/Tx.
get vpn ipsec stats crypto Crypto stats per component (ASIC/software) of
the Fortigate: encryption algorithm, hashing
algorithm. Useful to see if unwanted situation of
software encryption/decryption occurs.
Command Description
get vpn ssl monitor List logged in SSL VPN users with allocated IP
address, username, connection duration.
diagnose vpn ssl debug-filter criteria Limit debug output according to the criteria
below:
diagnose debug app sslvpn -1 Debug SSL VPN connection. Shows only SSL
protocol negotiation and set up. That is - ciphers
used, algorithms and such, does NOT show user
names, groups, or any client related info.
11
Static Routing Debug
Table 6. Static and Policy Based Routing debug & diagnostics
Command Description
get router info kernel View the kernel routing table (FIB). This is the
list of resolved routes actually being used by the
get router info6 kernel FortiOS kernel.
get router info routing-table all Show RIB - active routing table with installed
and actively used routes. It will not show routes
get router info6 routing with worse priority, multiple routes to the same
destination if unused.
get router info routing database Show ALL routes, the Fortigate knows of -
including not currently used.
get rotuer info6 routing database
get router info routing-table details <route> Show verbose info about specific route, e.g. get
router info routing-table details 0.0.0.0/0
get firewall proute Get all configured Policy Based Routes on the
Fortigate.
get firewall proute6
exe ping-options [data-size bytes / df-bit / Set various options before running pings.
interface if-name / interval seconds / repeat-
count integer / reset / view-settings / timeout
seconds / source ip / ttl integer / use-sdwan yes]
exe ping6-options see available options above Set various ping6 options before running it.
for ipv4
Interfaces
Table 7. Interafces of all kinds diagnostics
Command Description
get hardware nic <inerface name> Hardware info of the interface: MAC address,
state (up/down), duplex (full, half), Rx/Tx
packets, drops.
get sys interface transceiver List all SFP/SFP+ transceivers installed with info
on: vendor name, serial number, temperature,
voltage consumed, and, most important -
Transmit (TX) and Receive (RX) signal power in
dBm.
get hardware npu np6 port-list Show on which interfaces the NPU offloading is
enabled.
fnsysctl ifconfig <interface name> Gives the same info as Linux ifconfig. The only
way to see the actual MTU of the interface.
diagnose sys gre list Show configured GRE tunnles and their state.
13
Command Description
diagnose netlink aggregate list List all aggregate interfaces in the current
VDOM, shows names, state (up/down), LACP
mode and algorithm used
diagnose netlink aggregate name <aggregate Shows details of the given aggregate interface
interface name> under the entry actor state (preferred state is
ASAIEE): LACP Mode (Active/Passive), LACP
Speed mode (Slow [default]/Fast), Synced or Out
of Sync, minimal physical interfaces to be up for
the whole aggregate to be up, Aggregator ID (has
to be identical on both sides), own and peer’s
MAC addresses, link failure count.
diagnose sniffer packet any "ether proto Sniffer to see all LACP traffic on this Fortigate:
0x8809" 6 0 a 0x8809 LACP Ethernet protocol designation, 6 -
maximum verbosity, 0 - do not limit number of
captured packets, a - show time in UTC format,
rather than delta from the 1st packet seen. LACP
packets should arrive from the peer’s MAC
address on the aggregate logical interface name,
and should leave from the physical interface(s)
destined to the peer’s MAC address. This capture
will also show LACP actor state in
arriving/leaving packets - for working LACP
aggregate it should be ASAIEE in both directions.
Command Description
execute dhcp/dhcp6 lease-clear all/start-end- Clear DHCP allocations on the Fortigate. This will
IP-address-range NOT cause clients that already have IP addresses
to release them, but will just clear Fortigate
DHCP database and will start over allocating
again. You can either clear all IP addresses in the
database, or only specific IPs.
diagnose debug application dhcps/dhcp6s -1 Enable real-time debug of DHCP server activity.
This will show DHCP messages sent/received,
DHCP options sent in each reply, details of
requesting hosts.
diagnose debug application dhcprelay/dhcp6r Enable real-time debug of the DHCP relay agent,
-1 dhcp6r is for DHCPv6.
diagnose debug application dhcpc/dhcp6c -1 Enable real-time debug when Fortigate is itself a
DHCP Client.
15
Command Description
dia sni pa any 'port 67 or port 68' 6 and for Run packet sniffer for DHCP or DHCPv6 packets
DHCPv6 dia sni pa any 'port 546 or port 547' 6 reaching the Fortigate.
NTP debug
Table 9. NTP daemon diagnostics and debug
Command Description
Command Description
show system snmp community Show SNMP community and allowed hosts
configuration
BGP
Table 11. BGP debug
Command Description
diagnose ip router bgp level info Set BGP debug level to INFO (the default is
ERROR which gives very little info) and enable
diagnose ip router bgp all enable the BGP debug.
exec router clear bgp all Disconnect all BGP peering sessions and clear
BGP routes in BGP table and RIB. Use with care,
involves downtime.
get router info bgp summary State of BGP peering sessions with peers, one per
line.
get router info bgp network <prefix> Detailed info about <prefix> from the BGP
process table. Output includes all learned via
BGP routes, even those not currently installed in
RIB. E.g. get router info bgp network 0.0.0.0/0.
The <prefix> is optional, if absent shows the
whole BGP table.
get router info routing-table bgp Show BGP routes actually installed in the RIB.
get router info bgp neighbors Detailed info on BGP peers: BGP version, state,
supported capabilities, how many hops away,
reason for the last reset.
get router info bgp neighbors <IP of the Show all routes advertised by us to the specific
neighbor> advertised-routes neighbor.
get router info bgp neighbors <IP of the Show all routes learned from this BGP peer. It
neighbor> routes shows routes AFTER filtering on local peer, if
any.
get router info bgp neighbors <IP of the Show all received routes from the neighbor
neighbor> received-routes BEFORE any local filtering is being applied. It
only works if set soft-reconfiguration enable is
set for this peer under router bgp configuration.
diagnose sys tcpsock | grep 179 List all incoming/outgoing TCP port 179 sessions
for BGP.
OSPF
Command Description
get router info ospf status Info about OSPF for the whole Fortigate: Router
ID, Hello timer, stats of LSA originated/received,
OSPF Areas configured, number of neighbors for
each Area, whether authentication is enabled
per Area.
get router ospf Show all general OSPF process settings, default
and not.
get router info ospf neighbor Get info on all neighbors of this Fortigate - their
IPs, state, Dead Interval timers.
get router info ospf interface inerface-name Show OSPF info for a given interface - Area,
Router ID, timers, DR/BDR for broadcast nets,
MTU as seen by OSPF.
get router info ospf route Show OSPF routes installed into RIB.
17
Command Description
diagnose ip router ospf level info Set OSPF debug level to the highest - info. A must
before running OSPF debug to see relevant
information.
diagnose ip router ospf option Enable specific debug option. If in doubt just use
dia ip router ospf all -1 to enable all OSPF
debug options. Some options expect integer as
info level as the last parammeter, some options
need enable as the last parameter. Options:
Admin sessions
Table 12. Admin sessions management
Command Description
get sys info admin status List logged in administrators showing INDEX
value for each session
Command Description
diagnose firewall auth list List all authenticated and known by firewall
usernames. It does not matter what the source is
- LDAP/SSO/etc. Also shows client’s IP, idle time,
duration.
diagnose debug app fnbamd -1 Enable debug for authentication daemon, valid
for ANY remote authentication - RADIUS, LDAP,
TACACS+.
diagnose test authserver ldap <LDAP server Test user authenticaiton on Fortigate CLI against
name in FG> <username> <password> Active Directory via LDAP. E.g. test user Tara
Addison against LDAP server configured in
Fortigate as LDAP-full-tree having password
secret: diagnose test authserver ldap LDAP-
full-tree "Tara Addison" secret.
diagnose test authserver radius <RADIUS Test user authentication against the configured
server object> <chap/pap/mschap2> RADIUS server. E.g. to test user adminad1 with
<username> <password> password secr3t against RADIUS server RAD1:
diagnose test authserver radius RAD1 mschap2
adminad1 secr3t Output: authenticate 'adminad1'
against 'mschap2' succeeded, server=primary
assigned_rad_session_id=9839905755137
session_timeout=0 secs idle_timeout=0 secs!
diagnose debug authd fsso list List logged in users the Fortigate learned via
FSSO
diagnose debug authd fsso server-status Show status of connections with FSSO servers.
Note: it shows both, local and remote FSSO
Agent(s). The local Agent is only relevant when
using Direct DC Polling, without installing FSSO
Agent on AD DC, so it is ok for it to be waiting
for retry … 127.0.0.1 if you don’t use it. The
working state should be connected.
diagnose debug authd fsso refresh-group Refresh groups and groups memberships info.
19
Command Description
diag autoupdate versions Show current databases versions and the last
time they were updated.
diagnose debug application update -1 Enable updates debug and then force the
updates.
dia debug enable
execute update-now
diag webfilter fortiguard cache dump Show cache of web sites ratings responses from
the Fortiguard.
dia test app urlfilter 99 Restart the URL filtering daemon, causes short
downtime.
dia deb app urlfilter -1 Enable URL filering daemon debug, showing all
processing steps, a lot of output as shows
everything.
Command Description
get log fortianalyzer filter Verify if any log sending filtering is being done,
look for values of filter and filter-type. If
there are any filters, it means not all logs are
sent to FAZ.
exec telnet <IP of Fortianalyzer> 514 Test connectivity to port 514 on the
Fortianalyzer. If pings are allowed between
them, you can also try pinging.
diagnose sniffer packet any 'port 514' 4 Run sniffer on Fortigate to see if devices
exchange packets on port 514. Click in GUI on
Test Connectivity to initiate connection.
Command Description
diagnose sys sdwan health-check (6.4 and Show state of all the health checks/probes.
newer) Successful probes are marked alive, failed
probes are marked dead. Also displays packet-
diagnose sys virtual-link health-check (5.6 up loss, latency, jitter for each probe.
to 6.4)
diagnose sys sdwan service List configured SD-WAN rules (aka services),
except the Implied one which is always present
diagnose sys virtual-wan-link service (5.6 up and cannot be disabled, but is editable for the
to 6.4) default load balancing method used. Shows
member interfaces and their status alive or dead
for this rule.
diag sys sdwan intf-sla-log <interface name> Print log of <interface name> usage for the last
10 minutes. The statistics shown in bps:
diag sys virtual-wan-link intf-sla-log inbandwidth, outbandwidth, bibandwidth, tx bytes,
<interface name> (5.6 up to 6.4) rx bytes.
21
Command Description
diag netlink interface clear <interface name> Clear traffic statistics on the interface, this resets
statistics of the SD-WAN traffic passing over this
interface. Needed, if, for example, you changed
SD-WAN rules, but not sure if it’s already active.
E.g. diag netlink interface clear port1.
diagnose firewall proute list List ALL Policy Based Routes (PBR). SD-WAN in
Fortigate, after all, is implemented as a variation
of PBR. This command lists manual (classic) PBR
rules, along with SD-WAN created via SD-WAN
rules. Important: Manually created PBR rules
(via Network → Policy Routes or on CLI config
route policy always have preference over the
SD-WAN rules, and this command will show
them higher up.
Command Description
get sys status | grep -i lic Get status of the license (for VM only). The corect
status is Valid.
diagnose hardware sysinfo vm full Show license data as seen by FortiGuard: status
(should be valid=1), last time it was checked
(recv), answer code, should be code: 200, code:
401 is for duplicate license found, code: 502 is for
VM cannot connect to FortiGuard, and code: 400
is for invalid license.
config sys settings Show the current SIP inspection mode. If the
output is default-voip-alg-mode: proxy-based
get | grep alg then the full Layer 7 proxy SIP inspection is on
(ALG inspection). If the output is default-voip-
alg-mode: kernel-helper-based then the Layer 4
helper inspection is on. In both modes Fortigate
does IP address translation inside SIP packets (if
needed), and opens dynamically high ports for
incoming media/voice streams ports. In ALG
mode, the Fortigate additionally does RFC
compliance verification and more. So, the ALG
mode is more prone to cause issues but also
provides more security.
show system session-helper | grep sip -f If using SIP helper and not ALG, make sure there
is an entry for SIP in the helpers list, usually on
port 5060, but may be custom as well.
diagnose debug application sip -1 Display SIP debug in real-time (lots of output). It
shows IP replacement inside SIP packets if NAT
involved, all SIP communication requests
(REGISTER,INVITE etc.), and reply codes.
get system dns Show configured DNS servers, DNS cache limit
and TTL, source IP used, timeout and retry,
whther NDS over TLS is enabled.
diagnose test app dnsproxy Will present all debug options for dnsproxy.
Belowi are some of more useful of them.
diagnose test app dnsproxy 2 Show the following statatistics: number of DNS
process workers (if multiple), DNS latency
against each server used, Secure DNS IP and
latency - DNS server used for DNS filtering and
Botnet detections, DNS cache usage, UDP vs TCP
requests statistics, name of DNS Filter applied if
any.
diagnose test app dnsproxy 3 Display detailed statistics for each DNS/SDNS
server used and those that could be used.
23
Command Description
diagnose test app dnsproxy 6|4|5 Work with FQDN resolved objects:
diagnose test app dnsproxy 10 Show active SDNS, i.e. DNS Filter Policy used.
Shows Categories as numbers, so not easily
readable.
diagnose test app dnsproxy 12 Reload configuration of DNS Filter, in case the
changes made do not take effect immediately.
diagnose test app dnsproxy 15 Show cached responses and their rating of the
DNS Filter for each URL/domain scanned.
diagnose test app dnsproxy 16 Clear the DNS Filter responses and ratings cache.
diagnose test app dnsproxy -1 Enable all possible debug, a lot of output.
diagnose debug application httpsd -1 Enable diagnostics for administrator and remote
REST API access via api-user. When debugging
API automation, refrain from working in admin
GUI as it will produce a lot of unrelated output.
dia debug cli 8 Nice trick: this will print CLI commands the
Fortigate runs when you do something in the
GUI. This way we can find CLI commands
without long search in Google or documentation.
diagnose wireless-controller wlac -c ap-status Show list of all Access Points (APs) this Fortigate
is aware of with their BSSID (MAC), SSID, and
Status (accepted, rogue, suppressed)
diagnose wireless-controller wlac -c vap Show list of APs with their BSSIDs, broadcasted
SSIDs, IDs, and unlike wlac -c ap-status above,
also shows management IP and port which can
be later used for real-time debug.
FortiTokens
Command Description
25
Command Description
exec ping fds1.fortinet.com Verify that Fortigate can resolve and ping the
FortiGuard servers responsible for FortiToken
exec ping directregistration.fortinet.com activation/license validation.
diagnose automation test stitch-name log-if- Run the specified stitch name, optionally adding
needed log when using Log based events.
dia debug app alertmail -1 Enable sessions debug for sending alerts by
mail. This will show the configured settings, like
from/to email address, as well as SMTP session
log of connecting to the remote mail server and
received/sent SMTP session codes.
fnsysctl cat <filename> Show contents of a file, not all files in the
filesystem are accessible. Some examples:
fnsysctl kill <process-id> Kill a process by its ID (PID). The only option
accepted is -s N where N is the signal number to
send as per Linux.
27
Command Description
fnsysctl grep <regex> <file> Search contents of a file/files. The usual grep
options are available:
-q Quiet