How To Performance Logs or NC Pulse Windows
How To Performance Logs or NC Pulse Windows
Performance Logs on
Windows
How-to
Contents
Introduction 4
Part 1: Client Prerequisites 4
Step 1.1: Packet Sniffer 4
Step 1.2: Output of IPs, Routes, Ping, and Trace Route 4
Step 1.3: Delete Client-Side Logging 6
Step 1.4: Timezone of the client 6
Part 2: Server Prerequisites 7
Step 2.1: Enable Relay Protection 7
Step 2.2: Disable the Upload Logs feature 7
Step 2.3: Bandwidth Management 8
Step 2.4: FTP 9
Step 2.5: Additional Information 10
Part 3: Log Gathering Procedure 10
Step 3.1: Enable Server-Side Logging 10
Step 3.1.1: Enable Server-Side logging for NC client log files 10
Step 3.1.2: Debuglog 11
Step 3.1.3: SA TCP Dump 12
Step 3.2: Enable Client-Side Logging 13
Step 3.2.1: Start Packet Sniffer on the Local Client Adapter 13
Step 3.2.2: Start Packet Sniffer on the Pulse Secure Virtual Adapter 14
Step 3.3: Application Packet Sniffer 15
Step 3.4: Reproduce the Issue 15
Step 3.5: Turn off Server-Side logging and Collect Log files 17
Step 3.5.1: SA TCP Dump 17
Step 3.5.2: System Snapshot 17
Step 3.5.3: User Access Log, Event Log, and Administrator Access Log 18
Step 3.5.4: Disable Server-Side logging for NC client log files 19
Step 3.6: Turn off Client-Side logging and Collect Log files 19
Step 3.6.1: Stop Packet Sniffer on the Local Client Adapter
Introduction
The procedure outlined in this document describes the process of enabling and collecting the required log
files which are required by Pulse Secure Support in order to troubleshoot slowness issues over the Network
Connect (NC) or Pulse VPN tunnel. These log collection requirements outlined in this document do NOT apply
to installation, launch, or access related issues. Pulse combines the features of Odyssey Access Client for LAN
access, Network Connect or the SRX client software for WAN access, and WX client software for application
acceleration services. Users of mobile devices (smartphones) can also enable Pulse for secure connectivity to a
Secure Access Appliance. In this case, please note that any and all references made to Pulse within the contents
of this document specifically refer to the feature which allows for a Layer 3 VPN connection to the Pulse Secure
Access Appliance on a Windows Desktop.
While Pulse Secure does not advocate or condemn the use of any such packet sniffer in particular, we commonly
suggest the use of Wireshark as the full version is freely available for commercial use and it is supported on a
wide range of platforms (Windows, MAC OS, Linux, and so forth). See Wireshark’s FAQ for additional details.
A print out of the IP Addressing assigned to the interface(s) and on the client will allow us to determine the
network details for each interface currently in use on the client prior to setting up the VPN tunnel.
Routes
The Routing Table will confirm where data packets traveling over the client’s network is normally directed when
the Pulse Secure VPN is not in use.
Ping
In order to determine if there is any latency of the SA over the Wide Area Network (WAN), we will need to run the
ping command to measure the response times.
Trace Route
Because the number of hops in between the client and the server can have an impact on latency, we suggest
a. Windows Vista/Windows 7: Select Start and type cmd in the search bar
b. Windows XP: Select Start > Run and type cmd in the run prompt
2. Create text file outputs of IPs, Routes, Ping, and Trace Route:
b. Type route print > routes_before.txt and select enter on your keyboar
c. Type ping -c 5 <Secure-Access-IP> > ping.txt and select enter on your keyboard
d. Type traceroute <Secure-Access-IP> > traceroute.txt and select enter on your keyboard.
1. In the Admin Console choose Resource Policies > Network Connect > NC Connection Profiles
3. Ensure that Replay Protection is checked. If not, please put a check by Relay protection to enable it.
Note: Even if you manually reverse the logging back to 3 in the registry, the NC\Pulse UI will continue
to be set to “detailed” level if the Upload Log feature is enabled on the role.
The reason we ask that you disable this option for all performance based testing is because extended use of this
option enables verbose (Level 5) logging and can create large log files which may impact performance.
If you are running 6.5R10, 7.0R6, and 7.1R2 and above, then this behavior has been changed so that logging
level is automatically set to 3, and it is not required for you to disable the upload logs feature.
Note: You will need to disable this option on each role assigned to the user account currently used
for the subsequent test.
1. In the admin console, choose Users > Resource Policies > Network Connect > NC Bandwidth
Management
2. On the Network Connect Network Connect Bandwidth Management page, click on your existing Bandwidth
Management policy
3. In the Roles section, specify Policy applies to SELECTED roles. Then remove any and all possible roles
which your NC user could have assigned. If the user is assigned multiple (and you are implementing the Merge
settings for all assigned roles setting in your Realm See Users > User Realms > Your Realm > Role Mapping)
roles, then this policy will continue to be enforced if it applies to even one your user’s assigned roles.
If you are reporting a performance issue which is directly related to bandwidth management (For Example:
Users are receiving less than the minimum bandwidth guaranteed by the policy. Or users are receiving more
than the maximum bandwidth allotted by their policy), then please keep in mind that Part 3 of this document
will need to be performed twice: Once with the Bandwidth Management policy enabled and again with the
Bandwidth Management policy disabled.
a) Both download and upload to this FTP server is allowed and functioning for the designated test
user.
b) If your application uses a GUI interface, you observe a “slight” improvement of the command line
FTP application as it lacks the additional overhead associated with a GUI. If you do not observe at
least equal or slight improvement in performance when using command line ftp, then ftp will not be
of any assistance to us. Likely, there are other external factors which could be introducing additional
latency to the network hosting the FTP server and or the Application server. Please rule out any such
factors before proceeding with ftp.
Assuming the above criteria have been met, we can then use command line ftp as a baseline to compare
the performance of your application. This can also help us to identify and isolate if the issue lies between
the client and the SA or between the SA and the Application Server for the problem application. Please note
that we do not “require” the use of FTP (as we understand this option cannot be enabled in some production
environments), but it is strongly recommended for the reasons already stated above.
2. If this is in an A/A Cluster, can you replicate without the Load Balancer?
4. Screenshot of the Dashboard\Central Manager Graph on the SA (System > Status > Overview)
5. Network Topology Diagram which labels each device by name and IP Address.
7. What are we comparing NC slowness to (Third Party VPN, Internal Corporate Access, or similar Secure
Access setup)?
Client-side logging is useful for debugging problems with an SA Series Appliance client-side feature such as
Network Connect. When you enable logging Network Connect, the Appliance writes a log to any client computer
that uses either Network Connect. These settings are global, which means that the SA Series Appliance writes
a log file to all clients that use the enabled feature. The Appliance then appends to the log file each time the
feature is invoked during subsequent user sessions. Please note that logging on the client will be automatically
enabled at level 3.
1. In the Admin Console choose System > Log/Monitoring > Client Side Logs > Settings
The Server-Side debug logging requested in this document may increase overall system load, so we ask that you
enable this option only if one or both criteria are met:
a) The Secure Access currently in use for the following test does not have a heavy Network Connect
load.
Debug logs are particularly important in the event of a problem. You will need to set the debug log at a certain
level and add the events list as directed below. We can include both the Debuglog and the system configuration
in the System Snapshot. The debuglog is encrypted; you cannot view it.
To enable Debuglog:
1. In the Admin Console choose Maintenance > Troubleshooting > Monitoring > Debug
3. Set Max Debug Log Size to 50, Debug Log Detail Level to 30, and Event Codes to ipsec,dhcp (Comma
separated, no spaces)
TCP Dump is a packet sniffer which is the built-in to the Secure Access Appliance. This packet capture will be
required by Pulse Secure Support in order to validate the traffic pattern(s) and behavior on the local network of
the Appliance.
1. In the Admin Console choose Maintenance > Troubleshooting > Tools > TCP Dump
2. Make sure that Promiscuous mode is turned on, and that you are sniffing on the Internal Port and
External Port (if enabled).
Note: If you have a large number of users logged in to your Appliance, please create a filter for the
assigned Network Connect IP and the Source IP of the client. Example: host X.X.X.X or host X.X.X.X. If
you are unsure what the NC IP and WAN IP of the client will be, then you can log into the Appliance
from the client in order to obtain this information from the User Access Log.
Unless there are fewer IP Addresses than the number of NC users, the user is not getting mapped to
the same roles, or the user has not used NC in the last 24 hours then the user will receive the last NC
IP he was assigned from the Appliance.
If you have opted to use a packet sniffer other than the built in “tcpdump” application, please see Vendor
instructions for enabling the packet capture on the appropriate interface. Otherwise, if you have chosen to
proceed with tcpdump, you can enable it using the instructions provided below.
a. Windows Vista/Windows 7: Select Start and type wireshark in the search bar
b. Windows XP: Select Start > Run and type wireshark in the run prompt
3. Select the Pulse Secure Virtual Adapter in order to start the capture.
Step 3.2.2: Start Packet Sniffer on the Pulse Secure Virtual Adapter
If you have opted to use a packet sniffer other than Wireshark, please see Vendor instructions for enabling the
packet capture on the appropriate interface. Otherwise, if you have chosen to proceed with Wireshark, you can
enable it using the instructions provided below. Please note that if Wireshark was installed before you installed
NC\Pulse, then the Pulse Secure Virtual Adapter may not register with Wireshark until after you perform a
reboot.
a. Windows Vista/Windows 7: Select Start and type wireshark in the search bar
b. Windows XP: Select Start > Run and type wireshark in the run prompt
3. Select the Pulse Secure Virtual Adapter in order to start the capture.
a. Windows Vista/Windows 7: Select Start and type cmd in the search bar
b. Windows XP: Select Start > Run and type cmd in the run prompt
b. Browse to the directory designated for testing (Example: Type cd /pub/incoming and press enter
on your keyboard)
g. Highlight the entire FTP output and save in a text file titled ftp.txt in the Client_Logs folder
Step 3.5: Turn off Server-Side logging and Collect Log files
Once you have replicated the issue, please collect the following log files while NC is still connected. Do not
disconnect from NC until after all of the Server-Side and Client-Side logs files have been collected.
1. In the Admin Console choose Maintenance > Troubleshooting > Tools > TCP Dump
3. Under the Dump File section choose Raw from the drop down, then select Get to save the file as ive-
<date>-<time>.dmp
When you use this option, the Appliance runs various utilities to gather details on the system state, such as the
amount of memory in use, paging performance, the number of processes running, system uptime, the number
of open file descriptors, ports in use, and FIPS log messages. We can include both the Debuglog and the system
configuration in the System Snapshot. The system snapshot and debuglog is encrypted; you cannot view it.
To take System Snapshot, turn off debuglog, save the snapshot (which includes the debuglog):
1. In the Admin Console choose Maintenance > Troubleshooting > Monitoring > Debug
Step 3.5.3: User Access Log, Event Log, and Administrator Access Log
To collect the User Access Log, Event Log, and Administrator Access Log all at one time:
10. Select Save All Logs to save all three log files as Pulse Securelogs.tar.gz
1. In the Admin Console choose System > Log/Monitoring > Client Side Logs > Settings
Step 3.6: Turn off Client-Side logging and Collect Log files
Once you have replicated the issue, please collect the following log files while NC is still connected. Do not
disconnect from NC until after all of the Server-Side and Client-Side logs files have been collected.
Step 3.6.1: Stop Packet Sniffer on the Local Client Adapter and save it in pcap format
If you have opted to use a packet sniffer other than Wireshark, please see Vendor instructions for enabling the
packet capture on the appropriate interface. Otherwise, if you have chosen to proceed with Wireshark, you can
enable it using the instructions provided below.
To stop Wireshark on the Local Client Adapter and save it in pcap format:
Step 3.6.2: Stop Packet Sniffer on the Pulse Secure Virtual Adapter and save it in pcap format
If you have opted to use a packet sniffer other than Wireshark, please see Vendor instructions for enabling the
packet capture on the appropriate interface. Otherwise, if you have chosen to proceed with Wireshark, you can
enable it using the instructions provided below.
To stop Wireshark on the Pulse Secure Virtual Adapter and save it in pcap format:
To collect client-side logs for Network Connect or Pulse, browse to the following locations and save the
debuglog.log and debuglog.old:
c. Please save the debuglog.log and debuglog.old in the Client_Logs folder. In the case of Windows
Vista\Windows 7, please rename the logs in the public folder to something which is unique. Example:
debuglog_public.log and debuglog_public.old
A print out of the IP Addressing assigned to the interface(s) and on the client will allow us to determine the
network details for the NC interface while the VPN tunnel is connected. The Routing Table will confirm where
data packets traveling over the client’s network when the Pulse Secure VPN is in use.
a. Windows Vista/Windows 7: Select Start and type cmd in the search bar
b. Windows XP: Select Start > Run and type cmd in the run promptCreate text file outputs of IPs
and Routes:
2. Create text file outputs of your local Route Table and IP Address:
a. Type ipconfig /all > ipconfig_after.txt and select enter on your keyboard.
b. Type route print > routes_after.txt and select enter on your keyboard
Step 3.7: Turn off Application Packet Sniffer and Save Capture
You can save this file as ApplicationServer.pcap.
You can now disconnect from NC on the client machine. Please confirm that you have the following log
files both from the Server-Side and the Client-Side:
1. ipconfig_before.txt
2. routes_before.txt
3. ping.txt
4. tracert.txt
5. client_timezone.txt
8. ive-<date>-<time>.dmp
9. Pulse Secure-state-admin-<date>-<time>
11. ipconfig_after.txt
12. routes_after.txt
13. NC_Adapter.pcap
14. LAN_Adapter.pcap
16. In the case of Windows Vista/Windows 7 only you should also have a second set of client logs.
(Example: debuglog.log, debuglog.old, debuglog_public.log, debuglog_puplic.old)
17. ApplicationServer.pcap
For any questions or issues relating to the procedures outlined in this document, please contact support. For
details on how to engage support, please refer to the following link: https://www.pulsesecure.net/support