092-2.

book Page 173 Wednesday, July 14, 1999 1:52 PM

CHAPTER
4

General Troubleshooting Tools

This chapter presents an overview of the various tools that are available for troubleshooting
networks that include Cisco internetworking devices. These tools include network
management applications; third-party hardware tools such as digital test equipment, time
domain reflectometers (TDRs), and digital interface testing tools; and software tools such
as network monitors, protocol analyzers, and simulation/modeling tools.
Each of these tools has a specific purpose and works at specific OSI reference model layers.
Understanding what each tool can do and which tool is appropriate for each troubleshooting
task will help you become a more efficient network technician.
When troubleshooting, you should start at the physical layer. Use cable testers and other
low-level testers to ensure that there are no problems with the media, such as noise, too
much attenuation, improper cable lengths, improper connectors, and so forth. If the
physical layer seems fine, then move up the layers to the data link layer. You can use a
protocol analyzer to check for excessive collisions on Ethernet, beaconing on Token Ring
or FDDI networks, excessive soft errors on Token Ring, and other link-layer issues. If the
data link layer seems fine, check for routing errors or misconfigurations at the network
layer, using a protocol analyzer and Cisco IOS commands. Finally, you can look for
upper-layer problems such as misconfigurations, software bugs, and user errors.

Low-End Cable Test Equipment

At the low-technology end of the spectrum of test equipment are volt-ohm meters
and digital multimeters. These devices measure parameters such as AC and DC voltage,
current, resistance, capacitance, and cable continuity. They can be used to check physical
connectivity.
092-2.book Page 174 Wednesday, July 14, 1999 1:52 PM

174 Chapter 4: General Troubleshooting Tools

Cable testers (that is, scanners) can also be used to check physical connectivity. Cable testers
give users access to physical-layer information and are available for shielded twisted-pair
(STP), unshielded twisted-pair (UTP), 10BaseT, and coaxial and twinax cables. These testers
can test and report cable conditions including near-end crosstalk (NEXT), attenuation, and
noise. Some of them also have TDR, traffic monitoring, and wire map functions. In addition,
some handheld network testers display Media Access Control (MAC) layer information about
LAN traffic, provide statistics such as network utilization and packet error rates, and perform
limited protocol testing (for example, TCP/IP tests such as ping).
Similar testing equipment is available for fiber-optic cable. Due to the relatively high cost of
fiber cable and its installation, it is recommended that fiber-optic cable be tested before
installation (that is, on-the-reel testing) and after installation. Continuity testing of the fiber
requires either a visible light source or a reflectometer. Light sources capable of providing light
at the three predominant wavelengths—850 nm, 1300 nm, and 1550 nm—are used with power
meters that can measure the same wavelengths and test attenuation and return loss in the fiber.
Figure 4-1 shows one of the cable scanners available from Microtest: the OMNI Scanner. The
OMNI Scanner has the functionality to test cables complying with current and upcoming
standards with an extremely wide dynamic range of 100 dB and the ability to support up to
300 MHz bandwidth. The OMNI Scanner can test all the way up to 300 MHz on Category 7
cables.

NOTE Microtest, Inc., is located at 4747 N. 22nd St., Phoenix, AZ 85016-4708, and can be reached at
602-952-6400.

Figure 4-1 Microtest’s OMNI Scanner handheld cable scanners can test a wide range of cable.
092-2.book Page 175 Wednesday, July 14, 1999 1:52 PM

High-End Cable Testers 175

High-End Cable Testers

At the most technologically advanced end of the cable testing spectrum are TDRs. These
devices can quickly locate opens, shorts, crimps, kinks, sharp bends, impedance mismatches,
and other defects in metallic cables.
A TDR works by “bouncing” a signal off the end of the cable, much like radar. Opens, shorts,
and other problems reflect the signal back at different amplitudes, depending on the problem.
A TDR measures how much time it takes for the signal to reflect (that is, round-trip time) and
uses the principle
Distance = Rate of propagation × Time to calculate the distance to a fault in the cable
When a signal reaches the end of a cable, it reflects at a very low amplitude, so TDRs can also
be used to measure the length of a cable. Some TDRs can also calculate the propagation rate
based on a configured cable length.
Fiber-optic measurement is performed by an optical TDR (OTDR). These devices can
accurately measure the length of the fiber, locate cable breaks, measure the fiber attenuation,
and measure splice or connector losses by measuring the reflections that occur. Pulse reflections
that are generated at breaks or joints, and backscatter reflections that are generated uniformly
throughout the cable, are used to measure the fiber attenuation. One way in which the OTDR
can be put to good use is to take the signature of a particular installation, noting attenuation and
splice losses. This baseline measurement can then be compared with future signatures when a
problem in the system is suspected.
Figure 4-2 shows a TDR made by Biddle.

NOTE For more information on this product, you can contact AVO International at www.avointl.com/
contact/index.html.
092-2.book Page 176 Wednesday, July 14, 1999 1:52 PM

176 Chapter 4: General Troubleshooting Tools

Figure 4-2 The Biddle 510B is a handheld TDR that can find trouble on twisted-pair, coaxial, and power cable.

Digital Interface Testing Tools

Several test tools can be used to measure the discrete digital signals that are present at PCs,
modems, printers, and other peripheral interfaces. Examples of this type of test equipment
include breakout boxes, fox boxes, and bit/block error rate testers (BERTs/BLERTs). These
devices can monitor data line conditions, analyze and trap data, and diagnose problems
common to data communication systems. Traffic from data terminal equipment (DTE) through
data communications equipment (DCE) can be examined to help eliminate problems, identify
bit patterns, and ensure that the proper cabling has been installed.
Figure 4-3 shows the line-powered Blue Box 100 breakout box from IDS, Inc. The Blue Box
100 is a breakout box and cable tester that is compact, handheld, and fully 100 LED. It accesses
and monitors all 25 conductors of the RS-232-C, EIA-232-D, CCITT, and V.24, and any other
single-ended interface such as the Centronics parallel printer interface. One hundred red and
green LEDs monitor and display high, low, off, and signal activity conditions for each of 25
conductors on the DTE and DCE sides of the interface.
092-2.book Page 177 Wednesday, July 14, 1999 1:52 PM

Network Monitors 177

NOTE For more information on the Blue Box 100, contact IDS, Inc., at 800-IDS-DATA or 401-737-
9900; e-mail [email protected].

Figure 4-3 The Blue Box 100 breakout box is a useful tool for troubleshooting serial cables and connections.

Network Monitors
Network monitors continuously track packets crossing a network, providing an accurate picture
of network activity at any moment or a historical record of network activity over a period of
time. Monitors collect information such as packet sizes, the number of packets, error packets,
overall usage of a connection, the number of hosts and their MAC addresses, and details about
communications between hosts and other devices. Correlation of this data allows network
administrators to create profiles of their LAN traffic and find traffic overloads, plan for network
expansion, detect intruders, establish baseline performance, and distribute traffic more
efficiently.
Not only must the monitor collect information about frames, but it must also be able to warn
users if any frames are dropped or flag users if certain events such as bad frames, protocol
errors, or illegal addresses occur. Visible and audible alarms for the entire network or for
individual stations can be set, allowing the network manager to be informed when certain
parameters have exceeded predetermined thresholds.
092-2.book Page 178 Wednesday, July 14, 1999 1:52 PM

178 Chapter 4: General Troubleshooting Tools

The concept of baselining is becoming very important to network managers. To create a

baseline, the activity on a network is sampled over a period of time, and averages, means, and
other statistical calculations are used to establish a normal performance profile, or baseline.
This baseline can then be used as a reference if any abnormal performance is noted in the
network, or it can be used to plan expansion options.
Network monitors further enhance network management by gathering information from remote
sites and sending it back to a central management location.
Apart from gathering the standard traffic information, many monitors implement Simple
Network Management Protocol (SNMP), Remote Monitoring (RMON), and Management
Information Bases (MIBs) to gather information for central management stations. CiscoWorks
can also supply network monitoring functions.
Figure 4-4 shows some of the monitor screens on a Sniffer Pro product. These charts and graphs
enable you to easily build graphical baseline reports on your network.

Figure 4-4 The Sniffer Pro can provide network monitoring services.

Protocol Analyzers
A protocol analyzer records, interprets, and analyzes how a communication protocol operates
in a particular network architecture. It captures frames as they travel across the network. It then
decodes the various layers of protocol in the recorded frame contents and presents them as
readable abbreviations or summaries, detailing what layer is involved (physical, data link, and
some protocol analyzers, right up to the application layer) and what function each byte or byte
content serves. With LAN/WAN networks that involve multiple protocols, it is important that
a protocol analyzer be able to detect and decode all the protocols used in the network
environment.
092-2.book Page 179 Wednesday, July 14, 1999 1:52 PM

Protocol Analyzers 179

In capture mode, filters can be set to record only traffic that meets certain criteria; for example,
if a particular unit is suspected of inconsistent protocol behavior, then a filter can be configured
that captures all traffic to and from that unit. The analyzer should have the capability to
timestamp all the captured data. This can be extremely important when determining the effects
of peak traffic periods and when analyzing network performance—for example, determining
protocol response times by measuring the delta time between frames.
In display mode, an analyzer interprets the captured traffic, presenting the protocol layers in an
easily readable form. Filters can be set to allow only those captured frames that meet certain
criteria to be displayed.
It is also important that the analyzer be able to generate frames and transmit them onto the
network in order to perform capacity planning or load testing of specific devices such as servers,
bridges, routers, and switches. The analyzer should be able to send multiple captured frames in
succession, as well as allow network managers to tailor the frames by being able to edit the
frames prior to generation.
Figure 4-5 shows a packet that is decoded by the Sniffer Pro protocol analyzer. Sniffer Pro
analyzers include the Expert System that identifies fault symptoms and provides a diagnosis of
the network problems. Sniffer Pro provides decodes for more than 250 protocols.

NOTE For more information on Sniffer Pro, see the Network Associates Web site at www.nai.com.

Figure 4-5 The Sniffer Pro can decode frame and packet information.

DLC: ----- DLC Header -----

DLC:
DLC: Frame 1 arrived at 15:05:33.389; frame size is 62 (003E hex) bytes.
DLC: AC: Frame priority 0, Reservation priority 0, Monitor count 0
DLC: FC: LLC frame, PCF attention code: None
DLC: FS: Addr recognized indicators: 00, Frame copied indicators: 00
DLC: Destination = Station cisco A05903
DLC: Source = Station IBM 0AE59
DLC:
Summary Delta T DST SRC
LLC: ----- LLC Header ----- 1 DCE DTE HDLC SABM P/F=1
LLC:
LLC: DSAP = AA, SNAP = AA, C 2 0.0412 DTE DCE HDLC UA P/F=1
LLC:
3 0.0492 DCE DTE HDLC I NR=0 NS=0 P/F=0
SNAP: ----- SNAP Header ----- 4 0.0408 DTE DCE HDLC RR NR=1 P/F=0
SNAP:
SNAP: Type = 0800 (IP) 5 0.0438 DTE DCE HDLC I NR=1 NS=0 P/F=0
SNAP: 6 0.0287 DCE DTE HDLC RR NR=1 P/F=0
7 9.8700 DCE DTE HDLC I NR=1 NS=1 P/F=0
8 0.0379 DTE DCE HDLC RR NR=2 P/F=0
9 0.3000 DTE DCE HDLC I NR=2 NS=1 P/F=0
092-2.book Page 180 Wednesday, July 14, 1999 1:52 PM

180 Chapter 4: General Troubleshooting Tools

Portability of the analyzer is also an important factor because networks are not physically
located in one place, and the analyzer must be moved from segment to segment as problems
arise. Several manufacturers provide tools that allow for the remote gathering (and in some
cases, analysis) of data and transmission back to a central console or master station.
The ability of the analyzer to use a set of rules and knowledge of the network operation to
diagnose network problems is the emergent feature of an expert system. The expert system
gleans its knowledge from theoretical databases (that is, from standards information), from
network-specific databases (that is, topological information relating to the network), and from
users’ previous results and experience. From these repositories, the expert system generates a
hypothesis about the problem it has detected and offers a plan of action to resolve it.
Protocol analyzers are generally available in three categories:
• Software-based analyzers are software packages that are installed on personal computers
(usually portable notebook PCs) that are equipped with appropriate LAN interface
adapters.
• General-purpose analyzers offer a wide range of uses, such as traffic monitoring,
reasonably extensive protocol capture and decode support, and some network traffic
modeling during the network design phase.
• High-end analyzers offer a range of advanced features and can typically capture traffic at
higher rates and provide a more comprehensive protocol decode than can the other
analyzers. They also support generate-and-capture capabilities, which means you can use
them to stress-test parts of the network.

Network Management Systems

As networks grow larger and more complex, there is a greater chance of network failures that
can disable the entire network or degrade performance to an unacceptable level. The complexity
of such large networks makes the use of automated network management tools a critical factor
in efficient management. It is important that the continued addition of users, interfaces,
protocols, and vendor equipment to the network does not result in the network manager losing
control of these resources and how they are used. It is also important that as network resources
become more critical in an organization’s operations, downtime be reduced. To ensure
maximum network availability, network managers should include network management in their
internetwork designs.
The International Organization for Standardization (ISO) has defined five key functional areas
of network management: fault management, accounting management, configuration
management, performance management, and security management.
The functions of fault, performance, and configuration management are most applicable to a
troubleshooting environment. To achieve maximum network availability, all individual
components of a network must be maintained in working order. A key ingredient to achieving
this is having a mechanism in place that reports a fault immediately as it occurs. A fault can be
092-2.book Page 181 Wednesday, July 14, 1999 1:52 PM

Network Management Systems 181

defined as an abnormal network event, usually indicated by network components failing to

operate correctly or causing excessive errors. It is therefore important to be able to do the
following:
• Determine exactly where the fault has occurred.
• Isolate the failed area from the rest of the network so that the rest of the network can
continue operating.
• Reconfigure or modify the network or its configuration to minimize the impact of
operating without the failed component or affected portions of the network.
• Repair or replace the failed components to restore normal network operation.
Configuration management involves several functions. The network manager should be able to
set up the network by initial configuration of the network components and interactively control
these components by changing their configuration in response to performance evaluation or in
response to network upgrades or fault recovery.
SNMP is an application-layer protocol that facilitates the exchange of management information
between network devices. It is part of the TCP/IP protocol suite. SNMP enables network
administrators to manage network performance, find and solve network problems, and plan for
network growth. An SNMP network consists of SNMP agents (managed devices) and an SNMP
management station (manager).
Figure 4-6 shows a typical SNMP design where an SNMP manager queries an SNMP agent on
a router to obtain operational statistics from the agent.

Figure 4-6 The SNMP manager sends queries to the SNMP agent in order to obtain management statistics.

SNMP get Request

Manager

SNMP get
Agent
Request
Agent

SNMP get
Reply

SNMP get Reply

092-2.book Page 182 Wednesday, July 14, 1999 1:52 PM

182 Chapter 4: General Troubleshooting Tools

Simulation and Modeling Tools

Simulation/modeling software is useful for purposes such as initial network design, analysis of
a network reconfiguration or redesign, and stress-testing a network.
This type of software usually uses object-oriented design to predict the performance of
networks, ranging from departmental LANs up to complex, enterprisewide internetworks and
WANs.
By selecting numerous objects that represent network topology, protocols in use, traffic, and
routing algorithms, Netsys Baseliner attempts to simulate the operation of the network. Most
types of LAN, MAN, and WAN technologies can be modeled by these tools. The output gives
measures of the network performance such as response times; network throughput; node, link,
and LAN utilization; packets dropped; and other performance data.
Many analyzer vendors offer the capability to export the data from their analyzers into the
simulation/modeling tools, thus providing a source of real network data.
These simulation/modeling tools allow the network manager to see and test network
performance before committing to proposed designs or changes.

Summary
In this chapter you have learned about several different troubleshooting tools that are used at
various times when troubleshooting and managing internetworks.
Network modeling and simulation tools help you plan a new design or redesign. When
implementing the design, you can use cable testers and other low-level testers to certify the
installation of the cabling. You can use network management tools to simplify the configuration
of routers, switches, and other devices.
When a network is operational, you can use network monitors and network management tools,
including RMON-based applications, to monitor the network for errors and performance
problems. When serious performance problems occur or when the network ceases to operate,
you need low-level testers as well as a protocol analyzer or an RMON tool that lets you capture
and display frames. Many problems can also be diagnosed by using the tools and commands
built in to the Cisco Internetwork Operating System (IOS) software. We will talk more about
these tools in Chapter 5, “Cisco Management and Diagnostic Tools.”
092-2.book Page 183 Wednesday, July 14, 1999 1:52 PM

Chapter 4 Test 183

Chapter 4 Test
General Troubleshooting Tools
Estimated Time: 15 minutes
Complete all the exercises to test your knowledge of the materials contained in this chapter.
Answers are listed in Appendix A, “Chapter Test Answer Key.”
Use the information contained in this chapter to answer the following questions.

Question 4.1
Match the most appropriate tool with the required task.
Answer Task Tool
_____ 1. Analyze network design. a. Cable testers
_____ 2. Examine DTE-to-DCE b. BERT/BLERT testers
communications.
_____ 3. Capture and decode packets. c. Network monitors
_____ 4. Locate crosstalk. d. Modeling tools
_____ 5. Bounce signal off end of e. TDRs
cable to locate distance to fault.
_____ 6. Profile LAN traffic. f. Protocol analyzers

Question 4.2
At which OSI reference model layer should troubleshooting start?
__________________________________________________________________________

Question 4.3
You are concerned about broadcast overhead on the network. Which tool should you use to
determine the current broadcast rate?
__________________________________________________________________________

Question 4.4
You suspect that intermittent disconnections are due to a cable problem on your network. What
tool would help you troubleshoot this most efficiently?
__________________________________________________________________________
092-2.book Page 184 Wednesday, July 14, 1999 1:52 PM

184 Chapter 4: General Troubleshooting Tools

Question 4.5
You are designing a new campus LAN for a client. What tool can you use to review your design
before implementing it?
__________________________________________________________________________