C H A P T E R

Troubleshooting the SIPs and SSC

This chapter describes techniques that you can use to troubleshoot the operation of your SIPs. It includes the following sections:

General Troubleshooting Information, page 5-1 Using the Cisco IOS Event Tracer to Troubleshoot Problems, page 5-2 Troubleshooting Oversubscription on the Cisco 7600 SIP-400, page 5-3 Preparing for Online Insertion and Removal of SIPs, SSCs, and SPAs, page 5-3

The first section provides information about basic interface troubleshooting. If you are having a problem with your SPA, use the steps in the Using the Cisco IOS Event Tracer to Troubleshoot Problems section to begin your investigation of a possible interface configuration problem. To perform more advanced troubleshooting, see the other sections in this chapter.

General Troubleshooting Information

This section describes general information for troubleshooting SIPs, SSCs, and SPAs. It includes the following sections:

Interpreting Console Error Messages, page 5-1 Using debug Commands, page 5-2 Using show Commands, page 5-2

Interpreting Console Error Messages

To view the explanations and recommended actions for Cisco 7600 series router error messages, including messages related to Cisco 7600 series router SIPs and SSCs, refer to the following documents:

Cisco 7600 Series Cisco IOS System Message Guide, 12.2SX (for error messages in Release 12.2SX) System Error Messages for Cisco IOS Release 12.2S (for error messages in Release 12.2S)

System error messages are organized in the documentation according to the particular system facility that produces the messages. The SIP and SSC error messages use the following facility names:

Cisco 7600 SIP-200C7600_SIP200 Cisco 7600 SIP-400SIP400

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide OL-5070-07

5-1

Chapter 5 Using the Cisco IOS Event Tracer to Troubleshoot Problems

Troubleshooting the SIPs and SSC

Cisco 7600 SIP-600SIP600 Cisco 7600 SSC-400C7600_SSC400

Using debug Commands

Along with the other debug commands supported on the Cisco 7600 series router, you can obtain specific debug information for SIPs and SSCs on the Cisco 7600 series router using the debug hw-module privileged EXEC command. The debug hw-module command is intended for use by Cisco Systems technical support personnel. For more information about the debug hw-module command, see Chapter 40, SIP, SSC, and SPA Commands.

Caution

Because debugging output is assigned high priority in the CPU process, it can render the system unusable. For this reason, use debug commands only to troubleshoot specific problems or during troubleshooting sessions with Cisco technical support staff. Moreover, it is best to use debug commands during periods of lower network traffic and fewer users. Debugging during these periods decreases the likelihood that increased debug command processing overhead will affect system use. For information about other debug commands supported on the Cisco 7600 series router, refer to the Cisco IOS Debug Command Reference, Release 12.2 and any related feature documents for Cisco IOS Release 12.2SX.

Using show Commands

There are several show commands that you can use to monitor and troubleshoot the SIPs and SSCs on the Cisco 7600 series router. This chapter describes using the show hw-module slot command to perform troubleshooting of your SPA. For more information about show commands to verify and monitor SIPs and SSCs, see the following chapters of this guide:

Chapter 4, Configuring the SIPs and SSC Chapter 40, SIP, SSC, and SPA Commands

Using the Cisco IOS Event Tracer to Troubleshoot Problems

Note

This feature is intended for use as a software diagnostic tool and should be configured only under the direction of a Cisco Technical Assistance Center (TAC) representative. The Event Tracer feature provides a binary trace facility for troubleshooting Cisco IOS software. This feature gives Cisco service representatives additional insight into the operation of the Cisco IOS software and can be useful in helping to diagnose problems in the unlikely event of an operating system malfunction or, in the case of redundant systems, Route Processor switchover.

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide

5-2

OL-5070-07

Chapter 5

Troubleshooting the SIPs and SSC Troubleshooting Oversubscription on the Cisco 7600 SIP-400

Event tracing works by reading informational messages from specific Cisco IOS software subsystem components that have been preprogrammed to work with event tracing, and by logging messages from those components into system memory. Trace messages stored in memory can be displayed on the screen or saved to a file for later analysis. The SPAs currently support the spa component to trace SPA OIR-related events. For more information about using the Event Tracer feature, refer to the following URL: http://www.cisco.com/en/US/products/sw/iosswrel/ps1829/products_feature_guide09186a0080087164 .html

Troubleshooting Oversubscription on the Cisco 7600 SIP-400

As of Cisco IOS Release 12.2(18)SXF, when using the Cisco 7600 SIP-400 with the 2-Port Gigabit Ethernet SPA or the 1-Port OC-48c/STM-16 ATM SPA, consider the following oversubscription guidelines:

The Cisco 7600 SIP-400 only supports installation of one 1-Port OC-48c/STM-16 ATM SPA without any other SPAs installed in the SIP. The Cisco 7600 SIP-400 supports installation of up to two 2-Port Gigabit Ethernet SPAs without any other SPAs installed in the SIP. The Cisco 7600 SIP-400 supports installation of any combination of OC-3 or OC-12 POS or ATM SPAs, up to a combined ingress bandwidth of OC-48 rates. The Cisco 7600 SIP-400 supports installation of any combination of OC-3 or OC-12 POS or ATM SPAs up to a combined ingress bandwidth of OC-24 rates, when installed with a single 2-Port Gigabit Ethernet SPA.

Configurations on the Cisco 7600 SIP-400 with an unsupported aggregate SPA bandwidth greater than OC-48 rates generates the following error message:
SLOT 3: 00:00:05: %SIPSPA-4-MAX_BANDWIDTH: Total SPA bandwidth exceeds line card capacity of 2488 Mbps

Preparing for Online Insertion and Removal of SIPs, SSCs, and SPAs
The Cisco 7600 series router supports online insertion and removal (OIR) of the SPA interface processor (SIP) or SPA services card (SSC), in addition to each of the shared port adapters (SPAs). Therefore, you can remove a SIP or SSC with its SPAs still intact, or you can remove a SPA independently from the SIP or SSC, leaving the SIP or SSC installed in the router. This section includes the following topics on OIR support:

Preparing for Online Removal of a SIP or SSC, page 5-3 Verifying Deactivation and Activation of a SIP or SSC, page 5-5 Preparing for Online Removal of a SPA, page 5-5 Verifying Deactivation and Activation of a SPA, page 5-7 Deactivation and Activation Configuration Examples, page 5-7

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide OL-5070-07

5-3

Chapter 5 Preparing for Online Insertion and Removal of SIPs, SSCs, and SPAs

Troubleshooting the SIPs and SSC

Note

For simplicity, any reference to SIP in this section also applies to the SSC.

Preparing for Online Removal of a SIP or SSC

The Cisco 7600 series router supports OIR of the SIP and the SSC. To do this, you can power down a SIP (which automatically deactivates any installed SPAs) and remove the SIP with the SPAs still intact. Although graceful deactivation of a SIP is preferred using the no power enable module command, the Cisco 7600 series router does support removal of the SIP without deactivating it first. If you plan to remove a SIP, you can deactivate the SIP first, using the no power enable module global configuration command. When you deactivate a SIP using this command, it automatically deactivates each of the SPAs that are installed in that SIP. Therefore, it is not necessary to deactivate each of the SPAs prior to deactivating the SIP. Either a blank filler plate or a functional SPA should reside in every subslot of a SIP during normal operation. For more information about the recommended procedures for physical removal of the SIP, refer to the Cisco 7600 Series Router SIP, SSC, and SPA Hardware Installation Guide.

Deactivating a SIP or SSC

To deactivate a SIP or SSC and its installed SPAs prior to removal of the SIP, use the following command in global configuration mode: Command Router(config)# no power enable module slot Purpose Shuts down any installed interfaces, and deactivates the SIP in the specified slot, where:

slotSpecifies the chassis slot number where the SIP is installed.

For more information about chassis slot numbering, refer to the Identifying Slots and Subslots for SIPs, SSCs, and SPAs section in this guide.

Reactivating a SIP or SSC

Once you deactivate a SIP or SSC, whether or not you have performed an OIR, you must use the power enable module global configuration command to reactivate the SIP. If you did not issue a command to deactivate the SPAs installed in a SIP, but you did deactivate the SIP using the no power enable module command, then you do not need to reactivate the SPAs after an OIR of the SIP. The installed SPAs automatically reactivate upon reactivation of the SIP in the router. For example, consider the case where you remove a SIP from the router to replace it with another SIP. You reinstall the same SPAs into the new SIP. When you enter the power enable module command on the router, the SPAs will automatically reactivate with the new SIP.

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide

5-4

OL-5070-07

Chapter 5

Troubleshooting the SIPs and SSC Preparing for Online Insertion and Removal of SIPs, SSCs, and SPAs

To activate a SIP and its installed SPAs after the SIP has been deactivated, use the following command in global configuration mode: Command Router(config)# power enable module slot Purpose Activates the SIP in the specified slot and its installed SPAs, where:

slotSpecifies the chassis slot number where the SIP is installed.

For more information about chassis slot numbering, refer to the Identifying Slots and Subslots for SIPs, SSCs, and SPAs section in this guide.

Verifying Deactivation and Activation of a SIP or SSC

To verify the deactivation of a SIP or SSC, enter the show module command in privileged EXEC configuration mode. Observe the Status field associated with the SIP that you want to verify. The following example shows that the Cisco 7600 SIP-400 located in slot 13 is deactivated. This is indicated by its PwrDown status.
Router# show module 13 Mod Ports Card Type Model Serial No. --- ----- -------------------------------------- ------------------ ----------13 0 4-subslot SPA Interface Processor-400 7600-SIP-400 JAB0851042X

Mod MAC addresses Hw Fw Sw Status --- ---------------------------------- ------ ------------ ------------ ------13 00e0.aabb.cc00 to 00e0.aabb.cc3f 0.525 12.2(PP_SPL_ 12.2(PP_SPL_ Ok Mod Online Diag Status --- ------------------13 PwrDown

To verify activation and proper operation of a SIP, enter the show module command and observe Ok in the Status field as shown in the following example:
Router# show module 2 Mod Ports Card Type Model Serial No. --- ----- -------------------------------------- ------------------ ----------2 0 4-subslot SPA Interface Processor-200 7600-SIP-200 JAB074905S1 Mod MAC addresses Hw Fw Sw Status --- ---------------------------------- ------ ------------ ------------ ------2 0000.0000.0000 to 0000.0000.003f 0.232 12.2(2004082 12.2(2004082 Ok Mod Online Diag Status --- ------------------2 Pass

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide OL-5070-07

5-5

Chapter 5 Preparing for Online Insertion and Removal of SIPs, SSCs, and SPAs

Troubleshooting the SIPs and SSC

Preparing for Online Removal of a SPA

The Cisco 7600 series router supports OIR of a SPA independently of removing the SIP or SSC. This means that a SIP can remain installed in the router with one SPA remaining active, while you remove another SPA from one of the SIP subslots. If you are not planning to immediately replace a SPA into the SIP, then be sure to install a blank filler plate in the subslot. The SIP should always be fully installed with either functional SPAs or blank filler plates. The interface configuration is retained (recalled) if a SIP or SPA is removed and then replaced with one of the same type. This is not the case if you replace a Cisco 7600 SIP-200 with a Cisco 7600 SIP-400 or vice versa. If you are planning to remove a SIP along with its SPAs, then you do not need to follow the instructions in this section. To remove a SIP, see the Preparing for Online Removal of a SIP or SSC section on page 5-3.

Deactivating a SPA
Although graceful deactivation of a SPA is preferred using the hw-module subslot shutdown command, the Cisco 7600 series router does support removal of the SPA without deactivating it first. Before deactivating a SPA, ensure that the SIP is seated securely into the slot before pulling out the SPA itself.

Note

If you are preparing for an OIR of a SPA, it is not necessary to independently shut down each of the interfaces prior to deactivation of the SPA. The hw-module subslot shutdown command automatically stops traffic on the interfaces and deactivates them along with the SPA in preparation for OIR. In similar fashion, you do not need to independently restart any interfaces on a SPA after OIR of a SPA or SIP. To deactivate a SPA and all of its interfaces prior to removal of the SPA, use the following command in global configuration mode: Command Router(config)# hw-module subslot slot/subslot shutdown [powered | unpowered] Purpose Deactivates the SPA in the specified slot and subslot of the SIP, where:

slotSpecifies the chassis slot number where the SIP is installed. subslotSpecifies subslot number on a SIP where a SPA is installed. powered(Optional) Shuts down the SPA and all of its interfaces, and leaves them in an administratively down state with power enabled. This is the default state. unpowered(Optional) Shuts down the SPA and all of its interfaces, and leaves them in an administratively down state without power.

For more information about chassis slot and SIP subslot numbering, refer to the Identifying Slots and Subslots for SIPs, SSCs, and SPAs section in this guide.

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide

5-6

OL-5070-07

Chapter 5

Troubleshooting the SIPs and SSC Preparing for Online Insertion and Removal of SIPs, SSCs, and SPAs

Reactivating a SPA
Note

You do not need to reactivate a SPA after an OIR of either the SIP or a SPA if you did not deactivate the SPA prior to removal. If the router is running, then the SPAs automatically start upon insertion into the SIP or with insertion of a SIP into the router. If you deactivate a SPA using the hw-module subslot shutdown global configuration command and need to reactivate it without performing an OIR, you need to use the no hw-module subslot shutdown global configuration command to reactivate the SPA and its interfaces. To activate a SPA and its interfaces after the SPA has been deactivated, use the following command in global configuration mode: Command Router(config)# no hw-module subslot slot/subslot shutdown Purpose Activates the SPA and its interfaces in the specified slot and subslot of the SIP, where:

slotSpecifies the chassis slot number where the SIP is installed. subslotSpecifies subslot number on a SIP where a SPA is installed.

Verifying Deactivation and Activation of a SPA

When you deactivate a SPA, the corresponding interfaces are also deactivated. This means that these interfaces will no longer appear in the output of the show interface command. To verify the deactivation of a SPA, enter the show hw-module subslot all oir command in privileged EXEC configuration mode. Observe the Operational Status field associated with the SPA that you want to verify. In the following example, the SPA located in subslot 1 of the SIP in slot 2 of the router is administratively down from the hw-module subslot shutdown command:
Router# show hw-module subslot all oir Module Model Operational Status -------------- ------------------ ------------------------subslot 2/0 SPA-4XOC3-POS ok subslot 2/1 SPA-4XOC3-ATM admin down

To verify activation and proper operation of a SPA, enter the show hw-module subslot all oir command and observe ok in the Operational Status field as shown in the following example:
Router# show hw-module subslot all oir Module Model Operational Status -------------- ------------------ ------------------------subslot 2/0 SPA-4XOC3-POS ok subslot 2/1 SPA-4XOC3-ATM ok

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide OL-5070-07

5-7

Chapter 5 Preparing for Online Insertion and Removal of SIPs, SSCs, and SPAs

Troubleshooting the SIPs and SSC

Deactivation and Activation Configuration Examples

This section provides the following examples of deactivating and activating SIPs and SPAs:

Deactivation of a SIP Configuration Example, page 5-7 Activation of a SIP Configuration Example, page 5-8 Deactivation of a SPA Configuration Example, page 5-8 Activation of a SPA Configuration Example, page 5-8

Deactivation of a SIP Configuration Example

Deactivate a SIP when you want to perform OIR of the SIP. The following example deactivates the SIP that is installed in slot 5 of the router, its SPAs, and all of the interfaces. The corresponding console messages are shown:
Router# configure terminal Router(config)# no power enable module 5 1w4d: %OIR-6-REMCARD: Card removed from slot 5, interfaces disabled 1w4d: %C6KPWR-SP-4-DISABLED: power to module in slot 5 set off (admin request)

Activation of a SIP Configuration Example

Activate a SIP if you have previously deactivated it. If you did not deactivate the SPAs, the SPAs automatically reactivate with reactivation of the SIP. The following example activates the SIP that is installed in slot 5 of the router, its SPA, and all of the interfaces (as long as the hw-module subslot shutdown command was not issued to also deactivate the SPA):
Router# configure terminal Router(config)# power enable module 5

Notice that there are no corresponding console messages shown with activation. If you re-enter the power enable module command, a message is displayed indicating that the module is already enabled:
Router(config)# power enable module 5 % module is already enabled

Deactivation of a SPA Configuration Example

Deactivate a SPA when you want to perform OIR of that SPA. The following example deactivates the SPA (and its interfaces) that is installed in subslot 0 of the SIP located in slot 2 of the router and removes power to the SPA. Notice that no corresponding console messages are shown:
Router# configure terminal Router(config)# hw-module subslot 2/0 shutdown unpowered

Activation of a SPA Configuration Example

Activate a SPA if you have previously deactivated it. If you have not deactivated a SPA and its interfaces during OIR of a SIP, then the SPA is automatically reactivated upon reactivation of the SIP. The following example activates the SPA that is installed in slot 2 of the router and all of its interfaces.

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide

5-8

OL-5070-07

Chapter 5

Troubleshooting the SIPs and SSC Preparing for Online Insertion and Removal of SIPs, SSCs, and SPAs

Router# configure terminal Router(config)# no hw-module subslot 2/0 shutdown Router#

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide OL-5070-07

5-9

Chapter 5 Preparing for Online Insertion and Removal of SIPs, SSCs, and SPAs

Troubleshooting the SIPs and SSC

Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide

5-10

OL-5070-07