NPM Guide

NetOp Policy Manager Troubleshooting Guide
Release Number 6.1.5 Part Number 5/1543-CRA 119 1030/1 Uen
Corporate Headquarters Redback Networks Inc. 100 Headquarters Drive San Jose, CA 95134-1362 USA http://www.redback.com Tel: +1 408 750 5000
2008 to 2008, Ericsson AB. All rights reserved. Redback and SmartEdge are trademarks registered at the U.S. Patent & Trademark Office and in other countries. AOS, NetOp, SMS, and User Intelligent Networks are trademarks or service marks of Redback Networks Inc. All other products or services mentioned are the trademarks, service marks, registered trademarks or registered service marks of their respective owners. All rights in copyright are reserved to the copyright owner. Company and product names are trademarks or registered trademarks of their respective owners. Neither the name of any third party software developer nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission of such third party.
Rights and Restrictions

All statements, specifications, recommendations, and technical information contained are current or planned as of the date of publication of this document. They are reliable as of the time of this writing and are presented without warranty of any kind, expressed or implied. In an effort to continuously improve the product and add features, Redback Networks Inc. (Redback) or Ericsson AB (Ericsson) reserves the right to change any specifications contained in this document without prior notice of any kind. Neither Redback or Ericsson nor its parent or affiliate companies shall be liable for technical or editorial errors or omissions which may occur in this document. Neither Redback or Ericsson nor its affiliate companies shall be liable for any indirect, special, incidental or consequential damages resulting from the furnishing, performance, or use of this document.
Disclaimer
No part of this document may be reproduced in any form without the written permission of the copyright owner. The contents of this document are subject to revision without notice due to continued progress in methodology, design and manufacturing. Redback or Ericsson shall have no liability for any error or damage of any kind resulting from the use of this document.
-ii
Contents
Chapter 1: Verify the NetOp PM System Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1 Verify That the NetOp PM System Is Operational . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1 Verify the NetOp PM Software Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2 View Software License Details with the show_licenses.sh Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2 Troubleshoot NetOp PM Software License Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2 Verify Deployments Using PPP Subscriber Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3 Verify Deployments Using Dynamic CLIPS Subscriber Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6 Verify SmartEdge Router-to-RADIUS Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-10 Verify Communication Between a SmartEdge Router and the NetOp PM RADIUS Servers . . . . . . . . . . . . . . . . . . 1-11 Display test aaa Command Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-11 Interpret test aaa Command Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12 Chapter 2: Troubleshoot the NetOp PM System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1 System Diagnostics Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1 NetOp PM System Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2 Configure the NetOp PM System Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3 Identify Component Names in the NetOp PM System Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4 NetOp PM RADIUS Server Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4 Test NetOp PM RADIUS Server Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4 Generate NetOp PM RADIUS Server Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5 Verify That the NetOp PM RADIUS Servers Are Operational . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6 Set Up E-mail to Monitor the NetOp PM RADIUS Server Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7 View Statistics for NetOp PM RADIUS Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7 Run the show_radius.sh Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7 Output Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8 Interpret the Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9 External RADIUS Server Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10 Configuration Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10 Subscribers Using the Default Realm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11 Subscribers Using a Specific Realm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11 Troubleshoot External RADIUS Server Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11 Subscriber Not Authenticated and Not Assigned an IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12 Subscriber Not Authenticated but Is Assigned an IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12 Accounting Data Is Missing in External RADIUS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-15 NetOp PM Database Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-15 Verify That the Primary and Standby Databases Are Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-15 Determine the Name and Status of a Current Fast-Start Failover Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-17 Determine the Status of the Standby Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-18 Determine the Latest Primary and Standby Redo Log Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-19
Contents
iii
View the Oracle Data Guard Broker Log File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuration Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Subscriber Browser Not Redirected to Portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Change of Service Not Applied . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP Redirect Not Working . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Logon Fails When External LDAP Server Does Not Permit Multiple Searches from Same LDAP Connection . . . Logon Fails When an External LDAP Server Uses CHAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . No Reauth Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RADIUS Flow-Through Attributes Not Received . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wrong Circuit Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NetOp PM API Server Calls Time Out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NetOp PM Database Failover Configuration Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Verify That the Observer Process on a Solaris Host Is Operational . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Verify That the Observer Process on a PC Host Is Operational . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Move the Observer Process to Another Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stop and Restart the Observer Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Errors Connecting to the Standby Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . View the Oracle Alert Log File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Convert Manual Failover to Fast-Start Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Check the Status of the NetOp PM Databases in Fast-Start Failover Configuration . . . . . . . . . . . . . . . . . . . . . . . . . Disable Fast-Start Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Performance Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Monitor System Performance on a Solaris Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Determine if the NetOp PM RADIUS Server Is Limiting Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Monitor LAN Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Determine if the NetOp PM Database Is Limiting Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Monitor the NetOp PM Database and Generate E-Mail Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Troubleshoot the NetOp PM Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Generate the NetOp PM Database Performance Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2-20 2-20 2-20 2-21 2-21 2-24 2-25 2-25 2-26 2-26 2-27 2-27 2-28 2-28 2-29 2-29 2-30 2-30 2-30 2-31 2-33 2-34 2-34 2-35 2-36 2-36 2-36 2-37 2-38
Chapter 3: Troubleshoot System Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1 NetOp PM Installation Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2 NetOp PM System-Wide Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2 NetOp PM Service Manager Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3 License for NetOp PM Complex Services not Found . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3 Database Username or Password Is Incorrect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3 Database Name Is Incorrect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4 Database Host Is Wrong . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4 SNMP Errors When Trying to Change a Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4 Bad Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-5 Invalid IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-5 No Response Received . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-5 Resource Unavailable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-6 Unknown NAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7 NetOp PM Lightweight Web Portal Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7 Attempt to Modify Services Before Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8 Unknown Proxy Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8 Invalid Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-9 Error While Attempting to Log You In . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-9 Error While Attempting to Log You Out . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-9 Login Failed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10 Incorrect Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11 Inactive Account Logon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11
-iv
Invalid Location Logon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11 Invalid Logon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11 Reached Your Maximum Login Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12 Reached Maximum Number of Logons Permitted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12 Error While Attempting to Modify Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12 Incompatible Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13 Must Have JavaScript Enabled to Change Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13 Refused Access to Unknown IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13 Select at Least One Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-14 You Are Trying to Remove Your Access Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-14 You Are Trying to Select More Services Than Allowed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-15 Service Offerings Specify Different Contexts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-15 NetOp PM RADIUS Server Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16 Conflicting Flow-Through Attributes Encountered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16 Conflicting Service Attributes Encountered . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16 License for NetOp PM EAP Authentication Feature not Found . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17 License for NetOp PM Third-Party Vendor Support not Found . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17 Location Check Failed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18 Maximum Subscriber Sessions Exceeded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18 Request from Unknown Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18 PPP Client Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18 PPP Session Limit Exceeded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19 Location not Allowed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19 Authentication Failed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19 Subscriber Account Is Inactive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19 NetOp PM API Server Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20 License for Admission Control Function Feature not Found . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20 Bad Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-21 Busy NetOp PM API Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-21 Invalid IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-21 No Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-22 No Response Received . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-22 Invalid Credentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-23 Resource Unavailable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-23 Unable to Retrieve Database Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-24 Unknown NAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-24 NetOp PM API Server and External RADIUS Server Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-24 Failure Proxying Authentication Request to All RADIUS Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-25 No Response Received from External RADIUS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-25 Unable to Send Access-Request Packet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-26 Unknown Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-26 NetOp PM Database Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-27 License for NetOp PM Manual Failover Database Feature not Found . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-27 Connection Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-28 End of File on Communication Channel Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-28 Integrity Constraint Violated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29 Invalid Changes to the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29 Switchover Latent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-30 Older Backup Is Already Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-30 NetOp PM Script Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-31 Appendix A: RADIUS Attribute 49 Termination Error Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1
Contents
-v
-vi
DraftApril 9 2009
Chapter 1
Verify the NetOp PM System Configuration
This chapter describes how to contact Redback technical support, and verify and troubleshoot the NetOp Policy Manager (PM) system configuration. In this guide, the term node refers to a SmartEdge multiservice edge router. For information about verifying that the NetOp PM database is operational, see the Verify That the Primary and Standby Databases Are Running section on page 2-15. This chapter includes the following topics: Verify That the NetOp PM System Is Operational Verify SmartEdge Router-to-RADIUS Communication Note For troubleshooting purposes, before performing these tests, set logging to report debug level messages by navigating to the /usr/local/npm/ directory and running the config_logging.sh script with the -level debug construct; for the syntax and usage guidelines for this script, see the Configure and the NetOp PM System chapter in the NetOp Policy Manager Installation Guide. We recommend turning off debug level messages after the NetOp PM system is verified. This level of reporting can produce large amounts of log information, which can reduce the performance of the NetOp PM system.
Note
To use EAP authentication, you need to purchase a license for the NetOp PM IEEE 802.1x EAP authentication feature.
Verify That the NetOp PM System Is Operational

This topic describes how to verify that the NetOp PM system is operational after it has been installed and configured, and troubleshoot problems that may occur. This section includes the following topics: Verify the NetOp PM Software Licenses Verify Deployments Using PPP Subscriber Access Verify Deployments Using Dynamic CLIPS Subscriber Access
1-1
DraftApril 9 2009
Note
It is assumed for these procedures that you have installed the NetOp PM components, using the NetOp Policy Manager Installation Guide, and you have configured them and the SmartEdge router using this guide.
For procedures to configure a node and install the SmartEdge router configuration file, see the Configure the Node for the NetOp PM System chapter in the NetOp Policy Manager Configuration Guide.
Verify the NetOp PM Software Licenses

This section describes how to verify that your software licenses for NetOp PM software features are properly installed using the show_licenses.sh script and troubleshoot license problems using the /var/log/netop/npm.log file. This section includes the following topics: View Software License Details with the show_licenses.sh Script Troubleshoot NetOp PM Software License Issues
View Software License Details with the show_licenses.sh Script

To use the show_licenses.sh script to display details on the software licenses installed, perform the following steps: 1. Log on to any NetOp PM server host as root. 2. Open a terminal window and navigate to the NetOp PM installation directory: cd /usr/local/npm 3. Run the show_licenses.sh script according to the following syntax: ./show_licenses.sh Output such as the following is displayed:
NetOp PM License Viewer Feature Name Expiry Date Count =========================================================== NetOp PM Complex Services No Expiry 500 k NetOp PM Oracle Standby Database No Expiry 500 k NetOp PM Admission Control No Expiry 500 k NetOp PM Right-to-Use No Expiry 500 k NetOp PM EAP Authentication No Expiry 500 k NetOp PM Third-Party Vendor Support No Expiry 500 k Number of licenses: 6
Troubleshoot NetOp PM Software License Issues

You may encounter one of the following error messages when you attempt to use the NetOp PM complex services or the NetOp PM ACF feature to set up or run a standby database or configure subscribers:
No license found for the Right-to-Use feature. No license found for Complex Services.
1-2
DraftApril 9 2009
No license found for Admission Control Function. No license found for NetOp PM Oracle Standby Database feature. No license found for NetOp PM EAP Authentication feature. No license found for NetOp PM Third-Party Vendor Support feature. Verify that the required license has been configured
Verify that your licenses for NetOp PM software features are valid by performing the following steps: 1. If you have not installed a software license for the NetOp PM complex services, the NetOp PM ACF feature, the NetOp PM Oracle standby database feature, or the NetOp PM subscriber right-to-use (RTU) feature, you must install them using the config_licenses.sh script; for the procedure to run it, see the Install NetOp PM Software Licenses chapter in the NetOp Policy Manager Installation Guide. 2. If you have installed the licenses, but you are still unable to set up or run the standby database or use the other software features, run the show_licenses.sh script or check the /var/log/netop/npm.log file for messages that indicate license errors.
Verify Deployments Using PPP Subscriber Access

This topic describes how to verify and troubleshoot the newly installed and configured NetOp PM system with Point-to-Point Protocol (PPP) subscriber access using either local or external RADIUS authentication. For procedures to verify and troubleshoot a system using clientless IP service selection (CLIPS) subscriber access, see the Verify Deployments Using Dynamic CLIPS Subscriber Access section on page 1-6. For procedures to troubleshoot communication with external RADIUS servers, see the Test NetOp PM RADIUS Server Status section on page 2-4. To verify that the NetOp PM system using PPP is operational, perform the following steps: 1. Start all the NetOp PM components, including the NetOp PM database, the RADIUS servers, the application programming interface (API) servers, the NetOp PM service manager, and the lightweight web portal. 2. If you are using a modified sample configuration file (for example, the sample-ser-ppp.cfg or sample-ser.cfg file) and have not installed it, perform the following steps: a. Use the copy command in exec (10) mode (for the SmartEdge OS) or administrator exec mode (for the AOS software) to copy the modified configuration file to the location of your other node configuration files; for example, to copy the previously modified sample-ser-ppp.cfg file from the NetOp PM directory to a SmartEdge router, enter the following command: copy ftp: //netop:[email protected]//usr/local/npm/config/sample-ser-ppp.cfg /flash/sample-ser-ppp.cfg
1-3
DraftApril 9 2009
b. Use the configure url command in exec (10) mode (for the SmartEdge OS) or administrator exec mode (for the AOS software) to instruct the system to read the new configuration file; for example, to add the contents of the sample-ser.cfg or sample-sms.cfg file to the configuration of the SmartEdge router, enter the following command: configure /flash/sample-config where the /flash/sample-config construct is the URL (path and name) of the configuration file you have modified. For syntax and usage guidelines for these commands, see the chapters on file and release operations and software operations in the Basic System Operations Guide for the SmartEdge OS. For information on configuration file management, see the Managing Configuration Files document in the SmartEdge OS library. For system images and configuration files, see the AOS Configuration Guide. 3. Connect a Windows computer (with a PPP client installed on it) directly to the port that is configured to receive PPP traffic on the node; for example, in the sample-ser-ppp.cfg file, slot 12, port 2, is configured to receive PPP traffic, therefore connect the computer to slot 12, port 2. 4. With the PPP client software on the PC, connect to the node using the configured username and password (by default, the username/password combination, joe/joe, is set in the sample database). If you are using external RADIUS authentication, use the username and password configured on the external RADIUS server. Note If the NetOp PM system is configured to use a specified realm for authentication, such as @AOL, the subscriber should log on with a username like joe@AOL. For the settings, see the Subscribers Using a Specific Realm section on page 2-11.
5. If the subscriber does not get authenticated or is not assigned an IP address, perform the following steps to investigate the problem: a. Run the test_radius.sh and show_radius.sh scripts to obtain a high-level view of the RADIUS server events: Use the test_radius.sh script to simulate the RADIUS messages generated by the node. Use the show_radius.sh script to determine if the NetOp PM RADIUS servers are processing the messages and the username and password combination has generated an Access-Accept or Access-Reject message.
If the show_radius.sh script indicates that the NetOp PM RADIUS servers encountered no errors, more information can be obtained from the test_radius.sh script. b. Run the test_radius.sh script to see if the NetOp PM RADIUS servers responded with an Access-Accept or an Access-Reject message. For example, for the sample subscriber account joe/joe, run the script according to the following syntax: test_radius.sh -user joe -password joe -f -trace c. If an Access-Reject message was returned, confirm that the sample subscriber account joe/joe has been configured in the NetOp PM database.
1-4
DraftApril 9 2009
d. If an Access-Accept message was returned, but the node did not bring up the session, to obtain further details on the RADIUS attributes returned by the NetOp PM RADIUS server, run the test_radius.sh script according to the following syntax: test_radius.sh -user joe -password joe -f -trace The resulting output displays the actual RADIUS attributes being returned for the subscriber, joe/joe, to the node. e. Confirm that all the profiles specified in the Access response message have been configured on the node. f. If everything looks correct, to obtain more detailed information, enter the following command (if you have not already set logging to display debugging messages): /usr/local/npm/config_logging.sh -level debug g. Shut down the NetOp PM RADIUS servers and restart them. The NetOp PM RADIUS servers now log the RADIUS messages being received from and sent to the node. h. Try to connect to the node using the PPP client software on the PC. i. Access the /var/log/netop/npm.log file and review the details of the RADIUS messages between the NetOp PM RADIUS server and the node. j. Pay particular attention to the reason code in any Accounting-Stop message recorded by the node. k. Correct the source of the error and try again to log on. For syntax descriptions and usage guidelines for this script, see the Run the show_radius.sh Script section on page 2-7. 6. If the subscriber does not succeed in logging on or is not assigned an IP address and the deployment uses external RADIUS authentication, see the Verify That the NetOp PM RADIUS Servers Are Operational section on page 2-6. 7. When the PPP subscriber has logged on and received an IP address, navigate to the NetOp PM lightweight web portal with the following URL: http://n.n.n.n/NPM-6.x.x.x/portal.php where n.n.n.n is the IP address for the web server host and NPM-6.x.x.x is the NetOp PM software version. Because the PPP session is already implicitly logged on, the web portal should redirect the subscriber to the service offering page. 8. If the page does not appear (the above URL does not display the NetOp PM lightweight web portal), verify the network connectivity by performing the following steps: a. From the NetOp PM API host, enter the UNIX ping command with the IP address of the SmartEdge router multibind interface for PPP; for example, as in the sample configurations, if the IP address is 10.192.45.1, enter the following command: ping 10.192.45.1
1-5
DraftApril 9 2009
b. If the ping command is not successful, from the NetOp PM API host, enter the following command: netstat -rv The output should contain the following line:
10.192.45.0 1500* 0 255.255.255.0 1 UG 0 10.192.100.6 0
9. If the ping command is successful, verify that both the NetOp PM lightweight web portal and the NetOp PM API server are running by entering the following commands for each service: For the NetOp PM API server: ps -ef | grep java The following output indicates that the NetOp PM API server is operational:
root 3579 3575 0 14:23:55 pts/2 2:45 /opt/Redback/jre/n.n.n_n/bin/java -Djava.security.auth.login.config==/usr/local
where n.n.n_n is the latest JRE release. For the NetOp PM lightweight web portal service: ps -ef | grep http The following output indicates that the NetOp PM lightweight web portal is operational:
nobody -k 2056 2052 0 Aug 24 ? 0:00 /usr/local/apache2/bin/httpd
10. If the services are running, check that the NetOp PM API server software version is correct: a. Navigate to the /usr/local/apache-tomcat-n.n.n/webapps directory; where n.n.n is the latest release of the Tomcat software. b. Verify that the NPM_API-6.n.n.n directory and the NPM_API-6.n.n.n.xml file exist; where n.n.n is the latest release of the NetOp PM software. 11. If the page appears, then disconnect the PPP session using the PPP client software. Note Set the logging level back to the desired level after verifying the system, to avoid large numbers of debugging messages being logged in the npm.log file in a live network; for example, enter the following command: /usr/local/npm/config_logging.sh -level notice where the notice keyword is the default logging level. For syntax and usage guidelines for this script, see the Configure the NetOp PM System chapter in the NetOp Policy Manager Installation Guide.
Verify Deployments Using Dynamic CLIPS Subscriber Access

This topic describes how to verify and troubleshoot the newly installed NetOp PM system with dynamic (Dynamic Host Configuration Protocol (DHCP)-based) CLIPS subscribers using either local or external RADIUS authentication.
1-6
DraftApril 9 2009
For procedures to verify communications with external RADIUS servers, see the Verify That the NetOp PM RADIUS Servers Are Operational section on page 2-6. To verify that the NetOp PM system using dynamic CLIPS is operational, perform the following steps: 1. Start all the NetOp PM components, including the NetOp PM database, RADIUS servers, API servers, and lightweight web portal. 2. If you are using a modified sample configuration file (for example, the sample-ser-clips.cfg file) and have not installed it, perform the following steps: a. Use the copy command in exec (10) mode to copy the modified configuration file to the location of your other node configuration files; for example, to copy the previously modified file from the NetOp PM directory to a SmartEdge router, enter the following command: copy ftp: //netop:[email protected]//usr/local/npm/config/ sample-ser-clips.cfg /flash/sample-ser-clips.cfg b. Use the configure url command in exec (10) mode to instruct the system to read the new configuration file; for example, to add the contents of the sample-ser-clips.cfg file to the configuration of the SmartEdge router, enter the following command: configure /flash/sample-ser-clips.cfg where /flash/sample-ser-clips.cfg is the URL (path and name) of the modified configuration file. For syntax and usage guidelines for these commands, see the chapters on file and release operations and software operations in the Basic System Operations Guide for the SmartEdge OS. See also the chapter on configuration file management in the Basic System Configuration Guide for the SmartEdge OS. 3. Connect a Windows computer directly to the port that is configured to receive CLIPS traffic on the node. For example, in the sample-ser-clips.cfg file, slot 12, port 1 is configured to receive CLIPS traffic; therefore, connect the computer to slot 12, port 1. 4. On the computer, enter the following command: ipconfig /renew If everything is configured correctly, the computer is assigned an IP address; output similar to the following results:
Windows IP Configuration Ethernet adapter Local Area Connection 3: Connection-specific : sample.com IP Address. . . . . Subnet Mask . . . . Default Gateway . . DNS Suffix .
. . . . . . . : 10.192.43.214 . . . . . . . : 255.255.248.0 . . . . . . . : 10.192.43.1
5. If the PC of the subscriber is not assigned an IP address, perform the following steps to investigate the problem: a. Run the test_radius.sh and show_radius.sh scripts to obtain a high-level view of the RADIUS server events: Use the test_radius.sh script to simulate the RADIUS messages generated by the node.
1-7
DraftApril 9 2009
Use the show_radius.sh script to determine if the NetOp PM RADIUS servers are processing the messages and the username and password combination has generated an Access-Accept or Access-Reject message.
If the show_radius.sh script indicates that the NetOp PM RADIUS servers encountered no errors, more information can be obtained from the test_radius.sh script. b. Verify if the NetOp PM RADIUS servers responded with an Access-Accept or Access-Reject message by running the test_radius.sh script with the following syntax: test_radius.sh -f -verbose The NetOp PM RADIUS server places the subscriber in the Captive Portal by returning an Access-Accept message to the node. c. If an Access-Accept message was returned, but the node did not bring up the session, to obtain further details on the RADIUS attributes returned by the NetOp PM RADIUS server, run the test_radius.sh script again with the following syntax: test_radius.sh -f -trace The resulting output displays the actual RADIUS attribute being returned for the MAC address to the node. d. Confirm that all the profiles specified in the Access response message have been configured on the node. e. If everything looks correct, to obtain more detailed information, enter the following command (if you have not already set logging to display debugging messages): /usr/local/npm/config_logging.sh -level debug f. Shut down the NetOp PM RADIUS servers and restart them. The NetOp PM RADIUS servers now log the RADIUS messages being received from and sent to the node. g. Reissue the ipconfig /renew command to attempt to assign an IP address to the PC of the subscriber. h. Access the /var/log/netop/npm.log file to review the details of the RADIUS messages between the NetOp PM RADIUS server and the node. i. Pay particular attention to the reason code in any Accounting-Stop message recorded by the node. j. Correct the source of the error and try again to assign an IP address to the PC of the subscriber by entering the ipconfig /renew command. 6. If the NetOp PM system did not receive an Access-Request message from the node, check the RADIUS configuration on the node; for the required commands, see one of the sample configurations. 7. If the reason for the failure is NO DHCP Server, perform the following steps: a. Verify that you have started the Dynamic Host Configuration Protocol (DHCP) server. b. Verify connectivity to the DHCP server by entering the ping ip-addr command in exec mode from the context; where the DHCP server is configured on the node (in the sample configurations, the BASIC context); for example, if the IP address of the DHCP server is 10.192.100.30, enter the following command: ping 10.192.100.30
1-8
DraftApril 9 2009
8. When the PC of the CLIPS subscriber has received an IP address, the subscriber is still not authenticated with the NetOp PM system, so navigate to the NetOp PM lightweight web portal with the following URL: http://n.n.n.n/NPM-6.n.n.n/portal.php where n.n.n.n is the web server host IP address and NPM-6.n.n.n is the NetOp PM software version. 9. Using a browser, log on to the portal with the default username and password, joe/joe. Note If the NetOp PM system is configured to use a specified realm for authentication, such as @AOL, the subscriber should log on with a username like joe@AOL.
When the subscriber is authenticated properly, the web portal should redirect the subscriber to the service offering page. 10. If the subscriber is redirected to a page reporting the error There was an error while attempting to log you in. Please contact Customer Service, consult the /var/log/netop/npm.log file for the debugging messages. Note To view debugging messages, you must have entered the /usr/local/npm/config_logging.sh -level debug command and shut down and restarted the NetOp PM components.
11. If the error is related to SNMP, perform the following steps, depending on the node platform: On a SmartEdge router, open a SmartEdge OS CLI session and enter the show configuration snmp command in any mode; in the output, ensure that the SNMP settings are correctly configured. For the required SNMP settings, see a sample configuration file for the SmartEdge router. On an SMS device, open an AOS CLI session and enter the show configuration command in administrator exec mode; in the output, ensure that the SNMP settings are correctly configured. For the required settings, see the sample-sms.cfg configuration file.
12. If the SNMP settings are not correct, correct them and try to log on to the portal again. 13. If the subscriber is redirected to a page reporting the error You have reached your maximum Login limit. Please Logoff one or more sessions and try again, in the NetOp client, check the services subscribed by that username and check the max_login field of the service. 14. If the page does not appear (the URL that resulted from the ping command does not display the NetOp PM lightweight web portal), verify the network connectivity by performing the following steps: a. From the NetOp PM API host, enter the UNIX ping command again with the IP address of the SmartEdge router multibind interface for CLIPS; for example, as in the sample configurations, if the IP address is 10.192.43.1, enter the following command: ping 10.192.43.1 b. If the ping command is not successful, enter the following command from the NetOp PM API host: netstat -rv The output should contain the following line:
10.192.43.0 1500* 255.255.255.0 0 1 UG 0 10.192.100.4 0
1-9
DraftApril 9 2009
Verify SmartEdge Router-to-RADIUS Communication
15. If the ping command is successful, verify that both the NetOp PM lightweight web portal and the NetOp PM API server are running by entering the following commands for each service: For the NetOp PM API server: ps -ef | grep java The following output indicates that the NetOp PM API server is operational:
root 3579 3575 0 14:23:55 pts/2 2:45 /opt/Redback/jre/n.n.n_n/bin/java -Djava.security.auth.login.config==/usr/local
where n.n.n_n is the latest JRE release. For the NetOp PM lightweight web portal service: ps -ef | grep http The following output indicates that the NetOp PM lightweight web portal is operational:
nobody -k 2056 2052 0 Aug 24 ? 0:00 /usr/local/apache2/bin/httpd
16. If the services are running, check that the NetOp PM API server software version is correct: a. Navigate to the /usr/local/apache-tomcat-n.n.n/webapps directory; where n.n.n is the latest release of the Tomcat software. b. Verify that the NPM_API-6.n.n.n directory and the NPM_API-6.n.n.n.xml file exist; where n.n.n is the latest release of the NetOp PM software. Note Set the logging level back to the desired level after verifying the system, to avoid large numbers of debugging messages being logged in the npm.log file in a live network; for example, enter the following command: /usr/local/npm/config_logging.sh -level notice where the notice keyword is the default logging level. For syntax and usage guidelines for this script, see the Configure the NetOp PM System chapter in the NetOp Policy Manager Installation Guide.

With the SmartEdge OS software, Release 4.0.5 or later, you can verify communication between a SmartEdge router and the NetOp PM RADIUS servers using the test aaa command (in exec mode) with a privilege level of at least 10. This section describes how to run the test aaa command, gives an example of the output, and provides information to interpret the output in the following sections: Verify Communication Between a SmartEdge Router and the NetOp PM RADIUS Servers Display test aaa Command Output Interpret test aaa Command Output
1-10
DraftApril 9 2009
Verify Communication Between a SmartEdge Router and the NetOp PM RADIUS Servers
To verify communication between the NetOp PM RADIUS servers and a SmartEdge router, perform the following steps: 1. Start a CLI session in the SmartEdge OS software. 2. Navigate to the local context in which the NetOp PM RADIUS servers have been configured. 3. Enter the test aaa command (in exec mode with privilege level of at least 10) with one of the following syntaxes: test aaa authentication username name password password protocol radius [server-ip ip-addr port] test aaa accounting username name protocol radius [server-ip ip-addr port] where the authentication keyword specifies that an Authentication-Request message is sent, the accounting keyword specifies that an Accounting-Request message is sent, the username name construct is the subscriber or administrator name (including the domain name), the protocol radius construct specifies that the communication to a NetOp PM RADIUS server is tested, and the optional server-ip ip-addr port construct specifies the IP address and User Datagram Protocol (UDP) ports for the NetOp PM RADIUS servers to be tested. If they are not specified, the command sends NetOp PM RADIUS requests to all the NetOp PM RADIUS servers configured in the context.
Display test aaa Command Output

The following example displays output for the command when it sent an Authentication-Request message to the RADIUS server at 10.192.100.10, UDP port 1812:
[local]Redback#test aaa authentication username b-32@local password redback protocol radius server-ip 10.192.100.10 1812 Apr 29 20:22:03: %AAA-7-AUTHEN: aaa_idx 92: Sending RADIUS_SERVER_TEST to radius. Pid 146: Testing radius server ... [local]Redback# Radius authentication test response: Server: 10.192.100.10/1812 Server response: Accepted. -----------------------------Attributes list: User-Name = b-32@local NAS_Identifier = Redback NAS-Port = 16842817 NAS-Port-Type = 0 Platform_Type = 2 OS_Version = 4.0.5 Service-Type = 2 Framed-IP-Address = 255.255.255.254 -----------------------------Send count: 1 Send time: Apr 29 20:22:03 2005 Response time: Apr 29 20:22:03 2005
1-11
DraftApril 9 2009
Interpret test aaa Command Output

Table 1-1 describes the fields in the test aaa command output.
Table 1-1
Field Server Server response
test aaa Command Field Descriptions

Description IP address and UDP port for the server. Either one of the following values: AcceptedThe server accepted the request. Timed OutThe request timed out with no response.
Attributes list
List of the values for the RADIUS attributes configured for the server: User-Name = b-32@local NAS_Identifier = Redback NAS_Port = 16842817 NAS_Port_Type = 0 Platform_Type = 2 OS_Version = 4.0.5 Service-Type = 2 Framed-IP-Address = 255.255.255.254
Send count Send time Response time
Number of times a message was sent to the server. Time at which the message was sent. Time at which the response was received.
1-12
DraftApril 9, 2009
Chapter 2
Troubleshoot the NetOp PM System
This chapter provides an overview of system logging, describes monitoring and troubleshooting of the following components: the NetOp Policy Manager (PM) system, the NetOp PM Remote Authentication Dial-In User Service (RADIUS) servers, including how to troubleshoot problems when proxying to an external server, and the high availability NetOp PM database feature. It also provides troubleshooting tips for configuration and performance issues. Note All references to a network access server (NAS) refer to the SmartEdge router or the SMS device used with the NetOp PM software.
This chapter includes the following topics: System Diagnostics Collection NetOp PM System Logging NetOp PM RADIUS Server Issues External RADIUS Server Issues NetOp PM Database Issues Configuration Issues RADIUS Flow-Through Attributes Not Received Wrong Circuit Type NetOp PM API Server Calls Time Out NetOp PM Database Failover Configuration Issues Performance Issues
System Diagnostics Collection

The NetOp PM system includes a script you can run if you encounter an operational problem that requires troubleshooting. The report_npm_diagnostic.sh script collects diagnostics from a single NetOp PM database server and stores the information as a .tar.gz file. You can send this file to Redback Technical Support for troubleshooting.
2-1
DraftApril 9, 2009
NetOp PM System Logging
Before you run the script, ensure there is at least 500 MB free space in the /tmp directory on the NetOp PM database host. This script will not run if there is not 500 MB of free disk space. To collect system diagnostics with the report_npm_diagnostic.sh script, perform the following steps: 1. Log on to the NetOp PM database server host as root. 2. Open a terminal window and navigate to the /usr/local/npm/diagnostic directory. 3. Run the report_npm_diagnostic.sh script according to the following syntax: ./report_npm_diagnostic.sh [-h][-lines line number] Command options and argument descriptions for the report_npm_diagnostic.sh script are shown in Table 2-1.
Table 2-1
Command Options -lines -h
Arguments for the report_npm_diagnostic.sh Script

Argument line number n/a Description Optional. The number of lines to extract from the npm.log, processCPUUsage.log, and /var/adm/messages log files. Optional. Displays usage information and exits.
Information collected by the system diagnostics collection tool includes the following: NetOp PM configuration files Tomcat (API server) configuration files NetOp PM log files Oracle alert and listener log files, and ora files Cricket files System resources log Java server information RADIUS server information (only if the RADIUS server is running on the host from which you launched the script) Database availability status Database statistics Oracle AWR and ADDM database reports

This section describes NetOp PM logging of system events in the following sections: Configure the NetOp PM System Logging Identify Component Names in the NetOp PM System Logging
2-2
Book-Title
DraftApril 9, 2009
Configure the NetOp PM System Logging

By default, the NetOp PM system logs events from all components to the Solaris system logging daemon (syslogd), which stores the messages in the /var/log/netop/npm.log file on each host. Optionally, you can configure all the NetOp PM components to log system events to a remote Solaris host. By default, the NetOp PM software logs events that are severity level of notice or above. It ignores debug and information messages. Using the config_logging.sh script, you can modify the severity threshold for events to be stored (notice, warning, error, critical, alert, or emergency level). When the logging level is changed, the NetOp PM components must be restarted to pick up the change in logging level. 11/28/06, per Dan Cachola: Track EV: 72818 config_logging.sh should dynamically change logging level of all NPM components. Change this, when fixed. To configure the NetOp PM system logging, perform the following steps: 1. Log on as root. 2. Open a terminal window and navigate to the NetOp PM installation directory: cd /usr/local/npm 3. Run the config_logging.sh script with the following syntax: config_logging.sh [-level level] [-host host | -nohost] [-f] [-h] Table 2-2 describes the syntax and usage guidelines for this script.
Table 2-2
Syntax -level level
Syntax for the config_logging.sh Script

Description Optional. Logging threshold of NetOp PM system events at one of the following severity levels: debug info notice (the default value) warning err crit alert emerg For information on the severity levels, see the UNIX man page for syslog.conf(5).
-host host
Optional. Hostname or IP address of the remote host to which the NetOp PM system redirects log messages to the syslogd program. If you use the hostname, ensure that it is in the /etc/hosts file. Optional. Restores the default behavior of logging messages to the /var/log/netop/npm.log file on the local host. The default behavior is to log to a file. Optional. Updates configuration without prompting the user. Optional. Prints usage information and exits.
-nohost
-f -h
Note
To configure the NetOp PM components to log system events to a remote Solaris host, run the config_logging.sh -host script on all hosts except the host that stores the messages.
2-3
DraftApril 9, 2009
NetOp PM RADIUS Server Issues
The standard Solaris logadm program administers the NetOp PM logs and clears the log files when they reach 10 MB in size. Once per night, the logadm program examines the current npm.log file to determine if it has reached its size limit. If so, the logadm program rotates the log files. It renames the current npm.log file to npm.log0 and creates a fresh npm.log file. For all other existing log files, it increments the number at the end of the filename by one (npm.log0 becomes npm.log1, and so on).
Identify Component Names in the NetOp PM System Logging

Each log message is tagged with a label indicating the message source. Table 2-3 lists the NetOp PM component names used in logging.
Table 2-3 Component Names in Logging
Name httpd API_SERVER WEB_PORTAL RADIUS_SERVER SERVICE_MANAGER
Message Source Apache web server and API load balancer NetOp PM API server (including Tomcat) Web portal NetOp PM RADIUS servers NetOp PM service manager (including plug-ins)
For example, error messages in the npm.log file that originate from the NetOp PM service manager are identified by the date, time, hostname, message ID, and NetOp PM component name SERVICE_MANAGER such as in the following example:
Dec 6 13:57:36 <hostname>: [ID 647684 local0.error] SERVICE_MANAGER -

When problems arise on the NetOp PM RADIUS servers, test the status of the servers to confirm that they are active and responding to RADIUS requests, that they can connect to the database, and the database configuration information is valid. You can also view server statistics to troubleshoot the problems or use the SmartEdge OS test aaa command to test communication between the node and the NetOp PM RADIUS servers. This section includes the following topics: Test NetOp PM RADIUS Server Status View Statistics for NetOp PM RADIUS Servers
Test NetOp PM RADIUS Server Status

This section describes how to test the NetOp PM RADIUS server status and confirm that the NetOp PM RADIUS servers are communicating properly with the NetOp PM database, external RADIUS server, and Lightweight Directory Access Protocol (LDAP) server. If one of the RADIUS ports is not responding, you can run a script to view the NetOp PM RADIUS server statistics for more evidence about the problem; see the View Statistics for NetOp PM RADIUS Servers section on page 2-7.
2-4
Book-Title
DraftApril 9, 2009
Perform the procedures in this section in the following order: Generate NetOp PM RADIUS Server Statistics Verify That the NetOp PM RADIUS Servers Are Operational Set Up E-mail to Monitor the NetOp PM RADIUS Server Status
Generate NetOp PM RADIUS Server Statistics

The NetOp PM system automatically runs a test_radius.sh script every five minutes that generates statistics and writes the results to a log file located in the /var/log/netop/npm.log file directory. You can use these results to monitor the status of the NetOp PM RADIUS servers. We recommend that you periodically monitor the log file for errors. To generate statistics on the NetOp PM RADIUS servers, run the test_radius.sh script by performing the following steps: 1. Log on as root to the NetOp PM RADIUS server host. 2. Open a terminal window and navigate to the NetOp PM RADIUS directory: cd /usr/local/npm/radius 3. Run the test_radius.sh script with the following syntax: ./test_radius.sh [-acct_port port[,port2,eap_port1,eap_port2]] [-auth_port port[,port2,eap_port1,eap_port2]] [-email e-mail-address] [-f] [-h] [-password user-password] [-user username] [-verbose] By default, the authentication port is 1812 and the accounting port is 1813. You can redefine the RADIUS or EAP authentication and accounting ports when you run the config_radius.sh script with the -auth_port and -acct_port keywords. You can optionally override either the default or the custom settings when you run the show_radius.sh script. It is recommended that you use dedicated ports for processing EAP messages. Table 2-4 describes the syntax and usage guidelines for this script.
Table 2-4
Syntax -acct_port port
Syntax for the test_radius.sh Script

Description Optional. Port for accounting messages; by default, port 1813. If you do not specify any ports when you run the test_radius.sh script, defaults to the ports configured when you ran the config_radius.sh script. To specify additional ports, use the optional port2 and port3 arguments, with no spaces between the port numbers and commas; for EAP ports, use the eap_port1 or eap_port2 arguments. All ports must be unique on the same host. Optional. Port for authentication messages; by default, port 1812. If you do not specify any ports when you run the test_radius.sh script, defaults to the ports configured when you ran the config_radius.sh script. To specify additional ports, use the optional port2 and port3 arguments, with no spaces between the port numbers and commas; for EAP ports, use the eap_port1 or eap_port2 arguments. All ports must be unique on the same host. Optional. E-mail address for failure notifications. Optional. Tests the server without prompting the user. Optional. Prints usage information and exits. Optional. User password.
-auth_port port
-email e-mail-address -f -h -password user-password
2-5
DraftApril 9, 2009
Table 2-4
Syntax -user username -verbose
Syntax for the test_radius.sh Script (continued)

Description Optional. Username. Optional. Displays the response for each server port specified.
Verify That the NetOp PM RADIUS Servers Are Operational

Run the test_radius.sh script on the same NetOp PM host as the NetOp PM RADIUS servers that are being tested; the script sends an Access-Request message to each authentication port and an Accounting-Stop message to each accounting port. The script detects which authentication and accounting ports have been configured by the config_radius.sh script. An authentication port is considered to have responded if the script receives either an Access-Accept, Access-Reject, or Access-Challenge message from the NetOp PM RADIUS server. An accounting port is considered to have responded if the script receives an Accounting-Response message from the NetOp PM RADIUS server. The following example displays the output when the script detects that the NetOp PM RADIUS servers are functioning properly:
/usr/local/npm/radius/test_radius.sh -f NetOp Policy Manager Test Radius Server Authentication Port(s): 1812 Accounting Port(s): 1813 Database Name: npm Database Server: 10.192.100.10 Port(s): 1812 1813 Responding Test Radius Server complete
The following example displays the output when the script detects that an NetOp PM RADIUS server accounting port is not responding:
/usr/local/npm/radius/test_radius.sh -f NetOp Policy Manager Test Authentication Port(s): Accounting Port(s): Database Name: Database Server: Port: 1812 Access-Accept Port: 1813 No response Test Radius Server complete Radius Server 1812 1813 npm 10.192.100.10
2-6
Book-Title
DraftApril 9, 2009
If the NetOp PM RADIUS ports are all functioning normally, the script exits with a status of 0. If the script determines that one or more NetOp PM RADIUS server ports are not responding, the script exits with a status of 2. If the script encounters any other errors, the script exits with a status of 1.This behavior allows you to easily integrate the script into third-party monitoring tools.
Set Up E-mail to Monitor the NetOp PM RADIUS Server Status

By default, the status of the NetOp PM RADIUS servers are monitored by the test_radius.sh script which is automatically configured to run periodically using the UNIX cron application. An entry in the npm.log file is generated each time the test_radius.sh script is run. The script determines if one or more NetOp PM RADIUS server ports are not responding. Use the optional -email e-mail address construct to send an e-mail notification to the specified address if the script detects that one or more NetOp PM RADIUS server ports are not responding. To report by e-mail if one or more NetOp PM RADIUS server ports are not responding, perform the following steps: 1. Set up and start the crontab editor by entering the following command: EDITOR=vi export EDITOR crontab -e netop 2. Update the following line in the crontab file with your e-mail address:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /usr/local/npm/radius/test_radius.sh -f -email [email protected] > /dev/null 2>&1
where the 0, 5, 10, up to 55 keywords are the minute intervals, and the four asterisks in each line are (from left to right) the hour, day of the month, month, and day of the week for the script to run. The asterisks are wildcards that equal any for the field where specified. In this case, if the script determines that one or more NetOp PM RADIUS server ports are not responding, it sends an e-mail to the specified address in addition to logging an entry in the npm.log file.
View Statistics for NetOp PM RADIUS Servers

This section describes how to generate and interpret statistics to troubleshoot problems with the NetOp PM RADIUS servers. The show_radius.sh script prints the statistics for the NetOp PM RADIUS servers, providing a current picture of RADIUS message processing. This section describes how to run the show_radius.sh script in the following topics: Run the show_radius.sh Script Output Example Interpret the Statistics
Run the show_radius.sh Script

By default, the show_radius.sh script aggregates the statistics for all the configured NetOp PM RADIUS servers or all the authentication ports and accounting ports specified on the command line. Use the -verbose keyword to view the statistics for each of the individual servers running on the NetOp PM host.
2-7
DraftApril 9, 2009
To view statistics for the NetOp PM RADIUS servers, perform the following steps: 1. Log on as root. 2. Open a terminal window and navigate to the NetOp PM RADIUS directory: cd /usr/local/npm/radius 3. Run the show_radius.sh script with the following syntax: ./show_radius.sh [-acct_port port[,port2,eap_port1,eap_port2]] [-auth_port port[,port2,eap_port1,eap_port2]] [-f] [-h] [-verbose] By default, the authentication port is 1812 and the accounting port is 1813. You can redefine the RADIUS or EAP authentication and accounting ports when you run the config_radius.sh script with the -auth_port and -acct_port keywords. You can optionally override either the default or the custom settings when you run the show_radius.sh script. All ports must be unique on the same NetOp PM host. We recommended that you use dedicated ports for processing EAP messages. Note Running the reinit_radius.sh script resets the NetOp PM RADIUS server statistics. Resetting the server statistics enables you to start monitoring the server statistics from a particular point onward.
Table 2-5 describes the syntax and usage guidelines for this script.
Table 2-5
Syntax -acct_port port
Syntax for the show_radius.sh Script

Description Optional. Port for accounting messages; by default, port 1813. If you do not specify any ports when you run the show_radius.sh script, defaults to the ports configured when you ran the config_radius.sh script. To specify additional ports, use the optional port2 and port3 arguments, with no spaces between the port numbers and commas; for EAP ports, use the eap_port1 or eap_port2 arguments. All ports must be unique on the same host. Optional. Ports for authentication messages; by default, port 1812. If you do not specify any ports when you run the show_radius.sh script, defaults to the ports configured when you ran the config_radius.sh script. To specify additional ports, use the optional port2 and port3 arguments, with no spaces between the port numbers and commas; for EAP ports, use the eap_port1 or eap_port2 arguments. All ports must be unique on the same host. Optional. Displays the statistics without prompting the user. Optional. Prints usage information and exits. Optional. Displays the statistics for each of the individual servers running on the NetOp PM host.
-auth_port port
-f -h -verbose
Output Example
The show_radius.sh script generates output similar to the following example for the NetOp PM RADIUS server (configured on ports 1812 and 1813):
/usr/local/npm/radius/show_radius.sh -f NetOp Policy Manager Radius Server status Authentication Port(s): 1812 Accounting Port(s): 1813 Database Name: npm Database Server: localhost
2-8
Book-Title
DraftApril 9, 2009
NetOp PM RADIUS Server Issues Status for port(s): 1812 1813 Radiator Radius server version 4.2 [NPM 6.1.4.0 Radius Server (1.186)] 0 Requests in the last second 0 Access accepts 0 Access challenges 0 Access rejects 0 Access requests 0 Accounting requests 0 Accounting responses 0 Bad authenticators in authentication requests 0 Bad authenticators in accounting requests 0 Total Bad authenticators in requests 0 Database errors 0 Dropped access requests 0 Dropped accounting requests 0 Total dropped requests 0 Duplicate access requests 0 Duplicate accounting requests 0 Total duplicate requests 0 Unknown client requests 0 Malformed acccess requests 0 Malformed accounting requests 0 Total proxied requests with no reply 0 Total proxied requests 0 Total requests 0 Average response time Show server status complete
Interpret the Statistics

If you run the show_radius.sh script immediately after installing and configuring the NetOp PM system, use it to verify system configuration. Table 2-6 lists in the output of this script the statistics that are used to indicate trouble areas.
Table 2-6
show_radius.sh Script Statistics Statistics Indicating Trouble Areas Access rejects Dropped access requests Dropped accounting requests Total dropped requests Total proxied requests with no reply Database installation, upgrade or operation Node configured in the NetOp PM database Database errors Unknown client requests
Configuration External RADIUS and LDAP servers
2-9
DraftApril 9, 2009
External RADIUS Server Issues
Table 2-6
show_radius.sh Script Statistics (continued) Statistics Indicating Trouble Areas Bad authenticators in accounting requests Bad authenticators in authentication requests Duplicate access requests Duplicate accounting requests Malformed access requests Malformed accounting requests Total bad authenticators in requests Total duplicate requests
Configuration Authentication and accounting requests
Note
If you encounter increased access reject counts when you run the show_radius.sh script, verify that the database on the external RADIUS server or external LDAP server contains a subscriber account called npm-ping with the password set to npm-ping. Increased access reject counts occur when the show_radius.sh script attempts to authenticate a default PPP subscriber account name called npm-ping. Access reject counts also increase when npm-ping has a location lock or is deactivated. If the NetOp PM system is configured to proxy to an external RADIUS server or external LDAP server and the Inactive Account Login, Invalid Location Login, or Invalid Login Redirect features are disabled (set to null), the NetOp PM system returns an access reject packet to the show_radius.sh script because the external RADIUS or LDAP server does not have a corresponding subscriber account entry for the npm-ping subscriber account. This causes the external RADIUS server or external LDAP server to reject the authentication request.

After performing the steps to verify a newly installed NetOp PM system in the Change of Service Not Applied section on page 2-21, if the subscriber is still not logged on and you are using external RADIUS authentication, use the tasks in this section to troubleshoot the problem; the steps differ, depending on whether subscribers are using the default realm or a specified realm (for example, AOL). This section includes the following topics: Configuration Assumptions Troubleshoot External RADIUS Server Issues
Configuration Assumptions
The procedures in this section assume that you have configured the NetOp PM system to proxy RADIUS messages to external RADIUS servers, using either the default or a specific realm: Subscribers Using the Default Realm Subscribers Using a Specific Realm
For more information about configuring the NetOp PM system to proxy RADIUS messages to external RADIUS servers, see the Configure External RADIUS and LDAP Servers chapter in the NetOp Policy Manager Configuration Guide.
2-10
Book-Title
DraftApril 9, 2009
Before proceeding to the troubleshooting tasks, verify the realm configuration in the NetOp PM database.
Subscribers Using the Default Realm

If subscribers in your NetOp PM deployments are using the default realm, these troubleshooting steps assume that you have performed the following steps using a database management tool: 1. Edited the proxy_config table to change the value for the default realm row in the proxy_login_access_request and proxy_srvc_chng_access_request columns from N to Y. 2. Edited the radius_proxy_server table to correct the IP address and listening port of each RADIUS server. For example, if the external RADIUS servers are running at IP addresses 10.192.201.10 and 10.192.201.20, and both RADIUS servers are listening on port 1812 for authentication, you edited the Host column to correct the IP address of the external RADIUS servers, and edited the auth_port column to correct the RADIUS listening port numbers.
Subscribers Using a Specific Realm

If subscribers in your NetOp PM deployments are using a specific realm (for example, AOL), these troubleshooting steps assume that you have performed the following steps using a database management tool: 1. Added a new row to the proxy_config table with the following values: realmAOL access_request_proxy_typeRADIUS proxy_algorithmfirst or round robin proxy_login_access_requestY proxy_srvc_chng_access_requestY
2. Added a new row to the radius_proxy_server table with the following values: realmAOL sequence1 hostIP address of the external RADIUS server secretsecret password configured on the external RADIUS server auth_portRADIUS listening port
Troubleshoot External RADIUS Server Issues

Note from FLH: Consider using the Problem/Action structure that you have elsewhere in this chapter for the issues in this section. Use the Troubleshooting element. Note Use the output of the show_radius.sh script and debugging messages in the /var/log/netop/npm.log file to perform this procedure. For information about running the script, see the Run the show_radius.sh Script section on page 2-7. For information about managing and monitoring the NetOp PM log files, see the Configure the NetOp PM System chapter in the NetOp Policy Manager Installation Guide.
2-11
DraftApril 9, 2009
This section provides tips to troubleshoot subscriber authentication problems related to configuration of the external RADIUS servers. After testing the NetOp PM system using PPP or CLIPS, if the subscriber still fails to authenticate and obtain an IP address and you are using external RADIUS authentication, use the following tasks to troubleshoot communication to external RADIUS servers: Subscriber Not Authenticated and Not Assigned an IP Address Subscriber Not Authenticated but Is Assigned an IP Address Accounting Data Is Missing in External RADIUS Server
Subscriber Not Authenticated and Not Assigned an IP Address

If the subscriber is not authenticated and is not assigned an IP address (and if the session should be redirected), verify that the proxy_config table has a value for the default_invalid_login_svc_id column (if the subscriber session is not intended to be redirected, the field should be empty).
Subscriber Not Authenticated but Is Assigned an IP Address

If the subscriber is not authenticated but is assigned an IP address, you are sent to the InvalidLogin web page or you see the following error:
Logon failed. Check your username and password and try logging on again. If logon fails with the correct username and password, it is possible that you are not permitted to connect from your current location. Contact your service provider to determine whether you are permitted to connect from your current location.
To fix this, check the show_radius.sh script output or the /var/log/netop/npm.log file debugging message and troubleshoot the problem with the following tasks: Subscriber Is Attempting to Log On from the Wrong Location NetOp PM RADIUS Server Received an Access-Request Message from the Node NetOp PM RADIUS Server Did Not Receive an Access-Request Message from the Node RADIUS Response Was an Access-Accept Message RADIUS Response Was an Access-Reject Message NetOp PM API Server or NetOp PM RADIUS Server Received No RADIUS Response External RADIUS Server Dropped the Access-Request Message
Subscriber Is Attempting to Log On from the Wrong Location

A subscriber, whose service definition includes a Location Lock, may be attempting to log on at a location from which he or she is not permitted to log on. If you verify that the subscriber is attempting to log on from his or her assigned location, continue with the steps in this section.
NetOp PM RADIUS Server Received an Access-Request Message from the Node

If the NetOp PM RADIUS server received an Access-Request message from the node, verify that the NetOp PM RADIUS server proxied the Access-Request message to the external RADIUS server in one of the following ways:
2-12
Book-Title
DraftApril 9, 2009
For deployments using the default realmDetermine if the NetOp PM API server sent the Access-Request message to the external RADIUS server; for example, to IP address 10.192.201.10 or 10.192.201.20 on port 1812. In the log, look for the message Sending to x.x.x.x port x, where the x.x.x.x argument is the IP address and the port x argument is the port of the appropriate NetOp PM RADIUS server. Note If the username is not followed by a domain (in the format @realm), the NetOp PM RADIUS server should have sent the Access-Request message to the external RADIUS server for the default realm.
For deployments using a specified realm, such as @AOLDetermine if the NetOp PM RADIUS server sent the Access-Request message to the external RADIUS server belonging to the specified realm (for example, @AOL, not the default realm). In the log, look for the message Sending to x.x.x.x port x, where x.x.x.x port x is the IP address and port of the appropriate RADIUS server. Note If the username is followed by a domain, the NetOp PM RADIUS server should have sent the Access-Request message to the external RADIUS server for that realm.
NetOp PM RADIUS Server Did Not Receive an Access-Request Message from the Node
If the NetOp PM RADIUS server did not receive an Access-Request message from the node, check the RADIUS configuration on the node; for the required commands, see one of the sample configurations.
RADIUS Response Was an Access-Accept Message

If the NetOp PM API server did send the Access-Request message to the external RADIUS server and the RADIUS response was an Access-Accept message, troubleshoot the problem in one of the following ways: If the NetOp PM API server received an Access-Accept message, by default, it sent an SNMP bounce message, which triggered the node to send another Access-Request message to the NetOp PM RADIUS server. If the double access suppression field is enabled for the realm of the subscriber, the NetOp PM RADIUS server responded with an Access-Accept message to the node. Otherwise, the message is proxied to the external RADIUS server. Note During web logon authentication, the NetOp PM API server sends the Access-Request message to the external RADIUS server.
If the NetOp PM API server or the NetOp PM RADIUS server received an Access-Accept message from the external RADIUS server, verify the attributes of the access response returned by the NetOp PM RADIUS server. To determine whether the node rejected the attributes, in the log, look for the Accounting-Stop request from the node, and view the Acct_Terminate_cause and Session_error_msg fields for a clue.
RADIUS Response Was an Access-Reject Message

If the NetOp PM API server or the NetOp PM RADIUS server did send the Access-Request message to the external RADIUS server, but the RADIUS response was an Access-Reject message, check the username and password defined in the external RADIUS database in one of the following ways:
2-13
DraftApril 9, 2009
For deployments using the default realmIf the username is username@realm in the NetOp PM database, but the external RADIUS database does not have that username, add username@realm to the external RADIUS database. For deployments using a specified realmIf the username is username@AOL in the NetOp PM database, but the external RADIUS database does not have that username, add username@AOL to the external RADIUS database.
NetOp PM API Server or NetOp PM RADIUS Server Received No RADIUS Response

If the NetOp PM API server or NetOp PM RADIUS server did send the Access-Request message to the external RADIUS server but received no RADIUS response, look for one of the following log messages: For the NetOp PM API serverAPI_SERVER - Error proxying authentication for user account: username Error: No response received from: 127.0.0.1 or Failure proxying authentication request to all servers! For the NetOp PM RADIUS serverRADIUS_SERVER[6553]: [ID 702911 local0.error] Error authenticating subscriber: username. Failure proxying authentication request to all servers!
Perform the following steps[added leadin to make structure valid - FLH]: 1. Verify network connectivity between the NetOp PM API server or NetOp PM RADIUS server and the external RADIUS server in one of the following ways: For the NetOp PM API serverEnter the UNIX ping ip-addr command on the NetOp PM API server host; where the ip-addr argument is the IP address of the external RADIUS server host. For the NetOp PM RADIUS serverEnter the UNIX ping ip-addr command on the NetOp PM RADIUS server host; where the ip-addr argument is the IP address of the external RADIUS server host.
2. If the ping command verifies that network connectivity is up, verify that the external RADIUS server processes are up and listening at the correct UDP port by entering the following command from the external RADIUS server host: netstat -a | grep port where the port argument is the auth port of the external RADIUS server.
External RADIUS Server Dropped the Access-Request Message

If the external RADIUS server is up, but the NetOp PM API server or the NetOp PM RADIUS server received no RADIUS response, check the output of the show_radius.sh script or debugging messages in the /var/log/netop/npm.log file. If the external RADIUS server received the Access-Request message but dropped it, troubleshoot the problem in one of the following ways: Verify that the secret password configured in the radius_proxy_server column of the proxy_config table matches the password in the external RADIUS server database. If the external RADIUS server dropped the message because of an unknown client error, verify that the NetOp PM RADIUS server is in the list of valid clients of the external RADIUS server configuration.
2-14
Book-Title
DraftApril 9, 2009
NetOp PM Database Issues
Accounting Data Is Missing in External RADIUS Server

ProblemRedback accounting data missing in the external RADIUS server. This occurs when the NetOp PM system is configured to proxy to an external RADIUS server and the server does not have all vendor-specific attributes (VSAs) present. ActionTwo possible actions exist for troubleshooting this issue: Confirm that all VSAs, including the RB-NPM-Service-Id attribute, exist in the external RADIUS server dictionary. Confirm that Redback VSAs are sent and received in accounting messages by closely examining the Ethereal captures. The following example shows the declaration of the RB-NPM-Service-Id VSA in the /usr/local/npm/radius/dictionary_redback.cfg file:
VENDORATTR 2352 RB-NPM-Service-Id 106 string
For the RB-NPM-Service-Id RADIUS attribute, look for attribute number 106 in the Ethereal trace to determine whether it is present. Follow this general method to determine if other Redback VSAs are being sent or received in the packets captured by the Ethereal application.

This section provides techniques for troubleshooting problems you may encounter with your NetOp PM high availability database configuration and includes the following topics: Verify That the Primary and Standby Databases Are Running Determine the Name and Status of a Current Fast-Start Failover Database Determine the Status of the Standby Database Determine the Latest Primary and Standby Redo Log Numbers View the Oracle Data Guard Broker Log File
Verify That the Primary and Standby Databases Are Running

The verify_npm_standby.sh script verifies that the primary and standby NetOp databaseNetOp PM databases are operating properly. The script sends information from the primary database to the standby database and then checks the status of the changes. Because the time required by this transmission varies depending on network traffic and database host loads, use the -time_delay minutes_to_wait construct to configure the interval between transmitting data from the primary database and checking the status of the changed data on the standby database. If you are planning to run this script after running the setup_primary_db.sh script, wait at least five minutes before you run the verify_npm_standby.sh script to allow for all processing to be completed on the primary database host.
2-15
DraftApril 9, 2009
Caution Risk that the verify_npm_standby.sh script fails following a failover. After a failover, the database on the failed database host is not operational. If you run the verify_npm_standby.sh script from the failed primary database host while it is not operational, it fails. To avoid this risk, fix the problem that caused the failover and restore the failed primary database as the standby database before you try to run the script on this host. To verify that both the primary and standby NetOp databaseNetOp PM databases are working properly, perform the following steps: 1. Log on to the standby NetOp databaseNetOp PM database host as oracle. 2. Open a terminal window and navigate to the /u01/app/netop/releaseId/scripts/db/admin directory, where releaseId is the four-digit identifier for the current NetOp EMS software release. 3. Open a terminal window and navigate to the NetOp PM database administration directory: cd /usr/local/npm/db/admin 4. Run the verify_npm_standby.sh script with the following syntax: verify_npm_standby.sh [-db database_name] [-h] -sys_passwd system_account_password [-time_delay minutes_to_wait] The script takes as much time to run as the value of the -time_delay minutes_to_wait construct. You are prompted to enter the password for the primary database. A message is displayed when the script terminates to indicate whether the standby database is working properly. Table 2-7 describes the syntax and usage guidelines for this script.
Table 2-7
Syntax -db database_name -h -sys_passwd system_account_password -time_delay minutes_to_wait
Syntax for the verify_npm_standby.sh Script

Description Optional. Name of the NetOp databaseNetOp PM database. The default value is emsnpm. Optional. Prints usage information and exits the script. Password for the database system account. Optional. Time to wait (in minutes) before checking for applied data at the standby NetOp databaseNetOp PM database. Valid values are 1 to 15; the default value is 5.
If the standby NetOp databaseNetOp PM database is not working, view the standby NetOp databaseNetOp PM database Oracle Data Guard Broker log file. For information on viewing these log files, see the View the Oracle Data Guard Broker Log File section on page 2-20. Note If network communication is not optimal, you run the risk of experiencing problems with the standby database. If you experience an unusually high number of issues related to administering the standby database, verify the following: Network communication between the primary and standby database is operating optimally. Network changes applied to the standby database are not affecting optimal network communication.
2-16
Book-Title
DraftApril 9, 2009
Determine the Name and Status of a Current Fast-Start Failover Database

To determine the name and status of a fast-start failover database, perform the following steps: 1. Log on to the NetOp databaseNetOp PM database on which you want to identify the name and status. 2. Start a DataGuard broker CLI session by entering the following command at the UNIX command line: dgmgrl The Data Guard broker CLI appears. 3. Enter the following lines in the Data Guard broker CLI: connect sys/password@database-name; show configuration where the value of the password argument is the password of the sys Oracle database account (the default value is redback), and the value of the database-name argument is the database name (the default value is emsnpm). Information similar to the following example appears:
--------------------------------------------------------DGMGRL> show configuration Configuration Name: npmcold_DG Enabled: YES Protection Mode: MaxAvailability Fast-Start Failover: ENABLED Databases: npm - Primary database npm_clone - Physical standby database - Fast-Start Failover target Current status for "npmcold_DG": SUCCESS
In this example, the output identifies the primary database as npm and the standby database as npm_clone. Table 2-8 describes the warning and error messages associated with the current status of the fast-start failover configuration.
Table 2-8
Status SUCCESS
Warnings and Error Messages from the show configuration Command

Description All three fast-start failover components are properly configured and available Action Required None
2-17
DraftApril 9, 2009
Table 2-8
Status Warning: ORA-16608: one or more databases have warnings
Warnings and Error Messages from the show configuration Command (continued)
Description One or both of the following scenarios: The DataGuard observer is either not set up or is not running. The standby database was stopped using the stop_npm_db.sh script. To determine which scenario is in effect, see the Check the Status of the NetOp PM Databases in Fast-Start Failover Configuration section on page 2-31. Action Required If the: DataGuard observer is not set up, see the NetOp PM Database Failover Configuration chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide. DataGuard observer is not started, restart it with the setup_observer.sh script; for more information, see the NetOp PM Database Failover Configuration chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide. Standby database is stopped, restart it with the start_npm_db.sh; for more information, see the NetOp PM Database Maintenance chapter in the NetOp Database Administration Guide. If the primary database is: StoppedRestart it with the start_npm_db.sh; for more information, see the NetOp PM Database Maintenance chapter in the NetOp Database Administration Guide. FailedRecover from the failure; for more information, see the NetOp PM Database Failover Management chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide. Recover from the failure; for more information, see the NetOp PM Database Failover Management chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide.
Error: ORA-16625: cannot reach the database
The primary database has: Been stopped using the stop_npm_db.sh script. Failed. To determine which scenario is in effect, see the Check the Status of the NetOp PM Databases in Fast-Start Failover Configuration section on page 2-31.
Warning: ORA-16607: one or more databases have failed
The standby database has failed.
4. If the status field shows an error, enter the following command to investigate: show database verbose database-name where the value of the database-name argument is the database name of either the primary database or standby database. You can also view the Oracle Data Guard Broker log files. If you do not understand an error message, contact your local Redback technical support representative and provide a copy of the diagnostic tar file and broker log files. For information on viewing Oracle broker log files, see the View the Oracle Data Guard Broker Log File section on page 2-20.
Determine the Status of the Standby Database

If the verify_npm_standby.sh script reports that the standby database is not operational, perform the following steps: 1. Log on to the primary NetOp databaseNetOp PM database as the Oracle system administrator (for example, sys) using the following command: sqlplus sys/password@database-name as sysdba where the value of the password argument is the password of the system database account (the default value is redback), and the value of the database-name argument is the database name (the default value is emsnpm).
2-18
Book-Title
DraftApril 9, 2009
2. In the SQL*Plus session, enter the following command: select STATUS from v$archive_dest where DEST_NAME='STANDBY_ARCHIVE_DEST'; 3. If the status reported is anything other than valid, record the content of the error column. 4. Exit the SQL*Plus session. 5. On both the primary and standby databases, examine the most recent section of the alert log files to check for errors (for example, see the /u01/app/oracle/product/10g/admin/npmems/bdump/alert_npmems.log file). Send the error columns from the status query and the alert_ems.log filesdiagnostic tar file to your local Redback technical support representative.
Determine the Latest Primary and Standby Redo Log Numbers

To determine the latest primary and standby redo log file numbers, perform the following steps: 1. Determine the number of the redo log file archived most recently by the primary database: a. Log on to the primary database host as oracle. b. Log on to the NetOp databaseNetOp PM database as the Oracle system administrator (for example, sys) using the following command: sqlplus sys/password@database-name as sysdba where the value of the password argument is the password of the system database account (the default value is redback), and the value of the database-name argument is the database name (the default value is emsnpm). c. In the SQL*Plus session, enter the following query: select max(sequence#) from v$archived_log; 2. Determine the number of the latest redo log file received and the last redo log file processed by the standby database: a. Log on to the standby database host as oracle. b. Log on to the NetOp databaseNetOp PM database as the Oracle system administrator (for example, sys) using the following commands: sqlplus sys/password@database-name as sysdba where the value of the password argument is the password of the system database account (the default value is redback), and the value of the database-name argument is the database name (the default value is emsnpm). c. Determine the last redo log file received from the primary database; in the SQL*Plus session, enter the following query: select max(sequence#) from v$archived_log;
2-19
DraftApril 9, 2009
Configuration Issues
d. Determine the last redo log file processed by the standby database; enter the following query: select max(sequence#) from v$log_history; The numbers of these log files should match the numbers on the primary database. If not, consult the Oracle alert log file on the standby database host for errors. If the log numbers do not match, send the error message and the alert_database_name.logdiagnostic tar file to your local Redback technical support representative. e. Exit the SQL*Plus session.
View the Oracle Data Guard Broker Log File

If you have a fast-start failover configuration, the Oracle Data Guard Broker records NetOp PM database behavior and status information in a broker log file. To view the Oracle database guard broker log file, perform the following steps: 1. Log on to the NetOp PM database host as oracle. 2. Open a terminal window and navigate to the /u01/app/oracle/product/10g/admin/npm/bdump directory. 3. Open the drc_database_name.log file (for example, the drc_npm.log file) and view the latest error messages at the bottom of the file. If you do not understand an error message, contact your local Redback technical support representative and provide a copy of your broker log file.
This section describes system problems that might occur in the NetOp PM software if it is not correctly configured. The following topics describe the problems and help identify possible solutions: Subscriber Browser Not Redirected to Portal Change of Service Not Applied HTTP Redirect Not Working Logon Fails When External LDAP Server Does Not Permit Multiple Searches from Same LDAP Connection Logon Fails When an External LDAP Server Uses CHAP No Reauth Messages
Subscriber Browser Not Redirected to Portal

ProblemThe web browser of the subscriber is pointed to the correct URL for the web portal but cannot reach the default router interface. The web browser displays the following message:
Please wait while you are redirected
2-20
Book-Title
DraftApril 9, 2009
This message can result from a misconfigured policy access control list (ACL). For example, the policy access list captiveportalacl must be configured to point to the NetOp PM web portal. ActionEnsure that all policy ACLs are correctly configured. For configuration details, see the Configurations on the Node for the NetOp PM System chapter in the NetOp Policy Manager Configuration Guide.
Change of Service Not Applied

ProblemWhen a subscriber attempts to modify a service, the change is not applied. The following RADIUS attributes cannot be changed without terminating and restarting a subscriber session. ActionTo change these attributes, clear the subscriber session: Framed-IP-Address RB-Context-Name RB-Client-DNS-Primary RB-Client-DNS-Secondary RB-Client-NBNS-Primary RB-Client-NBNS-Secondary RB-DHCP-Max-Leases RB-IP-Address-Pool-Name RB-PPPoE-IP-Route RB-PPPoE-IP-Route RB-PPPoE-URL
HTTP Redirect Not Working

ProblemThe subscriber is not forced to the captive portal web page for logon. ActionThree possible actions exist for correcting this configuration problem: For SmartEdge routers, check the service_radius table in the NetOp PM database to ensure that the value for the http_redirect_profile_name attribute matches the value for the http_redirect_profile_name attribute configured on the node. For SMS devices, check the http-redirect url line in the configuration file; for example:
http-redirect url http://10.192.100.20/NPM-6.0.3.0/portal.php
Ensure that HTTP redirect is properly configured on the node. For example, on the SmartEdge router, the following configuration is valid within the definition of a context in the node configuration file:
! ! Define an ACL which will only allow subscriber web login traffic to the ! NetOp PM Lightweight Web Portal server
2-21
DraftApril 9, 2009
Configuration Issues ! policy access-list captiveportalacl ! ! Permit dhcp traffic to flow from subscriber to the router. Required so ! the router can receive a DHCP release for a subscriber which has not Web ! logged in. ! seq 10 permit udp any any eq bootps class BOOTPS ! ! Permit DNS traffic to flow from subscriber to the router. Required so ! the browser can resolve web site names. ! seq 20 permit udp any any eq domain class DNS seq 30 permit udp any any eq netbios-ns class DNS ! ! Permit HTTP packets to access the URL configured under the ! "http-redirect profile <profile-name>". The IP address is ! the address of the network web server hosting the URL. ! seq 35 permit tcp any host 10.192.100.20 eq www class WEB ! ! Permit http traffic to flow from subscriber to the NetOp PM Lightweight ! Web Portal server. ! seq 40 permit tcp any any eq www class CAPTIVE_PORTAL ! ! Classify all other IP traffic so we can drop it. ! seq 60 permit ip any class IP ! ! Configure HTTP redirect profile names and the URLs to which subscriber ! sessions will be redirected ! http-redirect profile web_login_redirect url http://10.192.100.20/NPM-XXXX/portal.php
On the SmartEdge router, the following configuration is valid outside the definition of a context in the node configuration file:
! ! Define a forwarding policy which will force subscribers to the NetOp PM ! Lightweight Web Portal server (the HTTP packets will be redirected locally ! - i.e. to the HTTP server running on the XCRP card - and then to the
2-22
Book-Title
DraftApril 9, 2009
Configuration Issues ! NetOp PM Lightweight Web Portal server specified in the configured url under ! http-redirect profile name matching the profile name received from NPM ! VSA 107). ! forward policy captiveportal access-group captiveportalacl BASICG class CAPTIVE_PORTAL redirect destination local class IP drop ! ! Enable the HTTP redirect server running on the XCRP card and listening on ! port 80 (by default) ! http-redirect server
On the SMS device, the following configuration is valid within the definition of a context in the node configuration file:
! ! Enable HTTP redirect server and set the URL where the subscriber will be ! redirected ! Note that there could only be *one* http-redirect in one context, so ! just enable one http-redirect URL in this context ! http-redirect url http://10.192.100.20/NPM-XXXX/portal.php ! ! Define redirect ACL which will force subscribers to the NetOp PM Lightweight ! Web Portal server ! ip access-list captiveportal ! ! Permit HTTP packets with destination address 10.192.100.20 to flow from ! subscriber to the SMS ! permit tcp any host 10.192.100.20 eq 80 ! ! Redirect HTTP packets with any source/destination addresses to the local ! HTTP redirect server running on the node ! redirect pool 10.192.42.1 tcp any any eq 80 !
2-23
DraftApril 9, 2009
Configuration Issues ! Permit dhcp traffic to flow from subscriber to the SMS. Required so the ! SMS can receive a DHCP release for a subscriber which has not Web logged ! in. ! permit udp any any eq 67
Logon Fails When External LDAP Server Does Not Permit Multiple Searches from Same LDAP Connection
ProblemSubscriber logon fails when the external LDAP server does not permit multiple searches from the same LDAP connection. ActionComment out the HoldServerConnection attribute in the npm/radius/npm_radiator.cfg configuration file. For example, the following configuration is valid when the external LDAP server supports multiple searches for each connection
<AuthBy SQLLDAPCONFIGURABLE> Identifier proxy_ldap_request . . . # # Permit multiple searches on the same LDAP connection # HoldServerConnection . . . </AuthBy>
The following configuration is valid when the external LDAP server does not support multiple searches for each connection:
<AuthBy SQLLDAPCONFIGURABLE> Identifier proxy_ldap_request . . . # # Permit multiple searches on the same LDAP connection # # HoldServerConnection . . . </AuthBy>
2-24
Book-Title
DraftApril 9, 2009
Logon Fails When an External LDAP Server Uses CHAP

ProblemSubscriber logon fails when Challenge Handshake Authentication Protocol (CHAP) is used by the external LDAP server. ActionEnsure that the node is configured with the bind authentication pap or bind authentication pap chap command (in port configuration mode) and not the bind authentication chap or bind authentication chap pap command. Need modes for these commands? For example, on the SmartEdge router, the following configuration is valid:
port ethernet 12/2 no shutdown encapsulation pppoe bind authentication pap chap context BASIC maximum 8000
or:
bind authentication pap context BASIC maximum 8000
The following configuration is invalid:

port ethernet 12/2 no shutdown encapsulation pppoe bind authentication chap pap context BASIC maximum 8000
or:
bind authentication chap context BASIC maximum 8000
On the SMS device, the following configuration is valid:

port ethernet 3/1 encapsulation ppp over-ethernet bind authentication pap chap
or:
bind authentication pap
The following configuration is invalid:

port ethernet 3/1 encapsulation ppp over-ethernet bind authentication chap pap
or:
bind authentication chap
No Reauth Messages
ProblemNo SNMP reauth messages appear in the trace logs. ActionTwo possible actions exist for correcting this configuration problem:
2-25
DraftApril 9, 2009
RADIUS Flow-Through Attributes Not Received
Check the SmartEdge router configuration by checking the sample-ser.cfg file: Ensure that the AAA authentication and accounting settings are configured for the NetOp PM system to use the RADIUS servers configured in the local context, as in the following example: aaa global accounting reauthorization subscriber radius context local Ensure that the NetOp PM system is configured to be notified of reauthorization events, as in the following example: aaa global accounting event reauthorization Ensure that AAA reauthentication is configured for the local context so that an interim message is generated after a bulk reauthentication, as in the following example: aaa reauthorization bulk global Ensure that the SmartEdge routers are configured to accept SNMP reauth operations, as in the following example: snmp view npm_view rbnSubsReauthRadiusID included
Check the NetOp PM RADIUS server configuration to ensure that it is using the same port for authentication requests as the authentication port configured in the node. For information on starting the NetOp PM RADIUS servers using the script, see the Configure External RADIUS and LDAP Servers chapter in the NetOp Policy Manager Configuration Guide.
RADIUS Flow-Through Attributes Not Received

ProblemNetOp PM does not receive RADIUS flow-through attributes from the external RADIUS server. ActionPerform the following steps: 1. Verify that the double_access_suppression field in the proxy_config table is set to N. If it is set to Y, flow-through attributes are suppressed. 2. Verify that the missing flow-through attributes are configured in the radius_proxy_attributes or proxy_attribute_mapping table under the correct realm.
Wrong Circuit Type

ProblemThe NetOp PM system cannot determine the correct circuit type. ActionCheck the nas_port_id field in the accounting table. Ensure that the default format of the NAS-Port-Id attribute was not modified when the SmartEdge routers were configured to use external RADIUS servers for the NetOp PM system. By default, SmartEdge routers generate both the physical and logical specifications for a circuit and send the NAS-Port-Id attribute in RADIUS Access-Request and Accounting-Request packets. The NetOp PM system uses this attribute to determine the correct circuit type and uses the logical specifications to determine if the circuit is a static CLIPS or a dynamic CLIPS circuit. To ensure that the NetOp PM system can differentiate between circuit types, do not use the radius
2-26
Book-Title
DraftApril 9, 2009
NetOp PM API Server Calls Time Out
attribute nas-port-id format physical command (in context configuration mode) because it truncates the logical specifications from the NAS-Port-Id attribute and returns only the physical specifications for the circuit. Do not change the default format; leave it as all.
NetOp PM API Server Calls Time Out

ProblemThe host for the NetOp PM API server that is serving as the load balancer may be down (due to a hardware, an OS, or a power failure), causing all NetOp PM API calls to time out. ActionTo restore NetOp PM API server functionality, perform the following steps: Attempt to restore the load balancer host. If it is not possible to restart the hardware of the load balancer, designate another NetOp PM API server as the load balancer by performing the following steps: 1. Log on to the host of the NetOp PM lightweight web portal as root. 2. Open a terminal window and navigate to the NetOp PM installation directory: cd /usr/local/npm 3. Run the script, with the following syntax: ./config_portal.sh -npmapi_host api_server where the value of the api_server argument identifies the hostname or IP address of the NetOp PM API server you want to serve as the load balancer. If you use a hostname, it must be found in the DNS or the /etc/hosts file 4. Designate this NetOp PM API server as the load balancer for your Simple Object Access Protocol (SOAP) client. For syntax and usage guidelines for this script, see the section on configuring the NetOp PM lightweight web portal in the Initial Configuration chapter in the NetOp Policy Manager Configuration Guide.
NetOp PM Database Failover Configuration Issues

This section describes procedures to use when you investigate database failover configuration issues. It includes the following topics: Verify That the Observer Process on a Solaris Host Is Operational Verify That the Observer Process on a PC Host Is Operational Move the Observer Process to Another Host Stop and Restart the Observer Process Errors Connecting to the Standby Database View the Oracle Alert Log File Convert Manual Failover to Fast-Start Failover
2-27
DraftApril 9, 2009
Check the Status of the NetOp PM Databases in Fast-Start Failover Configuration Disable Fast-Start Failover
Verify That the Observer Process on a Solaris Host Is Operational

To verify that the Data Guard Observer host is operational, perform the following steps: 1. Log on to the primary NetOp PM database host as oracle. 2. Log on to the NetOp PM database as the Oracle user sys using the following command: sqlplus sys/password@database-name as sysdba where the value of the password argument is the password of the sys Oracle database account (the default value is redback), and the value of the database-name argument is the database name (the default value is emsnpm). 3. Enter the following command at the SQL*Plus command line: select FS_FAILOVER_STATUS, FS_FAILOVER_THRESHOLD, FS_FAILOVER_OBSERVER_PRESENT, FS_FAILOVER_OBSERVER_HOST from v$database; Table 2-9 describes the information that this query provides for each item specified in the command.
Table 2-9
Query Item FS_FAILOVER_STATUS FS_FAILOVER_THRESHOLD FS_FAILOVER_OBSERVER_PRESENT
Operational Information About the Observer Process

Information The current status of the fast-start failover configuration. The length of time the observer process waits before starting fast-start failover. One of the following values: YES NO Note: If the connection to the observer process is lost, the value for FS_FAILOVER_OBSERVER_PRESENT is NO.
FS_FAILOVER_OBSERVER_HOST
Hostname of the Data Guard observer host.
Verify That the Observer Process on a PC Host Is Operational

To verify that the Data Guard Observer host on a PC is operational, perform the following steps: 1. Start a Data Guard broker CLI session by entering the following command at the PC prompt in the command window: dgmgrl The Data Guard broker CLI appears. 2. To verify that all components are operational, enter the following line in the Data Guard broker CLI: show configuration; If the output shows that the current status for the configuration is anything other than SUCCESS, see Table 2-8 on page 2-17.
2-28
Book-Title
DraftApril 9, 2009
Move the Observer Process to Another Host

You can move the observer process to another host at any time, such as after an unrecoverable crash of the host. You do not need to disable fast-start failover to move the Data Guard Observer host. To move the observer process to another host, perform the following steps: 1. Prepare the new host machine; for more information, see the NetOp PM Database Failover Configuration chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide. 2. Configure and start the observer process on the new host; for more information, see the NetOp PM Database Failover Configuration chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide. 3. Verify that the new observer process is operational; see the NetOp PM Database Failover Configuration chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide.
Stop and Restart the Observer Process

When you restart the observer process, you must first stop the observer process to close the connections established between the previous observer process and DataGuard broker on the primary and standby database hosts. There are two methods you can use to stop and restart the observer process: Script methodThis is the preferred method due to the scripts error handling logic. Run the setup_observer.sh script to restart the observer process; for more information, see the NetOp PM Database Failover Configuration chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide. DataGuard broker CLI methodPerform the following steps: 1. Log on to a fast-start failover component host as the oracle user. 2. Start a DataGuard broker CLI session by entering the following command at the UNIX command line: dgmgrl The Data Guard broker CLI appears. 3. Enter the following lines in the Data Guard broker CLI: connect sys/password@database-name; where the value of the password argument is the password of the sys Oracle database account (the default value is redback), and the value of the database-name argument is the database name (the default value is emsnpm). stop observer; start observer; 4. To verify that all components are operational, enter the following line in the DataGuard broker CLI: show configuration; If the output shows that the current status for the configuration is anything other than SUCCESS see Table 2-8 on page 2-17.
2-29
DraftApril 9, 2009
Errors Connecting to the Standby Database

When the Data Guard broker cannot connect to the standby database, you may see the following error message from the Oracle DBMS:
Error: ORA-16796: one or more properties could not be imported from the database
Take the following steps to resolve the error: Ensure that you edited the Oracle Net configuration file, listener.ora, on the standby database host; for more information, see the NetOp PM Database Failover Configuration chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide. Ensure that the database has been started on the standby database host.
View the Oracle Alert Log File

To view the Oracle alert log file, perform the following steps: 1. Log on to the NetOp PM database host as oracle. 2. Open a terminal window and navigate to the /u01/app/oracle/product/10g/admin/npm/bdump directory. 3. Open the alert_database_name.log file (for example, the alert_npm.log file) and view the latest error messages at the bottom of the file. If you do not understand an error message, contact your local Redback technical support representative.
Convert Manual Failover to Fast-Start Failover

If you have set up a manual failover configuration, you have the option to convert to fast-start failover by running the enable_fast_start_failover.sh script. In this configuration, make sure that both the primary and standby databases are configured with the same database name or Oracle system identifier (SID) so that client applications are not required to change their references to the database following a failover. The enable_fast_start_failover.sh script configures the: Primary NetOp PM database for fast-start failover Standby NetOp PM database for fast-start failover Data Guard broker on the primary NetOp PM database
You do not need to run the enable_fast_start_failover.sh script if you have already run the setup_primary_db.sh script with the -enable_fast_start_failover option. Note After you set up the fast-start failover configuration, do not use the scripts provided for switching databases in a manual failover configuration. You must use the DataGuard broker CLI instead.
To configure the primary and standby NetOp PM databases for fast-start failover, perform the following steps: 1. Open a terminal window and log on to the primary NetOp PM database host as root.
2-30
Book-Title
DraftApril 9, 2009
2. Navigate to the /u01/app/netop/releaseId/scripts/db/admin directory, where releaseId is the identifier for the current release of the NetOp EMS software; for example, 6.0.3.3. 3. Navigate to the NetOp PM database administration directory: cd /usr/local/npm/db/admin 4. Run the enable_fast_start_failover.sh script according to the following syntax: enable_fast_start_failover.sh [-db database_name] [-h] [standby_db_acct standby_database_account_name] -standby_db_passwd standby_database_account_password Table 2-10 describes the syntax and usage guidelines for this script.
Table 2-10
Syntax -db database_name -h -standby_db_acct standby_database_account_name -standby_db_passwd standby_database_account_password
Syntax for the enable_fast_start_failover.sh Script

Description Optional. Name of the NetOp PM database. The default value is emsnpm. Optional. Prints usage information and exits. Optional. Account name for the standby database sysdba. The default value is sys. Password for the standby database sysdba account.
Check the Status of the NetOp PM Databases in Fast-Start Failover Configuration

After you have reviewed the status of fast-start failover configuration and discovered problems with either the primary or standby database, you can use the show database command of the DataGuard broker CLI to gather additional information by performing the following steps: 1. Log on to either the primary or standby NetOp PM database host as oracle. 2. Start a DataGuard broker CLI session by entering the following command at the UNIX command line: dgmgrl The Data Guard broker CLI appears. 3. Connect to the database whose status you want to determine: connect sys/password@database-name; where the value of the password argument is the password of the sys Oracle database account (the default value is redback) and the value of the database-name argument is the database name (the default value for the primary database is npm and standby database is npm_clone).
2-31
DraftApril 9, 2009
4. Request the status of the database: show database database-name where the value of the database-name argument is the database name (the default value for the primary database is npm and standby database is npm_clone). Table 2-11 describes the warning and error messages associated with the current status of the fast-start failover configuration.
Table 2-11
Status SUCCESS
Warnings and Error Messages from the show database Command

Connected to Database Either Description All three fast-start failover components are properly configured and available The DataGuard observer is either not set up or is not running. Action Required None
Warning: ORA-16819: Fast-Start Failover observer not started
Either
If the: DataGuard observer is not set up, see the NetOp PM Database Failover Configuration chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide. DataGuard observer is not started, restart it with the setup_observer.sh script; for more information, see the NetOp PM Database Failover Configuration chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide.
Error: ORA-16625: cannot reach the database
Primary
The primary database has: Been stopped using the stop_npm_db.sh script. Failed.
If the primary database is: StoppedRestart it with the start_npm_db.sh; for more information, see the NetOp PM Database Maintenance chapter in the NetOp Database Administration Guide. FailedRecover from the failure; for more information, see the NetOp PM Database Failover Management chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide.
Warning: ORA-16817: unsynchronized Fast-Start Failover configuration
Primary
The standby database was stopped using the stop_npm_db.sh script.
Restart the standby database with the start_npm_db.sh; for more information, see the NetOp PM Database Maintenance chapter in the NetOp Database Administration Guide. Recover from the failure; for more information, see the NetOp PM Database Failover Management chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide. Recover from the failure; for more information, see the NetOp PM Database Failover Management chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide.
Standby
The primary database has failed.
Error: ORA-16825: Fast-Start Failover and other errors or warnings detected for the database
Primary
2-32
Book-Title
DraftApril 9, 2009
Table 2-11
Status Warning: ORA-16818: Fast-Start Failover suspended
Warnings and Error Messages from the show database Command (continued)
Connected to Database Standby Description The primary database has been stopped using the stop_npm_db.sh script. Action Required Restart the primary database with the start_npm_db.sh; for more information, see the NetOp PM Database Maintenance chapter in the NetOp Database Administration Guide. Restart the standby database with the start_npm_db.sh; for more information, see the NetOp PM Database Maintenance chapter in the NetOp Database Administration Guide. Recover from the failure; for more information, see the NetOp PM Database Failover Management chapter in the NetOp Policy Manager Database Redundancy and Recovery Guide.
SHUTDOWN
Standby
The standby database was stopped using the stop_npm_db.sh script.
Error: ORA-01034: ORACLE not available Or Error: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor
Standby
Disable Fast-Start Failover

When a network outage isolates the primary database host from both the Data Guard Observer host and standby database host while the databases are synchronized, the primary database may stall to prevent any further transactions from being committed because a fast-start failover may have occurred while it was isolated. If you expect the network to be disconnected for a long time, disable fast-start failover on the primary database to allow processing to continue (without waiting for the connection to the target standby database or the observer computer to be restored). Disable fast-start failover on the primary database host only under the following circumstances: During a prolonged network outage when all the following conditions apply: A prolonged network outage has isolated all three of the following hosts from one another: the primary database host, standby database host, and observer host. In this case, the primary database stalls, preventing new transactions from being committed, and the standby database is unable to fail over to the primary role. You expect the network outage to continue. You want processing on the primary database to continue without waiting for any of the network connections to be restored. When you want to manually fail over to the standby database under either of the following conditions: The primary database is no longer synchronized with the standby database. The observer process is not running or is not communicating with the standby database. To disable the fast-start configuration on the primary database host, perform the following steps: 1. Log on to the primary NetOp PM database host as oracle.
2-33
DraftApril 9, 2009
Performance Issues
2. Start a DataGuard Broker CLI session by entering the following command at the UNIX command line: dgmgrl The Data Guard broker CLI appears. 3. Enter the following command to connect to the primary database: connect sys/password@database-name; where the value of the password argument is the password of the sys Oracle database account (the default value is redback), and the value of the database-name argument is the database name (the default value is emsnpm). 4. Enter the following command to disable the fast-start configuration: disable fast_start failover; In an emergency situation in which you run the disable fast_start failover command and it fails to disable the fast-start failover function, you can use the force option by running the following command: disable fast_start failover force; Caution Risk of both database hosts assuming the primary role, resulting in two divergent primary databases. After you disable fast-start failover on the primary host during a network outage, if the connection between the standby database host and the observer host is restored first, the standby host transitions to the primary role. This results in both databases assuming the primary role. To avoid this risk, remove the connection between the standby database host and observer host until the network outage is resolved and the connection between primary database host and observer host is restored.
Performance Issues
This section describes procedures to use when you investigate potential performance issues. It includes the following topics: Monitor System Performance on a Solaris Server Determine if the NetOp PM RADIUS Server Is Limiting Traffic Flow Monitor LAN Utilization Determine if the NetOp PM Database Is Limiting Traffic Flow
Monitor System Performance on a Solaris Server

To monitor system performance on a Solaris server, perform the following steps: 1. Log on as root. 2. Open an xterm or a terminal window and navigate to a disk partition that is not very active; for example, /var/tmp.
2-34
Book-Title
DraftApril 9, 2009
Performance Issues
3. Enter the following command to capture system activity: sar -o filename interval iteration where the value of the filename argument is the name of the file in which to report system activity, the value of the interval argument is the frequency of the system activity snapshot in seconds, and the value of the iteration argument is the number of snapshots of system activity to capture. For example, enter the following command to capture 10 snapshots at intervals of 180 seconds (3 minutes):
sar -o snapshot-file.sar 180 10
We recommend that you do not configure the interval value to be too low; a low value can negatively affect performance. To stop the captures at any time, type Ctrl+C. For more information on the sar command, see the UNIX man page for sar(1). 4. Check the data logged by command by opening another xterm window and performing one of the following tasks: To check the CPU, enter the sar -u -f filename command To check the disk, enter the sar -b -f filename command To check the memory, enter the sar -r -f filename command To get a report of all data, enter the sar -A -f filename command
You can check the sar data using these commands at any point while the sar command is running or after it has finished. 5. Open another xterm or terminal window and navigate to a disk partition that is not very active; for example, /var/tmp. 6. Enter the following command to capture active process statistics: prstat -c interval iteration > filename where the value of the interval argument is the frequency of the active process statistics snapshot (in seconds), the value of the iteration argument is the number of snapshots of active process statistics to capture, and the value of the filename argument is the name of the file in which to report active process statistics. For example, enter the following command to capture 10 snapshots at intervals of 180 seconds (3 minutes):
prstat -c 180 10 > snapshot-file.prstat
We recommend that you do not configure the value of the interval argument to be too low; a low value can negatively affect performance. To stop the captures at any time, type Ctrl+C. 7. Check the data logged by the prstat command by opening another xterm window and entering the following command to scroll through the file from the beginning: more filename You can check the prstat data using this command at any point while the command is running or when it has finished. For information on the fields in the output, read the prstat man pages.
Determine if the NetOp PM RADIUS Server Is Limiting Traffic Flow

To determine if the NetOp PM RADIUS server is limiting traffic flow, perform the following steps:
2-35
DraftApril 9, 2009
Performance Issues
1. Clear all RADIUS counters on the node by entering one of the following commands: On the SmartEdge router, enter the following command in exec mode: clear radius counters {acct_server | authen_server} ip-addr port-num On the SMS device, enter the following command in exec mode: clear radius counter ip-addr port-num where the value of the ip-addr argument is the IP address of the RADIUS host running the NetOp PM RADIUS servers, and the value of the port-num argument is the authentication or accounting port for which you want to clear counters. For example, on the SmartEdge router, enter the following commands to clear the authentication and accounting counters on the node communicating with the RADIUS servers hosted at 10.192.100.10 on ports 1812 and 1813:
clear rad count auth 10.192.100.10 1812 clear rad count acct 10.192.100.10 1813
2. Check the counters periodically to see if any messages are dropped or timing out, which may indicate that the NetOp PM RADIUS server is limiting traffic flow and, therefore, causing performance issues.
Monitor LAN Utilization

To monitor LAN utilization, use a packet sniffer or some other hardware or software tool independent of the NetOp PM system to ensure that: Network collisions are not excessive. The LAN speed is at least 100 Mbps for all segments.
Determine if the NetOp PM Database Is Limiting Traffic Flow

This section describes how to determine if the NetOp PM database is limiting traffic flow, and provides instructions on how to run the monitor_npm_db.sh, report_npm_db_operations.sh, and report_npm_db_performance.sh scripts to monitor and troubleshoot the NetOp PM database in the following sections: Monitor the NetOp PM Database and Generate E-Mail Alerts Troubleshoot the NetOp PM Database Generate the NetOp PM Database Performance Report
Monitor the NetOp PM Database and Generate E-Mail Alerts

Use the monitor_npm_db.sh script to monitor the NetOp PM database and generate e-mail alerts if the specified threshold for tablespace usage is reached. When a threshold is reached an alert is generated, and the script sends an e-mail alert to the specified e-mail address. The script also checks Oracle trace files and alert log for errors. If errors are found, the NetOp PM system sends an alert e-mail to the administrator. Schedule the monitor_npm_db.sh script to run regularly: weekly at least.
2-36
Book-Title
DraftApril 9, 2009
Performance Issues
Note
In order for the monitor_npm_db.sh script to send e-mail to a non-local e-mail address (an e-mail account that is not on the NetOp PM database itself), you must ensure that the hosts e-mail configuration is correct. One common issue is that the default Solaris installation requires the hostname, mailhost, to exist on the local network domain, either in the /etc/hosts file or in Domain Name System (DNS) or Network Information Service (NIS), and so on. For example, if your database hostname is dbhost.lab.mycompany.com, there must be a definition for the mailhost.lab.mycompany.com hostname that refers to a valid mail server.
To monitor the NetOp PM database and generate e-mail alerts, perform the following steps: 1. Log on to the active database host as oracle. 2. Open a terminal window and navigate to the NetOp PM database maintenance directory: cd /usr/local/npm/db/maintenance 3. Run the monitor_npm_db.sh script according to the following syntax: ./monitor_npm_db.sh [-db database_name] [-db_passwd database_account_password] [-email email_address] [-h] [-table_usage tablespace_usage] Table 2-12 describes the syntax and usage guidelines for this script.
Table 2-12
Syntax -db database_name -db_passwd database_account_password -email email_address
Syntax for the monitor_npm_db.sh Script

Description Optional. Name of the NetOp PM database. The default value is npm. Optional. NetOp PM database account password. Optional. E-mail address of the administrator to which alerts are sent. Defined when you run the config_db.sh script. The default value is oracle. Optional. Prints usage information and exits. Optional. Tablespace full threshold in percentage form. When the threshold is reached, an alert is generated. Defined when you run the config_db.sh script. The default value is 80. Optional. Number of days to retain old trace files. Defined when you run the config_db.sh script. The default value is 15.
-h -table_usage tablespace_usage
-trace_file_lifetime days_to_retain_trace_files
Troubleshoot the NetOp PM Database

Use the report_npm_db_operations.sh script to collect information about the NetOp PM database for troubleshooting purposes. Run it when the NetOp PM database fails or appears to be malfunctioning. The script reports on various OS and database information. The script output should be analyzed by an Oracle database administrator (DBA). To troubleshoot the NetOp PM database, perform the following steps: 1. Log on to the active database host as oracle. 2. Open a terminal window and navigate to the NetOp PM database maintenance directory: cd /usr/local/npm/db/maintenance
2-37
DraftApril 9, 2009
Performance Issues
3. Run the report_npm_db_operations.sh script according to the following syntax: ./report_npm_db_operations.sh [-db database_name] [-h] Table 2-13 describes the syntax and usage guidelines for this script.
Table 2-13
Syntax -db database_name -h
Syntax for the report_npm_db_operations.sh Script

Description Optional. Name of the NetOp PM database. The default value is npm. Optional. Prints usage information and exits.
Generate the NetOp PM Database Performance Report

The report_npm_db_performance.sh script monitors the general performance of the NetOp PM database and advises how to resolve some database issues. Run it regularly. An Oracle DBA should analyze the output of this script. To generate a report of the NetOp PM database performance, perform the following steps: 1. Log on as oracle. 2. Open a terminal window and navigate to the NetOp PM database maintenance directory: cd /usr/local/npm/db/maintenance 3. Run the report_npm_db_performance.sh script according to the following syntax: ./report_npm_db_performance.sh [-db database_name] [-db_passwd database_account_password] [-h] Table 2-14 describes the syntax and usage guidelines for this script.
Table 2-14
Syntax -db database_name -db_passwd database_account_password -h
Syntax for the report_npm_db_performance.sh Script

Description Optional. Name of the NetOp PM database. The default value is npm. Optional. NetOp PM database account password. Optional. Prints usage information and exits.
2-38
Book-Title
DraftApril 9, 2009
Chapter 3
Troubleshoot System Errors
This chapter describes system error messages and provides solutions to the problems that caused the errors. Error messages are logged in the /var/log/netop/npm.log file on each Solaris host, unless remote logging is enabled; see the NetOp PM System Logging section on page 2-2. We recommend that you monitor this log file regularly. All references to a network access server (NAS) refer to the SmartEdge router or the SMS device used with the NetOp PM software. Note If you cannot resolve the issue by following the procedures described in this guide, contact your Redback technical support representative.
This chapter includes the following topics: NetOp PM Installation Error Messages NetOp PM System-Wide Error Messages NetOp PM Service Manager Error Messages NetOp PM Lightweight Web Portal Error Messages NetOp PM RADIUS Server Error Messages PPP Client Error Messages NetOp PM API Server Error Messages NetOp PM API Server and External RADIUS Server Error Messages NetOp PM Database Error Messages NetOp PM Script Error Messages
For instructions on modifying the error messages contained in the portal.xsl and services.xsl style sheets, see the Configure the NetOp PM Lightweight Web Portal chapter in the NetOp Policy Manager Configuration Guide. Note Although the subscriber account scripts are available with Release 2.0 and later, you can access the Subscriber Account NPM Properties panel in the NetOp client for information on subscriber accounts, such as the subscribed services, service order history, static IP addresses, and active sessions.
3-1
DraftApril 9, 2009
NetOp PM Installation Error Messages
NetOp PM Installation Error Messages

All errors that occur during installation are displayed on the screen and more data is logged and stored in a file in the /var/log/netop/install/releaseID directory; where releaseID is the release of the NetOp PM software. The log file is named component-date.log; where component is the name of the software component (for example, npm), and date is the date the installation occurred. A separate log file is created for each component you install each time you install it. MessageThe following error message may occur during the NetOp client installation:
One or more errors occurred during the copying of the files (file1). Please refer to the install log for additional information. Java error.
DescriptionAn InstallShield error occurred during the installation of the NetOp client. The error varies depending on the platform: For Solaris installations, the error is logged in the NetOp client log file in the following location: /tmp/NetOpInstall.log. For Windows installations, the error is logged in the NetOp client log file in the following location: %TEMP%; where %TEMP% is the following location by default on Windows XP machines: C:\Documents and Settings\Windows-username\Local Settings\Temp.
ActionExamine the NetOp client log file for the InstallShield error to determine the root cause of the error. For example, the following error message in the NetOp client log may indicate that you were not logged on as root when you attempted to install the NetOp client:
java.io.IOException: Could not create directory: /opt/Redback/NetOpClt
NetOp PM System-Wide Error Messages

This section describes the action to take when you encounter the following error message in the /var/log/netop/npm.log file on each Solaris host, unless remote logging is enabled; see the NetOp PM System Logging section on page 2-2. MessageIf there is a mismatch between the version of the NetOp PM application generating the message and the version of the NetOp PM database, the following error message may be logged in the npm.log file:
PLS-00201: identifier 'NPM_MAIN2_0_5.<some database stored procedure name>' must be declared
DescriptionThis error could be generated by the NetOp PM RADIUS server, the NetOp PM application programming interface (API) server, or the synch_npm_with_node.sh script. In this example, the NetOp PM application was Version 2.0.5 and it was expecting the NetOp PM database to be the same version, but the database had not yet been upgraded to the new version. ActionUpgrade the software of the NetOp PM database; see the chapter about upgrading the NetOp PM software in the NetOp Policy Manager Installation Guide.
3-2
DraftApril 9, 2009
NetOp PM Service Manager Error Messages

This section describes actions to take when you encounter the following error messages: License for NetOp PM Complex Services not Found Database Username or Password Is Incorrect Database Name Is Incorrect Database Host Is Wrong SNMP Errors When Trying to Change a Service
You can identify error messages in the npm.log file that originate from the NetOp PM service manager by the date, time, hostname, message ID, and the NetOp PM component name, SERVICE_MANAGER, such as in the following example:
Dec 6 13:57:36 <hostname>: [ID 647684 local0.error] SERVICE_MANAGER -
License for NetOp PM Complex Services not Found

MessageThe following example displays an error message that may be logged in the npm.log file when you try to use the NetOp PM complex services:
No license found for Complex Services.
DescriptionThe required license for the NetOp PM complex services feature has not been installed or an error occurred when installing it. ActionPerform one or more of the following steps: Install the license using the config_licenses.sh script; for the procedure to run it, see the Install NetOp PM Software Licenses chapter in the NetOp Policy Manager Installation Guide. To verify the installed license, run the show_licenses.sh script. To run the show_licenses.sh script, log on to any NetOp PM server host as root and run the script. If you installed the license, but you are still receiving the error message, check the /var/log/netop/npm.log file for messages that indicate license errors.
Database Username or Password Is Incorrect

MessageIf the NetOp PM service manager is unable to make a database connection, the following error message may be logged in the npm.log file:
Failed to establish a database connection. The userid or the password is incorrect.
DescriptionIn this case, the username or password provided was incorrect. ActionVerify the database username and password and reattempt to log on. In the servicemanager.cfg file, see the db_acct field for the username and the db_passwd field for the password.
3-3
DraftApril 9, 2009
Database Name Is Incorrect

Failed to get database connection. Original message: Failed to establish a database connection. The database name is incorrect or it is not running.
DescriptionIn this case, the problem may be that the database name specified was incorrect or the database may not be running. ps -ef | grep -v grep | grep ora_smon_database_name ActionCheck the spelling of the database name, verify that the database is running and reattempt to log on. To verify that the Oracle application is running, log on to the database as oracle and issue the following command: where the value of the database_name argument is the name of the database; for example, npm. For more information, see the db parameter in the servicemanager.cfg file.
Database Host Is Wrong

Failed to get database connection. Cannot establish a network connection with the database server. The listener may be down or the given ip address cannot be reached.
DescriptionIn this case, the problem may be that the listener may be down or no network connection exists. ActionPerform the following steps: 1. Verify that the listener is up and attempt to log on: a. Log on to the Oracle application as oracle and enter the following command: lsnrctl status b. If the listener is offline, start it by entering the following command: lsnrctl start 2. Verify that a network connection exists between the NetOp PM service manager and database.
SNMP Errors When Trying to Change a Service

This section provides tips on dealing with the following Simple Network Management Protocol (SNMP) errors, which may occur when you attempt to change a service using the NetOp PM service manager: Bad Value Invalid IP Address No Response Received
3-4
DraftApril 9, 2009
Resource Unavailable Unknown NAS
Bad Value
MessageThe following example displays an error message logged in the npm.log file:
Error changing user service! SNMP Error: badValue SNMP OID: 1.3.6.1.4.1.2352.2.27.1.1.3.4.0 SNMP Value: 0B00FFFF9000000E-3F468240 NAS: ser-1 Subscriber: 00:00:00:00:00:00 Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240
DescriptionThe NetOp PM service manager attempted to change services, but the session record in the NetOp PM database is not synchronized with the node because the NetOp PM software did not receive a RADIUS packet that it was supposed to receive. ActionRun the synch_npm_with_node.sh script in verify mode to check whether the NetOp PM database is synchronized with the node. If synchronization problems are found, run the synch_npm_with_node.sh script in the mode to resynchronize the sessions in the NetOp PM database with the sessions on the SmartEdge router. For syntax and usage guidelines for this script, see the Data Management chapter in the NetOp Database Administration Guide.
Invalid IP Address
Error changing user service! SNMP Error: Invalid IP: 10.192.100.355 NAS: rickys Subscriber: 00:00:00:00:00:00 Account: joe Session Id: 0106FFFF9000001D-3DE330E3 IP Address: 172.19.36.240
DescriptionThis message is generated if invalid syntax is used for the nas_ip_address field in the nas_info table. ActionCorrect the value of the nas_ip_address field; for more information, see the nas_info Table Structure section in the NetOp PM Database Schema appendix in the NetOp Database Administration Guide.
No Response Received
Error changing user service! SNMP Error: No response received SNMP OID: 1.3.6.1.4.1.2352.2.27.1.1.3.4.0 SNMP Value: 0B00FFFF9000000E-3F468240
3-5
DraftApril 9, 2009
NetOp PM Service Manager Error Messages NAS: ser-1 NAS User: 00:00:00:00:00:00 Subscriber Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240
DescriptionThe NetOp PM service manager attempted to change services, but the NetOp PM API server cannot communicate with the node. This error message occurs in three possible scenarios: Node communication is unavailable. SNMP is incorrectly configured in the NetOp PM database. SNMP is incorrectly configured on the node.
ActionPerform one or more of the following actions: Verify communication between the NetOp PM API server and the node. Check the values for the following attributes in the nas_info table: nas_ip_address, nas_identifier, and snmp_community. Check the SNMP settings.
Resource Unavailable
Error changing user service! SNMP Error: resourceUnavailable SNMP OID: 1.3.6.1.4.1.2352.2.27.1.1.3.4.0 SNMP Value: 0B00FFFF9000000E-3F468240 NAS: ser-1 NAS: 00:00:00:00:00:00 Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240 Error changing user service! SNMP Error: resourceUnavailable until: Wed Aug 27 15:14:49 PDT 2003 SNMP OID: 1.3.6.1.4.1.2352.2.27.1.1.3.9.0 SNMP Value: 1 NAS: ser-1 Subscriber: 00:00:00:00:00:00 Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240
DescriptionThe NetOp PM service manager attempted to change services while a node was undergoing a dynamic switchover between the active and standby controller cards. ActionWhen the dynamic switchover is completed, the condition corrects itself. No action is required.
3-6
DraftApril 9, 2009
NetOp PM Lightweight Web Portal Error Messages
Unknown NAS
Error changing user service! SNMP Error: Unknown NAS NAS: ser-2 Account: redback Session Id: 09000064-3F50270E IP Address: 10.192.42.3
DescriptionThe NetOp PM service manager attempted to change services for a subscriber, but the subscriber is connected to a node that has not been configured in the NetOp PM database. ActionAdd the node to the NetOp PM database by defining a record in the nas_info table. The nas-identifier attribute represents the name of the node that is requesting user authentication or providing accounting information. Note On the node, the NetOp PM software defines the name associated with this attribute by adding the following line to the SmartEdge router or SMS device configuration: system hostname hostname

NetOp PM API server error messages tell subscribers who are using the NetOp PM lightweight web portal to log on, log off, or change services. These error messages are reported by the web server but caused by the NetOp PM API server. This section describes the following NetOp PM lightweight web portal error messages: Attempt to Modify Services Before Authentication Unknown Proxy Server Invalid Network Configuration Error While Attempting to Log You In Error While Attempting to Log You Out Login Failed Incorrect Password Inactive Account Logon Invalid Location Logon Invalid Logon Reached Your Maximum Login Limit Reached Maximum Number of Logons Permitted Error While Attempting to Modify Services Incompatible Service
3-7
DraftApril 9, 2009
Must Have JavaScript Enabled to Change Services Refused Access to Unknown IP Address Select at Least One Service You Are Trying to Remove Your Access Service You Are Trying to Select More Services Than Allowed Service Offerings Specify Different Contexts Note Subscribers can access a Help page from the NetOp PM lightweight web portal by clicking the Help button in the header of the web portal. Some error messages also include a question mark (?) button. When the subscriber clicks the ? button, the Help page opens to provide more details on the error.
Error messages in the npm.log file, which originate from the web portal, are identified by the date, time, hostname, message ID, and the NetOp PM component name WEB_PORTAL as in the following example:
Dec 6 13:57:36 <hostname> WEB_PORTAL : [ID 647684 local0.error] -
Revising in all error messages to View the appropriate log files due to change in portal log location: In the /usr/local/jakarta-tomcat-4.1.27/logs directory, view the appropriate log files.
Attempt to Modify Services Before Authentication

MessageIf JavaScript is enabled in the subscribers browser, the following message may occur:
You must log on before accessing any service profiles. In 5 seconds, you will be redirected to the portal login page. If you are not automatically redirected: click here to view the portal page and log on. watch if this changes before 2.0 or else in 2.0 (EV 56091)
If JavaScript is disabled in the browser of the subscriber, the following message appears:
You must log on before accessing any service profiles. Click here to view the portal page and log on.
DescriptionThe subscriber has not yet logged on but attempted to access the web portal service change page. ActionInstruct the subscriber to log on with a username and password on the portal logon page before attempting to modify services.
Unknown Proxy Server

Admin only - displayed only in log MessageThe following example displays an error message logged in the npm.log file:
Unknown proxy server: 172.19.12.70 Refused access to unknown ip: 172.19.12.70
3-8
DraftApril 9, 2009
DescriptionThe web browser of the subscriber is configured to use a proxy server that is not configured in the NetOp PM lightweight web portal. ActionPerform one of the following actions: Configure the proxy server in the NetOp PM lightweight web portal; see the Configure the NetOp PM Lightweight Web Portal chapter in the NetOp Policy Manager Configuration Guide. Instruct the subscriber to reconfigure the web browser to use a proxy server configured in the NetOp PM lightweight web portal.
Invalid Network Configuration

MessageThe following example displays an error message that subscribers may encounter:
You have an invalid network configuration. Please contact customer service.
DescriptionThe NetOp PM API server is unable to find an IP address entry in the session_ip_address table for the PC of the subscriber. ActionSee the Refused Access to Unknown IP Address section on page 3-13 for detailed descriptions and recommended actions.
Error While Attempting to Log You In

There was an error while attempting to log you in. Please contact Customer Service.
DescriptionThe NetOp PM API server sent an SNMP message to the node in an attempt to change the subscriber from the Captive Portal service to a selected service, but the NetOp PM API server is unable to connect to the node and, therefore, cannot remove the Captive Portal service. ActionTwo possible actions exist for troubleshooting this error message: View the appropriate log files. Check the values for the following attributes in the nas_info table: nas_ip_address, nas_identifier, and snmp_community to ensure that SNMP is properly enabled on the node.
For more details on troubleshooting NetOp PM API server error messages, see the NetOp PM API Server Error Messages section on page 3-20.
Error While Attempting to Log You Out

There was an error while attempting to log you out. Please contact Customer Service.
DescriptionThe NetOp PM API server sent an SNMP message to the node in an attempt to change the subscriber from a selected service to the Captive Portal service, but the NetOp PM API server is unable to connect to the node and, therefore, cannot remove the selected service.
3-9
DraftApril 9, 2009
ActionTwo possible actions exist for troubleshooting this error message: View the appropriate log files. Check the values for the following attributes in the nas_info table: nas_ip_address, nas_identifier, and snmp_community to ensure that SNMP is properly enabled on the node.
See the NetOp PM API Server Error Messages section on page 3-20 for more details on troubleshooting NetOp PM API server error messages.
Login Failed
Logon failed. Check your username and password and try logging on again. If logon fails with the correct username and password, it is possible that you are not permitted to connect from your current location. Contact your service provider to determine whether you are permitted to connect from your current location.
DescriptionThis error message can occur in multiple situations. I changed the wording to validate and format this correctly. Can these 6 be broken out into separate Troubleshooting elements? This is the first time running into a new formatting bug.- IR Check for the following possible scenarios: The subscriber account or password is incorrect.A subscriber whose service definition includes a location lock is attempting to log on from a different location than what has been assigned. The subscriber account record has been removed from the NetOp PM database. The activated field in the subscriber_account table is set to N, meaning that web-based authentication is not allowed for the subscriber account. If RADIUS proxying is enabled, the external RADIUS server does not know about the subscriber account and password. If LDAP authentication is enabled, the external LDAP server does not know about the subscriber account and password.
ActionPerform one of the following steps: Attempt to log on again. Instruct that the subscriber go to the assigned location and try again to log on. Note Sometimes the location should be understood as being logical, that is, if the location lock is configured to match the agent ID, hostname, or some other logical attribute within the NAS-Port-Id or Calling-Station-Id attribute. The lock is not necessarily bound to a physical location.
Verify that the subscriber_account table has not been accidentally deleted. Make the subscriber account available by changing the activated field in the subscriber_account table to Y.
3-10
DraftApril 9, 2009
Update the configuration for the external RADIUS server; see the section on proxying RADIUS messages to external RADIUS servers in the Configure External RADIUS and LDAP Servers chapter in the NetOp Policy Manager Configuration Guide. Update the configuration for the external LDAP server; see the section on querying external LDAP servers in the Configure External RADIUS and LDAP Servers chapter in the NetOp Policy Manager Configuration Guide.
Incorrect Password
The password you entered was incorrect. Please check your password and try again.
DescriptionThe password that the subscriber entered while trying to modify services was incorrect. ActionInstruct the subscriber to enter the correct password to log on.
Inactive Account Logon

Contact your service provider to activate your subscriber account. When activated, attempt to connect again.
DescriptionThe subscriber attempted to log on with an inactive account. ActionInstruct the subscriber to contact the service provider to reactivate their subscriber account prior to the subscriber attempting to log in again.
Invalid Location Logon

Contact your service provider to confirm that you are permitted to connect from your correct location. If you do have permission, attempt to connect again. If you do not have permission, login is denied until you request this service.
DescriptionThe subscriber attempted to log on from an invalid location. ActionInstruct the subscriber to contact the service provider to obtain instructions to reset the location lock prior to the subscriber attempting to log in again.
Invalid Logon
Contact your service provider for your correct username and password, then attempt to connect again with your username and password.
DescriptionThe subscriber attempted to log on with an invalid username or incorrect password.
3-11
DraftApril 9, 2009
ActionInstruct the subscriber to contact the service provider to obtain the correct username and password prior to the subscriber attempting to log in again.
Reached Your Maximum Login Limit

You have reached your maximum Login limit. Please Logoff one or more sessions and try again.
DescriptionToo many active web logon sessions exist for the same subscriber account. ActionFour possible actions exist for troubleshooting this error message: Log off one or more sessions and try again. Access the Subscriber Account NPM Properties panel in the NetOp client for information on subscriber accounts; see the Manage Subscribers chapter in the NetOp Policy Manager Configuration Guide for instructions. Log off the subscriber session to redirect the subscriber to the service; see the Manage Subscribers chapter in the NetOp Policy Manager Configuration Guide for instructions. Run the script to synchronize the NetOp PM database with a node; for syntax and usage guidelines, see the Synchronize the NetOp PM Database with a Node section in the NetOp Database Administration Guide. Note Although the subscriber account scripts are available with Release 2.0 and later, you can access the Subscriber Account NPM Properties panel in the NetOp client for information on subscriber accounts, such as the subscribed services, service order history, static IP addresses, and active sessions.
Reached Maximum Number of Logons Permitted

You have reached the maximum number of logons permitted. Please log off one or more sessions before changing services.
DescriptionThe subscriber attempted to subscribe to a service that allows fewer logon sessions than the current number of active logon sessions for the subscriber. For example, the subscriber currently has two active sessions logged on and changes services to a service that allows a maximum of one logon session. ActionInstruct the subscriber to log off one or more sessions and then attempt the service change again.
Error While Attempting to Modify Services

There was an error while attempting to modify your services. Please contact Customer Service.
3-12
DraftApril 9, 2009
DescriptionThe NetOp PM API server sent an SNMP message to the node in an attempt to change the services of the subscriber, but the NetOp PM API server is unable to connect to the node and, therefore, cannot change the services. ActionTwo possible actions exist for troubleshooting this error message: View the appropriate log files. Check the values for the following attributes in the nas_info table: nas_ip_address, nas_identifier, and snmp_community to ensure that SNMP is properly enabled on the node.
For more details on troubleshooting NetOp PM API server error messages, see the NetOp PM API Server Error Messages section on page 3-20.
Incompatible Service
You are trying to add an incompatible service. Please contact Customer Service.
DescriptionThe service being added has an attribute that conflicts with an attribute of another service applied to the subscriber session. This message may appear, for example, if the subscriber attempts to subscribe to two service offerings with different contexts. ActionInstruct the subscriber not to subscribe to the selected combination of services. To avoid this condition, do not create service offerings with conflicting attributes, such as different contexts.
Must Have JavaScript Enabled to Change Services

You must have JavaScript enabled in order to change services.
DescriptionJavaScript must be enabled for the subscriber to select or deselect services on the Services page. ActionInstruct the subscriber to enable JavaScript in the browser and then attempt to modify services again. The error message in the browser of the subscriber links to instructions on enabling JavaScript in the supported browsers.
Refused Access to Unknown IP Address

Refused access to unknown ip: 172.19.39.252
The NetOp PM lightweight web portal is unable to find an IP address entry in the nas_session table for the PC of the subscriber. This error message corresponds to the Invalid Network Configuration NetOp PM lightweight web portal error message displayed to subscribers. This error message occurs in three possible scenarios. DescriptionThis error message occurs in multiple scenarios. I changed the structure and wording for this multiple troubleshooting element to validate it. Can this multiple one be broken down? Check for the following three possible scenarios:
3-13
DraftApril 9, 2009
The subscriber is attempting to log on from a PC that bypassed the node, and the NetOp PM software does not permit subscribers to bypass the node to access the portal. The carrier is attempting to use the NetOp PM system with a web proxy server and has not configured the web proxy server to allow the session record containing the IP address of the subscriber PC to be sent to the NetOp PM lightweight web portal. The web browser of the subscriber is set to a web proxy server that is configured for anonymity, which means that the IP address of the subscriber PC is not broadcast over the Internet and only the address of the web proxy server is known. Consequently, the NetOp PM system cannot find the IP address of the subscriber PC in the NetOp PM database because the address of the web proxy server is used instead.
ActionPerform one of the following steps: Check that the session record containing the IP address of the subscriber PC was not accidentally deleted from the session table in the NetOp PM database. Run the script in verification mode to check whether the NetOp PM database is synchronized with the node. If synchronization problems are found, run the script in fix mode to resynchronize the sessions in the NetOp PM database with the sessions on the SmartEdge router. For syntax and usage guidelines, see the Synchronize the NetOp PM Database with a Node section in the NetOp Database Administration Guide. Correct the web proxy server settings to allow the IP address of the subscriber PC to be sent to the NetOp PM portal; see the Initial Configuration chapter in the NetOp Policy Manager Configuration Guide. The web proxy server must send the HTTP_X_FORWARDED_FOR attribute. Either the service provider should configure the web proxy server to forward the IP address of the subscriber PC, or the subscriber should change the browser configuration to point to the version of a web proxy server of the carrier that does not use anonymity.
Select at Least One Service

You must have at least one service selected.
DescriptionThe subscriber clicked Modify on the service page, but no services were selected. ActionTo select a service on the NetOp PM lightweight web portal, the subscriber clicks a service name in the Available Services list, clicks the >> button, enters a password, and then clicks Modify. Note Subscribers can access a Help page from the NetOp PM lightweight web portal by clicking the Help button in the header of the web portal. Some error messages also include a question mark (?) button. When the subscriber clicks the ? button, the Help page opens to provide more details on the error.
You Are Trying to Remove Your Access Service

MessageThe following example displays an error message that subscribers encounter if the subscribers attempt to change the service that provides access:
You are trying to remove your access service.
3-14
DraftApril 9, 2009
DescriptionIf a subscriber removes the access service, a connection to the node can no longer be established. ActionDo not perform this service change.
You Are Trying to Select More Services Than Allowed

You are trying to select more services than allowed.
DescriptionThe subscriber is attempting to perform a service change that adds more services than the maximum number of selected services that the NetOp PM software supports. ActionDo not subscribe to more than four services for each subscriber.
Service Offerings Specify Different Contexts

MessageThe following example displays an error message that an operator may encounter in the npm.log file:
Mar 31 10:30:01 npmserver11.lab.van.redback.com WEB_PORTAL[7393]: [ID 800047 local0.error] SOAP 'changeAndApplyServicesToSubSession' request Failed Mar 31 10:30:01 npmserver11.lab.van.redback.com Request: changeAndApplyServicesToSubSession Mar 31 10:30:01 npmserver11.lab.van.redback.com URL: http://:@localhost:8080/NPM_API-2.0.2.0.0330/service Mar 31 10:30:01 npmserver11.lab.van.redback.com Client IP: 10.192.45.55 Mar 31 10:30:01 npmserver11.lab.van.redback.com Parameters: a:2:{s:8:"String_1";s:12:"10.192.45.55";s:15:"arrayOfString_2";a:3:{i:0;s:7:"Basic";i: 1;s:23:"TurboBoost";}} Mar 31 10:30:01 npmserver11.lab.van.redback.com Fault Code: env:Server Mar 31 10:30:01 npmserver11.lab.van.redback.com Mar 31 10:30:01 npmserver11.lab.van.redback.com WEB_PORTAL[7393]: [ID 800047 local0.error] Fault String: com.redback.npm.common.IncompatibleServiceException Mar 31 10:30:01 npmserver11.lab.van.redback.com Fault Detail: a:1:{s:28:"IncompatibleServiceException";a:1:{s:7:"message";s:110:"The following service offerings specify different contexts: Basic, TurboBoost";}}
DescriptionThe Turbo Boost service is an add-on service for the Basic access service; therefore, it must be in the same context. ActionThe subscriber should not subscribe to the selected combination of services. To avoid this condition, do not create service offerings with conflicting attributes, such as different contexts.
3-15
DraftApril 9, 2009
NetOp PM RADIUS Server Error Messages

This section describes the following error messages: Conflicting Flow-Through Attributes Encountered Conflicting Service Attributes Encountered License for NetOp PM EAP Authentication Feature not Found License for NetOp PM Third-Party Vendor Support not Found Location Check Failed Maximum Subscriber Sessions Exceeded Request from Unknown Client
Error messages in the npm.log file that originate from the RADIUS server are identified by the date, time, hostname, message ID, and the NetOp PM component name RADIUS_SERVER as in the following example:
Dec 6 13:57:36 <hostname> RADIUS_SERVER : [ID 647684 local0.error] -
Conflicting Flow-Through Attributes Encountered

MessageThe following example displays a RADIUS error message:
Conflicting flow-through attributes encountered. NAS: $nas_identifier Subscriber: $subscriber_account with services: @services ignoring $attribute_name: $attribute_value
Where the value of the nas_identifier argument is the name of the node, the value of the subscriber_account argument is the subscriber account name, the value of the services argument is a list of services to which the subscriber is subscribed, the value of the attribute_name argument is the name of the flow-through attribute with a conflict, and the value of the attribute_value argument is the value of the flow-through attribute with a conflict. DescriptionThe subscriber does not receive the expected service because the external RADIUS server attempted to specify a flow-through attribute that conflicts with the current services of the subscriber in the service_subscription table of the NetOp PM database. ActionEnsure that the service offerings do not conflict with any attributes returned by the external RADIUS server.
Conflicting Service Attributes Encountered

Conflicting service attributes encountered! NAS: nas_identifier Subscriber: account_name with services: services ignoring attribute_name: attribute_value
3-16
DraftApril 9, 2009
where the value of the nas_identifier argument is the node ID, the value of the account_name argument is the subscriber name, the value of the services argument is the id field in the service_offering table, the value of the attribute_name argument is a field in the service_radius table that represents a standard or Redback VSA, and the value of the attribute_value argument is the RADIUS attribute value. DescriptionThe subscriber does not receive the expected service because two service offerings share the same service attributes. The NetOp PM system does not permit the same attribute in more than one service offering; therefore, if the priority for both service IDs is the same, the system chooses one value for both services and ignores the other value. ActionIssue the show subscribers command (in any mode) on the nas_identifier node, with the optional active sub-name construct; where the value of the sub-name argument is either the PPP username or the media access control (MAC) address. Issuing this command displays a list of active subscribers and the details on the specified subscriber. For detailed syntax and usage guidelines, see the AOS Command Reference publication or the Basic System Operations Guide for the SmartEdge OS. If you find that not all the attributes are applied, check the npm.log file to see if it contains this error message.
License for NetOp PM EAP Authentication Feature not Found

MessageThe following example displays an error message that may be logged in the npm.log file when you try to use EAP authentication:
No license found for the NetOp PM EAP Authentication feature.
DescriptionThe required license for the NetOp PM EAP Authentication feature has not been installed or an error occurred when installing it. ActionPerform one or more of the following steps: Install the license using the config_licenses.sh script; for the procedure to run it, see the Install NetOp PM Software Licenses chapter in the NetOp Policy Manager Installation Guide. To verify the installed license, run the show_licenses.sh script. To run the show_licenses.sh script, log on to any NetOp PM server host as root and run the script. If you installed the license, but you are still receiving the error message, check the /var/log/netop/npm.log file for messages that indicate license errors.
License for NetOp PM Third-Party Vendor Support not Found

MessageThe following example displays an error message that may be logged in the npm.log file when you try to use multivendor RADIUS VSAs:
No license found for the NetOp PM Third-party Vendor Support.
DescriptionThe required license for the NetOp PM multivendor VSA support feature has not been installed or an error occurred when installing it. ActionPerform one or more of the following steps: Install the license using the config_licenses.sh script; for the procedure to run it, see the Install NetOp PM Software Licenses chapter in the NetOp Policy Manager Installation Guide. To verify the installed license, run the show_licenses.sh script. To run the show_licenses.sh script, log on to any NetOp PM server host as root and run the script.
3-17
DraftApril 9, 2009
PPP Client Error Messages
If you installed the license, but you are still receiving the error message, check the /var/log/netop/npm.log file for messages that indicate license errors.
Location Check Failed

MessageThe following error message appears if a location check fails[FLH-Added Leadin to make valid]:
Location check did not pass. Location Lock: location_lock, NAS Port ID: nas_port_id, Calling Station ID: calling_station_id
DescriptionThe subscriber attempted to log on from a location at which no permission access exists. ActionConfirm that the location lock of the subscriber is correct. If the location lock is correct, instruct the subscriber to stop attempting to come in from an invalid location.
Maximum Subscriber Sessions Exceeded

NAS: $nas_identifier, Subscriber: $nas_user_name, Session Count: $session_count, Session Limit: $session_limit
DescriptionThis message appears if a subscriber exceeds the limit for the MaxLogins attribute and an Access-Reject packet is sent to the node. ActionThe subscriber is notified if the maximum is reached and must log off an active PPP session.
Request from Unknown Client

Date Time: NOTICE: Request from unknown client IP_address: ignored
DescriptionThe NetOp PM RADIUS server cannot communicate with nodes that do not exist in the nas_info table. ActionConfirm all RADIUS clients (SmartEdge routers or SMS devices) are in the nas_info table.

PPP client error messages appear in the PPP client of the subscriber. This section describes the following error messages: PPP Session Limit Exceeded Location not Allowed Authentication Failed Subscriber Account Is Inactive
3-18
DraftApril 9, 2009
PPP Session Limit Exceeded

PPP session limit exceeded.
DescriptionThe NetOp PM system restricts the number of PPP sessions allowed for a single PPP username with the MaxLogins attribute specified by the service the subscriber selects. If a subscriber attempts to start another PPP client using the same username and password and has exceeded the MaxLogins value, the connection from this PPP client is rejected. (circuit is blocked) ActionShut down a PPP client. Run the synch_npm_with_node.sh script in verification mode to check whether the NetOp PM database is synchronized with the node. If synchronization problems are found, run the synch_npm_with_node.sh script in fix mode to resynchronize the sessions in the NetOp PM database with the sessions on the SmartEdge router. For syntax and usage guidelines, see the Synchronize the NetOp PM Database with a Node section in the NetOp Database Administration Guide. Use the delete_nas_sessions.sh script if no PPP clients are running; for instructions, see the Remove Sessions from the NetOp PM Database section in the NetOp Database Administration Guide.
Location not Allowed

MessageThe following example displays an error message that subscribers can encounter:
Request denied: Location not allowed.
DescriptionThe subscriber is attempting to log on from a location where there is no permission access. ActionInstruct the subscriber to contact the service provider to obtain instructions to reset the location lock and then restart the Point-to-Point Protocol over Ethernet (PPPoE) client with the valid PPP username and password to log on.
Authentication Failed
Request denied: Authentication failed.
DescriptionThe username and password combination is not valid. ActionInstruct the subscriber to contact the service provider to obtain the correct username and password, and then restart the Point-to-Point Protocol over Ethernet (PPPoE) client with the valid PPP username and password to log on.
Subscriber Account Is Inactive

Request denied: Subscriber account is inactive.
DescriptionThe subscriber account is inactive. ActionInstruct the subscriber to contact the service provider to reactivate the subscribers account, and then restart the Point-to-Point Protocol over Ethernet (PPPoE) client with the valid PPP username and password to log on.
3-19
DraftApril 9, 2009
NetOp PM API Server Error Messages

This section provides information on dealing with the following NetOp PM API server messages caused by SNMP errors: License for Admission Control Function Feature not Found Bad Value Busy NetOp PM API Server Invalid IP Address No Access No Response Received Invalid Credentials Resource Unavailable Unable to Retrieve Database Connection Unknown NAS
Error messages in the npm.log file that originate from the NetOp PM API server are identified by the date, time, hostname, message ID, and the NetOp PM component name API_SERVER such as in the following example:
Dec 6 13:57:36 <hostname>: [ID 647684 local0.error] API_SERVER -
License for Admission Control Function Feature not Found

Note The following error message does not indicate that it originated from the NetOp PM API server; however, it is associated with it.
MessageThe following example displays an error message that may be logged in the npm.log file when you try to use the NetOp PM admission control function (ACF) feature:
No license found for Admission Control Function.
DescriptionThe required license for the NetOp PM ACF feature has not been installed or an error occurred when installing it. ActionPerform one or more of the following steps: Install the license using the config_licenses.sh script; for the procedure to run it, see the Install NetOp PM Software Licenses chapter in the NetOp Policy Manager Installation Guide. To verify the installed license, run the show_licenses.sh script. To run the show_licenses.sh script, log on to any NetOp PM server host as root and run the script. If you installed the license, but you are still receiving the error message, check the /var/log/netop/npm.log file for messages that indicate license errors.
3-20
DraftApril 9, 2009
Bad Value
Error changing user service! SNMP Error: badValue SNMP OID: 1.3.6.1.4.1.2352.2.27.1.1.3.4.0 SNMP Value: 0B00FFFF9000000E-3F468240 NAS: ser-1 Subscriber: 00:00:00:00:00:00 Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240
DescriptionThe subscriber attempted to log on, log off, or change services, but the session record in the NetOp PM database is not synchronized with the node because the NetOp PM software did not receive a RADIUS packet that it was supposed to receive. ActionRun the synch_npm_with_node.sh script in verification mode to check whether the NetOp PM database is synchronized with the node. If synchronization problems are found, run the synch_npm_with_node.sh script in fix mode to resynchronize the sessions in the NetOp PM database with the sessions on the SmartEdge router. For syntax and usage guidelines, see the Synchronize the NetOp PM Database with a Node section in the NetOp Database Administration Guide.
Busy NetOp PM API Server

MessageThe following error message appears if the NetOp PM API server is busy[FLH-Added Leadin to make valid]:
The NetOp PM API server is busy. As a result, the NetOp PM system failed to proxy the Access-Request message for realm realm after n attempts because it cannot successfully proxy to external servers using the round-robin algorithm.
DescriptionThe NetOp PM API server could not find a round-robin server to which to proxy the Access-Request packet. ActionWait for a while and then try again.
Invalid IP Address
Error trying to login a user! SNMP Error: Invalid IP: 10.192.100.355 NAS: rickys Subscriber: 00:00:00:00:00:00 Account: joe Session Id: 0106FFFF9000001D-3DE330E3 IP Address: 172.19.36.240
DescriptionThis message is generated if invalid syntax is used for the nas_ip_address field in the nas_info table.
3-21
DraftApril 9, 2009
ActionCorrect the nas_ip_address field; for more information, see the nas_info Table Structure section in the NetOp PM Database Schema appendix in the NetOp Database Administration Guide.
No Access
Apr 7 14:41:41 db-dev4 : [ID 312646 local0.error] API_SERVER - Error changing user service! SNMP Error: noAccess Apr 7 14:41:41 db-dev4 SNMP OID: 1.3.6.1.4.1.2352.2.27.1.1.3.8.0 Apr 7 14:41:41 db-dev4 SNMP Value: 0001FFFF70000006-424A559C Apr 7 14:41:41 db-dev4 NAS: blackhawk Apr 7 14:41:41 db-dev4 NAS User: joe Apr 7 14:41:41 db-dev4 Subscriber Account: joe Apr 7 14:41:41 db-dev4 Session Id: 0001FFFF70000006-424A559C Apr 7 14:41:41 db-dev4 IP Address: 10.192.45.6 Apr 7 14:41:41 db-dev4 Context Name: BASIC Apr 7 14:41:41 db-dev4 MAC Address: 00-d0-b7-bd-2c-74 Apr 7 14:41:41 db-dev4 : [ID 420605 local0.error] API_SERVER - Error changing and applying services for ip address: 10.192.45.6
DescriptionWhen a subscriber unsuccessfully attempts to change service, this message appears in the npm.log file. The error occurs when the node contains missing SNMP configuration items. ActionVerify that all required SNMP settings are included in the configuration file of the node.
No Response Received
Error changing user service! SNMP Error: No response received SNMP OID: 1.3.6.1.4.1.2352.2.27.1.1.3.4.0 SNMP Value: 0B00FFFF9000000E-3F468240 NAS: ser-1 NAS User: 00:00:00:00:00:00 Subscriber Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240
DescriptionThe subscriber attempted to log on, log off, or change services, but the NetOp PM API server cannot communicate with the node. This error message occurs in three possible scenarios: Node communication is unavailable. SNMP is incorrectly configured in the NetOp PM database. SNMP is incorrectly configured on the node.
ActionPerform one or more of the following actions: Verify communication between the NetOp PM API server and the node.
3-22
DraftApril 9, 2009
Check the values for the following attributes in the nas_info table: nas_ip_address, nas_identifier, and snmp_community. Verify that all required SNMP settings are included in the configuration file of the node.
Invalid Credentials
[ID 996943 local0.error] API_SERVER - Error: Invalid Credentials LDAP Host: server_ip_address DN: username_attr=subscriber_account,base_dn
DescriptionThe subscriber attempted to log on with an incorrect password when the NetOp PM system was configured to query an external LDAP server. ActionThe username_attr and base_dn fields in the ldap_proxy_server table of the NetOp PM database define the names of the fields expected in the external LDAP server. Ensure that the values in the username_attr and base_dn fields of the NetOp PM database reference fields that exist in the database of the external LDAP server.
Resource Unavailable
MessageThe following examples display error messages logged in the npm.log file:
Error trying to logout a user! SNMP Error: resourceUnavailable SNMP OID: 1.3.6.1.4.1.2352.2.27.1.1.3.4.0 SNMP Value: 0B00FFFF9000000E-3F468240 NAS: ser-1 NAS: 00:00:00:00:00:00 Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240 Error trying to logout a user! SNMP Error: resourceUnavailable until: Wed Aug 27 15:14:49 PDT 2003 SNMP OID: 1.3.6.1.4.1.2352.2.27.1.1.3.9.0 SNMP Value: 1 NAS: ser-1 Subscriber: 00:00:00:00:00:00 Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240
DescriptionThe subscriber attempted to log on, log off, or change services while a node was undergoing a dynamic switchover between the active and standby controller cards. ActionWhen the dynamic switchover is completed, the condition corrects itself. No action is required.
3-23
DraftApril 9, 2009
NetOp PM API Server and External RADIUS Server Error Messages
Unable to Retrieve Database Connection

ERROR com.redback.npm.api.APIImpl getStatement.90 - Unable to retrieve database connection java.sql.SQLException: Cannot load JDBC driver class 'null'
DescriptionThis message is generated if an incorrect URL is used for the NetOp PM API server. ActionCheck the URL of the NetOp PM API server, correct the URL, and run the operation again.
Unknown NAS
Error trying to login a user! SNMP Error: Unknown NAS NAS: ser-2 Account: redback Session Id: 09000064-3F50270E IP Address: 10.192.42.3
DescriptionThe subscriber attempted to log on, log off, or change services, but the PC of the subscriber is connected to a node that has not been configured in the NetOp PM database. ActionAdd the node to the NetOp PM database by defining a record in the nas_info table. The nas_identifier attribute represents the name of the node that is requesting user authentication or providing accounting information. Note On the node, the NetOp PM software defines the name associated with this attribute by adding the following line to the SmartEdge router or SMS device configuration: system hostname hostname

These NetOp PM API server error messages occur only if authentication is run on a RADIUS server external to the NetOp PM software. This section describes the following error messages: Failure Proxying Authentication Request to All RADIUS Servers No Response Received from External RADIUS Server Unable to Send Access-Request Packet Unknown Host
Error messages in the npm.log file that originate from a NetOp PM API server or a RADIUS server are identified by the date, time, hostname, message ID, and the NetOp PM component name, as in the following examples:
3-24
DraftApril 9, 2009
NetOp PM API Server and External RADIUS Server Error Messages Dec Dec 6 13:57:36 <hostname>: [ID 647684 local0.error] API_SERVER -(message) 6 13:57:36 <hostname> RADIUS_SERVER : [ID 647684 local0.error] -(message)
Failure Proxying Authentication Request to All RADIUS Servers

com.redback.npm.portal.VerifyException: Failure proxying authentication request to all Radius servers!
DescriptionThe subscriber attempted to log on or change services, but no RADIUS packet was received from or sent to any of the configured RADIUS servers. ActionPerform the actions specified in the No Response Received from External RADIUS Server section or the Unable to Send Access-Request Packet section on page 3-26.
No Response Received from External RADIUS Server

Error proxying authentication for user account: redback Error: No response received from: 10.192.13.253 com.theorem.radius3.ClientReceiveException: Packet Receive Failed Receive timed out 2003-08-26 10:04:46 Error changing user service! NAS: ser-1 Subscriber: 00:00:00:00:00:00 Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240 com.redback.npm.portal.VerifyException: Failure proxying authentication request to all Radius servers!
DescriptionThe subscriber attempted to log on or change services, but no RADIUS Access-Response packet was received from the RADIUS server. This error occurs only if authentication is run on a RADIUS server external to the NetOp PM software. ActionFive possible actions exist for troubleshooting this error message: Verify that the hostname or IP address of the external RADIUS server is correct in the radius_proxy_server table of the NetOp PM database. Verify that the external RADIUS server is running at the IP address specified. Verify that no communication problems exist between the external RADIUS server and the NetOp PM API server using the ping or traceroute command. Confirm that the external RADIUS server is sending RADIUS Access-Response packets. If the external RADIUS server or network is busy, increase the timeout period by changing the value for the retry_timeout attribute in the radius_proxy_server table of the NetOp PM database.
3-25
DraftApril 9, 2009
Note
The NetOp PM API server attempts to receive RADIUS packets from the external RADIUS server three times and logs an error for each falied attempt. After three failed attempts, if multiple external RADIUS servers are defined, the NetOp PM API server attempts to contact the other external RADIUS servers. The appearance of this error message in the log file does therefore not necessarily impact subscriber operations. When the NetOp PM API server has exhausted all attempts on all configured RADIUS servers, the Failure proxying authentication request to all RADIUS servers message appears in the log file.
Unable to Send Access-Request Packet

Error proxying authentication for user account: redback Error: Unable to send request to: 10.192.13.253 2003-08-26 10:04:46 Error trying to login a user! NAS: ser-1 Subscriber: 00:00:00:00:00:00 Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240
DescriptionThe subscriber attempted to log on or change services, but the NetOp PM lightweight web portal or NetOp PM API server cannot send a RADIUS Access-Request packet to any of the configured RADIUS servers. This error occurs only if authentication is run on a RADIUS server external to the NetOp PM software. ActionFour possible actions exist for troubleshooting this error message: Verify that the hostname or IP address of the external RADIUS server is correct in the radius_proxy_server table of the NetOp PM database. Verify that the external RADIUS server is running at the IP address specified. Verify that no communication problems exist between the external RADIUS server and the NetOp PM API server using the ping or traceroute command. Confirm that the external RADIUS server is receiving RADIUS Access-Request packets. Note The NetOp PM API server attempts to receive RADIUS packets from the external RADIUS server three times and logs an error for each falied attempt. After three failed attempts, if multiple external RADIUS servers are defined, the NetOp PM API server attempts to contact the other external RADIUS servers. The appearance of this error message in the log file does therefore not necessarily impact subscriber operations. When the NetOp PM API server has exhausted all attempts on all configured RADIUS servers, the Failure proxying authentication request to all RADIUS servers message appears in the log file.
Unknown Host
Error proxying authentication for user account: redback Error: Unknown host: host-1
3-26
DraftApril 9, 2009
NetOp PM Database Error Messages 2003-08-26 10:00:27 Error trying to login a user! NAS: ser-1 Subscriber: 00:00:00:00:00:00 Account: redback Session Id: 0B00FFFF9000000E-3F468240 IP Address: 172.19.36.240
DescriptionThe subscriber attempted to log on or change services, but an invalid external RADIUS server host is configured. This error occurs only if authentication is run on a RADIUS server external to the NetOp PM software. ActionUpdate the NetOp PM configuration in the radius_proxy_server table with the correct hostname or check the Domain Name System (DNS) settings to confirm that the hostname can be translated to an IP address. Note If multiple RADIUS servers are defined, the NetOp PM API server attempts to contact the other defined RADIUS servers. This error condition, therefore, does not necessarily impact subscriber operations.
NetOp PM Database Error Messages

This section provides information on handling the following NetOp PM database errors: License for NetOp PM Manual Failover Database Feature not Found Connection Errors End of File on Communication Channel Error Integrity Constraint Violated Invalid Changes to the Database Switchover Latent Older Backup Is Already Running
License for NetOp PM Manual Failover Database Feature not Found

MessageThe following example displays an error message that you may encounter when you try to set up or run a NetOp PM disaster recovery database using a manual failover database configuration:
No license found for NetOp PM Oracle Standby Database feature. Verify that the required license has been configured.
DescriptionThe required license for the NetOp PM manual failover database feature (known in previous releases as the warm standby feature) was not installed or an error occurred during installation. ActionPerform one or more of the following steps: Install the license using the config_licenses.sh script; for the procedure to run it, see the Install NetOp PM Software Licenses chapter in the NetOp Policy Manager Installation Guide.
3-27
DraftApril 9, 2009
To verify the installed license, run the show_licenses.sh script. To run the show_licenses.sh script, log on to any NetOp PM server host as root and run the script. If you installed the license, but you are still receiving the error message, check the /var/log/netop/npm.log file for messages that indicate license errors.
Connection Errors
MessageThe following example displays an error message that may be logged in the npm.log file when subscribers unsuccessfully attempt to connect to the NetOp PM lightweight web portal:
Could not connect to database npm! ORA-00257: archiver error. Connect internal only, until freed.
DescriptionThe NetOp PM database host has no remaining free disk space. This situation is likely because the database was not being backed up and the archive logs filled all available space. ActionSchedule regular backups, which preserve the database in case of failure, and delete obsolete archive log files. In addition, to save disk space on the database host, configure backup sets to be saved to tape or other space off the database host. For more information, see the Back Up the NetOp PM Database section in the NetOp Database Administration Guide.
End of File on Communication Channel Error

MessageThe following example displays an error message generated by the verify_npm_standby.sh script:
ORA-03113: end-of-file on communication channel
DescriptionThis error can occur if the primary database loses its connection to the standby database when the script is running. ActionPerform the following steps: 1. Log on to the NetOp PM database as the Oracle system administrator (for example, npmadmin) using the following command: sqlplus npmadmin/npmadmin_password@DATABASE_NAME' where the value of the npmadmin_password argument is the password of the npmadmin database account (the default value is redback), and the value of the DATABASE_NAME argument is the database name (the default value is npm). 2. Enter the following command: alter system switch logfile; 3. Wait for ten minutes and run the script again.
3-28
DraftApril 9, 2009
Integrity Constraint Violated

MessageThe following error message appears if an integrity constraint is violated:
ORA-20000: NPM error - from procedure:add_or_update_nas_user During Oracle message:ORA-02291: integrity constraint (NPMADMIN.NAS_USR_NAS_INFO_NAS_ID_FK) violated - parent key not found.
DescriptionThis error occurs if a mismatch occurs between the hostname configured on the node and the node name that is specified in the nas_identifier field of the nas_info table in the NetOp PM database. The node name must match exactly; it is case sensitive. ActionEnsure that the value in the nas_identifier field of the nas_info table in the NetOp PM database matches exactly the hostname configured on the node.
Invalid Changes to the Database

The following error messages are generated by the NetOp PM database when you attempt to make an invalid change to the data in the database: MessageCannot remove the default realm. DescriptionYou have attempted to delete the default realm, which cannot be deleted. Messageproxy_srvc_chng_access_request cannot be true when proxy_login_access_request is false. DescriptionYou are attempting to change a service without being logged on. MessageDefault realm must exist in radius_proxy_server before enabling RADIUS proxy. DescriptionYou must add an entry for the associated realm to the radius_proxy_server table before enabling a RADIUS proxy server. MessageDefault realm must exist in ldap_proxy_server before enabling LDAP proxy. DescriptionYou must add an entry for the associated realm to the ldap_proxy_server table before enabling an LDAP proxy server. MessageSequence number for realm realm must start with one. DescriptionThe sequence numbers for realms must start with one and be contiguous. MessageSequence number must be next-sequence-number for realm realm. DescriptionThe sequence numbers for realms must start with one and be contiguous. MessageCannot change existing sequence number. DescriptionIf you change a sequence number, it (and those following it) is (or becomes) out of sequence. MessageYou can only delete record with sequence number max-sequence-number for the realm. DescriptionIf you delete a number in the sequence, the numbers following the changed one is (or becomes) out of sequence. For information on field descriptions and database constraints, see Appendix A, NetOp PM Database Schema in the NetOp Database Administration Guide.
3-29
DraftApril 9, 2009
Error messages, such as the following, are generated by the NetOp PM database if you attempt to insert an invalid value into a field: MessageDelete on accounting off must be Y or N. DescriptionYou are attempting to insert a value other than Y or N in the field; the field is case-sensitive. MessageAccess request proxy type must be RADIUS or LDAP. DescriptionYou are attempting to insert a value other than RADIUS or LDAP in the field. MessageProxy algorithm must be first or round robin. DescriptionYou are attempting to insert a value other than first or round-robin in the field. MessageRecurrence must be Once, Daily, Weekly or Monthly. DescriptionYou are attempting to insert an unsupported value in the field. For information on field descriptions and database constraints, see Appendix A, NetOp PM Database Schema in the NetOp Database Administration Guide.
Switchover Latent
MessageThe following message identifies a switchover latent error[FLH-Added Leadin to make valid]:
SWITCHOVER LATENT
DescriptionThis message occurs if the primary NetOp PM database cannot communicate with the standby NetOp PM database to tell it to become a primary database. As a result, the primary database switches back to become a primary database. ActionVerify the network configuration by running the verify_npm_standby.sh script. The most likely cause of the error message is if the primary NetOp PM database cannot communicate with the standby NetOp PM database. When you run the verify_npm_standby.sh script, the script either resets the status of the NetOp PM database hosts or confirms that the primary database cannot communicate with the standby database. After the problem is fixed, try running the primary_to_standby.sh and standby_to_primary.sh scripts again.
Older Backup Is Already Running

MessageThe following error message appears if two backup processes are running simultaneously[FLH-Added Leadin to make valid]:
There is a RMAN process running already. Is older backup still running? Aborting backup.
DescriptionThis message occurs if two backup processes overlap; for example, if another backup is running when you start a new backup process. Investigate the error carefully to determine the root cause of the error before starting another backup job. ActionVerify that the old process is still active by looking at the CPU load level and I/O rates. If the old process is active, wait for the process to finish. If the old process is not active, contact your local Redback technical support representative. If the old backup takes too long to run, the cause should be investigated. Contact your local Redback technical support representative.
3-30
DraftApril 9, 2009
NetOp PM Script Error Messages

MessageThe following error message may appear when you run NetOp PM scripts:
Account USER_ID does not have root permission.
DescriptionThis message occurs when you run a script that does not have root permission. ActionTo run this script, log on to the workstation as root.
3-31
DraftApril 9, 2009
3-32
Appendix A
RADIUS Attribute 49 Termination Error Codes

This appendix describes the RADIUS attribute 49 (Acct-Terminate-Cause) error codes that are included in Accounting-Stop messages; Table A-1 describes termination error codes 0 to 209.
Table A-1 RADIUS Attribute 49 Termination Error Codes 0 to 209
Attribute 49 Error Code and Code Description 18 9 9 9 0 5 15 9 9 17 15 15 15 15 15 9 9 0 6 6 6 Host_Request NAS_Error NAS_Error NAS_Error INVALID Session_Timeout Service_Unavailable NAS_Error NAS_Error User_Error Service_Unavailable Service_Unavailable Service_Unavailable Service_Unavailable Service_Unavailable NAS_Error NAS_Error INVALID Admin_Reset Admin_Reset Admin_Reset Session cleared by administrator Circuit cleared by administrator Port shutdown by administrator Session Timeout Couldn't establish a session within session timeout period Received packet with bad session ID Remote peer sent general/unknown error for this session Authentication Failure Failed to bind subscriber Subscriber provisioning failed Cannot find AAA SESSION Clearing stale AAA SESSION Clearing unstable session due to XC switchover State Machine Timeout State Machine Error Session Error Message No error was recorded No error was recorded No termination cause code was given by peer Vendor Specific Error
Redback Terminate Error Code and Code Description 0 1 2 3 419 20 21 22 23 24 25 26 27 28 29 30 31 3239 40 41 42 No Error Unknown Error Error Not Specified Vendor Specific Error Not used Session Timeout Setup_Timeout Bad_Session_ID Unknown_Remote_Session_Error Authentication_Failed Bind_Failed Provision_Failed No_Session Stale_Session Aging_Session FSM_Timeout FSM_Error Not used Session_Cleared CCT_Cleared Port_Admin_Down
A-1
Table A-1
RADIUS Attribute 49 Termination Error Codes 0 to 209 (continued)

Attribute 49 Error Code and Code Description 6 6 6 Admin_Reset Admin_Reset Admin_Reset Admin_Test Admin_Test 0 6 6 6 3 3 3 3 6 6 0 2 9 8 9 9 2 2 2 2 0 17 17 17 17 17 INVALID Admin_Reset Admin_Reset Admin_Reset Lost_Service Lost_Service Lost_Service Lost_Service Admin_Reset Admin_Reset INVALID Lost_Carrier NAS_Error Port_Error NAS_Error NAS_Error Lost_Carrier Lost_Carrier Lost_Carrier Lost_Carrier INVALID User_Error User_Error User_Error User_Error User_Error Bad peer configuration, negotiation failed Peer rejected required option(s) Peer refused to negotiate IPCP Peer required outbound authentication Peer refused to negotiate an authentication protocol Lost carrier Busy signal, try again later Incorrect or missing framing No dialtone detected No carrier detected LMI declared PVC down or LMI control channel lost Port down HDLC down on circuit Circuit down A 'no bind' was executed on this circuit by the administrator Circuit removed from configuration Circuit reset because of encapsulation change Context removed from configuration Interface removed from configuration Interface configuration changed Dynamic shaping profile was deleted Shutdown configured on port. Shutdown configured on circuit. Session Error Message Port removed from configuration Circuit disabled by administrator Peer disabled by administrator Radius test by administrator Auto radius connectivity test
Redback Terminate Error Code and Code Description 43 44 45 46 47 4859 60 61 62 63 64 65 66 67 68 6999 100 101 102 103 104 105 106 107 108 109129 130 131 132 133 134 Port_Admin_Deleted CCT_Admin_Down Peer_Admin_Down Admin_RAD_Test Admin_RAD_Test Not used CCT_Unbound CCT_Deleted Encaps_Changed Context_Deleted Intf_Deleted Intf_Changed Profile_Deleted Port_Shutdown CCT_Shutdown Not used Lost_Carrier EC_Busy Bad_Framing No_Dialtone No_Carrier LMI_Down Port_Down HDLC_Down CCT_Down Not used Bad_Peer_Config Confrej_By_Peer Rej_IPCP Peer_Req_Auth No_Auth_Protocol
A-2
Table A-1

Attribute 49 Error Code and Code Description 17 17 17 2 2 1 2 4 5 9 1 6 9 17 1 9 10 1 0 15 15 15 6 3 3 15 9 9 User_Error User_Error User_Error Lost_Carrier Lost_Carrier User_Request Lost_Carrier Idle_Timeout Session_Timeout NAS_Error User_Request Admin_Reset NAS_Error User_Error User_Request NAS_Error NAS_Request User_Request INVALID Service_Unavailable Service_Unavailable Service_Unavailable Admin_Reset Lost_Service Lost_Service Service_Unavailable NAS_Error NAS_Error PPPoEd initiated cleanup, stuck pppoe session PPPoEd initiated cleanup, no PPP magic received PPPoEd Initiated cleanup, no subscriber binding received Tunnel was cleared Received a StopCCN from peer Control channel timeout - Remote peer dead Control packet received but no control channel exists Length field did not match packet size or packet size invalid A header field had an invalid value Session Error Message Peer refused to negotiate a callback LCP state machine timeout Authentication state machine timeout No LCP packets received from peer Link loopback detected Received PPP Terminate Request No response to PPP keepalive from peer No traffic within idle timeout period Session absolute timeout expired A PPP layer went down (LCP/IPCP/CHAP/PAP) Received PPPoE Active-Discovery Terminate from client The circuit transporting the PPPoE session was unbound No IP address was configured or pool was out of usable addresses No response to PPP Confreq from peer Peer rejected encryption, which is required by local policy Cannot find MP bundle Traffic limit exceeded Received LCP confreq from remote peer after LCP was up
Redback Terminate Error Code and Code Description 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153156 157 158 159 160 161 162 163 164 165 LCP_Rej_Callback LCP_Fsm_Timeout Auth_Fsm_Timeout No_LCP_Packets LCP_Looped Recv_Term_Req Echo_Timeout Idle_Timeout ABS_Timeout Layer_Down Recv_PADT PPPOE_Real_CCT_Unbound IPCP_No_Addr No_Confreq_Resp Rej_Crypto No_MP_Bundle Traffic_Limit_Exceeded Peer_LCP_Restart Not used PPP_PPPoE_Sync_Timeout PPPoE_Magic_Timeout PPPoE_Bind_Timeout Tun_Cleared Recv_Stopccn Rexmit_Timeout No_Ctrl_Conn Bad_Len Bad_Field
A-3
Table A-1

Attribute 49 Error Code and Code Description 15 15 10 9 9 0 15 15 15 15 9 0 10 0 1 5 15 6 0 6 0 Service_Unavailable Service_Unavailable NAS_Request NAS_Error NAS_Error INVALID Service_Unavailable Service_Unavailable Service_Unavailable Service_Unavailable NAS_Error INVALID NAS_Request INVALID User_Request Session_Timeout Service_Unavailable Admin_Reset INVALID Admin_Reset INVALID CCOD Idle down timer expired for circuit Client released DHCP lease DHCP lease expired DHCP server is unavailable DHCP IP-host cleared CLIPS circuit was reset due to a 'clips-bounce' request Reached configured max-tunnels limit Reached configured max-sessions limit Can't create session, Configured for LAC-only Can't create LNS or LTS session: no available card Wrong remote or local address received from peer Session Error Message Temporarily out of resource. Try later Remote peer permanently lacks sufficient resources for session Remote peer was too busy to accept session. Try another peer Unknown Mandatory AVP Destination Invalid
Redback Terminate Error Code and Code Description 166 167 168 169 170 171 172 173 174 175 176 177179 180 181189 190 191 192 193 194199 200 201209 Temp_Res_Fail Perm_Res_Fail Try_Another Unk_M_Avp Bad_Dest Not Used Max_Tunnels Max_Sessions Tunnel_Not_LNS No_Avail_Card Wrong_Endpoints Not used Clips_Bounce Not used DHCP_Lease_Released DHCP_Lease_Expired DHCP_Server_Unavailable DHCP_IPhost_Cleared Not used CCOD_Idle_Down Not used
A-4

NPM Guide

Uploaded by

Copyright:

Available Formats

NPM Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

NPM Guide

Uploaded by

Copyright:

Available Formats

NetOp Policy Manager Troubleshooting Guide

Release Number 6.1.5 Part Number 5/1543-CRA 119 1030/1 Uen

Rights and Restrictions

NetOp Policy Manager Troubleshooting Guide

NetOp Policy Manager Troubleshooting Guide

NetOp Policy Manager Troubleshooting Guide

Verify the NetOp PM System Configuration

Verify That the NetOp PM System Is Operational

Verify the NetOp PM System Configuration

Verify the NetOp PM Software Licenses

View Software License Details with the show_licenses.sh Script

Troubleshoot NetOp PM Software License Issues

NetOp Policy Manager Troubleshooting Guide

Verify Deployments Using PPP Subscriber Access

Verify the NetOp PM System Configuration

NetOp Policy Manager Troubleshooting Guide

Verify the NetOp PM System Configuration

Verify Deployments Using Dynamic CLIPS Subscriber Access

NetOp Policy Manager Troubleshooting Guide

. . . . . . . : 10.192.43.214 . . . . . . . : 255.255.248.0 . . . . . . . : 10.192.43.1

Verify the NetOp PM System Configuration

NetOp Policy Manager Troubleshooting Guide

Verify the NetOp PM System Configuration

Verify SmartEdge Router-to-RADIUS Communication

NetOp Policy Manager Troubleshooting Guide

Display test aaa Command Output

Verify the NetOp PM System Configuration

Interpret test aaa Command Output

test aaa Command Field Descriptions

Send count Send time Response time

NetOp Policy Manager Troubleshooting Guide

Troubleshoot the NetOp PM System

System Diagnostics Collection

Troubleshoot the NetOp PM System

Arguments for the report_npm_diagnostic.sh Script

NetOp PM System Logging

Configure the NetOp PM System Logging

Syntax for the config_logging.sh Script

Troubleshoot the NetOp PM System

Identify Component Names in the NetOp PM System Logging

NetOp PM RADIUS Server Issues

Test NetOp PM RADIUS Server Status

Generate NetOp PM RADIUS Server Statistics

Syntax for the test_radius.sh Script

-email e-mail-address -f -h -password user-password

Troubleshoot the NetOp PM System

Syntax for the test_radius.sh Script (continued)

Verify That the NetOp PM RADIUS Servers Are Operational

Set Up E-mail to Monitor the NetOp PM RADIUS Server Status

View Statistics for NetOp PM RADIUS Servers

Run the show_radius.sh Script

Troubleshoot the NetOp PM System

Syntax for the show_radius.sh Script

Interpret the Statistics

Configuration External RADIUS and LDAP servers

Troubleshoot the NetOp PM System

Configuration Authentication and accounting requests

External RADIUS Server Issues

Subscribers Using the Default Realm

Subscribers Using a Specific Realm