Introduction à

l'architecture de IOS-XR

Richard Lefebvre

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 1
 Agenda

1. Introduction to IOS-XR
2. Manageability
3. High Availability
4. Secure Domain Router (Logical Router)
5. RPL – Route Policy Language

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 2
Introduction to IOS XR

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 3
 Router OS Evolution

Control Plane Applications Control Plane Data Plane Management Plane

Management Plane Applications

Control Plane Data Plane Management Plane

SSH
Forwarding Plane Applications

HA Infrastructure
Control Plane Data Plane Management Plane

SSH
Host Service
L2 Drivers

Per.fMgmt
Interface
Routing

Netflow
SNMP

Alarm
LPTS
OSPF

IGMP

SSH
QoS
ACL
BGB

XML
ISIS

FIB
PIM

PFI
RIB
RIP

CLI
Network Stack

System Forward Checkpoint DB Multicast IPC System DB

Infrastructure Infrastructure Distributed Infrastructure

Scheduler Synch. Services IPC Mech Memory Mgmt

OS Scheduler

Kernel System Services

ƒ Monolithic Kernel ƒ Micro Kernel

ƒ Centralized Infrastructure ƒ Distributed Infrastructure
ƒ Integrated Network stack ƒ Independent Network stack
ƒ Centralized applications ƒ Distributed applications

Presentation_ID © 2007 Cisco Systems,

Presentation_ID © 2004 Cisco
Inc. Systems,
All rights Inc. All rights
reserved. Ciscoreserved.
Confidential 44
 IOS XR Key Features

• Modular – Runtime SW upgrade/downgrade support

• Distributed – scaleable with multi chassis support

• Platform Independent – POSIX compliant

• Management Interface – Unified Data Model (SNMP/XML)

• High Availability – Hot Standby and Process Restart

• Security – Control, Data and Management Plane

• Logical Router – Router Partitioning

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 5
 Modularity
IOS XR Software Packages

MPLS Multicast Security Manageability

IPSec, Encryption, ORB, XML,
MPLS, UCP PIM, MFIB, IGMP
Decryption Alarms management

Routing:
RIB, BGP, ISIS, OSPF, RPL

Forwarding Line Card

Platform independent Platform Dependent
FIB, ARP, QoS, ACL, etc LC ucode & drivers

Base Admin
Interface manager,
Resource Management:
System database, checkpoint services
Rack, Fabric, LR management
Configuration management, etc.

OS:
Kernel, file system, memory management, and other slow changing core
Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 6
 IOS XR Software Packages offer Flexibility
• Ability to upgrade independently
Multi-
MPLS, Multicast, Routing protocols and MPLS
cast
Line Cards
RPL BGP
• Ability to run different versions on Routing
Routing
different nodes Composite
Composite
OSPF ISIS

• Ability to release software packages

async Manageability

Security
• Ability to have composites into one
manageable unit if desired Forwarding
Host
Host
Composite
Composite
Base
• Notion of optional packages if
IOX Admin
technology not
OS
Desired on device (Multicast, MPLS)
Line card

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 7
 IOS XR Modular Packaged Software
RP DRP LC
Manage- Manage- Multi-cast Opt’l
Security Security GMPLS
ability ability
Opt’l Opt’l

GMPLS Multi-cast GMPLS Multi-cast

Line Card

Forwarding
RPL BGP RPL BGP
Mand
Base
OSPF ISIS OSPF ISIS

Forwarding Forwarding OS Mand

Mand Mand
Base Base
SC
Admin OS
Base
OS
Admin Mand

OS

ƒ Upgrade specific packages/Composites

– Across Entire system
• Useful once a feature is qualified and you want to roll it without lot of cmd
– Targeted Install to specific cards
• Useful while a feature is being qualified
• Reduces churn in the system to card boundary
ƒ Point Fix for software faults
Presentation_ID © 2007 Cisco Systems,
Presentation_ID © 2005 Cisco
Inc. Systems,
All rights Inc. All rights
reserved. Ciscoreserved.
Confidential 88
 Distributed Control Plane
BGP BGP MPLS Multi-cast
IS-IS RESILLIENT
SYSTEM PROCESS
DISTRIBUTION

RP1 RP2 RP3 RP4 RPn

ƒ Routing protocols and signaling protocols can

run in one or more (D)RP
ƒ Each (D)RP can have redundancy support
with standby (D)RP
ƒ Out of resources handling for proactive planning

Presentation_ID © 2007 Cisco Systems,

Presentation_ID © 2004 Cisco
Inc. Systems,
All rights Inc. All rights
reserved. Ciscoreserved.
Confidential 99
 Distributed Forwarding Infrastructure
RP RP LC-CPU
IP IM VLAN PPP
Stack IM Netflow
Drivers
Global
Int. Mgr. ARP HDLC Netflow
VLAN PPP ARP

Ingress CPU
FIB FIB

Global Ingress Egress

IDB & AIB CPU AIB & IDB FIB

Switching Fabric Switching Fabric Egress

CPU CPU AIB & IDB

LC LC LC LC

Single stage forwarding Two stage forwarding

ƒ Single global Adjacency Information ƒ Each line card has independent AIB only
Base (AIB) distributed to all line cards for local interfaces
ƒ Single global Interface Management DB ƒ Each line card has independent Interface
distributed to all line cards DB for local interfaces
ƒ Only Ingress FIB – forces forwarding ƒ Both Ingress and Egress FIB – allows
features to be run in RP forwarding features to be independently
run in LCs
Presentation_ID © 2007 Cisco Systems,
Presentation_ID © 2004 Cisco
Inc. Systems,
All rights Inc. All rights
reserved. Ciscoreserved.
Confidential 10
10
 IOS XR Software Architecture
IOS XR Software consists of:
ƒ Micro-Kernel OS, memory-protection, msg-passing, pre-emptive
ƒ All basic OS and router functionality implemented as processes
ƒ UNIX process model with separate, protected memory/address spaces

Applications

PO
S
Distributed Processing Microkernel I Message Queues
C Threads X
File System Scheduling

Lightweight Messaging
I Debug
S Timers Synchronization
Event Management C
O

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 11
 IOS XR Process Model
Message Passing Inter-Process Communication Model

“OS”

SysDB IM NetIO OSPF

BGP RIB IF mgr FIB…

microkernel

process manager
Separate, Restartable processes

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 12
 Manageability

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 13
 Manageability Architecture
“Industry Standard” Object Model

External EMS

Fault Configuration Accounting Performance Security

XML “Standards Derived” Object Model

Craft Works
RP Interface
CLI SNMP XML
Agent Agent Agent XML

Object Request Broker

ACL, QoS, Alarm and Perf and Test/

Inventory Routing IF
MPLS Log Accounting Diagnostic
Agent Agent Agent
Agent Agent Agent Agent

Common APIs to the rest of HFR S/W

DRP RP/Shelf Control Shelf Control

Line Card Fabric Card

Netflow

• Consistent data model independent of access schemes:

CLI, SNMP or XML
Presentation_ID © 2007 Cisco Systems,
Presentation_ID © 2004 Cisco
Inc. Systems,
All rights Inc. All rights
reserved. Ciscoreserved.
Confidential 14
14
 Manageability – Router CLI

ƒ Router config is based on two stage config model.

ƒ “running” or “active” config can not be modified
directly.
ƒ Instead, user config first enters a staging area (first
stage)
ƒ Must be explicitly promoted to be part of active config
(second stage).

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 15
 Craft Works Interface (CWI)

ƒ Graphical Configuration Desktop

Interface Clone
Protocols: BGP, ISIS, OSPF, TE

ƒ Validation/2 stage configuration

ƒ Configuration Editor
ƒ Value-added SSH/Telnet
ƒ Inventory and Rack View
ƒ Integrated Alarm Views
ƒ Metadata for fast feature
development

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 16
16
IOS XR Boot Overview

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 17
 IOS XR Boot Overview
ƒ IOS XR Boot is coupled with various Infra pieces
– Software Packaging
– Upgrades/Downgrades
– Installing patches for bug fixes

ƒ Major departure from IOS boot mechanism

– Image can be “baked”
– Image broken up into component pieces
– Different image naming conventions

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 18
 IOS XR Boot Overview
ƒ IOS XR Image:
• Base Image:
• Includes the following components:
• OS
• Admin
• Forwarding (IPv4 / IPv6 Unicast)
• Features
• PIEs (Package Installation Envelope)
• Unique PIE for each feature including
• MPLS, Multicast, Manageability and Security

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 19
 IOS XR Image components
IOS XR Modular Packages

Manage
Security Multicast / MPLS / Security and
ability
Manageability PIEs
Multi-
GMPLS
cast

RPL BGP
OSPF ISIS

Forwarding
Base or Mini Image
Base

IOX Admin

OS

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 20
 IOS XR Boot Overview – C12K Image Names

Type File Name Size

Bootable Image c12k-mini.vm-3.3.0 ~ 90 MB

mbiprp-rp.vm-3.3.0 ~ 10 MB
Optional PIEs c12k-mcast.pie-3.3.0 ~ 2.4MB
c12k-mgbl.pie-3.3.0 ~ 9.6MB
c12k-mpls.pie-3.3.0 ~ 2.4MB
c12k-k9sec.pie-3.3.0 ~ 1.3MB
Software Maintenance c12k-os-mbi- Variable
Update (SMU) 3.2.0.CSCei07321-1.0.0

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 21
 IOS XR Boot Overview – CRS Image
Names
Type File Name Size

Bootable Image comp-hfr-mini.vm-3.3.0 ~ 65MB

Upgrade PIE Image comp-hfr-mini.pie-3.2.0 ~ 53MB

Optional PIEs hfr-mcast-p.pie-3.2.0 ~ 2.4MB

hfr-mgbl-p.pie-3.2.0 ~ 9.6MB
hfr-mpls-p.pie-3.2.0 ~ 2.4MB
hfr-k9sec-p.pie-3.2.0 ~ 1.3MB
Maintenance Updates hfr-rout-p.pie-3.2.1 Variable
(Example)
Software Maintenance hfr-os-mbi-3.2.0.CSCei07321- Variable
Update (SMU) 1.0.0

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 22
Configuration Management

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 23
 IOS XR and IOS Config differences
IOS XR IOS

Configuration changes do NOT take place after

Configurations take place immediately after <CR>
<CR>

Configuration changes must be ‘committed’ before

No commit
they take effect

Allows you to verify your configuration before

No verification required
applying it

Two stage configuration model Not available

Configuration rollback Not available

Feature centric Interface centric

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 24
 IOS XR CLI - Location addressing format
ƒ CRS-1 is designed to scale 72 linecard chassis’s with a
potential of 1296 linecard and RP slots
ƒ Location identifiers use R/S/M/I format
R = Rack (applicable in multi-chassis systems)
S = Slot (physical slot the module is in)
M = module (0 for ‘fixed’ PLIMs, n for SPAs)
I = Interface

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 25
 IOS XR CLI: New CLI format
ƒ New CLI reflects the HW position in the system
– Introduces the Hierarchical location scheme
– Each linecard has three-level identification: Shelf/Slot/cpu #
– Interfaces have the Shelf/Slot/Bay/Interface scheme
ƒ Protocol referenced by address family type – v4/v6
ƒ Backward compatible command-set with IOS

RP/0/0/CPU0:Router-1#show ipv4 interface brief

Interface IP-Address Status

Protocol
MgmtEth0/0/CPU0/0 10.23.1.69 Up Up
MgmtEth0/0/CPU0/1 unassigned Shutdown Down
MgmtEth0/0/CPU0/2 unassigned Shutdown Down
GigabitEthernet0/2/0/0 100.12.1.1 Up Up
Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 26
 IOS XR CLI: New CLI format
ƒ Config modes include:
– Privileged exec mode
– Global config mode
– Config sub-mode
– Admin mode
ƒ Admin mode is newly introduced compared to IOS
ƒ Admin mode allows viewing / configuring shared
resources
– Fabric
– Logical Router
– Package installation

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 27
 IOS XR CLI: Config Modes

RP/0/0/CPU0:ios#config t
RP/0/0/CPU0:ios(config)#interface MgmtEth 0/0/CPU0/0
RP/0/0/CPU0:ios(config-if)#
RP/0/0/CPU0:ios#
RP/0/0/CPU0:ios#admin
RP/0/0/CPU0:ios(admin)#

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 28
 IOS CLI – Single Stage IOS config model

Configuration
Database

CLI user
Active Config

Target Config = Active Config

Changes take effect immediately

User establishes
config session

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 29
 IOS XR CLI: Two Stage Config model

First Stage Second Stage

Configuration
Database

CLI user Config Commit

Change Active Config

Target Config = Config Change + Active Config

User establishes Adds/deletes/modifies Promotes the changes to active

configuration; these changes: configuration; these changes:
config session • Are entered in the staging area • Are verified for semantic
• Are validated for syntax and correctness
authorized • Are check-pointed
• Can be reviewed and modified on the router

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 30
 IOS XR CLI: Config Commits
RP/0/0/CPU0:ios#show run int gi0/2/0/0
% No such configuration item(s)

RP/0/0/CPU0:iosxr1#conf t
RP/0/0/CPU0:iosxr1(config)#interface gig0/2/0/0
RP/0/0/CPU0:iosxr1(config-if)#ipv4 address 100.12.1.1/24
RP/0/0/CPU0:iosxr1(config-if)#commit
RP/0/0/CPU0:Apr 24 00:49:28.119 : config[65691]: %MGBL-CONFIG-6-
DB_COMMIT : Configuration committed by user 'root'. Use 'show
configuration commit changes 1000000036' to view the changes.
RP/0/0/CPU0:iosxr1(config-if)#end
RP/0/0/CPU0:Apr 24 00:49:30.701 : config[65691]: %MGBL-SYS-5-
CONFIG_I : Configured from console by root
RP/0/0/CPU0:iosxr1#
RP/0/0/CPU0:iosxr1#show run int gigabitEthernet 0/2/0/0
interface GigabitEthernet0/2/0/0
ipv4 address 100.12.1.1 255.255.255.0

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 31
 IOS XR CLI: Config Commits
• Each commit generates record with unique Commit ID or label
• Each commit ID is a rollback point
• Commit Database stores up to 100 rollback points
Config
Database

Running
Config

Target Config Commit

Config Log:
Commit ID# 100
Commit ID# 099
Commit ID# 098
…
Commit ID #001

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 32
 IOS XR CLI: Config Commits
ƒ Commit keyword writes config into Active Config
ƒ Supplies a commit ID to help in Config Rollback
– 1000000036 is the commit ID in previous illustration

ƒ List of commits can be viewed

– History list is maintained

ƒ Commits can be labeled with user-friendly ‘tags’

– Eliminates the cumbersome IDs

ƒ Config restrictions can be imposed based on user

– In previous illustration, the user “root” is indicated

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 33
 IOS XR CLI: Commit List
• Sample Commit List output

RP/0/0/CPU0:ios#show configuration commit list

SNo. Label/ID User Line Client Time Stamp
~~~~ ~~~~~~~~ ~~~~ ~~~~ ~~~~~~ ~~~~~~~~~~
1 1000000037 root con0_0_CPU CLI 01:39:03 UTC Mon Apr 24 2006
2 1000000036 root con0_0_CPU CLI 01:18:10 UTC Mon Apr 24 2006
3 1000000035 root con0_0_CPU CLI 01:00:54 UTC Mon Apr 24 2006

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 34
 IOS XR CLI: Config Rollback
RP/0/0/CPU0:iosxr1#conf t
RP/0/0/CPU0:iosxr1(config)#hostname iox-nw06
RP/0/0/CPU0:iosxr1(config)#commit
RP/0/0/CPU0:Apr 24 01:00:55.302 : config[65691]: %MGBL-CONFIG-6-DB_COMMIT
: Configuration committed by user 'root'. Use 'show configuration commit
changes 1000000034' to view the changes.
RP/0/0/CPU0:iox-nw06(config)#end
RP/0/0/CPU0:iox-nw06#
RP/0/0/CPU0:iox-nw06#rollback configuration to 1000000033
Loading Rollback Changes.
Loaded Rollback Changes in 1 sec
Committing.
3 items committed in 1 sec (2)items/sec
Updating.RP/0/0/CPU0:Apr 24 01:01:07.143 : config_rollback[65691]: %MGBL-
CONFIG-6-DB_COMMIT : Configuration committed by user 'root'. Use 'show
configuration commit changes 1000000035' to view the changes.

Updated Commit database in 1 sec

Configuration successfully rolled back to '1000000033'.
RP/0/0/CPU0:iosxr1#
RP/0/0/CPU0:iosxr1#

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 35
 IOS XR CLI: Config Session locks
ƒ More than one user can open a target config session at a time
ƒ Provision for multiple users to work on separate target
configurations.
ƒ Locking the router config prevents changes by other users while
someone is already committing the config.
RP/0/0/CPU0:ios#configure exclusive
RP/0/0/CPU0:ios(config)#hostname iox-nw06-1
RP/0/0/CPU0:ios(config)#commit
RP/0/0/CPU0:iox-nw06-1(config)#

Config
Database
CLI
Running
Config

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 36
 IOS XR CLI: Pre-config capabilities
ƒ Pre-config feature allows configuring physical interfaces before
they are inserted into the router.
ƒ Preconfigured interfaces are not verified or applied until the
actual interface with the matching location
ƒ Allows reduction down time and helps improve operational tasks

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 37
 IOS XR CLI: Pre-config capabilities

ƒ Prior to the LC being inserted

• Select the interface
• Configure the timing (e.g. for SONET controller)
• Configure the framing
• Configure the IP address

RP/0/0/CPU0:IOX-4(config)#interface preconfigure POS 0/4/1/0

RP/0/0/CPU0:IOX-4(config-if-pre)#ip address 1.1.1.1 255.255.255.0
RP/0/0/CPU0:IOX-4(config-if-pre)#encapsulation ppp
RP/0/0/CPU0:IOX-4(config)#controller preconfigure sonet 0/4/0/0
RP/0/0/CPU0:IOX-4(config-sonet)#clock source line

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 38
 IOS XR CLI: Config error handling
ƒ Two levels of config error handling
ƒ Parser/Syntax error
– Identified by the parser when the <return> key is entered
ƒ Commit error
– Syntactically correct but erroneous from config commit standpoint
– Error details viewed through “show configuration failed” command
– Common reasons for this error include:
• Non-atomic config sequence
• Lack of predecessor config
• Unsupported config from platform perspective

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 39
 IOS XR CLI: Config error handling
ƒ Sample config error from QoS feature

RP/0/0/CPU0:ios#conf t
RP/0/0/CPU0:ios(config)#policy p1
RP/0/0/CPU0:ios(config-pmap)#class c0
RP/0/0/CPU0:ios(config-pmap-c)#set precedence 0
RP/0/0/CPU0:ios(config-pmap-c)#commit
% Failed to commit one or more configuration items during an
atomic operation, no changes have been made.
Please use 'show configuration failed' to view the errors

RP/0/0/CPU0:ios(config-pmap-c)#show configuration failed

!! CONFIGURATION FAILED DUE TO SEMANTIC ERRORS
policy-map p1
class c0
set precedence routine
!!% Class-map not configured: c0

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 40
IOS XR: User Access
Privileges

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 41
 IOS XR: Base User Security Privileges

• IOS XR uses 3 hierarchical steps to implement user privileges:

• Configure task groups and associate task IDs to the group
• Configure user groups
• Give it permissions by associating the group to specific task group
• Configure users and assign them to one or more user groups.

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 42
 User Security Privileges: Task Group

• Task Group is defined by collection of Task IDs

• For instance, an OSPF task group might have only OSPF config rights
• Whereas a BGP task group might inherit all of OSPF rights in addition to
BGP config rights

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 43
 User Security Privileges: Task IDs

• Task IDs grant permissions to perform tasks namely:

• read or write or execute or debug
• combinations of all

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 44
 User Security Privileges: User Groups

• Each User Group is associated with a set of task groups

applicable to users in that group

• User task permissions are derived from Task Groups

associated with user groups to which that particular user belongs

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 45
 Pre-defined Task Groups: Permissions
Pre-defined User Groups’ Permission

• root-system Root-system users

Read and write all commands available on the router
• root-lr — Root-logical router users
Read and write all commands on the LR
Root-system owner tasks are read only
• netadmin — Network administrators
Read all commands except root-system owner commands
Write routing, forwarding, connectivity, VLAN, AAA, and so on
• sysadmin — System administrators
Read all commands except root-system owner commands
Write AAA, manageability, logging, and so on
• operator — Day-to-day activity users
Read and write basic operations
Read logs, CDP, and run diagnostics

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 46
 Base Security Flow

AAA Local or Remote Server

Users, User-Groups, and Task-Groups

Can Be Stored Locally in SysDB

Authentication Authorization
User User Group Task-Group Tasks
john bgp-users ospf-access ospf read/write
… … bgp-access …
… bgp read/write
…

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 47
 Limited Visibility of Two Users
Username: user1
Password:
RP/0/9/CPU0:iox1#
RP/0/9/CPU0:iox1#
RP/0/9/CPU0:iox1#
RP/0/9/CPU0:iox1#sh run router ospf Í SO, OSPF is accessible
router ospf 100
log adjacency changes
address-family ipv4 unicast
area 0
interface Loopback0
!
interface GigabitEthernet0/3/0/0
!
…….
RP/0/9/CPU0:iox1#sh run router bgp ÍSo, BGP is also accessible
router bgp 100
bgp router-id 11.11.11.11
address-family ipv4 unicast
!
!

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 48
 Limited Visibility of Two Users
Username: user2
Password:
RP/0/9/CPU0:iox1#
RP/0/9/CPU0:iox1#
RP/0/9/CPU0:iox1#sh run
Building configuration...
!! Last configuration change at 11:58:01 UTC Tue Apr 25 2006 by lab
!
…..
router ospf 100
log adjacency changes
address-family ipv4 unicast
area 0
interface Loopback0
!
interface GigabitEthernet0/3/0/0
!
…..
RP/0/9/CPU0:iox1#sh run router bgp
% This command is not authorized Í=
RP/0/9/CPU0:iox1#

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 49
 High Availability

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 50
 IOS XR High Availability Depth
ISSU

Non-Stop Forwarding

Separate Control and Data Planes

RP/DRP Redundancy Active/Standby Failover

Process Restartability: Active State Checkpointing

All processes: Separate Address Spaces

memory faults affect only 1 process, recovery = restart process

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 51
 IOS XR Software Architecture Overview
Process
Filesystem
Manager
TRUE Microkernel
(Mach, QNX)
MMU with full protection
Applications, drivers, and
protocols are protected d FAULT
FAULT
aine )
Application
FAULT Driver
n t d
Co
rtabl
a
e
on t aine )
( Rest C rtabl
e
ta
(Res

Monolithic Kernel n t ained Application

Co rtable) FAULT Application
ta
(BSD/Linux, NT) (Res
MMU with partial protection
Applications are protected r r uption
Kernel Co Filesystem Network
FAULT Driver
em Wide
Syst
Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 52
 High Availability in IOS XR

Many Aspects of High Availability in the Software &

Hardware
ƒ Process/Application level High Availability
ƒ Resource Monitoring and Validation
ƒ Fault Management
ƒ Data Traffic Protection and Restoration
ƒ In-Service Software Upgrade (ISSU)

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 53
 High Availability Infrastructure
Contained Contained

IP
BGP IS-IS RIB QoS FIB Stack CLI XML Alarm File System

OS Distributed Middleware

L2 Inter Process
OSPF PIM IGMP ACL PFI Drivers Netflow SNMP SSH
Communication

Contained

ƒ Distribution improves fault tolerance and recovery time

Database and system management functionality localized to each node
ƒ Granular process restart allows for fast recovery from failures
ƒ IOS XR is designed to optimize the switch over between redundant hardware
elements (RP, SC, PS, Fan C.)
IOS XR is designed to route around fabric failure
Line cards are protected by link bundling, APS, IPS, ECMP etc.

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved.

Presentation_ID Cisco Confidential 54
54
 NSF capabilities
ƒ Separate mgmt, control and data planes
Mgmt and control plane outages do not affect forwarding operation
ƒ Process-level fault containment (process restart)
ƒ Card-level fault mgmt (active->standby RP or DRP failover)
ƒ NSF achieved through Check pointing and Graceful Restart of
routing protocol
ƒ ISSU Capability where many processes can be upgraded without
impact
Uses process restart to install updated software on the active RP

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 55
 Process Restartability
ƒ Used for small/contained faults (individual or small groups of process
failures)
ƒ Processes support restarting with dynamic state recovery
Mirrored State via checkpoint or synchronization with peer
ƒ First line of defense- All Processes are restartable for fault recovery
ƒ Certain processes are ‘mandatory’ – must always be running. Failure of
mandatory processes can cause RP failover
ƒ Second line of defense - Card-level Redundancy is used when Process
Restart fails-
Contained Contained

IP
BGP IS-IS RIB QoS FIB Stack CLI XML Alarm File System

OS Distributed Middleware

L2 Inter Process
OSPF PIM IGMP ACL PFI Drivers Netflow SNMP SSH
Communication

Contained

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 56
 Checkpoint support for process restart
(checkpoint to RAM on local card)
ƒ State recovery: Processes with dynamic state may utilize
Checkpoint and Checkpoint Mirroring or
Obtain state from neighbors to regain state after a failover
ƒ Processes that restart reconcile inconsistencies with peer processes
after restart
ƒ In general restarting one process does not cause other processes to
restart

process Updates of running state

(process dies)
Checkpoint
new shared
instance memory store
of Recover state
process

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 57
 Process-level Redundancy

Active service-providing process

Active
process client

client
Standby
process Standby process
client

Active process uses checkpoint library

to share running state with standby.
Clients use active service-
Standby Process may or may not be providing process
running

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 58
 Process-level Redundancy (cont)
Active process dies

Standby Process becomes Active

process
process
process client
client

New Standby
client
Process is started
Clients use new primary
New primary starts service-providing process
sending updates to
standby process

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 59
RP/DRP Redundancy

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 60
 RP/DRP Redundancy
Card-level redundancy and failover
ƒ Used for handling larger error sets
control plane lock-up on active RP or
Hardware errors on active RP or
cases where critical processes will not restart
properly
Active Standby
ƒ “Warm” redundancy - Processes on standby RP RP
card use mirrored, checkpointed data.
ƒ Warm Standby processes do not process
checkpointed data until they become active

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 61
 RP Switch over
ƒ RP Switchover mechanism is optimized around two goals:
ƒ NSF- Non-Stop Forwarding : Continue forwarding packets and prevent route flaps
while reconverging Layer 3 protocols
ƒ NSR – Non- Stop Routing: Preserving routing adjacencies, avoiding network
topology change
GR- Graceful Restart : Reestablishes the routing information bases without churning the
network
State Information
Ethernet
FR
ATM
PPP Active Standby
HDLC.. RP RP

BGP
OSPF
ISIS Line Card Line Card
MPLS
Mcast..
Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 62
 Redundancy - Card-Level Switchover
When?

State Information ƒ HW errors detected on active RP

card
ƒ Control plane lockup on active
card
ƒ ‘Mandatory’ Process crashed and
not restartable or crashed
Active Standby repeatedly
RP RP
ƒ Kernel crash
ƒ User command “redundancy
failover”
Line Card Line Card ƒ HW watchdog timer

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 63
 Redundancy - Card-Level Switchover
When is RP switchover NOT allowed?

ƒ HW errors detected on standby

State Information card
ƒ Control plane lockup on standby
card
ƒ Mandatory Process(es) crashed
on standby card
Active Standby ƒ Software Install in progress
RP RP
ƒ Any process on standby indicates
“standby not ready” (usually
during standby card init)
Line Card Line Card

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 64
 Redundancy - Process state recovery
models
Hot Process A Process A

Check Check
Warm Process B
Point Point Process B
Server Server

Cold Process C Process C

ACTIVE RP STANDBY RP

ƒ Mixture of hot/warm/cold process restart/redundancy used depending on the

requirements of the individual software process
ƒ Process A: HOT standby: checkpoint data sent to standby peer continually
ƒ Process B: Warm standby: checkpoint data mirrored to standby card - read by
process on GoActive
ƒ Process C: Cold standby: no checkpointing - process able to init w/o requiring
saved state on GoActive
Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 65
 Switchover mechanics

ƒ Switchover mechanics Then MPLS, BGP,

optimized around two goals: 5 and “everything else”
ƒ Non-Stop Forwarding
4 Then OSPF infra, then OSPF
ƒ NSR (preserving routing
adjacencies, avoiding network
topology change)
First, infra needed for ISIS
3
is brought up, then ISIS,
then ISIS sends HELLOs
Active card fails,
2 Switchover begins…
x
Active
RP
Standby
RP

Standby Card boot, HW-init,

1 Standby mode entered, state
syncing from active started
Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 66
Non-Stop Forwarding

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 67
 Headless operation limits
How long will the LC’s operate in a ‘headless’-state if control plane
fails?
ƒ Case 1: Redundant RP card exists
Standby card switches to active role
Headless operation governed by individual protocol go-active, re-
convergence times
ƒ Case 2: no redundant RP
If headless state is due to process failure and restart, headless
operation will exist for as long as it take to restart process
If due to RP crash, LC’s forward until RP reboots and comes active

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 68
 NSF: Autonomous Hardware and Quick
Control Plane Recovery
State
ƒ IOX box (GSR/CRS) has dual Information
RPs
ƒ Each LC has dedicated packet
forwarding hardware (PSE) RP x
Active Standby
RP
ƒ Packet forwarding is not affected
by:
Control
ISIS, OSPF, BGP, MPLS, Updates But…
Multicast process restart Interrupted
Fwding
Infrastructure process restarts Ok!
RP failover
ƒ Packet forwarding on LCs can
function autonomously during LC LC
control plane outages

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 69
In Service Software
Upgrade- ISSU

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 70
 ISSU
• Ability to deliver point fixes (e.g. to fix a BGP bug,
only install/modify a patch to BGP code, nothing else)
or upgrade complete set of functionality whilst system
is operating
• Identify exactly what has changed, and its impact ‘fan-
out’
• Safety Checks conducted before proceeding with
upgrade
• Restart processes to activate new code
• Goal is to preserve NSF / NSR even on “live”
upgrades to active RP

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 71
 ISSU
ƒ Two main forms of ISSU
Software Maintenance Unit (SMU) based –
Patch code to resolve problem. SMU limited to single DDTS
component
Package Installation Envelope (PIE) based –
Complete set of binaries for a particular collection of functions/
Software Package

MPLS Multicast Security Manageability

IPSec, Encryption, ORB, XML,
MPLS, UCP PIM, MFIB, IGMP
Decryption Alarms management

Routing:
RIB, BGP, ISIS, OSPF, RPL

Forwarding Line Card

Platform independent Platform Dependent
FIB, ARP, QoS, ACL, etc LC ucode & drivers

Base Admin
Interface manager,
Resource Management:
System database, checkpoint services
Rack, Fabric, LR management
Configuration management, etc.

OS:
Kernel, file system, memory management, and other slow changing core

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 72
 Secure Domain Router
SDR/LR

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 73
 SDR
ƒ Routing system partitioned into discrete
configurable units
ƒ Includes at least one RP per LR.
ƒ Has a distinct RIB and FIB combination
with its own route processor and routing
protocol instances
ƒ Each LR has a secure and distinct mgmt
interface
ƒ Fabric and SC shared by all LRs
ƒ Can span across chassis

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 74
Terminology

SDR Secure Domain Router

LR Logical Router
dSC Designated Shelf Controller. RP which has been designated to do
shelf management functions.
Owner SDR Default SDR. Owns all slots unless configured otherwise. Hosts
the dSC. Has access to admin plane CLI command, to “run”
commands.
Non-owner SDR SDR explicitly defined in the admin configuration. Also referred to as
“Named SDR”
dSC Designated Shelf Controller: RP which has been designated to do
shelf management functions.
dSDRSC/dLRSC Designated SDR(LR) SW Controller: RP/DRP that manages a given
SDR. There is one and only one per SDR. In the owner SDR, the DSC
is also the dLRSC.

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 75
 What is a Secure Domain Router- SDR?
CRS -1 / IOX
ƒ Independent/isolated physical Router 1 Router 2
routing instance within a common BGP/IGP Multicast
(multi-) chassis Multicast
BGP/IGP
Multicast …n BGP/IGP

ƒ Each RP & LC in chassis

uniquely allocated to a specific

SPA/SIP
Switch

SPA/SIP

SPA/SIP

SPA/SIP
Fabric
LR
ƒ Resource sharing between LRs …n
is limited to fabric, power, cooling
ƒ Acts as an independent router Carrier Class IOS XR
Processors not shared – CPU Moving to Secure Domain Routers
resources not in contention to Complete
Physical & Logical (Memory Protected)
Memory not shared – Memory
Separation
leaks can only affect that SDR
ƒ Hardware Failures Isolated

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 76
 Resource allocation for SDR creation
SDR-A

ƒ Route Processor[s]
(RPs) & Line Card[s] D
F
A
F
A
R N N
P D D
ƒ Slot level granularity C C R R SDR-B
P P

ƒ Configure and
communicate via Admin
Plane
D D
ƒ All routing apps run in R R
P P R
P
R
P
LR Plane. SDR-C

Can not communicate

with processes outside
their LR.

Control Plane Separation -> Data Plane Separation -> No

inter-SDR communication (apart some exceptions) ->
Simple ->fault isolation
Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 77
 GSR IOX LR – General Architecture

LR0 (Owner) RP2 RP3

RP0 RP1

Mbus
SDR foo

SWITCH FABRIC AND Clock SCHEDULER CARDS

CSC0 CSC1 SFC0 SFC1 SFC2

PS0 Alarm LC1

Alarm FAN0 LC0 LC2 LC3

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 78
 Default/Owner SDR
ƒ Handles inventory of unassigned cards
ƒ When IOX router boots first time
All cards belongs to default LR
User configures to assign to diff LR (SDR)
ƒ No configuration needed
ƒ Head of Admin Plane - DSC
ƒ Head of LR Plane : dLRSC/dSDRSC

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 79
 Secure Domain Router Architecture
Admin vs. SDR Planes

ƒ LR Plane: Everything involved with running a router.

Generally more platform independent.
ƒ Admin Plane: Everything involved with keeping the
hardware running. More platform dependent. Also, this
is where LRs are created and provisioned.
LR 1 LR 2

RIB FIB RIB FIB

Protocol Config Protocol Config

LRd LRd

Chassis Config Fabric

Control Admin Plane Mgmnt

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 80
 Secure Domain Router Architecture
Overview
LR Planes with Admin Plane
ƒ SDRs operate like
SDR1 Plane SDR2 Plane physically separate
routers
RIB FIB RIB FIB
ƒ Admin and SDR
Protocol Protocol
planes provide Fault
isolation through
IPC Partitioning
LRd LRd

Chassis Fabric
Control Mgmnt
Admin Plane

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 81
 Secure Domain Router Architecture
Configuration Partitioning
• System owner configures fabric/chassis ƒ SDR Plane Config
• SDR owner configures LC/(D)RPs Local config stored within
(D)RP
• Fault Isolation through Config Partitioning
Interfaces, applications,
SDR Plane1 SDR Plane 2 (D)RP pairing
ƒ Must log into SDR to
RIB FIB RIB FIB configure it
ƒ Admin Plane Config
Protocol Config Protocol Config Shared resource config
stored in separate file
Accessible to CRS/GSR
LRd LRd owner only
ƒ Scoped within the Admin
Shared
Chassis resource Fabric plane (RPs, SPs and SCs)
Control Config Mgmnt
Admin
Plane
Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 82
 Secure Domain Router Connectivity
External Connections
External Connectivity:
ƒ Physically connected ports
ƒ Network connected ports
ƒ Support for logical
L1/L2/L3
interfaces per port
SDR1
Network

SDR2

CRS/GSR

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 83
 SDR Routers
External connections (Logical view)
Back Bone Routers POP Agg Routers

(SDR1)
Interconnect
Physical

(SDR2)

Single ChassisCRS-1

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 84
 RPL

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 85
 RPL - Motivation
ƒ Scaling
Using route-maps could lead to 100k – 1M lines of
configuration (e.g. 1000s of BGP peers).

ƒ Modularity
Exploit modularity to reuse common portions of
configuration.

ƒ Parameterization
For elements which are not exact copies of each other we
can add parameterization ( think variables ) to get further re-
use.

ƒ Improved Clarity
No Silently skipped statements.

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 86
 RPL Syntax – General Structure
ƒ RPL is used in 2 steps :
Define the policy in configuration mode:

Route-policy <Policy Name>

statement A
statement B
End-policy
Use the policy in BGP body:

Router bgp 99
neighbor 2.2.2.2 remote-as 1
address-family ipv4 unicast
route-policy <policy Name> in/out

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 87
 RPL Syntax – Salient Features

ƒ Simple if, then, elseif & else statements.

ƒ Boolean operators to combine multiple
statements.
ƒ Hierarchical and parameterized policy
definitions for modularity.
ƒ Unordered sets to form operand templates.
ƒ Protocol specific constructs: as-path,
community, med, tag, metric etc.

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 88
 RPL - Conditional Statements – if

An “if” statement uses a conditional expression to

decide which actions or dispositions should be
taken for the given route.

If as-path in as-path-set-1 then

drop
endif

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 89
 RPL - Conditional Statements – else if

The “if” statement also permits an “else” clause, which is

executed if the expression is false.

if med eq 150 then

set local-preference 10
elseif med eq 200 then
set local-preference 60
else
set local-preference 0
endif

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 90
 RPL - Conditional Statements - Nested If

The statements within an if statement may

themselves be if statements
if community matches-every(12:34,56:78) then
if med eq 8 then
drop
endif
set local-preference 100
endif

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 91
 RPL - Boolean Expressions
ƒ Boolean expressions evaluate as either true or false.
ƒ RPL provides means to build compound conditions by
means of boolean operators
ƒ There are three Boolean operators :
– negation (not)
– conjunction (and)
– disjunction (or).

if med eq 42 and next-hop in (1.1.1.1) then

if med eq 10 and not destination in (10.1.3.0/24) or community is (56:78)

if med eq 10 and (not destination in (10.1.3.0/24)) or community is (56:78 )

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 92
 Summary

• Modular – Runtime SW upgrade/downgrade support

• Distributed – scaleable with multi chassis support

• Platform Independent – POSIX compliant

• Management Interface – Unified Data Model (SNMP/XML)

• High Availability – Hot Standby and Process Restart

• Security – Control, Data and Management Plane

• Logical Router – Router Partitioning

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 93
 Merci!

ƒ Veuillez remplir votre formulaire d’évaluation!

ƒ Les présentations seront disponibles sur notre site Web.
ƒ N’oubliez pas de visiter le Salon des Exposants.
ƒ Nous vous invitons pour le cocktail à la fin de la journée dans le
Salon des Exposants.
ƒ Ne manquez pas Networkers 2008
Orlando, Floride du 22 juin au 26 juin 2008

Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 94
Presentation_ID © 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential 95