NOCTION

Intelligent Routing Platform

Lite (free version)

Installation and Configuration Guide

Version 1.0
 2

1294 Lawrence Station Rd, Sunnyvale, CA 94089

Tel: 1-650-618-9823 Fax: 1-650-618-8781

Email: [email protected]

Copyright ©2013 Noction Holdings Ltd, All Rights Reserved.

Noction logos, and trademarks or registered trademarks of Noction Holdings Ltd or its subsidiaries in
the United States and other countries.
Other names and brands may be claimed as the property of others. Information regarding third party
products is provided solely for educational purposes. Noction Holdings Ltd is not responsible for the
performance or support of third party products and does not make any representations or warranties
whatsoever regarding quality, reliability, functionality, or compatibility of these devices or products.
The Intelligent Routing Platform includes GeoIP data created by MaxMind, available from http://maxmind.com/
Copyright ©2013 Noction Holdings Ltd, All Rights Reserved.
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1 How to contact support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 What is IRP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.1 IRP Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.2 IRP Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.3 IRP Technical Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.4 IRP Operating modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.5 BGP Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.2.6 Outage detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2.7 VIP Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2.8 Retry Probing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2.9 Routing Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.2.10 Support for Centralized Route Reflectors . . . . . . . . . . . . . . . . . . . . . . . 21
1.2.11 Support for Internet Exchanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.2.12 Optimization for multiple Routing Domains . . . . . . . . . . . . . . . . . . . . . 22
1.2.13 Improvements confirmation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.2.14 Improvements weight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.2.15 Notifications and events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.2.16 IRP API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.2.17 IRP Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.2.18 Inbound (beta) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
1.2.19 Flowspec policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
1.2.20 Throttling excessive bandwidth use with Flowspec . . . . . . . . . . . . . . . . . . 40
1.2.21 Maintenance windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
1.3 IRP Optimization modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
1.3.1 Performance optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
1.3.2 Cost optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
1.3.3 Commit Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.1 Configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.2 Global and Core Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.3 Collector Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.1 Irpflowd Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.2 Irpspand Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.4 Explorer Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.4.1 Specific PBR configuration scenarios . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.5 BGPd Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
2.5.1 AS-Path behavior in IRP BGPd . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.5.2 BGPd online reconfiguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.6 Failover configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.6.1 Initial failover configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.6.2 Re-purpose operational IRP node into an IRP failover slave . . . . . . . . . . . . 83
2.6.3 Re-purpose operational IRP failover slave into a new master . . . . . . . . . . . . 83
2.6.4 Recover prolonged node downtime or corrupted replication . . . . . . . . . . . . . 83
2.7 Frontend Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
2.8 Administrative Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
2.9 IRP Initial Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

3
CONTENTS 4

2.10 IRP software management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

2.11 Starting, stopping and status of IRP components . . . . . . . . . . . . . . . . . . . . . . . 85
3 Using IRP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.1 Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.1.1 Accessing the system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.1.2 System dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.2 Frontend structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.2.1 Sidebar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.2.2 Failover status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.2.3 Global search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.2.4 Warning bars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.3 Customizing the dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.3.1 Custom dashboard gadgets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.4 Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.4.1 Platform overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.4.2 Platform performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.4.3 Loss Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.4.4 Latency Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.4.5 Probe conversion rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.4.6 Current improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.4.7 VIP Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.4.8 Improvements to Exchanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.4.9 Historical records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.4.10 Providers overall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.4.11 Provider performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.4.12 Cost overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.4.13 AS pattern problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.4.14 Top volume prefixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.4.15 Top volume AS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.4.16 Top problem AS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.4.17 Prefix statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.4.18 AS statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.4.19 Current inbound improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.5 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.5.1 Improvements by type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.5.2 Performance improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.5.3 Probed prefixes and Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.5.4 New and retry improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.5.5 Improvements by probing source . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.5.6 Prefixes rerouted from provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.5.7 Prefixes rerouted to provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.5.8 Improvements by provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.5.9 Performance improvements by provider . . . . . . . . . . . . . . . . . . . . . . . . 116
3.5.10 Commit improvements by provider . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.5.11 Total and Improved traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.5.12 Bandwidth usage by provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.5.13 Bandwidth usage and improvements . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.5.14 Total bandwidth usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.5.15 Inbound traffic distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.6 Routing Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
3.7 Flowspec policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.8 Throttling excessive bandwidth use with Flowspec . . . . . . . . . . . . . . . . . . . . . . 124
3.9 Maintenance windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
3.10 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
3.11 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
3.11.1 Traceroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
3.11.2 Looking glass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
3.11.3 Whois . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
CONTENTS 5

3.11.4 Prefix probing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

3.12 Configuration editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
3.12.1 Global configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
3.12.2 Core configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
3.12.3 Providers and Peers configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
3.12.4 Commit Control configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
3.12.5 BGP and Routers configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
3.12.6 Collector configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
3.12.7 Explorer configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
3.12.8 Notifications and Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
3.12.9 Security Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
3.12.10 Frontend configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
3.12.11 Inbound (beta) configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
3.12.12 Wizards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
4 Configuration parameters reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.1 Complete parameters list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.1.1 Common database credentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.1.2 Global parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
4.1.3 API daemon settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
4.1.4 BGPd settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
4.1.5 BGP sessions settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
4.1.6 Collector settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
4.1.7 Core settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
4.1.8 Explorer settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
4.1.9 Notification and events parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 226
4.1.10 Administrative settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
4.1.11 Providers settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
4.1.12 Inbound parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
4.1.13 Routing Policies settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
4.1.14 Exchanges settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
4.1.15 User Directory settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
5 Appendixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
5.1 Frontend labels for configuration parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 253
5.2 Configuration parameter index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Chapter 1

Introduction

1.1 How to contact support

If you encounter any problems while using or setting up the Intelligent Routing Platform, please contact
us.
You may contact us in the following ways:

• Email us at [email protected]
• Open a support request at https://helpdesk.noction.com

1.2 What is IRP

BGP is a fundamental technology for the fault-tolerance of the Internet, which chooses network paths
based on the number of hops traffic must traverse before it reaches its destination. However, BGP does
not take into account the important factors of network performance. Even if multi-homing does provide
some redundancy, multiple network outages have shown that multi-homing alone is not the solution for
risk diversity and business continuity. When major blackouts or even congestions happen, multi-homing
gives a fallback link for the “first-mile” connection, rather than providing a way to route around the
Internet “middle-mile” issues.
Noction Intelligent Routing Platform (IRP) is a product developed by Noction to help businesses to
optimize their multi-homed network infrastructure. The platform sits in the network and gets a copy
of the traffic from the edge routers. The system passively analyzes it for specific TCP anomalies and
actively probes remote destination networks for such metrics as latency, packet loss, throughput, and
historical reliability. It computes a performance- or a cost-improvement network traffic engineering policy
and applies the new improved route by announcing it to the network‘s edge routers via traditional BGP
session.
Noction IRP is a complete network monitoring and troubleshooting solution, which facilitates the detec-
tion, diagnosis, and automatic resolution performance issues. It delivers real-time views and dashboards
that allow to visually track network performance and generate triggers, alerts and notifications when
specific problems occur.

1.2.1 IRP Features

The Intelligent Routing Platform is designed to help Service Providers to improve the performance and to
reduce the costs of running a multi-homed BGP network. The system makes intelligent routing decisions
by analyzing various network performance metrics and selecting the best performing route for the traffic
to pass through. As a result, Noction IRP allows you to:

• Improve overall network performance

• Reroute congestion and outages

6
CHAPTER 1. INTRODUCTION 7

• Decrease network downtime

• Reduce latency and packet loss
• Get comprehensive network performance analytics
• Facilitate network troubleshooting
• Decrease network operational costs
• Monitor platform performance
• Reduce the risk of human errors during BGP configuration

1.2.1.1 IRP Lite limitations

IRP Lite is designed as a promotional and educational tool that might also be useful to optimize some
smaller networks that are not encumbered by IRP Lite limitations. IRP Lite use is governed by Terms
of Service that constrain further how IRP Lite can be used.
The high level list of technical limitations in IRP Lite include:

• Support for Internet Exchanges is missing

• Commit Control features are missing
• IRP API and consequently Frontend reports and graphs with the exception of Current Improve-
ments return only sample data
• At most 750 improvements, 3 transit providers and 3 policies are supported.

IRP Lite periodically sends event notifications to a designated @noction.com email address. The data
is collected for statistics purposes and is used to enhance this and other Noction services like outage
detection and confirmation and Internet health reports.

1.2.2 IRP Components

The IRP platform has a few interconnected components (see figure 1.2.1 on the following page) , which
are performing together to improve the routing best path selection and to eliminate various common
network issues.
A short description of each of the components is given below. The detailed information is available in
the next chapters.
The Core is the most important part of the system. It runs all the logical operations and interconnects
all the components. It handles the performance and cost improvements. It processes, stores and exports
data.
The Collector receives, analyzes and processes all the traffic passing the network. It has two ways to
gather data about the network: by mirroring network traffic or by using NetFlow/sFlow.
The Explorer runs all the probes and checks the metrics specified by the platform policies, such as
packet loss and latency. This information is sent back to the Core.
To inject the Improvements into the network, the platform needs to ‘tell’ the routers what exactly needs
to be changed in the routing table. The IRP BGP daemon announces the improved prefix with an
updated next-hop for the traffic to start flowing through the new path.
The Frontend represents a web interface with a comprehensive set of reports, graphs and diagnostic
information which can reflect the current and historical network state, as well as the benefits of IRP
network optimization.

1.2.3 IRP Technical Requirements

In order to plan the IRP deployment in your network, a series of requirements need to be met and specific
information has to be determined to configure IRP.
CHAPTER 1. INTRODUCTION 8

Figure 1.2.1: IRP Components

1.2.3.1 Hardware requirements

In production, a dedicated server for each IRP instance is strongly recommended. The system can
also be deployed on a Virtual Machine with matching specifications, provided that this is hardware-
or paravirtualization (Xen, KVM, VMware). Os-level virtualization (OpenVZ/Virtuozzo or similar)
is not supported.

1. CPU

• Recommended Intel® Xeon® Processor E3/E5 family, for example:

– 1x Intel® Xeon® Processor E3 family for up to 20 Gbps traffic;
– 1x Intel® Xeon® Processor E5 family for 40 Gbps or more traffic.

2. RAM

• if providing sFlow/NetFlow data at least 16 GB, recommended - 32 GB;

• if providing raw traffic data by port mirroring:
– minimum 16 GB for up to 10 Gbps traffic;
– minimum 32 GB for 40 Gbps traffic.

3. HDD

• At least 160GB of storage. Allocate at least 100GB to /var partition. SAS disks are recom-
mended for production environments. LVM setup is preferrable.
CHAPTER 1. INTRODUCTION 9

4. NIC

• if providing sFlow/NetFlow data - at least 1 x 1000Mbps NIC while two NICs are recom-
mended (one will be dedicated to management purposes).
• if providing raw traffic data by port mirroring - additional 10G interfaces are required for
each of the configured SPAN ports (Myricom 10G network cards with Sniffer10G license are
recommended to be used for high pps networks). When configuring multiple SPAN ports the
same number of additional CPU cores are needed to analyze traffic.

In very large networks carrying hundreds or more of Gbps of traffic and in IRP configurations
with very aggressive optimization settings configurations with 2x or 4x CPUs are recommended. The
IRP servers in these cases should also allocate double the recommended RAM and use SSD storage.

Noction can size, setup and mail to you an appliance conforming to your needs. The appliance is delivered
with OS installed (latest CentOS 7) and IRP software deployed.

1.2.3.2 Software requirements

Clean Linux system, with the latest CentOS 7 x86_64 version installed on the server.

IRP has a dependency on MariaDB (CentOS 7) and it expects the latest version from Official
CentOS Repositories. In case the DBMS has been installed from a different repository it is strongly
advised that the database instance and its configuration is purged before proceeding with IRP
installation.
IRP requires root access to local database instance during first installation. In case the root access
can’t be given, use the statements below to grant all necessary privileges to the ’irp’ user and
database.:
GRANT ALL ON $dbdbname.* TO ’$dbusername’@’$dbourhost’ IDENTIFIED BY
’$dbpassword’ WITH GRANT OPTION
GRANT SELECT ON $dbdbname_fe.* TO ’$dbusername_fe’@’$dbourhost’ IDENTI-
FIED BY ’$dbpassword_fe’
where $dbdbname (4.1.1), $dbusername (4.1.1), $dbpassword (4.1.1), $dbourhost
(4.1.1) are the corresponding parameters from /etc/noction/db.global.conf
$dbdbname_fe and $dbusername_fe from /etc/noction/db.frontend.conf

IRP’s Frontend is built on PHP 5.6 and Symfony 2.8 framework. These are not currently part of
mainstream CentOS . During installation of IRP validated packages for PHP 5.6 and Symfony 2.8
are retrieved from Noction’s repositories.

1.2.3.3 Network-related information and configuration

IRP is designed to help Service Providers (AS) optimize a multi-homed BGP network. This implies the
basic prerequisites for using IRP:

• Ownership of the AS for the network where IRP is deployed,

• BGP protocol is used for routing and,

• Network is multi-homed.

Eventually the following needs to be performed in order to deploy and configure IRP:
CHAPTER 1. INTRODUCTION 10

1. Prepare a network diagram with all the horizontal (own) as well as upstream (providers) and
downstream (customers) routers included. Compare if your network topology is logically similar
to one or more of the samples listed in section Collector Configuration for example Flow export
configuration.
2. Identify the list of prefixes announced by your AS that must be analyzed and optimized by IRP.
3. Review the output of commands below (or similar) from all Edge Routers:

• ‘sh ip bgp summary‘

• ‘sh ip bgp neighbor [neighboor-address] received-routes‘
• ‘sh ru‘ (or similar)

The settings relating to BGP configuration, prefixes announced by your ASN, the route maps,
routing policies, access control list, sFlow/NetFlow and related interfaces configurations are used
to setup similar IRP settings or to determine what settings do not conflict with existing network
policies.

4. Provide traffic data by:

(a) sFlow, NetFlow (v1, 5, 9) or jFlow and send it to the main server IP. Make sure the IRP
server gets both inbound and outbound traffic info.
Egress flow accounting should be enabled on the provider links, or, if this is not technically
possible, ingress flow accounting should be enabled on all the interfaces facing the internal
network.
NetFlow is most suitable for high traffic volumes, or in the case of a sophisticated network
infrastructure, where port mirroring is not technically possible.
Recommended sampling rates:
i. For traffic up to 1Gbps: 1024
ii. For traffic up to 10Gbps: 2048
(b) Or: configure port mirroring (a partial traffic copy will suffice). In this case, additional
network interfaces on the server will be required - one for each mirrored port.

See also: Collector Configuration

5. Setup Policy Based Routing (PBR) for IRP active probing.

• Apart from the main server IP, please add an additional alias IP for each provider and configure
PBR for traffic originating from each of these IPs to be routed over different providers.
• No route maps should be enforced for the main server IP, traffic originating from it should
pass the routers using the default routes.
• Define Provider↔PBR IP routing map

In specific complex scenarios, traffic from the IRP server should pass multiple routers before getting
to the provider. If a separate probing Vlan cannot be configured across all routers, GRE tunnels
from IRP to the Edge routers should be configured. The tunnels are mainly used to prevent
additional overhead from route maps configured on the whole IRP←→Edge routers path.
6. Configure and provide SNMP (a read-only community will suffice) for each provider link, and
provide the following information:

• SNMP interface name (or ifIndex)

• SNMP IP (usually the router IP)
• SNMP community

This information is required for the report generation, Commit Control decision-making and pre-
vention of overloading a specific provider with an excessive number of improvements.
CHAPTER 1. INTRODUCTION 11

The above is applicable in case of SNMP v2c. If SNMP v3 is used further details will be required
depending on security services used.

1. To setup cost related settings as well as the Commit Control mechanism, provide the maximum
allowed interface throughput for each provider link as well as the cost per Mbps for each provider.
CHAPTER 1. INTRODUCTION 12

1.2.4 IRP Operating modes

The IRP platform can operate in two modes, which can be used at different stages of the deployment
process. During the initial installation and configuration, it is recommended for the system not to inject
any improvements into the network until the configuration is completed.
After running several route propagation tests, the system can be switched to the full Intrusive mode.

1.2.4.1 Non-intrusive mode

While running in this mode, the system will not actually advertise any improvement to the network, and
will only reflect the network improvements and events in the platform reports and graphs.

1.2.4.2 Intrusive mode

After the system configuration is completed, and manual route propagation tests were performed in order
to ensure that the edge routers behavior is correct, the system can be switched to Intrusive mode. While
running in this mode, the system injects all the computed improvements into the edge router(s) routing
tables, allowing the traffic to flow through the best performing route.

1.2.4.3 Going Intrusive

While IRP operates in non-intrusive mode it highlights the potential improvements within client’s envi-
ronment. Going Intrusive will realize IRP potential.
The difference between Intrusive and Non-Intrusive operating modes is that IRP advertises improvements
to edge routers. In order to switch to Intrusive we follow a controlled process. The highlights of the
process are as follows:

1. The optimizing component of IRP (Core) is taken offline and existing improvements are purged.
The Core being offline guarantees IRP will not automatically insert new improvements into its
Current Improvement table and hinder the Go Intrusive process.

Listing 1.1: Stop IRP Core and purge existing improvements

root@server ~ $ service core stop
root@server ~ $ mysql irp -e ’delete from improvements;’

2. Enable Intrusive Mode and adjust IRP Core and BGPd parameters as follows:

Listing 1.2: Switch to Intrusive Mode and adjust IRP Core and BGPd parameters
root@server ~ $ nano /etc/noction/irp.conf
global.nonintrusive_bgp = 0
core.improvements.max = 100
bgpd.improvements.remove.next_hop_eq = 0
bgpd.improvements.remove.withdraw = 0

3. Improvements towards test networks are introduced manually so that client’s traffic is not affected.
The improvements are chosen so that they cover all client’s providers. Any public networks could
be used for test purposes. Just keep in mind that preferably your network shouldn’t have traffic
with chosen test network in order to do not re-route the real traffic. Use the template below in
order to insert the test improvements:

Listing 1.3: Inserting test improvements

mysql> insert into improvements
(ni_bgp, prefix, peer_new, ipv6, asn)
CHAPTER 1. INTRODUCTION 13

values
(0, ’10.10.10.0/24’, 1, 0, 48232),
(0, ’10.10.11.0/24’, 2, 0, 48232),
(0, ’10.10.12.0/24’, 3, 0, 48232);

4. Make sure that ’route-reflector-client’ is set for IRP BGP session.

5. Make sure that ’next-hop-self’ is not configured for IRP BGP session.
6. On iBGP sessions (between edge routers, route-reflectors; except session with IRP) where ’next-
hop-self’ is configured, the following route-map should be applied:

Listing 1.4: Remove next-hop-self route-map (RM-NHS) example

route-map RM-NHS
set ip next-hop peer-address
neighbor X.X.X.X route-map out RM-NHS

where X.X.X.X is the iBGP neighbor

route-map contents should be integrated into existing route-map in case other route-map
already configured on the iBGP session.

7. Use the commands below to restart IRP BGPd to use actual IRP configuration and establish BGP
session(s) and verify if BGP updates are being announced:

Listing 1.5: Restart IRP BGPd

root@server ~ $ service bgpd restart
root@server ~ $ tail -f /var/log/irp/bgpd.conf

Wait for the following lines for each BGP session:

NOTICE: Adding peer X
NOTICE: BGP session established X
INFO: N update(s) were sent to peer X

where X is the router name and N is the number of the updates sent towards the X router.
8. Verify if IRP BGP announcements are properly propagated across all the network. Run the fol-
lowing commands on each router (the commands vary depends on the router brand):

Listing 1.6: Show BGP information for specified IP address or prefix

show ip bgp 10.10.10.1
show ip bgp 10.10.11.1
show ip bgp 10.10.12.1

Analyze the output from all the routers. If the IRP BGP announcements are properly propagated,
you should see /25 (refer to 4.1.4.16) announcements and the next-hop for each announcement
should be the improved provider’s next-hop:
10.10.10.1 - provider 1 next-hop
10.10.11.1 - provider 2 next-hop
10.10.12.1 - provider 3 next-hop
(refer to 4.1.11.25, 4.1.11.32, 4.1.14.3).

Run the following commands in order to check if IRP improvements are announced and applied:
CHAPTER 1. INTRODUCTION 14

Listing 1.7: Traceroute destination networks

root@server ~ $ traceroute -nn 10.10.10.1
root@server ~ $ traceroute -nn 10.10.11.1
root@server ~ $ traceroute -nn 10.10.12.1

Again, you should see corresponding providers’ next-hops in the traces.

9. If the tests are successful perform the steps below:

(a) Delete test improvements

Listing 1.8: Delete test improvements

root@server ~ $ mysql -e "delete from improvements where prefix
like ’10.10.1%’;"

(b) Configure at most 100 improvements and revert BGPd configuration

Listing 1.9: Configure the maximum improvements limit and revert BGPd configuration
root@server ~ $ nano /etc/noction/irp.conf
core.improvements.max = 100
bgpd.improvements.remove.next_hop_eq = 1
bgpd.improvements.remove.withdraw = 1

(c) Restart IRP Core and BGPd

Listing 1.10: Restart IRP Core and BGPd

root@server ~ $ service core restart
root@server ~ $ service bgpd restart

10. If everything goes well, after 1-2 hours the maximum number of improvements announced is in-
creased to 1000 and after 24 hour to 10000.

As a rollback plan we have to revert the changes and switch the system to non-intrusive mode:

1. Delete test improvements

Listing 1.11: Delete test improvements

root@server ~ $ mysql -e "delete from improvements where prefix like
’10.10.1%’;"

2. Switch the system to non-intrusive mode

Listing 1.12: Switch the system to non-intrusive mode

root@server ~ $ nano /etc/noction/irp.conf
global.nonintrusive_bgp = 1

3. Restart IRP Core and BGPd

Listing 1.13: Restart IRP Core and BGPd

root@server ~ $ service core restart
root@server ~ $ service bgpd restart
CHAPTER 1. INTRODUCTION 15

1.2.5 BGP Monitoring

IRP uses two types of BGP monitors to diagnose and report the state of the BGP session between the
edge routers and the providers, as well as the network reachability through a specific provider. The
information provided by these monitors enables IRP to avoid announcing routing updates that would
result in traffic misrouting (sending improvements to a failed provider).
Internal BGP Monitor is checking the state of the Edge Router→Provider BGP session by regularly
polling the router via SNMP. When queried, the SNMP protocol returns variables describing the session
status to be used by the IRP’s Internal BGP Monitor. If the session between the edge router and the
provider is down, SNMP will return a value, representing session failure and IRP will react as follows:

• the provider will be marked as FAILED,

• all the improvements towards this provider will be withdrawn from the routing tables to avoid
creating black holes,
• new improvements towards this providers will not be made.

In some cases (e.g. DDoS attack or various factors causing router CPU over-usage) there may be no re-
sponse to the SNMP queries at all. In this case a timeout status will be reported to the Internal Monitor
and a 30 minutes timer (called longhold timer) (bgpd.mon.longholdtime) will be started. During this time
the monitor will be sending ICMP/UDP ping requests toward the configured provider’s next-hop IP ad-
dress (peer.X.ipv4.next_hop or peer.X.ipv6.next_hop). The requests will be sent once in keepalive period
(a parameter adjustable in the BGP daemon configuration interface) (bgpd.mon.keepalive). If the next-
hop stops responding to these requests, another 30 seconds timer (called hold timer) (bgpd.mon.holdtime)
will be started. If according to the ping response the session is reestablished during this time, the hold
timer will be discarded while the longhold timer continues. In case one of the timers expires, the provider
is switched to a FAIL state and all the improvements towards this provider will be withdrawn from the
routing table. However, if the BGP session with the provider is re-established, the system will start
rerouting traffic to this provider.
When the BGPd is started, the monitors are initialized and one SNMP query is sent towards each router,
in order to check the status of the BGP sessions with providers. If there is no reply, the Internal Monitor
will send up to two more SNMP requests, separated by a keepalive interval.

Internal monitors for Internet Exchange peering partners are not initialized until there is an
improvement made towards it. When running in non-intrusive mode internal monitors for IX peers
are not initialized at all.

If none of the SNMP queries returned a status confirming that the sessions with providers are up, the
provider will be assigned a FAIL status and the Internal Monitor will continue the periodical SNMP
polling (each 60 seconds), to recheck providers sessions’ status.
Then, the BGP session with the edge routers is initialized and BGPd starts retrieving the routing table
from the edge routers. While IRP retrieves the routing table, SNMP request may timeout due to the
high CPU usage on the edge routers.
External BGP Monitor analyzes the network reachability through a specific provider. It performs
ICMP/UDP ping requests towards the configured remote IP address(es) (peer.X.ipv4.mon or peer.X.ipv6.mon)
through the monitored provider. If any of the configured IP addresses are accessible, the monitor is
marked as OK. If the monitored remote IP addresses do not reply through the examined provider IRP
will react as follows:

• the provider will be marked as FAILED,

• all the improvements towards this provider will be withdrawn from the routing table,
• new improvements towards this providers will not be made.

If for some reason (e.g. when the provider’s interface goes down state), the Next-Hop of the Policy
Based Routing rule does not exist in the routing table, then the packets forwarding may return to the
CHAPTER 1. INTRODUCTION 16

default route. In that case, the External BGP Monitor will return a false-positive state. To avoid
that by properly configuring PBR, please consult “Specific PBR configuration scenarios” (Specific PBR
configuration scenarios).
The External BGP Monitor status does not depend on the state of the BGP session(s) between the edge
router and the provider (which is monitored by the Internal BGP Monitor). Therefore, in the case that
the BGP session with one of the providers goes down, the External Monitor still shows an OK state which
will remain unchanged as long as the packets are successfully routed towards the monitored destination.
We do recommend adding at least two remote IP addresses, in order to prevent false-positive alerts.
When both BGP monitors are enabled (peer.X.mon.enabled), they function in conjunction with each
other. If any of them fails, the provider will be declared as FAILED and IRP will react as described
above. The BGP monitors’ statuses are displayed on the system dashboard as shown in the screenshot
below.

Figure 1.2.2: System Dashboard

Note: Starting with version 1.8.5, IRP requires at least the Internal Monitor to be configured. Otherwise,
the system Frontend will return an error as shown below.

Figure 1.2.3: Error: Internal BGP Monitor not configured

For future reference, please check:

peer.X.mon.snmp.ip
peer.X.mon.snmp.ipv6
peer.X.mon.snmp.community
bgpd.mon.guardtime
bgpd.mon.keepalive
CHAPTER 1. INTRODUCTION 17

bgpd.mon.holdtime
bgpd.mon.longholdtime
peer.X.ipv4.mon
peer.X.ipv6.mon
peer.X.mon.ipv4.bgp_peer
peer.X.mon.ipv6.bgp_peer

1.2.6 Outage detection

A complete traffic path from source to destination typically passes through multiple networks, with
different AS numbers. This is reflected in the traceroute results. If the Outage detection is enabled
in the IRP configuration, the system gathers network performance information for all traceroute hops
over which the traffic passes to the remote networks. Next, each hop is translated into AS numbers. In
case any network anomalies are detected on a specific ASN, then this ASN and the immediate neighbor
ASN are declared a problematic AS-pattern. The system then re-probes the prefixes that pass through
this AS-pattern. In case the issue is confirmed, all related prefixes are rerouted to the best performing
alternate provider.
The Outage detection uses a statistical algorithm for selecting the best routing path. Rerouting will
occur for all the prefixes that are routed through the affected as-pattern, despite their current route .

Several improvements-related reports need to indicate the original route for a specific prefix. This
value is taken from the last probing results for this prefix, even if the results are outdated (but not
older than 24h). Since the outage-affected prefixes are rerouted in bulk by as-pattern, in some cases
the reports can show the same provider for both the old and the new route.

1.2.7 VIP Improvements

The VIP Improvements is a feature that allows manual specification of a list of prefixes or AS numbers
that will be periodically probed by IRP and optimized in compliance with the probing results. This
allows the system to monitor specific networks or Autonomous Systems, without reference to the data
provided by the IRP collector.
Possible usage scenarios include, but are not limited to:

• monitoring and optimizing traffic to commercial partners that should have access to your networks
via the best performing routes

• monitoring and optimizing traffic to your remote locations, operating as separate networks
• monitoring and optimizing traffic to AS, which are known for the frequent accessibility issues due
to geographical or technical reasons

If a prefix is being announced from multiple Autonomous Systems, you can see different ASNs in
Current improvements report in the prefixes translated from ASN

IRP performs the proactive (more frequent than regular probing) monitoring of the VIP prefixes/ASNs
that allows VIPs to be constantly improved.
For future reference, see:
core.vip.interval.probe

1.2.8 Retry Probing

Retry Probing is a feature that allows reconfirmation of the initial and already reconfirmed improvements
validity. The feature is applicable to all types of improvements made by the system (Performance, Cost
and Commit Control improvements). The improvements that were made more than a retry probing
CHAPTER 1. INTRODUCTION 18

period ago (core.improvements.ttl.retry_probe) are being sent to retry probing. If the probing results
confirm that the current improvement is still valid, it stays in the system and its description is updated.
Otherwise, it will be removed with the log message further described in this section.
During Retry Probing reconfirmation the improvement details will be updated in the following cases:

• Performance and Cost improvements

– An old provider has been removed from the system configuration.

Example: “Old provider and performance metrics not known. New packet loss 55%, avg rtt
105 ms.”

• Commit Control improvements

– An old provider’s has been removed from the system configuration.

Example: “Previous provider not known. Rerouted 1 Mbps to Peer5[5] (250 Mbps, 50%)”
– An old provider’s bandwidth statistics are not available.
Example: “Rerouted 6 Mbps from Peer1[1] to Peer5[5] (250 Mbps, 50%)”
– A new provider’s bandwidth statistics are not available.
Example: “Rerouted 6 Mbps from Peer1[1] (250 Mbps, 50%) to Peer5[5]”
– The old and new providers’ bandwidth statistics are not available.
Example: “Rerouted 6 Mbps from Peer1[1] to Peer5[5]”

Commit control improvements are reconfirmed based on their average bandwidth usage (and
not on current bandwidth usage). This way if performance characteristics allow it, even when
current bandwidth usage is low but the average is still relevant, the improvement is preserved thus
anticipating network usage cycles and reducing number of route changes.

During Retry Probing reconfirmation the improvements will be removed from the system and the details
will be logged into the log file (core.log) in the following cases:

• The Commit Control feature has been disabled.

Example: “Prefix 1.0.2.0/24 withdrawn from Improvements (Commit Control is disabled)“
• The low prefix traffic volume is less than the configured bandwidth limits (core.commit_control.agg_bw_min).
Example: “Prefix 1.0.2.0/24 withdrawn from Improvements (low traffic volume, irrelevant for the
Commit Control algorithm)”
• The system has been switched from the Cost mode to the Performance mode (applied for cost
improvements only).
Example: “Prefix 1.0.2.0/24 withdrawn from Improvements (Performance Improvements mode)”
• A prefix has been added to the ignored networks/ASN list.
Example: “Prefix 1.0.2.0/24 withdrawn from Improvements (added to ignored networks/ASN)”

• The improvement’s performance metrics are not the best ones anymore.
Example: “Prefix 1.0.2.0/24 withdrawn from Improvements (Performance Improvement not actual
anymore)”
• The maximum number of improvements limits (core.improvements.max, core.improvements.max_ipv6)
are exceeded.
Example: “Prefix 1.0.2.0/24 withdrawn from Improvements (no more available Improvement slots)”

For future references, see:

core.eventqueuelimit.retry_probe_pct
core.improvements.retry_probe.volume_top_n
core.improvements.ttl.retry_probe
CHAPTER 1. INTRODUCTION 19

1.2.9 Routing Policies

Routing Policies brings the capability of defining specific routing policies according to business objec-
tives. This feature permits denying or allowing providers to be used for probing and reaching a specific
prefix or ASN. It also provides the possibility to set a static route through a particular provider. Within
a policy you can choose between VIP or non-VIP (regular) probing mechanisms to be used.

Below you can find some typical scenarios of Routing Policies usage. For instance, there may be a
specific prefix that consumes a high volume of traffic and IRP redirects it through the most expensive
provider due to performance considerations. At the same time you have another two less costly available
providers. In this case, you could deny the expensive provider for the specific prefix and IRP will choose
the best-performing provider between the remaining two. This can be achieved by applying a Deny
policy to the expensive provider or an Allow policy to the less costly providers.

The table below shows which cases a specific policy can be applied in, depending on the number of
providers.
No. of providers \ Policy Allow Deny Static
1 provider No Yes Yes
2 providers or more (maximum: total Yes Yes No
number of providers - 1)
All providers (VIP probing disabled) No No No
All providers (VIP probing enabled) Yes No No
If a Static Route policy is applied to a prefix, VIP probing through each of the providers is unnecessary.
Regular probing will suffice for detection of a provider failure that would trigger IRP to reroute the
traffic to a different provider. Therefore IRP does not allow using VIP probing within a Static Route
policy.

Routing policies are designed to control outgoing paths for destination networks that are outside
your infrastructure. This feature should not be used to manipulate your infrastructure network
behavior.

Avoid setting up of policies that point to same prefix. When improvement by aggregate is enabled,
multiple prefixes can point to the same aggregate and this can cause unpredictable behaviour.

If a provider is added or removed, suspended or shutdown, the routing policies are adjusted by IRP in
the following way:

A new provider is added.

Policy Result
Allow all providers (VIP probing The new provider is automatically included into the
enabled) configured policy and probed by the VIP probing
mechanism.
Allow selected providers only The new provider is automatically ignored by the configured
policy and not probed by the probing mechanism.
Deny The new provider is automatically included into the
configured policy and probed by the selected probing
mechanism.
Static Route The new provider is automatically ignored by the configured
policy and probed by the Regular probing mechanism.

The provider under the policy is removed.

CHAPTER 1. INTRODUCTION 20

Policy Result
The new provider is automatically removed from the
configured policy and not probed by the VIP probing
Allow all providers (VIP probing
mechanism. If there is only one provider left, the policy is
enabled)
automatically deactivated.
The provider is automatically removed from the configured
policy and not probed by the probing mechanism. If there is
Allow selected providers only
only one provider left, the policy is automatically
deactivated.
The provider is automatically removed from the configured
policy and not proved by the probing mechanism. If there is
Deny
no any other provider under this policy, it is automatically
deactivated.
The provider is automatically not probed by the probing
Static Route
mechanism. The policy is automatically deactivated.
The provider under the policy is suspended.
Policy Result
Allow all providers (VIP probing The provider is temporarily removed from the configured
enabled) policy and not probed by the VIP probing mechanism. If
there is only one provider left, the policy is temporarily
deactivated.
Allow selected providers only The provider is temporarily removed from the configured
policy and not probed by the probing mechanism. If there is
only one provider left, the policy is temporarily deactivated.
Deny The provider is temporarily removed from the configured
policy and not probed by the probing mechanism. If there is
no any other provider under this policy, it is temporarily
deactivated.
Static Route The provider is temporarily not probed by the probing
mechanism. The policy is temporarily deactivated.
The provider under the policy is shutdown.
Policy Result
Allow all providers (VIP probing The provider is temporarily removed from the configured
enabled) policy and not probed by the VIP probing mechanism. If
there is only one provider left, the policy is temporarily
deactivated.
Allow selected providers only The provider is temporarily removed from the configured
policy and not probed by the probing mechanism. If there is
only one provider left, the policy is temporarily deactivated.
Deny The provider is temporarily removed from the configured
policy and not probed by the probing mechanism. If there is
no any other provider under this policy, it is temporarily
deactivated.
Static Route The provider is temporarily not probed by the probing
mechanism. The policy is temporarily deactivated.
Policies that target an AS can be cascaded. Cascading applies the same policy to AS that are downstream
to target AS, i.e. to AS that are transited by target AS.

The use case of cascading is to apply a policy to a remote AS that transits a few other AS. Still,
a cascading policy can cover a huge number of down-streams. This number is parameterized and
can be set to values that best fit customer’s needs. Refer for example 4.1.4.8.

All Routing Policies are stored in /etc/noction/policies.conf file, which is automatically generated by the
system.
CHAPTER 1. INTRODUCTION 21

Do not alter the /etc/noction/policies.conf file manually because any modifications will be over-
written by the system. Any changes can only be performed from the Configuration -> Routing
Policies section in the system Frontend.

The rule will be ignored by the platform if the Routing Policy contains a syntax/logical error.

Please check (4.1.13) for a detailed description of Routing Policies parameters.

1.2.10 Support for Centralized Route Reflectors

Figure 1.2.4: Support for Centralized Route Reflectors

IRP gives the possibility to advertise routes into one or more route reflectors which subsequently advertise
improvements into upper and/or lower routing layers (such as edge, core or distribution).
In case the iBGP sessions can’t be established between the IRP appliance and edge routers, a route
reflector is used. The following restrictions apply for such a solution:

• The Next-Hop-Self option should not be used. A direct iBGP session is required between IRP
and each of the divergence points (Reflector1, Edge2) in case it is enabled. This restriction is not
applicable between reflectors and core/distribution layers.

• Next-Hop addresses should be accessible (exist in the routing table) where next-hop-self is not
applied (Either static routes or IGP is used).
• An Internal Monitor should be configured to retrieve the eBGP session state from the device
where the corresponding eBGP session is terminated. For example, ISP1 should be monitored on
Reflector1, ISP2 on Edge1, and ISP3 and ISP4 on Edge2.

• Injecting routes to reflector(s) can cause temporary routing loops.

In order to announce improvements into route reflector, it should be configured as a BGP router in the
“Configuration”→”BGP and routers” section and should be assigned to all the related providers in the
“Configuration”→”Providers and Peers” section.
CHAPTER 1. INTRODUCTION 22

1.2.11 Support for Internet Exchanges

A transit provider can deliver traffic to any destination on the Internet. However, within an Internet
Exchange, a peering partner gives access only to the set of prefixes originated or transiting its network.
Therefore, when IRP evaluates the Exchange as a best path, it has to know the prefixes announced by
each peer, to avoid inefficient probing of paths that cannot lead to the desired destination.
With this purpose, IRP gets the routing table from the edge router containing the list of IPs and
the corresponding next-hop; this represents the next router’s IP address to which a packet is sent as it
traverses a network on its journey to the final destination. IRP matches the prefix with the corresponding
next-hop among the configured peers, allowing it to select for probing only those peers that have access
to a specific prefix. This process is also performed in the case of a transit provider that gives access only
to a limited set of prefixes, rather than the entire Internet.

Figure 1.2.5: IRP configuration in a multi-homed network connected to transit providers as well as and
Internet Exchange

In the case of multiple transit providers, there is an additional IP alias added on the IRP platform for
each provider. The edge router is configured in such a way that traffic originating from each of these IPs
is routed over different providers. This is done with the help of Policy Based Routing (PBR).
With PBR, a network engineer has the ability to dictate the routing behavior based on a number of
different criteria other than the destination network. These PBR rules are applied to make sure that
IRP probes are following the desired paths. However, when it comes to Internet Exchanges, configuring
hundreds of IP aliases on the platform would result in inefficient IP address usage and an unmanageable
setup.
To avoid this, a set of PBR rules are applied making sure that the probes to be sent through a specific
provider are originating from one of the configured IPs with a specific DSCP code assigned. DSCP -
Differentiated Services Code Point - is a field in an IP packet that enables different levels of service to
be assigned to network traffic. Since DSCP can take up to 64 different values, one configured IP can be
associated with up to 64 peers. Although, due to this mechanism, the number of required IP addresses
for aliases to be configured has decreased considerably, hard work would still be needed to configure the
PBR on the edge router as described above.
To solve this, IRP implemented a built-in PBR config-generator which provides the configuration code to
be used for a specific router model. By running this generated set of commands, network administrators
can easily configure the required PBR rules on the router.

1.2.12 Optimization for multiple Routing Domains

Overview
Some networks have multiple Points of Presence interconnected both internally via inter-datacenter
CHAPTER 1. INTRODUCTION 23

links and externally via multiple transit providers. The diagram below depicts an example diagram with
the available routes to one destination on the Internet.
IRP uses the concept of Routing Domains to separate the locations. A Routing Domain’s main charac-
teristic is that its routing tables are mainly built on data received from its locally connected providers
and the preferred routes are based on locally defined preferences.
The process of optimizing outbound network traffic in such a configuration is to mainly find better
alternative routes locally (within a Routing Domain) and only reroute the traffic to other Routing
Domains via inter-datacenter links when local routes are completely underperforming.
It must be noted that a multiple Routing Domain configuration works best if the Points of Presence are
not too far away (ex. a network with POPs in San Francisco, Palo Alto and Danville is perfectly suitable
under this scenario.

Figure 1.2.6: City wide network

POPs situated at larger distances, for example in Las Vegas and Salt Lake City are still supported by a
single IRP instance running in San Francisco.

Figure 1.2.7: Regional network

Intercontinental links for POPs in Tokyo and Melbourne are way too far away from the IRP instance in
San Francisco and in such a case multiple IRP instances are required.
CHAPTER 1. INTRODUCTION 24

Figure 1.2.8: Intercontinental network

Multiple Routing Domains implementation attributes To further detail the multiple routing
domain attributes the following diagram will be used:

Figure 1.2.9: Multiple routing domains

A multiple Routing Domain configuration has a series of attributes:

CHAPTER 1. INTRODUCTION 25

• Multiple locations belonging to the same network (AS) represented in the diagram by POP SJC,
POP SFO and POP LAX (of course, more than 3 routing domains are supported).
• The locations are distinguished by the different Routing Domains within which they operate (de-
picted by RD SJC, RD SFO, and RD LAX)

• The Routing Domains are managed by edge routers belonging to different locations
• Nearby locations that process routing data differently should be split into different Routing Do-
mains, even if they have the same upstream providers. In the diagram above RD SFO and RD
SFO’ are depicted as part of a single Routing Domain. A decision to split or keep in the same
routing domain should be made based on exact knowledge on how routing data is processed.
• Inter-datacenter loop interconnects the different locations (depicted by idc1, idc2 and idc3 seg-
ments)
• Data flows between locations take only the short path (in the example POP SJC can be reached
from POP SFO via idc2 path (short) or idc3 + idc1 path (long))

• Each Routing Domain has different providers and different preferred routes to reach a specific
destination (a1, b1, c1)
• A single IRP instance collects statistics about traffic (Irpflowd only), probes available destinations
and makes improvements towards specific prefixes/networks on the Internet.

• IRP assumes RTT of zero and unlimited capacity to route traffic within a Routing Domain
• IRP assumes that Sites are not physically too far away. It is ok to have different sites in the same
city or region as at this scale inter-datacenter links have predictable characteristics. When taking
intercontinental links into consideration this is quite probably not the case.

• Distances between sites (idc1, idc2, idc3 delays) are measured in advance and specified in IRP’s
configuration.

Inter-datacenter link characteristics Support for Multiple Routing Domains relies on existence
of inter-datacenter links. These links should be independent of upstream providers.
Example of inter-datacenter links that multiple routing domains is designed for are:

• private connections,
• L2 links with guaranteed service,
• MPLS links

VPNs via public Internet could be used with Multi Routing Domain feature but is a suboptimal
choice. Under such conditions IRP MUST be prevented from making Global Improvements. This way
IRP will do only local optimizations in each Routing Domain and will operate similarly to multiple
IRP instances (while probing excessively because it probes destinations via remote locations too).

Constraints At the moment IRP multiple Routing Domains implementation does not cover the fol-
lowing:

• IRP does not take measurements of inter-datacenter link delays (idc1, idc2 and idc3). This values
are configurable.
• IRP does not monitor if inter-datacenter links are operating normally. In case such a link is broken
it is expected IRP to loose BGP connectivity with routing domain routers and this will cause IRP
improvements to be withdrawn till the link is restored.
CHAPTER 1. INTRODUCTION 26

• IRP does not try to detect if the traffic is following long or short paths on the inter-datacenter
links. In the image above traffic from RD SJC can follow path idc1 (short) or idc2+idc3 (long).
IRP always assumes the short path is being followed internally.
• IRP does not take measurements of inter-datacenter link capacity and current bandwidth usage.
At this stage IRP assumes there is enough inter-datacenter link capacity to also carry the (few)
global improvements. Also, IRP tries to minimize usage of inter-datacenter links.

Routing domains
Routing domain is a generic term used to distinguish a logical location that works with different
routing tables. The differences are caused by the fact that a router composes its routing table according
to routes received from different providers. It is possible to have multiple routing domains in the same
datacenter if routing data is received by different routers (even from same or different sources) and data
flows are distributed via different routers by different policies. In the image above RD SFO and RD SFO’
can represent a single routing domain or multiple routing domains depending on what routing policies
are applied.
Different routing domains are assigned identifiers in the range 1-100. Routing Domain identifier is
assigned individually to each provider via parameter peer.X.rd. It must be noted the Routing domain
that hosts the IRP instance is considered as the first routing domain (RD=1).
Parameter global.rd_rtt gives the distances between routing domains. The format of the parameter is

rda:rdb:rtt

for example if RD SJC has Routing Domain id = 42, RD SFO - 1 (since it hosts IRP), RD LAX - 3 then
the idc1, idc2 and idc3 rtt is defined as the collection:

global.rd_rtt = 3:42:20 42:1:17 1:3:35

This parameter will be validated for correctness and besides the format above it requires that RD SJC
and RD SFO values are different and already configured (RD1 is always present).

Round trip time between one routing domain and another is calculated by executing PING
towards edge routers and taking average integer value:

$ ping X -c 10 -q
PING X (X) 56(84) bytes of data.
--- X ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9085ms
rtt min/avg/max/mdev = 40.881/41.130/41.308/0.172 ms

Flow agents
A very natural constraint for Multiple Routing Domain networks is that IRP can rely only on Flow
statistics - NetFlow or sFlow.

SPAN cannot be used because it does not carry attributes to distinguish traffic between different
providers

In big network quite often multiple physical interfaces are used to carry traffic for a single provider.
Flow collector need to know the exact details of such a configuration in order to correctly determine the
overall provider volume and active flows characteristics. The parameter is given in the form of

IPv4/interfaceID
CHAPTER 1. INTRODUCTION 27

where IPv4 is the source IP address of the packets coming from the agent and interfaceID is a numerical
identifies of the interface in the range 1- 4294967295. Interface ID usually represents SNMP Interface ID
from a subsequent router. A collection of such values is given if multiple physical interfaces are used for
example:

peer.X.flow_agents = 8.8.8.8/1 8.8.8.8/2 8.8.8.8/3 8.8.8.8/4

The value is set via parameter peer.X.flow_agents that is also available in Providers and Peers configu-
ration labeled "Flow agents with outgoing interfaces". Frontend will also retrieve and suggest a list of
available values for the provider.

Global and local improvements

Local improvements Local improvements represent better alternative routes identified within a rout-
ing domain. If in the example image above current routes are represented by black lines then local
improvements are depicted by orange lines b2 and c2. Keep in mind that a1 just reconfirmed an existing
current route and no improvements are made in such a case.
Local improvements are announced in their routing domains and this has the characteristic that local
traffic exits customer’s network via local providers. This also means that inter-datacenter interconnects
are free from such traffic.
IRP prefers local routes and Improvements to Global improvements.
Parameter bgpd.rd_local_mark specifies a community marker that distinguishes local improvements
from Global Improvements. A BGP speaker should not advertise these improvements outside its Routing
Domain. It must be noted that a single marker is used for all the routing domains and each of them
shall be configured to advertise local improvements within the domain and filter it out for inter-domain
exchanges.
Local improvements should be stopped from propagating across routing domains. A route map is used
to address this. Below are listed sample route maps for Cisco IOS and JUNOS 9.
Cisco IOS

Refer your router capabilities in order to produce the correct route map. The route map MUST
be integrated into existing route maps. It is not sufficient to simply append them.

neighbor <neighbor-from-another-RD> send-community (should be configured

for all iBGP sessions)

ip community-list standard CL-IRP permit 65535:1

route-map RM-IRP-RD deny 10
match community CL-IRP
route-map RM-IRP-RD permit 20

router bgp AS
neighbor <neighbor-from-another-RD> route-map RM-IRP-RD out

Refer Route-Maps for IP Routing Protocol Redistribution Configuration

JUNOS 9

Refer your router capabilities in order to produce the correct route map. The route map MUST
be integrated into existing route maps. It is not sufficient to simply append them.
CHAPTER 1. INTRODUCTION 28

policy-options{
policy-statement IRP-CL {
term 0 {
from {
protocol bgp;
community IRP-RD;
}
then reject;
}
term 1 {
then accept;
}
}
community IRP-RD members 65535:1;
}
protocols {
bgp {
group ebgp {
type external;
neighbor 10.0.0.1 {
export IRP-CL;
}
}
}
}

Refer Policy Framework Configuration Guide; Release 9.3

Global improvements Global improvements are made when IRP identifies an alternative route that

even after factoring in the latencies incurred by inter-datacenter interconnects are better than all existing
alternatives. Such an example can be represented by alternative route c2 in the image above. A global
improvement is made when one routing domain alternative is better than the best local alternatives in
all other routing domains even considering the latencies incurred by inter-datacenter interconnects. In
the image above c2 will become a global improvement if his loss characteristic is best to all alternatives
and its latency:

• (c2+idc1 - margin) is better than best local alternative a1 in RD SJC

• (c2+idc3 - margin) is better than best local alternative b2 in RD SFO

where:

• a1, b2 and c2 represent roundtrip times determined by IRP during probing of a destination.
• idc values are configurable and are set as one entry of global.rd_rtt parameter.
• margin is given by core.global.worst_ms.

Global improvements move traffic via inter-datacenter interconnects and as such are less desirable to
local routes. Global improvements make sense when defined as above and even more sense when packet
loss is taken in consideration and routing via a different datacenter reduces packet loss significantly.

1.2.13 Improvements confirmation

Improvements confirmation is a feature designed to address network variability over time. There are cases
when performance metrics towards a destination degrade significantly for very short periods of time and
subsequently return to normal. IRP probes all destinations continuously and there is a small but real
chance that it will detect situations like these. At this stage IRP will consider making an improvement.
CHAPTER 1. INTRODUCTION 29

Some of our customers need to address the concern of very short-lived network degradation. Improve-
ments confirmation feature schedules a potential improvement for confirmation after a short period of
time instead of immediately announcing it. This allows IRP to verify that a detected problem is short-
lived and there is no need to make the improvement. Only if IRP gets the same probing results during
confirmation, an improvement is considered relevant and will be announced.
This feature also helps with fluctuating network performance metrics of some very big volume desti-
nations. By fine-tuning Improvement confirmation delay IRP can be configured to significantly reduce
Improvement flapping - the situation when an improvement is moved back and forth between providers
due to the fact that performance metrics for the default route fluctuate between very good and signifi-
cantly worse.
Improvements confirmation represents a tradeoff since all improvements are deferred for some time and
problems persist a little longer.

1.2.14 Improvements weight

IRP assigns each improvement a weight. The weight takes into consideration many network and envi-
ronment aspects of the change such as policy or VIP destinations, loss and latency differences, cost or
commit control type of the improvement. Based on all of the above the improvement gathers more or
less weight as appropriate.
Later on, instead of replacing oldest improvements that might still bring significant benefits with new
improvements just because they are fresh, IRP relies on the weights to decide whether the benefit of the
new improvement is sufficient to replace an existing one. More so, besides preserving the most relevant
improvements this feature reduces route flapping by blocking announcement of new improvements and
withdrawal of existing ones if the changes are not offering a good enough return.

1.2.15 Notifications and events

IRP produces a huge number of various events and some of them are critical for customers’ awareness.
Notifications allow customers to subscribe to any of the available events using the following channels:

• SMS
• Email
• SNMP Traps

IRP service Irppushd provides this feature. In order for Notifications to be delivered correctly the cor-
responding channel configuration shall be provided. By default only email notifications can be delivered
since IRP uses the embedded system email service to send them.
More so, users should subscribe for specific events.

Only events for valid subscriptions using correctly configured channels will be delivered.

Refer section 3.12.8 for details about configuring, subscribing and contents of notifications.
Refer section 4.1.9 for details about individual configuration parameter.

Events
The list of events monitored by IRP that can generate notifications is provided below.
When one of the IRP components detects a transition form normal to abnormal traffic behavior or back
it fires these events:

• Abnormal correction: irpflowd

• Abnormal correction: irpspand
CHAPTER 1. INTRODUCTION 30

• Inbound traffic low: SPAN

• Inbound traffic low: Flow
• Inbound traffic normal: Flow

• Inbound traffic normal: SPAN

• Outbound traffic low: SPAN
• Outbound traffic low: Flow

• Outbound traffic normal: Flow

• Outbound traffic normal: SPAN

When Commit Control limits are exceeded per provider or overall one of the following events fires. Refer
section 4.1.9 for configuring the actual limits of the events.

• Commit Control overload by X Mbps

• Commit Control overload by X%

• Commit Control provider X overloaded by Y Mbps

• Commit Control provider X overloaded by Y%

When an IRP component (re)loads the configuration it validates it and depending on results fires one of
the following events:

• Configuration Invalid: BGPd

• Configuration Invalid: Core

• Configuration Invalid: Explorer
• Configuration Invalid: irpapid
• Configuration Invalid: irpflowd

• Configuration Invalid: irpspand

• Configuration Invalid: irpstatd
• Configuration Ok: BGPd

• Configuration Ok: Core

• Configuration Ok: Explorer
• Configuration Ok: irpapid
• Configuration Ok: irpflowd

• Configuration Ok: irpspand

• Configuration Ok: irpstatd

Outage detection algorithm fires one of the following events when it confirms congestion or outage
problems and reroutes traffic around it:

• Congestion or Outage

• Outage: Confirmed and rerouted

Explorer periodically checks the PBRs and its expected probing performance and triggers the following
events:
CHAPTER 1. INTRODUCTION 31

• Failed PBR (IPv6) check for provider

• Failed PBR (IPv4) check for provider
• Successful PBR (IPv4) check for provider
• Successful PBR (IPv6) check for provider
• Explorer performance low
• High number of VIP prefixes degrades IRP performance

IRP BGP Internal and External monitors fire the following events:

• ExternalMonitor (IPv4) Failed status for a provider. All improvements towards the provider will
be withdrawn.
• ExternalMonitor (IPv4) OK status for a provider. All improvements towards the provider will be
announced.
• ExternalMonitor (IPv6) Failed status for a provider. All improvements towards the provider will
be withdrawn.
• ExternalMonitor (IPv6) OK status for a provider. All improvements towards the provider will be
announced.
• InternalMonitor (IPv4) Failed status for a provider. All improvements towards the provider will
be withdrawn.
• InternalMonitor (IPv4) OK status for a provider. All improvements towards the provider will be
announced.
• InternalMonitor (IPv6) Failed status for a provider. All improvements towards the provider will
be withdrawn.
• InternalMonitor (IPv6) OK status for a provider. All improvements towards the provider will be
announced.

When statistics collection over SNMP is up or down IRP fires the following events:

• Provider SNMP stats down: X

• Provider SNMP stats up: X

BGPd raises these events when BGP sessions are established/disconnected:

• IRP BGP session disconnected

• IRP BGP session established

When IRP identifies conditions to re-route traffic (make an improvement) and additionally it considers
the differences to be excessive it raises these events:

• Excessive packet latency for prefix

• Excessive packet loss for prefix
• Low rate of announced IPv4 improvements
• Low rate of announced IPv6 improvements
• New improvement

Once an IRP component is started, stopped or restarted it raises the following events:

• Service started: BGPd

CHAPTER 1. INTRODUCTION 32

• Service started: Core

• Service started: Explorer
• Service started: irpapid
• Service started: irpflowd
• Service started: irpspand
• Service started: irpstatd
• Service stopped: BGPd
• Service stopped: Core
• Service stopped: Explorer
• Service stopped: irpapid
• Service stopped: irpflowd
• Service stopped: irpspand
• Service stopped: irpstatd

SNMP Traps
SNMP traps is a widely used mechanism to alert about and monitor a system’s activity.
IRP SNMP traps not only notify about some IRP platform event but also include the list of varbinds
which contain detailed information related to the thrown trap. The complete list of traps and varbinds
with their descriptions can be found at /usr/share/doc/irp/NOCTION-IRP-MIB.txt

1.2.16 IRP API

IRP exposes a web API that uses HTTP verbs and a RESTful endpoint structure. Request and response
payloads are formatted as JSON.
The API is running on the IRP instance and is reachable by default over SSL at port 10443. If
called directly from the IRP instance server the API can be accessed at https://localhost:10443 Use
https://hostname/api in order to access the API from elsewhere on the network.
An IRP user id is required to access most of the API services. Use IRP’s Frontend to manage users
or configure external User Directories (refer Security Configuration). IRP API uses an authenticating
mechanism based on authentication tokens. The token is passed as a query parameter for all API requests
that require authentication.
The API comes with an API Reference available from IRP’s Help menu and the API References also
includes a sample PHP application with source code to aid in development.
IRP’s API is powered by Irpapid service that can be started, stopped, configured like any other IRP
service. Refer to the 4.1.3 section for Irpapid configuration parameter details.

1.2.17 IRP Failover

Overview
IRP offers failover capabilities that ensure Improvements are preserved in case of planned or unplanned
downtime of IRP server.
IRP’s failover feature uses a master-slave configuration. A second instance of IRP needs to be deployed
in order to enable failover features. For details about failover configuration and troubleshooting refer
Failover configuration.
CHAPTER 1. INTRODUCTION 33

A failover license is required for the second node. Check with Noction’s sales team for details.

IRP’s failover solution relies on:

• slave node running same version of IRP as the master node,

• MySQL Multi-Master replication of ’irp’ database,
• announcement of the replicated improvements with different LocalPref and/or communities by
both nodes,
• monitoring by slave node of BGP announcements originating from master node based on higher
precedence of master’s announced prefixes,
• activating/deactivating of slave IRP components in case of failure or resumed work by master,
• syncing master configuration to slave node.

For exact details about IRP failover solution refer to configuration guides (2.6, 3.12.12.5), template
files, and (if available) working IRP configurations. For example, some ’irp’ database tables are not
replicated, ’mysql’ system database is replicated too, some IRP components are stopped.

IRP versions 3.5 and earlier do no offer failover capabilities for Inbound improvements. It is
advised that in these versions only one of the IRP instances is configured to perform inbound op-
timization in order to avoid contradictory decisions. In case of a failure of this instance inbound
improvements are withdrawn.

An overview of the solution is presented in the following figure:

Figure 1.2.10: Failover high level overview

The diagram highlights the:

CHAPTER 1. INTRODUCTION 34

• two IRP nodes - Master and Slave,

• grayed-out components are in stand-by mode - services are stopped or operating in limited ways.
For example, the Frontend detects that it runs on the slave node and prohibits any changes to
configuration while still offering access to reports, graphs or dashboards.

• configuration changes are pushed by master to slave during synchronization. SSH is used to connect
to the slave.
• MySQL Multi-Master replication is setup for ’irp’ database between master and slave nodes. Ex-
isting MySQL Multi-Master replication functionality is used.

• master IRP node is fully functional and collects statistics, queues for probing, probes and eventually
makes Improvements. All the intermediate and final results are stored in MySQL and due to
replication will make it into slave’s database as well.
• BGPd works on both master and slave IRP nodes. They make the same announcements with
different LocalPref/communities.

• BGPd on slave node monitors the number of master announcements from the router (master
announcements have higher priority than slave’s)
• Timers are used to prevent flapping of failover-failback.

Requirements
The following additional preconditions must be met in order to setup failover:

1. second server to install the slave,

2. MySQL Multi-Master replication for the irp database.

MySQL replication is not configured by default. Configuration of MySQL Multi-Master repli-

cation is a mandatory requirement for a failover IRP configuration. Failover setup, and specifically
MySQL Multi-Master replication should follow a provided failover script. Only a subset of tables in
irp database are replicated. Replication requires extra storage space, depending on the overall traffic
and platform activity, for replication logs on both failover nodes.

1. a second set of BGP sessions will be established,

2. a second set of PBR IP addresses are required to assign to the slave node in order to perform
probing,

3. a second set of improvements will be announced to the router,

4. a failover license for the slave node,
5. Key-based SSH authentication from master to slave is required. It is used to synchronize IRP
configuration from master to slave,

6. MySQL Multi-Master replication of ’irp’ database,

7. IRP setup in Intrusive mode on master node.

In case IRP failover is setup in a multiple Routing Domain configuration and IRP instances are
hosted by different RDs this must be specified in IRP configuration too. Refer Optimization for
multiple Routing Domains, global.master_rd, global.slave_rd.
CHAPTER 1. INTRODUCTION 35

Failover
IRP failover relies on the slave node running the same version of IRP to determine if there are issues
with the master node and take over if such an incident occurs.
Slave’s BGPd service verifies that announcements are present on a router from master. If announcements
from master are withdrawn for some reason the slave node will take over.

In order for this mechanism to work IRP needs to operate in Intrusive mode and master’s node
announcements must have higher priority then the slave’s.

During normal operation the slave is kept up to date by master so that it is ready to take over in case
of an incident. The following operations are performed:

• master synchronizes its configuration to slave. This uses a SSH channel to sync configuration files
from master to slave and process necessary services restart.
• MySQL Multi-Master replication is configured on relevant irp database tables so that the data is
available immediately in case of emergency,

• components of IRP such as Core, Explorer, Irppushd are stopped or standing by on slave to prevent
split-brain or duplicate probing and notifications,
• slave node runs BGPd and makes exactly the same announcements with a lower BGP LocalPref
and/or other communities thus replicating Improvements too.

It is imperative that master’s LocalPref value is greater than slave’s value. This ensures that
master’s announcements are preferred and enables slave to also observe them as part of monitoring.

In case of master failure its BGP session(s) goes down and its announcements are withdrawn.

Slave node only considers that master is down and takes over only if master’s Improvements are
withdrawn from all edge routers in case of networks with multiple edge routers.

The same announcements are already in router’s local RIB from slave and the router chooses them as
best.

This is true only if LocalPref and/or communities assigned to slave node are preferred. If other
most preferable announcements are sent by other network elements , no longer announcements from
slave node will be best. This defeats the purpose of using IRP failover.

At the same time, Failover logic runs a set of timers after master routes are withdrawn (refer global.failover_timer_fail).
When the timers expire IRP activates its standby components and resumes optimization.

Failback
IRP includes failback feature too. Failback happens when master comes back online. Once BGPd on the
slave detects announcements from master it starts its failback timer (refer global.failover_timer_failback).
Slave node will continue running all IRP components for the duration of the failback period. Once the
failback timer expires redundant slave components are switched to standby mode and the entire setup
becomes normal again. This timer is intended to prevent cases when master is unstable after being
restored and there is a significant risk it will fail again.
CHAPTER 1. INTRODUCTION 36

During failback it is recommended that both IRP nodes are monitored by network administrators
to confirm the system is stable.

Recovery of failed node

IRP failover configuration is capable to automatically restore its entire failover environment if downtime
of failed node is less than 24 hours.

Recovery speed is constrained by restoring replication of MySQL databases. On 1Gbps non-

congested links replication for a full day of downtime takes approximately 30-45 minutes with 200-
250Mbps network bandwidth utilization between the two IRP nodes. During this time the operational
node continues running IRP services too.

If downtime was longer than 24 hours MySQL Multi-Master replication is no longer able to synchronize
the databases on the two IRP nodes and manual MySQL replication recovery is required.

Upgrades
Failover configurations of IRP require careful upgrade procedures especially for major versions.

It is imperative that master and slave nodes are not upgraded at the same time. Update one
node first, give the system some time to stabilize and only after that update the second node.
CHAPTER 1. INTRODUCTION 37

1.2.18 Inbound (beta)

Inbound is a beta feature and has not been fully integrated with all IRP features and functionality.
Using this feature requires careful planning. Please coordinate with Noction’s support team if you
want to start using Inbound.

Starting with version 3.4 IRP introduced optimization of Inbound traffic. Inbound bandwidth control
reshapes the amount of traffic from different providers targeting your sub-prefixes .
IRP uses well known and proven BGP mechanisms to instruct your routers to adjust their advertise-
ments of your network segments to upstream providers and subsequently to the World. The adjusted
advertisements take advantage of existing BGP policies implemented by edge routers worldwide in order
to increase or decrease the preference of your internal network segments as advertised by one or another
AS to the world. This allows more traffic to lean towards some of your upstream providers and less
towards others. In case of an incident that your multihomed configuration is designed to be resilient
against, the entire world still knows of the alternative routes towards your network and will be able to
adjust accordingly.
Noction’s IRP Inbound feature:

• advises your edge routers to advertise different prefixes of your network with different counts of
BGP prepends to each of your upstream providers;

• monitors inbound and outbound traffic on your network interfaces using standard protocols (SNMP,
NetFlow, sFlow) to determine if there is need for action;
• continuously assesses network health and performance to ensure that when overloads are possible
it knows good and reliable alternative routes to use;

• uses a proprietary inferring mechanism that takes into account the inertial nature of the Internet
to react to inbound preference changes and thus dampens the urge to “act now” and destabilize
the network;
• provides you with configuration, monitoring and visualization tools that allow you to cross-check
and validate its actions.

The entire Inbound optimization solution covers:

1. Segmenting your network into prefixes that are controlled by IRP

2. Coordinating communication over BGP between IRP and routers
3. Setting network conditions for making Inbound Improvements
4. Reviewing Inbound optimization results

The use cases below highlight typical scenarios when IRP’s inbound bandwidth control capabilities might
come handy:

Lets start with the case of unbalanced inbound and outbound peering You have a peering
agreement with one of your neighbors to exchange traffic. Unfortunately the rest of the neighboring
network configuration significantly unbalances the volumes of inbound and outbound traffic.
CHAPTER 1. INTRODUCTION 38

Figure 1.2.11: Unbalanced inbound/outbound peering

‘

You rely on manipulating prefix announcements towards neighbors in order to shape the traffic. Un-
fortunately this is a reactive solution and consumes a lot of time while at the same time pushing the
balance either one way or another. Often this pushes the network into the second typical scenario.

Fluctuating traffic shape A multihomed configuration overwhelms some links while other links re-
main barely used. Your network administrators frequently get alerts during peak network use and they
manually add or remove prepends or altogether remove some inbound prefixes from being announced to
affected neighboring links. These changes are sometime forgotten or other times just push the bulk of
traffic towards other links and the problem re-appears.

Figure 1.2.12: Fluctuating inbound traffic

‘

For both above scenarios Inbound commit control in IRP can automate most of the inbound traffic
shaping operations. It uses a typical solution for traffic shaping and specifically manipulates prepend
counts announced to different providers and this eventually propagates across the Internet thus diverting
traffic as desired. IRP automates your routine day to day traffic shaping actions and probably makes
significantly more adjustments than you can afford to. At the same time it offers you reliable reviewing
and fine-tuning options that will allow you to adjust IRP’s work in a changing and evolving network.
Refer Inbound (beta) configuration, Current inbound improvements, Inbound traffic distribution, In-
bound parameters for details.
CHAPTER 1. INTRODUCTION 39

1.2.19 Flowspec policies

Flowspec policies can be used only in conjunction with Flowspec capable routers.

Starting with version 3.5 IRP has support of Flowspec policies. This means that Flowspec capability is
recognized and can be used accordingly for BGP sessions established by IRP. In short Flowspec defines
matching rules that routers implement and enforce in order to ensure that only desirable traffic reaches
a network. Flowspec by relying on BGP to propagate the policies uses a well understood and reliable
protocol.

As specified by BGP, when a session disconnects all the announcements received from that session
are discarded. The same is generally true for Flowspec policies. Still, since some vendors recognize
Flowspec policies but implement them using capabilities other than BGP a confirmation is needed
whether on session disconnect the specific router model indeed removes all the underlying constructs
and reverts to a known state.

IRP not being involved in direct packet forwarding expects that Flowspec policies are implemented at
least by your edge routers. If upstream providers also offer Flowspec support these policies can be
communicated upstream where their implementation is even more effective.
Eventually Flowspec policies help ensure that traffic entering or exiting your network conforms to your
plans and expectations. The main use cases that can be accomplished with Flowspec policies in IRP
allows:

• controlling bandwidth usage of your low priority traffic towards external services, for example
throttling bandwidth usage originating on your backup systems towards off-premises services.

• anticipating inbound traffic towards your services and shaping bandwidth use in advance, for
example anticipating low numbers of legitimate customers from Russia, China or India on your
e-commerce services and setting high but controllable rate limits on packets originating in those
networks.
• reacting on a packet flooding incident by dropping specific packets, for example dropping all packets
targeting port 53.
• redirecting some traffic for scrutiny or cleansing, for example forwarding port 80 packets through
an intelligent device capable of detecting RUDY, slow read or other low-bandwidth/amplification
attacks.

IRP Flowspec policies rely on a minimal set matching rules and actions that offer most of the capabilities
while keeping the learning curve low and integration simple:

• Source or destination IP address specified as either CIDR format prefix or direct IP address

• Traffic protocols, for example TCP, UDP or ICMP

• Source or destination TCP/UDP ports
• Throttle, drop and redirect actions.

It is important to note that IRP does not cross-validate Flowspec policies with improvements.
While it is possible that for example a Flowspec redirect action pushes some traffic a different way
to what an improvement advises, usually improvements cover many prefixes and while there will be
a contradiction for one prefix there will be many other prefixes that IRP improves to compensate
for these unitary abnormalities. It is recommended that Flowspec policies take precedence over
improvements in order to benefit from this compensating nature of improvements.
CHAPTER 1. INTRODUCTION 40

Consider that depending on whether source or destination prefix belongs to your network the policy
applies to either inbound or outbound traffic while the choice of ports allows targeting different traffic
types.
Compare Flowspec policies to the already well known Routing Policies . For further details regarding
Flowspec configuration refer Flowspec policies.

1.2.20 Throttling excessive bandwidth use with Flowspec

IRP can be configured to automatically add throttling Flowspec policies for prefixes that started using
abnormal volumes of traffic. This feature can be used only if your network has Flowspec capabilities.
Refer Flowspec policies for details about Flowspec.

In case thresholds for excessive bandwidth use are set to very aggressive levels IRP can create
large numbers of Flowspec policies.

This feature can be described as:

• configure excess threshold and throttling multipliers

• periodically determine current and average prefix bandwidth usage for this hour of the day
• verify if current usage exceeds the average by a larger factor then the threshold multiplier
• rate-limit excessive bandwidth usage prefixes at their average use times throttling multiplier.

Past throttling rules are revised if/when prefix abnormal usage pattern ends.
For example, when a prefix usually consumes 1-2Mbps of traffic and its current bandwidth spikes tenfold
and the spike is sustained for a significant period of time a throttling rule limiting usage by this prefix
at 5-6Mbps will still offer ample bandwidth for normal service use and will also protect other services
from network capacity starvation.
Refer Throttling excessive bandwidth use with Flowspec, Core configuration for details.

1.2.21 Maintenance windows

Maintenance works are on everybody’s agenda in current fast paced and continuously evolving networks.
During maintenance network engineers are very busy and will welcome any help their systems can offer
in carrying out those works with the least amount of headaches. IRP is clearly not in the top of network
engineer’s priorities and asking to suspend or shutdown a provider immediately before a maintenance
window starts and restart the provider back once the maintenance works end is not very helpful if not
even annoying.
Instead IRP offers the facility to plan maintenance windows in advance. Knowing when a maintenance
window starts and ends, IRP excludes that specific provider link from either performance optimization
or bandwidth control. More so, IRP has the capability to reshape the traffic flowing in and out of a
network to anticipate any downtime on a link.
Properly configured maintenance windows allows IRP time to move most of the outbound traffic and
deflect most of inbound traffic away from the provider link that is scheduled for maintenance. Having
only a small fraction of traffic or none at all on the maintenance link before the downtime starts avoids
any (shall we say, catastrophic) spikes, possible overloads and consequently unpredictable behavior of
the remaining live network equipment.
Specifically the following applies:

• a maintenance window is configured in advance and can be removed/revised at any time,

• a maintenance window sets details for single provider. If needed multiple maintenance windows
can be setup and even overlapping maintenance windows are OK.
CHAPTER 1. INTRODUCTION 41

• IRP highlights maintenance windows in IRP’s Frontend sidebar so that it is easy to spot current
maintenance window status.
• optionally IRP can preserve existing improvements so that once the maintenance window ends
improvements are reimplemented. It is advised that this feature is used only when the maintenance
window is very short (a few minutes).
• an unloading period can be setup. During unloading IRP actively re-routes outbound prefixes
through other available providers. While IRP is able to make most of the unloading improvements
fast consideration shall be given to the announcement rate limitations setup in BGPd in order for
all the improvements to reach network routers in time for maintenance window starting time.
• a prepend time can be setup. This is only applicable if Inbound optimization is operational. If
this time is setup then IRP will prepend configured inbound prefixes with the maximum allowed
number of prepends through the provider link under maintenance in order to deflect inbound traffic
towards other providers. Refer to Inbound (beta) for more details about Inbound optimization.

Refer Maintenance windows for details how to configure, review, create, edit, delete maintenance win-
dows.

1.3 IRP Optimization modes

1.3.1 Performance optimization

Figure 1.3.1: Performance optimization algorithm

By running in this mode the system tries to decrease packet loss and latency, while ignoring the transit
cost. The system analyzes each of the connected providers and compares the performance metrics among
all of them.
First of all, it compares the packet loss. If the loss is lower and the difference is greater than a predefined
value, then the system checks if sending the traffic through this provider will not cause any network
congestion. If it confirms that the traffic can flow freely, the system declares the provider as the best
one to route through.
That is how an improvement based upon loss is made.
However, if the loss values are equal or the difference between them is smaller than the predefined value,
the system goes further by comparing the latency values. If the latency is lower and the difference in
CHAPTER 1. INTRODUCTION 42

latency between the providers is greater than the predefined value, then the system declares a latency-
based improvement.
Thus the performance mode makes sure that the traffic flows through the best performing routes, without
checking the bandwidth cost.

1.3.2 Cost optimization

Figure 1.3.2: Cost optimization algorithm

By operating in this mode IRP decreases packet loss and improves the cost while not worsening latency
more than a preconfigured value. If it cannot improve the cost it tries to reduce latency. The platform
runs the same algorithm for loss comparison as in the performance mode.
Additionally, prior to comparing the latency values it compares the transit cost for each provider. If the
cost of the new provider is better, the system goes further by checking the latency and validates the cost
improvement only if the latency is not worsening more than the preconfigured value. However, if the
cost cannot be improved, system tries to make a latency improvement using the same algorithm as in
the performance mode. Thus, if there is no way to make a cost improvement, system reroutes the traffic
to the best performing provider.

1.3.3 Commit Control

Commit Control, allows to keep the commit levels for each provider at a pre-configured level. It in-
cludes bandwidth control algorithms for each provider as well as the active traffic rerouting, in case
bandwidth for a specific provider exceeds the configured limit. Commit Control also includes passive
load adjustments inside each provider group.
A parameter called “precedence” (see peer.X.precedence) is used to set the traffic unloading priorities,
depending on the configured bandwidth cost and providers throughput. The higher is the precedence,
CHAPTER 1. INTRODUCTION 43

the lower is the probability for the traffic to be sent to a provider,when its pre-configured 95th percentile
usage is higher. The platform will reroute excessive bandwidth to providers, whose current load is less
than their 95th percentile. If all providers are overloaded, traffic is rerouted to the provider with the
smallest precedence - usually this provider has either the highest available bandwidth throughput, or the
lowest cost.
See also: core.commit_control.

1.3.3.1 Flexible aggressiveness of Commit algorithm based on past overloads

Metering bandwidth usage by the 95th presents the following alternative interpretation - the customer
is allowed to exceed his limits 5% of times. As such, IRP assumes there’s a schedule of overloads based
on the current time within a month and an actual number of overloads already made.

Figure 1.3.3: More aggressive when actual exceeds schedule

The sample image above highlights the remaining scheduled and the actual amount of allowed overloads
for the month decreasing. Whenever IRP depicts that the actual line goes below schedule (meaning the
number of actual overloads exceeds what is planned) it increases its aggressiveness by starting unloading
traffic earlier than usual. For example, if the commit level is set at 1Gbps and IRP will start unloading
traffic at possibly 90% or 80% depending on past overloads count.
The least aggressive level is set at 99% and the most aggressive level is constrained by configuration
parameter core.commit_control.rate.low + 1%.
This is a permanent feature of IRP Commit Control algorithm.

1.3.3.2 Trigger commit improvements by collector

Commit control improvements are made after the flows carrying traffic are probed and IRP has fresh
and relevant probe results. Subsequent decisions are made based on these results and this makes them
more relevant. Still, probing takes some time and in case of fluctuating traffic patterns the improvements
made will have reduced impact due to flows ending soon and being replaced by other flows that have not
been probed or optimized.
CHAPTER 1. INTRODUCTION 44

For networks with very short flows (average flow duration under 5 minutes) probing represents a signif-
icant delay. In order to reduce the time to react to possible overload events IRP added the feature to
trigger commit control improvements on collector events. When Flow Collector detects possible overload
events for some providers, IRP will use data about past probed destinations in order to start unloading
overloaded providers early. This data is incomplete and a bit outdated but still gives IRP the opportu-
nity to reduce the wait time and prevent possible overloads. Later on, when probes are finished another
round of improvements will be made if needed.
Due to the fact that the first round of improvements is based on older data, some of the improvements
might become irrelevant very soon. This means that routes fluctuate more than necessary while on
average getting a reduced benefit. This is the reason this feature is disabled by default. Enabling this
feature represents a tradeoff that should be taken into consideration when weighing the benefits of a
faster react time of Commit Control algorithm.
This feature is configurable via parameter: core.commit_control.react_on_collector . After enabling/dis-
abling this feature IRP Core service requires restart.

1.3.3.3 Commit Control improvements on disable and re-enable

IRP up to version 2.2 preserved Commit Control improvements when the function was disabled globally
or for a specific provider. The intent of this behavior was to reduce route fluctuation. In time these
improvements are overwritten by new improvements.
We found out that the behavior described above ran contrary to customer’s expectations and needs.
Usually, when this feature is disabled it is done in order to address a more urgent and important need.
Past Commit Control improvements were getting in the way of addressing this need and was causing
confusion. IRP versions starting with 2.2 aligns this behavior with customer expectations:

• when Commit Control is disabled for a provider (peer.X.cc_disable = 1), this Provider’s Commit
Control improvements are deleted;
• when Commit Control is disabled globally (core.commit_control = 0), ALL Commit Control im-
provements are deleted.

It must be noted that the improvements are actually moved into a ’recycle bin’ of sorts and if need be
they can be restored after a short period of time. When Commit Control is enabled overall or for a
Provider IRP will need some time to make a series of improvements that level traffic flows. This is only
natural because IRP does not have statistics and has no knowledge of flow behaviors in the network.
Still, if Commit Control feature is re-enabled there’s the possibility to restore old improvements based
on past data and measurements. Unfortunately network characteristics fluctuate and IRP cannot restore
any past improvements since they might be no longer relevant or even altogether wrong. As such, IRP
will preserve and restore only improvements that are not older than the retry time interval which is
configurable via parameter: core.improvements.ttl.retry_probe

1.3.3.4 Provider load balancing

Provider load balancing is a Commit Control related algorithm, that allows a network operator to evenly
balance the traffic over multiple providers, or multiple links with the same provider.
For example, a specific network having an average bandwidth usage of 6Gbps has two separate ISPs. The
network operator wants (for performance and/or cost reasons) to evenly push 3Gbps over each provider.
In this case, both upstreams are grouped together (see peer.X.precedence), and the IRP system passively
routes traffic for an even traffic distribution. Provider load balancing is enabled by default via parameter
peer.X.group_loadbalance.
CHAPTER 1. INTRODUCTION 45

Figure 1.3.4: Provider load balancing

1.3.3.5 Commit control of aggregated groups

Customers can deploy network configurations with many actual links going to a single ISP. The additional
links can serve various purposes such as to provision sufficient capacity in case of very large capacity
requirements that cannot be fulfilled over a single link, to interconnect different points of presence
on either customer (in a multiple routing domain configuration) or provider sides, or for redundancy
purposes. Individually all these links are configured in IRP as separate providers. When the customer
has an agreement with the ISP that imposes an overall limitation on bandwidth usage, these providers
will be grouped together in IRP so that it can optimize the whole group.
The rationale of this feature as illustrated in the figure below is that if in the group overusages on one
provider are compensated by underusages on another provider there is no need to take any action since
overall the commitments made by the customer to the ISP have not been violated. Commit control
algorithm will take action only when the sum of bandwidth usage on all providers in the group exceed
the sum of bandwidth limits for the same group of providers.

Figure 1.3.5: Commit control of aggregated groups

The image above highlights that many overusages on the green line are compensated by purple line
underusages so that the group usage is below group total limits. Only when the traffic on the purple
line increases significantly and there are no sufficient underusages on the other providers in the group
to compensate the overusages, Commit Control identifies overusages (highlighted with a red x on the
drawing above) and takes action by rerouting some traffic towards providers outside the group.

It is important to note that in order for this feature to be effective there must be providers
configured in IRP that are not part of this group. This way when candidate improvements are
CHAPTER 1. INTRODUCTION 46

considered there are alternative routes via those providers that the traffic can be rerouted to.

In order to configure providers that are optimized as an aggregated group 1) first the providers will be
configured with the same precedence in order to form a group; 2) the overall 95th limitation will be
distributed across providers in the group as appropriate; 3) and finally load balancing for the group will
be Disabled by parameter peer.X.group_loadbalance.

1.3.3.6 95th calculation modes

Commit control uses the 95th centile to determine whether bandwidth is below or above commitments.
There are different ways to account for Outbound and Inbound traffic when determining the 95th value.
IRP supports the following 95th calculation modes:

• Separate 95th for in/out: The 95th value for inbound and outbound traffic are independent and
consequently bandwidth control for each is performed independently of each other. For this 95th
calculation modes IRP monitors two different 95th for each inbound and outbound traffic levels.
• 95th of combined in+out: The 95th is determined based on the sum for inbound and outbound
values at each time-point.
• 95th from greater of in, out: At each time-point the greater of inbound or outbound bandwidth
usage value is used to determine 95th.

• Greater of separate in/out 95th: 95th are determined separately for inbound and outbound traffic
and the larger value is used to verify if commitments have been met.

Refer 4.1.11.5.

1.3.3.7 Other commit control features

Current improvements report has been improved to keep CC improvements applied during an algorithm
cycle in a single sorted batch. This way the report no longer confuses by intermingling the improvements.

Figure 1.3.6: Sorted Commit Control improvements

The excerpt above highlights the new commit control improvements done at the same time. They unload
9.11, 6.09, 5.70, 4.26 Mbps from provider B to A thus unloading B from 109% to 58% and increasing
the load on A from 11% to 46%. The data would have been more confusing if the improvements in that
single batch were intermingled.
During retry probing IRP will delete Commit Control improvements with current bandwidth less than
the configured threshold given by core.commit_control.agg_bw_min.
Chapter 2

Configuration

2.1 Configuration files

IRP is currently configured using a set of textual Unix-style configuration files, located under the
/etc/noction/ directory. In-line comments can be added to any line, using the “#” symbol.
Several separate configuration files are presented as follows:

• /etc/noction/irp.conf - the main IRP configuration file contains configuration parame-

ters for all IRP components, including algorithm parameters, optimization modes definitions and
providers settings.
• /etc/noction/db.global.conf - database configuration file for all IRP components, except
the Frontend component.

• /etc/noction/db.frontend.conf - database configuration file for the Frontend component.

• /etc/noction/exchanges.conf - Exchanges configuration file (3.12.3.2).
• /etc/noction/inbound.conf - Inbound prefixes configuration file (3.12.11).
• /etc/noction/policies.conf - Routing Policies configuration file (1.2.9).

• /etc/noction/user_directories.conf - User Directories configuration file (3.12.9.2).

Additional configuration files can be used for several core, global and explorer preferences, as described
in sections 4.1.2.20, 4.1.6.14 and 4.1.8.5.
A comprehensive list of all the IRP parameters along with their description can be found in the Config-
uration parameters reference chapter.

2.2 Global and Core Configuration

The default values, specified for the Core service are sufficient for a proper system start-up. Several
parameters can be adjusted during the initial system deployment. For a comprehensive list please see
the Global parameters and Core settings sections.
During the initial setup and configuration stage, one must pay attention to the following configuration
parameters:

• global.nonintrusive_bgp - must be set to “1” until the configuration and route propagation tests
are completed.
• global.improve_mode - must be configured according to the specific network operator policies. See
also: IRP Optimization modes

47
CHAPTER 2. CONFIGURATION 48

• global.aggregate - in most cases, it is recommended to enable the aggregates, in order to reduce

the number of prefixes advertised by the IRP. Please consult the network infrastructure and con-
figuration.
• core.commit_control - should be configured according to the specific network operator policies, see
also: Commit Control
• core.outage_detection - in most cases it must be enabled. For more details see Outage detection
CHAPTER 2. CONFIGURATION 49

2.3 Collector Configuration

Depending on the preferred method of traffic data collection, one or both collector components should
be configured. As specified in the IRP Components section, IRP can gather traffic data using the Flow
(from now on: irpflowd) and Span collector (irpspand).
First of all, specific configuration to each collector will be described, along with the required router
configuration changes.

2.3.1 Irpflowd Configuration

Irpflowd is a NetFlow/sFlow collector that receives and analyzes network traffic information, generated
by your router(s).
NetFlow - an IP network statistics protocol developed by Cisco Systems, Inc. for collecting and trans-
ferring statistics regarding IP traffic information from network devices such as switches/routers to
network analysis applications. Irpflowd currently supports the following NetFlow versions: v1,
v5, v9.
sFlow - is a protocol designed for monitoring network, wireless and host devices. Developed by the
sFlow.org Consortium, it is supported by a wide range of network devices, as well as software
routing and network solutions.
To use the irpflowd collector, the following steps must be completed:
1. NetFlow/sFlow/jFlow must be configured on the router(s), which must send traffic flow information
to the main IRP server IP (Figure 2.3.1). See (2.3.1.1) for specific network device configuration
instructions

Figure 2.3.1: Flow export configuration

2. Irpflowd must be enabled (by setting the collector.flow.enabled parameter):

collector.flow.enabled = 1

3. A list of all the networks, advertised by the edge routers that IRP will optimize, should be added
to the configuration. This information should be specified in the collector.ournets parameter.
4. For security reasons, the list of valid Flow sending IP addresses must be configured in the collec-
tor.flow.sources, to protect irpflowd from unauthorized devices sending Flow data.
Example:
collector.flow.sources = 10.0.0.0/29

5. In case the Flow exporters are configured to use non-standard port numbers (2055 for Net-
Flow/jFlow and 6343 for sFlow), then collector.flow.listen.nf and collector.flow.listen.sf must be
adjusted accordingly:
collector.flow.listen.nf = 2055
collector.flow.listen.sf = 6343
CHAPTER 2. CONFIGURATION 50

2.3.1.1 Vendor-specific NetFlow configuration examples

The following sections contain vendor-specific configuration examples, along with possible pitfalls and
version-specific issues
NetFlow configuration on Cisco 7600/6500 series routers

Refer also to the Cisco NetFlow Software Configuration Guide

Listing 2.1: Global MLS settings configuration

(config)# mls netflow
(config)# mls flow ip interface-full
(config)# mls flow ipv6 interface-full
(config)# mls sampling packet-based 512 8192
(config)# mls nde sender version 7

Please replace the IP address and port with the actual IRP host IP and the collector UDP port
(2055 by default)

Listing 2.2: Global NetFlow settings and export configuration

(config)# ip flow-cache entries 524288
(config)# ip flow-cache timeout inactive 60
(config)# ip flow-cache timeout active 1
(config)# ip flow-export version 9
(config)# ip flow-export destination 10.11.12.14 2055

Ingress flow collection must be enabled on all interfaces facing the internal network.
MLS NetFlow sampling must be enabled to preserve router resources.

Listing 2.3: Per-interface NetFlow settings configuration

(config)# int GigabitEthernet 3/6
(config-if)# mls netflow sampling
(config-if)# ip flow ingress

Flexible NetFlow configuration on Cisco 6500 series routers IOS 15.0SY series
Listing 2.4: Flexible NetFlow monitor configuration
(config)# flow monitor IRP-FLOW-MONITOR
(config-flow-monitor)# record platform-original ipv4 full
(config-flow-monitor)# exporter IRP-FLOW-EXPORTER
(config-flow-monitor)# cache timeout inactive 60
(config-flow-monitor)# cache timeout active 60
(config-flow-monitor)# cache entries 1048576

Listing 2.5: Flexible NetFlow exporter configuration

(config)# flow exporter IRP-FLOW-EXPORTER
(config-flow-exporter)# destination 10.11.12.14
(config-flow-exporter)# source Loopback0
(config-flow-exporter)# transport udp 2055
(config-flow-exporter)# template data timeout 120
CHAPTER 2. CONFIGURATION 51

Please replace the IP address and port with the actual IRP host IP and the collector UDP port
(2055 by default). Also replace the source interface with the actual one.

Listing 2.6: Flexible NetFlow sampler configuration

(config)# sampler flow-sampler
(config-sampler)# mode random 1 out-of 1024

Listing 2.7: Per-interface Flexible NetFlow settings configuration

(config)# interface FastEthernet0/0
(config-if)# ip flow monitor IRP-FLOW-MONITOR sampler flow-sampler input
(config-if)# ip flow monitor IRP-FLOW-MONITOR sampler flow-sampler output

NetFlow configuration on Cisco 7200/3600 series routers

Please replace the IP address and port with the actual IRP host IP and the collector UDP port
(2055 by default)

Do not attempt to configure 7600/6500 series routers according to 7200/3600 router’s configura-
tion guide.

Listing 2.8: NetFlow configuration on Cisco 7200/3600 series routers

Router(config)# ip flow-cache entries 524288
Router(config)# ip flow-cache timeout inactive 60
Router(config)# ip flow-cache timeout active 1
Router(config)# ip flow-export version 9
Router(config)# ip flow-export destination 10.11.12.14 2055

Ingress/egress flow export configuration on peering interfaces

According to Cisco IOS NetFlow Command Reference regarding the "ip flow" command history in
IOS releases, this feature was introduced in IOS 12.3(11)T, 12.2(31)SB2, 12.2(18)SXE, 12.2(33)SRA.

NetFlow exporting must be configured on each peering interface which is used to send and/or receive
traffic:

Listing 2.9: Ingress/egress flow export configuration on peering interfaces

Router(config)#interface FastEthernet 1/0
Router(config-if)#ip flow ingress
Router(config-if)#ip flow egress

Ingress flow export configuration (earlier IOS releases)

Ingress flow export must be enabled on all interfaces facing the internal network. Prior to
IOS 12.3(11)T, 12.2(31)SB2, 12.2(18)SXE, 12.2(33)SRA, flow export can be enabled only for ingress
traffic, therefore it must be enabled on each interface that transmits/receives traffic from/to networks
that must be improved
CHAPTER 2. CONFIGURATION 52

Listing 2.10: Ingress flow export configuration

Router(config)#interface FastEthernet 1/0
Router(config-if)#ip route-cache flow

NetFlow/sFlow configuration examples for Vyatta routers (VC 6.3)

Do not configure both NetFlow and sFlow export for the same router interface. It will lead to
traffic statistics distortion.

Sampling interval must be set to at least 2000 for 10G links and 1000 for 1G links in order to
save resources.

Listing 2.11: Configuring NetFlow export on Vyatta

vyatta@vyatta# set system flow-accounting netflow server 10.11.12.14 port
2055
vyatta@vyatta# set system flow-accounting netflow version 5

Listing 2.12: Configuration of an interface for the flow accounting

vyatta@vyatta# set system flow-accounting interface eth0
vyatta@vyatta# commit

jFlow export configuration for Juniper routers

Routing Engine-based sampling supports up to eight flow servers for both version 5 and version 8 con-
figurations. The total number of collectors is limited to eight, regardless of how many are configured for
version 5 or version 8. During the sampling configuration, the export packets are replicated to all the
collectors, configured to receive them. If two collectors are configured to receive version 5 records, then
both collectors will receive records for a specified flow.

Default export interval for active/inactive flows on some Juniper routers is 1800 seconds. IRP
requires significantly more frequent updates. The export interval recommended by IRP is 60 seconds.
Refer your JunOS documentation on how to set export intervals. These parameters are named flow-
active-timeout and flow-inactive-timeout.

Listing 2.13: Juniper flow export configuration

forwarding-options {
sampling {
input {
family inet {
rate 1000;
}
}
}
family inet {
output {
flow-server 10.10.3.2 {
port 2055;
version 5;
source-address 10.255.255.1;
}
}
}
}
CHAPTER 2. CONFIGURATION 53

NetFlow export must be configured on all the interfaces facing the providers. In some cases, it may also
be necessary to enable NetFlow forwarding on the interfaces facing the internal network.

Listing 2.14: Per-interface NetFlow sampling configuration

interfaces {
xe-0/0/0 {
unit 0 {
family inet {
sampling {
input output;
}
}
}
}
}

2.3.2 Irpspand Configuration

Irpspand acts like a passive network sniffer, analyzing the traffic that is provided to one or more
dedicated network interfaces on the IRP server (defined in collector.span.interfaces) from a mirrored
port on your router or switch. For better results (higher performance and more analyzed traffic), as
specified in the IRP Technical Requirements section, we recommend using Myricom 10Gbps NICs, with
Sniffer10G license enabled.
To use the irpspand collector, the following steps must be completed:

1. Configure port mirroring on your router or switch, as shown in figures (2.3.2) and (2.3.3).

Figure 2.3.2: Span port configuration (on the router)

or:

Figure 2.3.3: Span port configuration (on the switch)

2. Enable the span collector by setting the collector.span.enabled parameter in the configuration file:
CHAPTER 2. CONFIGURATION 54

collector.span.enabled = 1

3. Define the list of network interfaces that receive mirrored traffic, by setting the collector.span.interfaces
parameter (multiple interfaces separated by space can be specified):
collector.span.interfaces = eth1 eth2 eth3

4. A list of all the networks advertised by the edge routers that IRP will optimize must be added to
the configuration. This information should be specified in the collector.ournets parameter.

5. In case blackouts, congestions and excessive delays are to be analyzed by the system, the collec-
tor.span.min_delay must be turned on as well
collector.span.min_delay = 1
CHAPTER 2. CONFIGURATION 55

2.4 Explorer Configuration

As described in the 1.2.2 section, the explorer is the actual service that performs active remote network
probing and tracing.
Explorer runs active and passive probings through all the available providers in order to determine the
best one for the particular prefix. Such metrics as packet loss, latency, link capacity and link load are
taken into consideration. In order to run a probe through a particular provider, Explorer needs the
following to be configured:

1. An additional IP alias for each provider should be assigned and configured on the IRP server. This
IP will be used as a source address during the probing process.

It is recommended to configure reverse DNS records for each IP using the following template:
performance-check-via-<PROVIDER-NAME>.
HARMLESS-NOCTION-IRP-PROBING.<YOUR-DOMAIN-NAME>.

2. Policy-based routing (PBR) has to be configured on the edge router(s), so that traffic originating
from each of these probing IP addresses will exit the network via specific provider. See specific
PBR configuration in the Specific PBR configuration scenarios section.
3. Policy-based routing has to be configured to drop packets rather than routing them through the
default route in case that the corresponding Next-Hop does not exist in the routing table.

2.4.1 Specific PBR configuration scenarios

PBRs can be setup in multiple ways, depending on the existing network infrastructure.
We will assume the following IP addresses/networks are used:

10.0.0.0/24 - used on the IRP server as well as the probing VLANs

10.0.0.2/32 - main IRP server IP address

10.0.0.3-10.0.0.5 - probing IP addresses
10.0.0.250-10.0.0.254 - router-side IP addresses for the probing VLANs
10.0.1.0/24 - used for GRE tunnel interfaces, if needed
10.10.0.0/24 - real edge routers IP addresses

10.11.0.0/30 - BGP session with the 1st provider, 10.11.0.1 being the ISP BGP neighbor IP
10.12.0.0/30 - BGP session with the 2nd provider, 10.12.0.1 being the ISP BGP neighbor IP
10.13.0.0/30 - BGP session with the 3rd provider, 10.13.0.1 being the ISP BGP neighbor IP
Vlan 3 - the probing Vlan
eth0 - the probing network interface on the IRP server

In a production network, please change the IP addresses used in these examples to the actual
addresses assigned and used in the network. Same goes for the Vlan interface.

Brocade routers use Cisco compliant configuration commands. Unless otherwise noted, the Cisco
configuration examples will also work on Brocade devices.

Case 1: Single router, two providers, and separate probing Vlan.

CHAPTER 2. CONFIGURATION 56

10.0.0.1/32 is configured on the edge router, on the probing vlan interface (ve3).

Figure 2.4.1: PBR configuration: single router and separate probing Vlan

In this case, a single PBR rule should be enforced on the Edge router for each of the probing IP addresses.

Listing 2.15: Cisco IPv4 PBR configuration: 1 router, 2 providers

access-list 1 permit ip host 10.0.0.3
access-list 2 permit ip host 10.0.0.4
!
route-map irp-peer permit 10
match ip address 1
set ip next-hop 10.11.0.1
set interface Null0
!
route-map irp-peer permit 20
match ip address 2
set ip next-hop 10.12.0.1
set interface Null0
!
interface ve 3
ip policy route-map irp-peer

Route-map entries with the destination set to Null0 interface are used for preventing packets to
flow through the default route in the case that the corresponding next-hop does not exist in the
routing table (typically when a physical interface administrative/operational status is down).

Cisco ASR9000 routers running IOS XR use a different PBR syntax.

Listing 2.16: Cisco IPv4 PBR configuration for IOS XR: 1 router, 2 providers
configure
ipv4 access-list irp-peer
10 permit ipv4 host 10.0.0.3 any nexthop1 ipv4 10.11.0.1 nexthop2 ipv4
169.254.0.254
11 permit ipv4 host 10.0.0.4 any nexthop1 ipv4 10.12.0.1 nexthop2 ipv4
169.254.0.254
end

router static
address-family ipv4 unicast
169.254.0.254 Null0
end

interface FastEthernet1/1
CHAPTER 2. CONFIGURATION 57

ipv4 access-group irp-peer ingress

end

For Juniper routers, a more complex configuration is required:

Listing 2.17: Juniper IPv4 PBR configuration: 1 router, 2 providers

[edit interfaces]
xe-0/0/0 {
unit 3 {
family inet {
filter {
input IRP-policy;
}
}
}
}
[edit firewall]
family inet {
filter IRP-policy {
term irp-peer1 {
from {
source-address 10.0.0.3/32;
}
then {
routing-instance irp-isp1-route;
}
}
term irp-peer2 {
from {
source-address 10.0.0.4/32;
}
then {
routing-instance irp-isp2-route;
}
}
term default {
then {
accept;
}
}
}
}
[edit]
routing-instances {
irp-isp1-route {
instance-type forwarding;
routing-options {
static {
route 0.0.0.0/0 next-hop 10.11.0.1;
}
}
}
irp-isp2-route {
instance-type forwarding;
routing-options {
static {
route 0.0.0.0/0 next-hop 10.12.0.1;
}
CHAPTER 2. CONFIGURATION 58

}
}
}
routing-options {
interface-routes {
rib-group inet irp-policies;
}
rib-groups {
irp-policies {
import-rib [ inet.0 irp-isp1-route.inet.0 irp-isp2-route.inet.0 ];
}
}
}

PBR configuration on Vyatta routers.

Unfortunately, prior to and including VC6.4, Vyatta does not natively support policy-based routing.
Thus, the PBR rules should be configured using the standard Linux ip toolset. To make these rules
persistent, they should be also added to /etc/rc.local on the Vyatta system.

Listing 2.18: Vyatta IPv4 PBR configuration example

ip route add default via 10.11.0.1 table 101
ip route add default via 10.12.0.1 table 102
ip rule add from 10.0.0.3 table 101 pref 32001
ip rule add from 10.0.0.4 table 102 pref 32002

Vyatta versions VC6.5 and up, natively support source-based routing. The following example can be
used:

Listing 2.19: Vyatta (VC6.5 and up) IPv4 PBR configuration example
# Setup the routing policy:
set policy route IRP-ROUTE
set policy route IRP-ROUTE rule 10 destination address 0.0.0.0/0
set policy route IRP-ROUTE rule 10 source address 10.0.0.3/32
set policy route IRP-ROUTE rule 10 set table 103
set policy route IRP-ROUTE rule 20 destination address 0.0.0.0/0
set policy route IRP-ROUTE rule 20 source address 10.0.0.4/32
set policy route IRP-ROUTE rule 20 set table 104
set policy route IRP-ROUTE rule 30 destination address 0.0.0.0/0
set policy route IRP-ROUTE rule 30 source address 0.0.0.0/0
set policy route IRP-ROUTE rule 30 set table main
commit

# Create static route tables:

set protocols static table 103 route 0.0.0.0/0 nexthop 10.11.0.1
set protocols static table 104 route 0.0.0.0/0 nexthop 10.12.0.1
commit

# Assign policies to specific interfaces, Vlan 3 on eth1 in this example:

set interfaces ethernet eth1.3 policy route IRP-ROUTE

# Verify the configuration:

show policy route IRP-ROUTE
show protocols static
show interfaces ethernet eth1.3

Case 2: Two edge routers, two providers, and a separate probing Vlan.
CHAPTER 2. CONFIGURATION 59

Following IP addresses are configured on the routers:

- 10.0.0.251 is configured on R1, VE3
- 10.0.0.252 is configured on R2, VE3

Figure 2.4.2: PBR configuration: two routers and separate probing Vlan

To reduce the number of PBR rules to one per each router, additional source based routing rules must be
configured on the IRP server. This can be done either by using the ip command, or by adjusting the stan-
dardized /etc/sysconfig/network-scripts/route-eth* and /etc/sysconfig/network-scripts/rule-e
configuration files. Both ways will be presented.

Listing 2.20: IRP server source-based routing using the ‘ip‘ command
ip route add default via 10.0.0.251 table 201
ip route add default via 10.0.0.252 table 202
ip rule add from 10.0.0.3 table 201 pref 32101
ip rule add from 10.0.0.4 table 202 pref 32102

Listing 2.21: IRP server source-based routing using standard CentOS configuration files
#/etc/sysconfig/network-scripts/route-eth0:
default via 10.0.0.251 table 201
default via 10.0.0.252 table 202
#/etc/sysconfig/network-scripts/rule-eth0:
from 10.0.0.3 table 201 pref 32101
from 10.0.0.4 table 202 pref 32102

Router configuration looks similar to the previous case. A Cisco/Brocade example will be provided.

Some Brocade routers/switches have PBR configuration limitations. Please refer to the “Policy-
Based Routing” → “Configuration considerations” section in the Brocade documentation for your
router/switch model.
For example, BigIron RX Series of switches do not support more than 6 instances of a route map,
more than 6 ACLs in a matching policy of each route map instance, and more than 6 next hops in
a set policy of each route map instance.
On the other hand, some Brocade CER/CES routers/switches have these limits raised up to 200
instances (depending on package version).

Listing 2.22: Cisco IPv4 PBR configuration: 2 routers, 2 providers, separate Vlan
#Router R1
access-list 1 permit ip host 10.0.0.3
!
route-map irp-peer permit 10
match ip address 1
set ip next-hop 10.11.0.1
CHAPTER 2. CONFIGURATION 60

set interface Null0

!
interface ve 3
ip policy route-map irp-peer

#Router R2
access-list 1 permit ip host 10.0.0.4
!
route-map irp-peer permit 10
match ip address 1
set ip next-hop 10.12.0.1
set interface Null0
!
interface ve 3
ip policy route-map irp-peer

Case 3: Complex network infrastructure, multiple routers, no probing VLAN

Figure 2.4.3: PBR configuration via GRE tunnels

In specific complex scenarios, traffic from the IRP server should pass multiple routers before getting to
the provider. If a separate probing Vlan cannot be configured across all routers, GRE tunnels from IRP
to the Edge routers should be configured.

It is also possible to configure the PBRs without GRE tunnels, by setting PBR rules on each
transit router on the IRP↔ provider path.

Brocade routers do not support PBR set up on GRE tunnel interfaces. In this case the workaround
is to configure PBR on each transit interface towards the exit edge router(s) interface(s).

GRE Tunnels configuration

Listing 2.23: IRP-side IPv4 GRE tunnel CLI configuration

modprobe ip_gre
ip tunnel add tun0 mode gre remote 10.10.0.1 local 10.0.0.2 ttl 64 dev eth0
ip addr add dev tun0 10.0.1.2/32 peer 10.0.1.1/32
ip link set dev tun0 up

Listing 2.24: IRP-side IPv4 GRE tunnel configuration using standard CentOS configs
#/etc/sysconfig/network-scripts/ifcfg-tun0
DEVICE=tun0
TYPE=GRE
ONBOOT=yes
MY_INNER_IPADDR=10.0.1.2
MY_OUTER_IPADDR=10.0.0.2
CHAPTER 2. CONFIGURATION 61

PEER_INNER_IPADDR=10.0.1.1
PEER_OUTER_IPADDR=10.10.0.1
TTL=64

Listing 2.25: Router-side IPv4 GRE tunnel configuration (Vyatta equipment)

set interfaces tunnel tun0
set interfaces tunnel tun0 address 10.0.1.1/30
set interfaces tunnel tun0 description "IRP Tunnel 1"
set interfaces tunnel tun0 encapsulation gre
set interfaces tunnel tun0 local-ip 10.10.0.1
set interfaces tunnel tun0 remote-ip 10.0.0.2

Listing 2.26: Router-side IPv4 GRE tunnel configuration (Cisco equipment)

interface Tunnel0 routers
ip address 10.0.1.1 255.255.255.252
tunnel mode gre ip
tunnel source Loopback1
tunnel destination 10.0.0.2

Listing 2.27: Router-side IPv4 GRE tunnel configuration (Juniper equipment)

interfaces {
gr-0/0/0 {
unit 0 {
tunnel {
source 10.0.0.2;
destination 10.10.0.1;
}
family inet {
address 10.0.1.1/32;
}
}
}
}

The above configuration is for the first edge router (R1). One more GRE tunnel should be configured
on the 2nd router (R2).
As soon as the GRE tunnels are configured, the source-based routing and the PBR rules should be
configured, similar to the previous section.
Listing 2.28: IRP server IPv4 source-based routing via GRE using the ‘ip‘ command
ip route add default dev tun0 table 201
ip route add default dev tun1 table 202
ip route add default dev tun2 table 203
ip rule add from 10.0.1.2 table 201 pref 32101
ip rule add from 10.0.1.6 table 202 pref 32102
ip rule add from 10.0.1.10 table 202 pref 32103

Listing 2.29: IRP server IPv4 source-based routing via GRE using standard CentOS configuration files
#/etc/sysconfig/network-scripts/route-tun0:
default dev tun0 table 201
default dev tun1 table 202
default dev tun2 table 203
#/etc/sysconfig/network-scripts/rule-tun0:
from 10.0.1.2 table 201 pref 32101
from 10.0.1.6 table 202 pref 32102
from 10.0.1.10 table 203 pref 32103
CHAPTER 2. CONFIGURATION 62

Router configuration looks similar to the previous cases. A Cisco/Brocade example will be provided.

Listing 2.30: Cisco IPv4 PBR configuration: 2 routers, 2 providers, separate Vlan
#Router R1
access-list 1 permit ip host 10.0.1.2
access-list 2 permit ip host 10.0.1.6
!
route-map irp-peer permit 10
match ip address 1
set ip next-hop 10.11.0.1
set interface Null0
!
route-map irp-peer permit 20
match ip address 2
set ip next-hop 10.12.0.1
set interface Null0
!

interface Tunnel0
ip policy route-map irp-peer
interface Tunnel1
ip policy route-map irp-peer

#Router R2
access-list 1 permit ip host 10.0.1.10
!
route-map irp-peer permit 10
match ip address 1
set ip next-hop 10.13.0.1
set interface Null0
!
interface Tunnel0
ip policy route-map irp-peer

Case 4: Internet Exchanges configuration examples

The PBR rules for Cisco routers/switches are generated by IRP, following the template below.

Listing 2.31: Cisco IPv4 PBR configuration template

!--- repeated block for each peering partner
no route-map <ROUTEMAP> permit <ACL>
no ip access-list extended <ROUTEMAP>-<ACL>

ip access-list extended <ROUTEMAP>-<ACL>

permit ip host <PROBING_IP> any dscp <PROBING_DSCP>

route-map <ROUTEMAP> permit <ACL>

match ip address <ROUTEMAP>-<ACL>
set ip next-hop <NEXT_HOP>
set interface Null0

!--- block at the end of PBR file

interface <INTERFACE>
ip policy route-map <ROUTEMAP>

The “<>” elements represent variables with the following meaning:

• <ROUTEMAP> represents the name assigned by IRP and equals the value of the Route Map
parameter in PBR Generator ("irp-ix" in Figure 4)
CHAPTER 2. CONFIGURATION 63

• <ACL> represents a counter that identifies individual ACL rules. This variable’s initial value is
taken from ACL name start field of PBR Generator and is subsequently incremented for each ACL
• <PROBING_IP> one of the configured probing IPs that IRP uses to probe link characteristics
via different peering partners. One probing IP is sufficient to cover up to 64 peering partners
• <PROBING_DSCP> an incremented DSCP value assigned by IRP for probing a specific peering
partner. This is used in combination with the probing IP
• <NEXT_HOP> represents the IP address identifying the peering partner on the exchange. This
parameter is retrieved during autoconfiguration and preserved in Exchange configuration
• <INTERFACE> represents the interface where traffic conforming to the rule will exist the Ex-
change router. This is populated with the Interface value of PBR Generator

The PBR rules for Brocade non-XMR router/switches are generated by IRP, following the template
below.

Listing 2.32: Cisco IPv4 PBR configuration template

!--- repeated block for each peering partner
no route-map <ROUTEMAP> permit <ACL>
no ip access-list extended <ROUTEMAP>-<ACL>

ip access-list extended <ROUTEMAP>-<ACL>

permit ip host <PROBING_IP> any dscp-matching <PROBING_DSCP>

route-map <ROUTEMAP> permit <ACL>

match ip address <ROUTEMAP>-<ACL>
set ip next-hop <NEXT_HOP>
set interface Null0

!--- block at the end of PBR file

interface <INTERFACE>
ip policy route-map <ROUTEMAP>

The “<>” elements represent variables with the same meaning as per Cisco example.

Brocade XMR routers use keyword “dscp-mapping” instead of “dscp-matching”.

The PBR rules for Juniper routers/switches are generated by IRP, following the template below.

It is important to note that the different sections of the PBR rules (load replace/merge relative
terminal) should be entered independently and not as a single file that is output by IRP. Also take
note that the last group includes a ’load merge’ combo and not a ’load replace’ as the first three
groups.

Listing 2.33: Juniper IPv4 PBR configuration template

load replace relative terminal
[Type ^D at a new line to end input]

interfaces {
<INTERFACE> {
unit <INTERFACE_UNIT> {
family inet {
filter {
replace:
CHAPTER 2. CONFIGURATION 64

input <ROUTEMAP>;
}
}
}
}
}

load replace relative terminal

[Type ^D at a new line to end input]

firewall {
family inet {
filter <ROUTEMAP> {
replace:
term <ROUTEMAP><ACL> {
from {
source-address <PROBING_IP>;
dscp <PROBING_DSCP>;
}
then {
routing-instance <ROUTEMAP><ACL>-route;
}
}
...
replace:
term default {
then {
accept;
}
}
}
}
}

load replace relative terminal

[Type ^D at a new line to end input]

routing-instances {
replace:
<ROUTEMAP><ACL>-route {
instance-type forwarding;
routing-options {
static {
route 0.0.0.0/0 next-hop <NEXT_HOP>;
}
}
}
...
}

load merge relative terminal

[Type ^D at a new line to end input]

routing-options {
interface-routes {
CHAPTER 2. CONFIGURATION 65

replace:
rib-group inet <ROUTEMAP>rib;
}
rib-groups {
replace:
<ROUTEMAP>rib {
import-rib [ inet.0 <ROUTEMAP><ACL>-route.inet.0 ... ];
}
}
}

The “<>” elements represent variables with the following meaning:

• <INTERFACE> represents the interface where traffic conforming to the rule will exist the Ex-
change router. This is populated with the Interface value of PBR Generator
• <INTERFACE_UNIT> is the value of the Interface Unit parameter in PBR Generator
• <ROUTEMAP>represents the name assigned by IRP and equals the value of the Route Map
parameter in PBR Generator
• <ACL> represents a combined counter like "00009" that identifies individual ACL rules. This
variable’s initial value is taken from ACL name start field of PBR Generator and is subsequently
incremented for each ACL
• <PROBING_IP> one of the configured probing IPs that IRP uses to probe link characteristics
via different peering partners. One probing IP is sufficient to cover up to 64 peering partners
• <PROBING_DSCP> an incremented DSCP value assigned by IRP for probing a specific peering
partner. This is used in combination with the probing IP
• <NEXT_HOP> represents the IP address identifying the peering partner on the exchange. This
parameter is retrieved during autoconfiguration and preserved in Exchange configuration

Note that Juniper routers/switches need an additional parameter in order to correctly configure
PBRs - Interface unit.

Verifying PBR configuration

To ensure that the PBRs are properly configured on the router(s), the standard *NIX traceroute com-
mand can be used, by specifying each probing IP as the source IP address for tracing the route. The
route should pass through a different provider (note the ISP BGP neighbor IP).

Listing 2.34: PBR validation using ‘traceroute‘

root@server ~ $ traceroute -m 5 8.8.8.8 -nns 10.0.0.3
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 10.0.0.1 0.696 ms 0.716 ms 0.783 ms
2 10.11.0.1 0.689 ms 0.695 ms 0.714 ms
3 84.116.132.146 14.384 ms 13.882 ms 13.891 ms
4 72.14.219.9 13.926 ms 14.477 ms 14.473 ms
5 209.85.240.64 14.397 ms 13.989 ms 14.462 ms

root@server ~ $ traceroute -m 5 8.8.8.8 -nns 10.0.0.4

traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 10.0.0.1 0.696 ms 0.516 ms 0.723 ms
2 10.12.0.1 0.619 ms 0.625 ms 0.864 ms
3 83.16.126.26 13.324 ms 13.812 ms 13.983 ms
4 72.14.219.9 15.262 ms 15.347 ms 15.431 ms
5 209.85.240.64 16.371 ms 16.991 ms 16.162 ms
CHAPTER 2. CONFIGURATION 66

Another useful method for checking that the PBRs are properly configured is to use Explorer self check
option. Make sure that the providers are added to IRP configuration before executing Explorer self
check.

Listing 2.35: PBR validation using Explorer self check

root@server ~ $ /usr/sbin/explorer -s
Starting PBR check

PBR check failed for provider A[2]. Diagnostic hop information: IP

=10.11.0.12 TTL=3
PBR check succeeded for provider B[3]. Diagnostic hop information: IP
=10.12.0.1 TTL=3

In order to ensure that the PBRs are properly configured for Internet Exchanges, the method below can
be used.

Listing 2.36: PBR validation using ‘iptables‘ and ‘traceroute‘

root@server ~ $ iptables -t mangle -I OUTPUT -d 8.8.8.8 -j DSCP --set-dscp
<PROBING_DSCP>

root@server ~ $ traceroute -m 5 -nns <PROBING_IP> 8.8.8.8

traceroute to 8.8.8.8, 30 hops max, 60 byte packets
1 ...
2 ...
3 <NEXT_HOP> 126.475 ms !X^C

where

• <NEXT_HOP> is a Peering Partner’s next-hop IP address in IRP configuration

• <PROBING_DSCP> is a Peering Partner’s DSCP value in IRP configuration
• <PROBING_IP> is a Peering Partner’s probing IP address in IRP configuration

The first IP of the trace leaving client infrastructure should be on the Exchange and the next-hop should
belong to the correct Peering Partner.

iptables rules should be deleted after all tests are done.

2.4.1.1 Current route detection

Current route detection algorithm requires the explorer.infra_ips parameter to be configured. All the
IP addresses and subnets belonging to the infrastructure should be added here. These IP addresses can
be easily determined by using the traceroute command.

2.4.1.2 Providers configuration

Before Explorer starts probing the prefixes detected by the Collector, all providers should be configured.

For the complete list of provider settings please see the Providers settings section.
Before configuring providers in IRP, the BGP sessions need to be defined, see BGPd Configuration

Please note that some Brocade router models do not support SNMP counters per Vlan, therefore
physical interfaces should be used for gathering traffic statistics.
CHAPTER 2. CONFIGURATION 67

For the sample configuration below, let’s assume the following:

Figure 2.4.4: PBR configuration: single router and separate probing Vlan

ISP1 - the provider’s name

10.0.0.1 - Router IP configured on the probing Vlan
10.0.0.3 - Probing IP for ISP1, configured on the IRP server
10.11.0.1, 10.11.0.2 - IP addresses used for the EBGP session with the ISP, 10.11.0.2 being
configured on the router
400Mbps - the agreed bandwidth
1Gbps - the physical interface throughput
’public’ - read-only SNMP community configured on R1
GigabitEthernet2/1 - the physical interface that connects R1 to ISP1

As presented in Listing 2.37, all the parameters are pretty self-explanatory. Please check the BGP
Monitoring section as well.

Listing 2.37: Sample provider configuration

peer.1.95th = 400
peer.1.95th.bill_day = 1
peer.1.bgp_peer = R1
peer.1.cost = 6
peer.1.description = ISP1
peer.1.ipv4.next_hop = 10.11.0.1
peer.1.ipv4.probing = 10.0.0.3
peer.1.ipv4.diag_hop = 10.11.0.1
peer.1.ipv4.mon = 10.11.0.1 10.11.0.2
peer.1.limit_load = 1000
peer.1.shortname = ISP1
peer.1.snmp.interface = GigabitEthernet2/1
peer.1.snmp.ip = 10.0.0.1
peer.1.snmp.community = public
peer.1.mon.ipv4.bgp_peer = 10.11.0.1

SNMP parameters validation

To make sure that the SNMP community, IP and interface name/index are correct, the ‘snmpwalk‘ tool
can be run on the IRP server:

Listing 2.38: SNMP parameters validation

root@server ~ $ snmpwalk -v2c -c irp-public 10.0.0.1 ifDescr
IF-MIB::ifDescr.1 = STRING: GigabitEthernet1/1
IF-MIB::ifDescr.2 = STRING: GigabitEthernet1/2
IF-MIB::ifDescr.3 = STRING: GigabitEthernet2/1
IF-MIB::ifDescr.4 = STRING: GigabitEthernet2/2
IF-MIB::ifDescr.5 = STRING: GigabitEthernet2/3
CHAPTER 2. CONFIGURATION 68

IF-MIB::ifDescr.6 = STRING: GigabitEthernet2/4

root@server ~ $ snmpwalk -v2c -c irp-public 10.0.0.1 ifIndex

IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.2 = INTEGER: 2
IF-MIB::ifIndex.3 = INTEGER: 3
IF-MIB::ifIndex.4 = INTEGER: 4
IF-MIB::ifIndex.5 = INTEGER: 5
IF-MIB::ifIndex.6 = INTEGER: 6
CHAPTER 2. CONFIGURATION 69

2.5 BGPd Configuration

For IRP to inject the improved prefixes to the routing tables, proper iBGP sessions have to be configured
between the edge routers and the IRP. The IRP BGP daemon acts similarly to a route-reflector-client.
Following criteria need to be met:

1. An internal BGP session using the same autonomous system number (ASN) must be configured
between each edge router and the IRP. BGP sessions must not be configured with next-hop-self
(route reflectors can’t be used to inject routes with modified next-hop) - the next-hop parameter
advertised by IRP BGPd should be distributed to other iBGP neighbors.
2. route-reflector-client must be enabled for the routes advertised by IRP BGPd to be distributed to
all non-client neighbors.
3. Routes advertised by IRP BGPd must have a higher preference over routes received from external
BGP neighbors.
This can be done by different means, on the IRP or on the router side:

• Local-pref can be set to a reasonably high value in the BGPd configuration

• Communities can be appended to prefixes advertised by BGPd

Avoid colisions of localpref or communities values assigned to IRP within both its configu-
ration and/or on customer’s network.

• Multi-exit-discriminator (MED) can be changed to affect the best-path selection algorithm

• Origin of the advertised route can be left unchanged or overridden to a specific value (incom-
plete, IGP, EGP )

LocalPref, MED and Origin attribute values are set with the first nonempty value in this
order: 1) value from configuration or 2) value taken from incoming aggregate or 3) default
value specified in RFC4271.
Communities attribute value concatenates the value taken from incoming aggregate with con-
figuration value. The router should be configured to send no Communities attribute in case it
is required that IRP announces Communities attribute that contain only the configured value.

4. BGP next-hop must be configured for each provider configured in IRP (please refer to Providers
configuration and Providers settings)

Special attention is required if edge routers are configured to announce prefixes with empty
AS-Path. In some cases, improvements announced by BGPd may have an empty AS-Path. Thus,
when edge router does not have any prefix-list filtering enforced, all the current improvements can
be improperly advertised to routers - this may lead to policy violations and session reset. Refer to
AS-Path behavior in IRP BGPd regarding the way to disallow announcements with empty AS-Path.
None of the improvements advertised by IRP should be advertised to your external peers

We recommend the routes to be injected into the edge router which runs the BGP session with
the provider. This ensures that the routes are properly redistributed across the network.
For example, there are two routers: R1 and R2. R1 runs a BGP session with Level3 and R2 with
Cogent. The current route is x.x.x.x/24 with the next-hop set to Level3 and all the routers learn this
route via R1. The system injects the x.x.x.x/24 route to R2 with the next-hop updated to Cogent.
In this case the new route is installed on the routing table and it will be properly propagated to R1
(and other routers) via iBGP.
CHAPTER 2. CONFIGURATION 70

However if the system injects the new route to R1 instead of R2, the route’s next-hop will point to
R2 while R2 will have the next-hop pointing to R1 as long as the injected route is propagated over
iBGP to other routers. In this case a routing loop will occur.

In the following BGP session configuration example, the IRP server with IP 10.0.0.2 establishes an
iBGP session to the edge router (IP: 10.0.0.1). The local-pref parameter for the prefixes advertised
by IRP is set to 190. BGP monitoring (see BGP Monitoring, BGPd settings) is enabled.

Listing 2.39: iBGP session configuration example

bgpd.peer.R1.as = 65501
bgpd.peer.R1.our_ip = 10.0.0.2
bgpd.peer.R1.peer_ip = 10.0.0.1
bgpd.peer.R1.listen = 1
bgpd.peer.R1.localpref = 190
bgpd.peer.R1.shutdown = 0
bgpd.peer.R1.snmp.ip = 10.0.0.1
bgpd.peer.R1.snmp.community = public

Vendor-specific router-side iBGP session configuration examples:

Vyatta routers:

Listing 2.40: Vyatta IPv4 iBGP session configuration example

set protocols bgp 65501 neighbor 10.0.0.2 remote-as ’65501’
set protocols bgp 65501 neighbor 10.0.0.2 route-reflector-client
set protocols bgp 65501 parameters router-id ’10.0.0.1’

Listing 2.41: Vyatta IPv6 iBGP session configuration example

delete system ipv6 disable-forwarding
commit
set protocols bgp 65501 neighbor 2001:db8:2::2 remote-as ’65501’
set protocols bgp 65501 neighbor 2001:db8:2::2 route-reflector-client
set protocols bgp 65501 neighbor 2001:db8:2::2 address-family ’ipv6-unicast
’
set protocols bgp 65501 parameters router-id ’10.0.0.1’

Listing 2.42: Vyatta IPv4 route-map for setting local-pref on the router
set protocols bgp 65501 neighbor 10.0.0.2 route-map import ’RM-IRP-IN’
set policy route-map RM-IRP-IN rule 10 action ’permit’
set policy route-map RM-IRP-IN rule 10 set local-preference ’190’

Listing 2.43: Vyatta IPv6 route-map for setting local-pref on the router
set protocols bgp 65501 neighbor 2001:db8:2::2 route-map import ’RM-IRP-IN’
set policy route-map RM-IRP-IN rule 10 action ’permit’
set policy route-map RM-IRP-IN rule 10 set local-preference ’190’

Cisco routers:

Listing 2.44: Cisco IPv4 iBGP session configuration example

router bgp 65501
neighbor 10.0.0.2 remote-as 65501
neighbor 10.0.0.2 send-community
neighbor 10.0.0.2 route-reflector-client
CHAPTER 2. CONFIGURATION 71

Listing 2.45: Cisco IPv6 iBGP session configuration example

router bgp 65501
neighbor 2001:db8:2::2 remote-as 65501
neighbor 2001:db8:2::2 send-community
neighbor 2001:db8:2::2 route-reflector-client

or

router bgp 65501

neighbor 2001:db8:2::2 remote-as 65501
no neighbor 2001:db8:2::2 activate
address-family ipv6
neighbor 2001:db8:2::2 activate
neighbor 2001:db8:2::2 send-community
neighbor 2001:db8:2::2 route-reflector-client

Listing 2.46: Cisco IPv4 route-map for setting local-pref on the router
router bgp 65501
neighbor 10.0.0.2 route-map RM-IRP-IN input
route-map RM-IRP-IN permit 10
set local-preference 190

Listing 2.47: Cisco IPv6 route-map for setting local-pref on the router
router bgp 65501
neighbor 2001:db8:2::2 route-map RM-IRP-IN input
route-map RM-IRP-IN permit 10
set local-preference 190

Listing 2.48: Limiting the number of received prefixes for an IPv4 neighbor on Cisco
router bgp 65501
neighbor 10.0.0.2 maximum-prefix 10000

Listing 2.49: Limiting the number of received prefixes for an IPv6 neighbor on Cisco
router bgp 65501
neighbor 2001:db8:2::2 maximum-prefix 10000

Juniper equipment:

The Cluster ID must be unique in multi-router configuration. Otherwise improvements will not
be redistributed properly. Cluster ID is optional for single router networks.

Listing 2.50: Juniper IPv4 iBGP session configuration example

[edit]
routing-options {
autonomous-system 65501;
router-id 10.0.0.1;
}
protocols {
bgp {
group 65501 {
type internal;
cluster 0.0.0.1;
CHAPTER 2. CONFIGURATION 72

family inet {
unicast;
}
peer-as 65501;
neighbor 10.0.0.2;
}
}
}

Listing 2.51: Juniper IPv6 iBGP session configuration example

[edit]
routing-options {
autonomous-system 65501;
router-id 10.0.0.1;
}
protocols {
bgp {
group 65501 {
type internal;
cluster 0.0.0.1;
family inet6 {
any;
}
peer-as 65501;
neighbor 2001:db8:2::2;
}
}
}

Listing 2.52: Juniper IPv4 route-map for setting local-pref on the router
[edit]
routing-options {
autonomous-system 65501;
router-id 10.0.0.1;
}
protocols {
bgp {
group 65501 {
type internal;
peer-as 65501;
neighbor 10.0.0.2 {
preference 190;
}
}
}
}

Listing 2.53: Limiting the number of received prefixes for an IPv4 neighbor on Juniper
protocols {
bgp {
group 65501 {
neighbor 10.0.0.2 {
family inet {
any {
prefix-limit {
maximum 10000;
CHAPTER 2. CONFIGURATION 73

teardown;
}
}
}
}
}
}
}

2.5.1 AS-Path behavior in IRP BGPd

Every advertised prefix contains an AS-Path attribute. However, in some cases this attribute can be
empty.
For compatibility purposes, the IRP BGPd has a few algorithms handling the AS-Path attribute:

1. The advertised prefix will be marked with a recovered AS-Path attribute.

Recovered AS-Path is composed of consecutive AS-Numbers that are collected during exploring
process. Please note that recovered AS-Path may differ from the actual BGP Path.

2. The advertised prefix will be marked with the AS-Path from the aggregate received via BGP.
3. If the advertised prefix, for whatever reason, has an empty AS-Path, it can be announced or ignored,
depending on the BGPd configuration.

For a detailed description refer to bgpd.as_path.

In case certain routes should be preferred over decisions made by IRP, use one of the following
ways:
- Routers may be configured to have more preferable localpref / weight values for such routes so the
best path algorithm always selects these routes instead of the routes injected by the BGPd daemon.
- Routes may be filtered or have lower localpref / weight set up, using incoming route-map applied
to BGP session with IRP.
- Networks that must be completely ignored by IRP can be specified in the global.ignorednets
parameter, so no probing / improving / announcing will be made by IRP.

If split announcements are enabled in IRP (refer 4.1.4.16) the routes announced by IRP will be
more specific.

2.5.2 BGPd online reconfiguration

BGPd can be reconfigured without BGP sessions restart. This feature increases network stability and
reduces routers’ CPU load during session initiation.
Use the commands below to inform IRP BGPd to reload its configuration:

Listing 2.54: Reload IRP BGPd configuration

root@server ~ $ service bgpd reload
CHAPTER 2. CONFIGURATION 74

2.6 Failover configuration

Failover relies on many operating system, networking and database components to work together. All
these must be planned in advance and require careful implementation.

We recommend that you request Noction’s engineers to assist with failover configuration.

Subsequent sections should be considered in presented order when configuring a fresh IRP failover setup.
Refer to them when troubleshooting or restoring services.

2.6.1 Initial failover configuration

Prerequisites
Before proceeding with failover configuration the following prerequisites must be met:

• One IRP node is configured and fully functional. We will refer to this node as $IRPMASTER.
• Second IRP node is installed with the same version of IRP as on $IRPMASTER. We will refer to
this node as $IRPSLAVE.

When troubleshooting problems, besides checking matching IRP versions ensure the same versions
of irp MySQL databases are installed on both failover nodes.

• IRP services, MySQL and HTTP daemons are stopped on $IRPSLAVE node.
• Network operator can SSH to both $IRPMASTER and $IRPSLAVE and subsequent commands
are assumed to be run from a $IRPMASTER console.

$IRPMASTER and $IRPSLAVE must have different hostnames.

Configure communication channel from $IRPMASTER to $IRPSLAVE

This channel is used during failover configuration and subsequently $IRPMASTER uses it to synchronize
IRP configuration changes when these occur. It uses key-based authentication without a passphrase to
allow automated logins on $IRPSLAVE by $IRPMASTER processes.

Adjust firewalls if any so that $IRPMASTER node can access $IRPSLAVE via SSH.

Generate SSH keys pair WITHOUT passphrase:

Listing 2.55: Generate keys on $IRPMASTER

root@IRPMASTER ~ # ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa -C "
failover@noction"

Default keys files are used. In case your system needs additional keys for other purposes we advise
that those keys are assigned a different name. If this is not possible then keys file name designated
for failover use should be also specified in IRP configuration parameter global.failover_identity_file.

Copy public SSH key from master to slave instance:

CHAPTER 2. CONFIGURATION 75

Listing 2.56: Install public key on $IRPSLAVE

root@IRPMASTER ~ # cat ~/.ssh/id_rsa.pub | while read key; do ssh $IRPSLAVE
"echo $key >> ~/.ssh/authorized_keys"; done

Check if $IRPMASTER can login to $IRPSLAVE without using a password:

Listing 2.57: Check SSH certificate-based authentication works

root@IRPMASTER ~ # ssh $IRPSLAVE

Install certificate and keys for MySQL Multi-Master replication between $IRP-
MASTER and $IRPSLAVE
MySQL Multi-Master replication uses separate communication channels that also require authentication.
IRP failover uses key-based authentication for these channels too.

Adjust firewalls if any so that $IRPMASTER and $IRPSLAVE can communicate with each other
bidirectionally.

Create Certificate Authority and server/client certificates and keys. Commands must be run on both
$IRPMASTER and $IRPSLAVE nodes:

Listing 2.58: Generate CA and certificates

# cd && rm -rvf irp-certs && mkdir -p irp-certs && cd irp-certs

# openssl genrsa 2048 > ‘hostname -s‘-ca-key.pem

# openssl req -new -x509 -nodes -days 3600 -subj "/C=US/ST=CA/L=Palo Alto/O
=Noction/OU=Intelligent Routing Platform/CN=‘/bin/hostname‘ CA/
[email protected]" -key ‘hostname -s‘-ca-key.pem -out ‘
hostname -s‘-ca-cert.pem

# openssl req -newkey rsa:2048 -days 3600 -subj "/C=US/ST=CA/L=Palo Alto/O=

Noction/OU=Intelligent Routing Platform/CN=‘/bin/hostname‘ server/
[email protected]" -nodes -keyout ‘hostname -s‘-server-
key.pem -out ‘hostname -s‘-server-req.pem

# openssl rsa -in ‘hostname -s‘-server-key.pem -out ‘hostname -s‘-server-

key.pem

# openssl x509 -req -in ‘hostname -s‘-server-req.pem -days 3600 -CA ‘

hostname -s‘-ca-cert.pem -CAkey ‘hostname -s‘-ca-key.pem -set_serial 01
-out ‘hostname -s‘-server-cert.pem

# openssl req -newkey rsa:2048 -days 3600 -subj "/C=US/ST=CA/L=Palo Alto/O=

Noction/OU=Intelligent Routing Platform/CN=‘/bin/hostname‘ client/
[email protected]" -nodes -keyout ‘hostname -s‘-client-
key.pem -out ‘hostname -s‘-client-req.pem

# openssl rsa -in ‘hostname -s‘-client-key.pem -out ‘hostname -s‘-client-

key.pem

# openssl x509 -req -in ‘hostname -s‘-client-req.pem -days 3600 -CA ‘

hostname -s‘-ca-cert.pem -CAkey ‘hostname -s‘-ca-key.pem -set_serial 01
-out ‘hostname -s‘-client-cert.pem
CHAPTER 2. CONFIGURATION 76

Verify certificates. Commands must be run on both $IRPMASTER and $IRPSLAVE nodes:

Listing 2.59: Verify certificates

# openssl verify -CAfile ‘hostname -s‘-ca-cert.pem ‘hostname -s‘-server-
cert.pem ‘hostname -s‘-client-cert.pem

server-cert.pem: OK
client-cert.pem: OK

Install certificates in designated directories. Commands must be run on both $IRPMASTER and $IRP-
SLAVE nodes:

Listing 2.60: Install certificates in designated directories

# mkdir -p /etc/pki/tls/certs/mysql/server/ /etc/pki/tls/certs/mysql/client
/ /etc/pki/tls/private/mysql/server/ /etc/pki/tls/private/mysql/client/

# cp ‘hostname -s‘-ca-cert.pem ‘hostname -s‘-server-cert.pem /etc/pki/tls/

certs/mysql/server/
# cp ‘hostname -s‘-ca-key.pem ‘hostname -s‘-server-key.pem /etc/pki/tls/
private/mysql/server/
# cp ‘hostname -s‘-client-cert.pem /etc/pki/tls/certs/mysql/client/
# cp ‘hostname -s‘-client-key.pem /etc/pki/tls/private/mysql/client/

# cd && rm -rvf irp-certs

Cross copy client key and certificates:

Listing 2.61: Cross copy client key and certificates

root@IRPMASTER ~# scp "/etc/pki/tls/certs/mysql/server/$IRPMASTER-ca-cert.
pem" "$IRPSLAVE:/etc/pki/tls/certs/mysql/client/"
root@IRPMASTER ~# scp "/etc/pki/tls/certs/mysql/client/$IRPMASTER-client-
cert.pem" "$IRPSLAVE:/etc/pki/tls/certs/mysql/client/"
root@IRPMASTER ~# scp "/etc/pki/tls/private/mysql/client/$IRPMASTER-client-
key.pem" "$IRPSLAVE:/etc/pki/tls/private/mysql/client/"

root@IRPMASTER ~# scp "$IRPSLAVE:/etc/pki/tls/certs/mysql/server/$IRPSLAVE-

ca-cert.pem" "/etc/pki/tls/certs/mysql/client/"
root@IRPMASTER ~# scp "$IRPSLAVE:/etc/pki/tls/certs/mysql/client/$IRPSLAVE-
client-cert.pem" "/etc/pki/tls/certs/mysql/client/"
root@IRPMASTER ~# scp "$IRPSLAVE:/etc/pki/tls/private/mysql/client/
$IRPSLAVE-client-key.pem" "/etc/pki/tls/private/mysql/client/"

Adjust file permissions. Commands must be run on both $IRPMASTER and $IRPSLAVE nodes:

Listing 2.62: Set file permissions for keys and certificates

# chown -R mysql:mysql /etc/pki/tls/certs/mysql/ /etc/pki/tls/private/mysql
/
# chmod 0600 /etc/pki/tls/private/mysql/server/* /etc/pki/tls/private/mysql
/client/*

Configure MySQL replication on $IRPSLAVE

Each node of MySQL Multi-Master replication is assigned its own identifier and references previously
configured certificates. There are also configuration parameters such as binary/relay log file names and
auto-increment and auto-increment-offset values.
CHAPTER 2. CONFIGURATION 77

IRP includes a template config file /usr/share/doc/irp/irp.my_repl_slave.cnf.template. The template

designates $IRPSLAVE as second server of the Multi-Master replication and includes references to ‘host-
name -s‘ that need to be replaced with the actual hostname of $IRPSLAVE before installing. Apply the
changes and review the configuration file. Alternatively a command like the example below can be used
to create $IRPSLAVE config file from template. Make sure to use the actual short hostname instead of
the provided variable:

Listing 2.63: Example $IRPSLAVE configuration from template

root@IRPSLAVE ~# sed ’s|‘hostname -s‘|$IRPSLAVE|’ < /usr/share/doc/irp/irp.
my_repl_slave.cnf.template > /etc/noction/mysql/irp.my_repl_slave.cnf

The config file created above must be included into $IRPSLAVE node’s MySQL config my.cnf. It is
recommended that line “!include /etc/noction/mysql/irp.my_repl_slave.cnf” is un-commented.
Check Multi-Master configuration on $IRPSLAVE:

Listing 2.64: Check MySQL on $IRPSLAVE works correctly

root@IRPSLAVE ~# service mysqld start
root@IRPSLAVE ~# tail -f /var/log/mysqld.log
root@IRPSLAVE ~# mysql irp -e "show master status \G"
root@IRPSLAVE ~# service mysqld stop

Configure MySQL replication on $IRPMASTER

Configuring $IRPMASTER is done with running services.

Configuring MySQL Multi-Master replication on $IRPMASTER should only be done after con-
firming it works on $IRPSLAVE.

Similarly to $IRPSLAVE above IRP comes with a template configuration file for $IRPMASTER -
/usr/share/doc/irp/irp.my_repl_master.cnf.template. The template designates $IRPMASTER as first
server of the Multi-Master replication and includes references to ‘hostname -s‘ that need to be replaced
with the actual hostname of $IRPMASTER before installing. Apply the changes and review the resulting
configuration file.
Alternatively a command like the example below can be used to create $IRPMASTER config file from
template. Make sure to use the actual hostname instead of the provided variable:

Listing 2.65: Set $IRPMASTER as a first node for Multi-Master replication

root@IRPMASTER ~# sed ’s|‘hostname -s‘|$IRPSLAVE|’ < /usr/share/doc/irp/irp
.my_repl_master.cnf.template > /etc/noction/mysql/irp.my_repl_master.cnf

Again, the config file created above must be included into $IRPMASTER node’s MySQL config my.cnf.
It is recommended that line "!include /etc/noction/mysql/irp.my_repl_master.cnf" is un-commented.
Check MySQL Multi-Master configuration on $IRPMASTER:

Listing 2.66: Check MySQL on $IRPMASTER works correctly

root@IRPMASTER ~# service mysqld restart
root@IRPMASTER ~# tail -f /var/log/mysqld.log
root@IRPMASTER ~# mysql irp -e "show master status \G"
CHAPTER 2. CONFIGURATION 78

If Multi-Master configuration on $IRPMASTER fails or causes unrecoverable errors, a first trou-

bleshooting step is to comment back the included line in /etc/my.cnf on master and slave and restart
mysqld service to revert to previous good configuration.

Create replication grants on $IRPMASTER

Replication requires a MySQL user with corresponding access rights to replicated databases. The user
must be assigned a password and is designated as connecting correspondingly from $IRPMASTER
or $IRPSLAVE. Unfortunately hostnames cannot be used for this and the exact IP addresses of the
corresponding nodes must be specified.

The user is created only once in our procedure since after being created the database on $IRP-
MASTER is manually transferred to $IRPSLAVE and the user will be copied as part of this process.

Grant replication access to replication user:

Listing 2.67: Replication user and grants

mysql> CREATE USER ’irprepl’@’<mysql_slave1_ip_address>’ IDENTIFIED BY ’<
replication_user_password>’;
mysql> GRANT REPLICATION SLAVE ON *.* TO ’irprepl’@’<
mysql_masterslave1_ip_address>’ REQUIRE CIPHER ’DHE-RSA-AES256-SHA’;
mysql> CREATE USER ’irprepl’@’<mysql_master2_ip_address>’ IDENTIFIED BY ’<
replication_user_password>’;
mysql> GRANT REPLICATION SLAVE ON *.* TO ’irprepl’@’<
mysql_slave2_ip_address>’ REQUIRE CIPHER ’DHE-RSA-AES256-SHA’;

Copy IRP database configuration and database to $IRPSLAVE

IRP failover copies the irp database and corresponding configuration file from $IRPMASTER to $IRP-
SLAVE before activating second server. This avoids the need to manually verify and synchronize iden-
tifiers.
Copy root’s .my.cnf config file if exists:

Listing 2.68: Copy database root user configuration file

root@IRPMASTER ~# scp /root/.my.cnf $IRPSLAVE:/root/

Copy config files:

Listing 2.69: Copy database configuration files

root@IRPMASTER ~# scp /etc/noction/db.global.conf /etc/noction/db.frontend.
conf $IRPSLAVE:/etc/noction/

Preliminary copy database files:

Listing 2.70: Copy database data files

root@IRPMASTER ~# rsync -av --progress --delete --delete-after --exclude="
master.info" --exclude="relay-log.info" --exclude="*-bin.*" --exclude
="*-relay.*" /var/lib/mysql/ $IRPSLAVE:/var/lib/mysql/
CHAPTER 2. CONFIGURATION 79

Preliminary copy ensures that large files that take a long time to copy are synced to $IRPSLAVE
without stopping MySQL daemon on $IRPMASTER and only a reduced number of differences will
need to by synced while MySQL is stopped. This operation can be rerun one more time to reduce
the duration of the downtime on $IRPMASTER even more.

Finish copy of database files (CentOS 7):

Listing 2.71: Copy differences of database files (CentOS 7)

root@IRPMASTER ~# systemctl stop httpd24-httpd mariadb
root@IRPMASTER ~# systemctl start irp-stop-nobgpd.target

systemctl start irp-shutdown-except-bgpd.target

systemctl start irp-shutdown.target

root@IRPMASTER ~# cd /var/lib/mysql && rm -vf ./master.info ./relay-log.

info ./*-bin.* ./*-relay.*
root@IRPMASTER ~# rsync -av --progress --delete --delete-after /var/lib/
mysql/ $IRPSLAVE:/var/lib/mysql/

The procedure above tries to reduce the downtime of MySQL daemon on $IRPMASTER.
During this time BGPd preserves IRP Improvements. Make sure this action takes less than
bgpd.db.timeout.withdraw.

Finish copy of database files (CentOS 6):

Listing 2.72: Copy differences of database files (CentOS 6)

root@IRPMASTER ~# service irp stop nobgpd

root@IRPMASTER ~# service httpd24-httpd stop

root@IRPMASTER ~# service mysqld stop

root@IRPMASTER ~# cd /var/lib/mysql && rm -vf ./master.info ./relay-log.

info ./*-bin.* ./*-relay.*
root@IRPMASTER ~# rsync -av --progress --delete --delete-after /var/lib/
mysql/ $IRPSLAVE:/var/lib/mysql/

The procedure above tries to reduce the downtime of MySQL daemon on $IRPMASTER.
During this time BGPd preserves IRP Improvements. Make sure this action takes less than
bgpd.db.timeout.withdraw.

Start MySQL on $IRPSLAVE and check if there are no errors at /var/log/mysqld.log

First $IRPSLAVE must be checked.

Start MySQL on IRP master and check if there are no errors at /var/log/mysqld.log
CHAPTER 2. CONFIGURATION 80

Start replication (Slaves) on both $IRPMASTER and $IRPSLAVE

MySQL Multi-Master replication uses a replication scheme where a replicating master acts as both
replication master and replication slave. The steps above configured IRP nodes to be capable of acting
as replication masters. Here we ensure that both nodes are capable of acting as replication slaves.
IRP provides a template command that needs to be adjusted for each $IRPMASTER and $IRPSLAVE
and will instruct MySQL daemon to take the replication slave role. The template is /usr/share/doc/ir-
p/changemasterto.template.

The template generates a different command for each $IRPMASTER and $IRPSLAVE nodes and
requires multiple values to be reused from configuration settings described above. The command
that is run on one node points to the other node as its master.

Make $IRPMASTER a replication slave for $IRPSLAVE:

Listing 2.73: Set $IRPMASTER as replication slave

$IRPMASTER-mysql>
CHANGE MASTER TO
MASTER_HOST=’$IRPSLAVE-ip-address’,
MASTER_USER=’irprepl’,
MASTER_PASSWORD=’$IRPSLAVE-password>’,
MASTER_PORT=3306,
MASTER_LOG_FILE= ’$IRPSLAVE--bin.000001’,
MASTER_LOG_POS= <$IRPSLAVE-bin-log-position>,
MASTER_CONNECT_RETRY=10,
MASTER_SSL=1,
MASTER_SSL_CAPATH=’/etc/pki/tls/certs/mysql/client/’,
MASTER_SSL_CA=’/etc/pki/tls/certs/mysql/client/$IRPSLAVE-ca-cert.pem’,
MASTER_SSL_CERT=’/etc/pki/tls/certs/mysql/client/$IRPSLAVE-client-cert.pem
’,
MASTER_SSL_KEY=’/etc/pki/tls/private/mysql/client/$IRPSLAVE-client-key.pem
’,
MASTER_SSL_CIPHER=’DHE-RSA-AES256-SHA’;

You must manually check what values to assign to

$IRPSLAVE-bin.000001 and <$IRPSLAVE-bin-log-position>
by running the following MySQL command on $IRPSLAVE
mysql> show master status
For the initial configuration the values for $IRPSLAVE-bin.000001 and <$IRPSLAVE-bin-log-
position> must be as follows:
Binlog file: $IRPSLAVE–bin.000001
Binlog position: 106

Run the commands below to run the replication and check the slave status:

Listing 2.74: Starting replication slave on $IRPMASTER

mysql> START SLAVE \G
mysql> show slave status \G
CHAPTER 2. CONFIGURATION 81

Check the Slave_IO_State, Last_IO_Errno, Last_IO_Error, Last_SQL_Errno,

Last_SQL_Error values for errors. Make sure there are no errors.

Make $IRPSLAVE a replication slave for $IRPMASTER:

Listing 2.75: Set $IRPSLAVE as replication slave

$IRPSLAVE-mysql>
CHANGE MASTER TO
MASTER_HOST=’$IRPMASTER-ip-address’,
MASTER_USER=’irprepl’,
MASTER_PASSWORD=’$IRPMASTER-password>’,
MASTER_PORT=3306,
MASTER_LOG_FILE= ’$IRPMASTER-bin.000001’,
MASTER_LOG_POS= <$IRPMASTER-bin-log-position>,
MASTER_CONNECT_RETRY=10,
MASTER_SSL=1,
MASTER_SSL_CAPATH=’/etc/pki/tls/certs/mysql/client/’,
MASTER_SSL_CA=’/etc/pki/tls/certs/mysql/client/$IRPMASTER-ca-cert.pem’,
MASTER_SSL_CERT=’/etc/pki/tls/certs/mysql/client/$IRPMASTER-client-cert.pem
’,
MASTER_SSL_KEY=’/etc/pki/tls/private/mysql/client/$IRPMASTER-client-key.pem
’,
MASTER_SSL_CIPHER=’DHE-RSA-AES256-SHA’;

You must manually check what values to assign to

$IRPMASTER-bin.000001and <$IRPMASTER-bin-log-position>
by running the following MySQL command on $IRPMASTER
mysql> show master status
For the initial configuration the values for $IRPMASTER-bin.000001 and <$IRPMASTER-bin-log-
position> must be as follows:
Binlog file: $IRPMASTER–bin.000001
Binlog position: 106

Run the commands below to run the replication and check the slave status:

Listing 2.76: Starting replication slave

mysql> START SLAVE \G
mysql> show slave status \G

Run IRP and HTTPD services on $IRPMASTER and $IRPSLAVE (CentOS 7):

Listing 2.77: Starting IRP services and Frontend (CentOS 7)

# systemctl start irp.target

Run IRP and HTTPD services on $IRPMASTER and $IRPSLAVE (CentOS 6):

Listing 2.78: Starting IRP services and Frontend (CentOS 6)

# service httpd24-httpd start
# service irp start
CHAPTER 2. CONFIGURATION 82

Start services on $IRPMASTER first if the actions above took very long in order to shorten
MySQL downtime.

Configure Failover using Wizard on $IRPMASTER

These steps should only be performed once SSH communication channel and MySQL Multi-Master irp
database replication have been setup and are verified to work. Refer subsections of Failover configuration
above.
Run failover wizard:
Login into IRP’s Frontend on $IRPMASTER node and run Configuration -> Setup wizard -> Setup
Failover.

A valid failover license should be acquired before failover configuration settings become available.

Configure IRP failover:

Follow Setup Failover wizard steps and provide the required details. Refer Setup Failover wizard for
details. Once your configuration changes are submitted IRP will validate configuration and if it is valid
the configuration will be synced to $IRPSLAVE.

Only after this synchronization step takes place will $IRPSLAVE node know what is its role in
this setup.

Apply configuration changes to edge routers:

Ensure designated probing IPs, BGP session(s) are also setup on edge routers. Refer to corresponding
sections of this document for details.
Enable failover:
Use IRP’s Frontend Configuration -> Global -> Failover to configure IRP failover on both nodes. Monitor
both IRP nodes and ensure everything is within expectations.

It is recommended that after finishing the preparatory steps above both IRP master and slave
nodes run with disabled failover for a short period of time (less than 1 hour) in order to verify that
all IRP components and MySQL Multi-Master replication work as expected. Keep this time interval
shortto avoid a split-brain situation when the two nodes make contradictory decisions.

Synchronize RRD statistics to $IRPSLAVE

RRD statistics is collected by both IRP failover nodes and needs not be synchronized during normal op-
eration. Still, when IRP failover nodes are setup at a large time interval between them it is recommended
that RRD files are synchronized too. This will ensure that past statistics is available on both IRP nodes.
During short downtimes of each of the nodes synchronization is less usefull but can be performed in order
to cover the short downtime gaps in RRD based graphs.
Sample command to synchronize IRP’s RRD statistics:

Listing 2.79: Synchronize RRD

root@IRPMASTER ~ # rsync -av /var/spool/irp/ $IRPSLAVE:/var/spool/irp
CHAPTER 2. CONFIGURATION 83

2.6.2 Re-purpose operational IRP node into an IRP failover slave

Sometimes an existing fully configured IRP node is designated to be re-purposed as a failover slave. Take
the following actions to make a node as an IRP slave:

1. upgrade IRP to the version matching IRP failover master node

2. create a backup copy of your configuration
3. delete the following configuration files: /etc/noction/irp.conf,/etc/noction/exchanges.conf,/etc/n
4. proceed with configuration as detailed in Initial failover configuration

2.6.3 Re-purpose operational IRP failover slave into a new master

If a master node fails catastrophically and cannot be recovered an operational IRP slave can be re-
purposed to become the new master. Once re-purposing is finished a new slave node can be configured
as detailed in Initial failover configuration.
Failover status bar (3.2.2) includes a context menu with functions to switch IRP failover node role.
Simply click on the failover status bar’s ’Slave node’ icon and when prompted if you want to switch role
confirm it.

If you switched a node’s role by error, simply change it one more time to revert to previous
configuration.

2.6.4 Recover prolonged node downtime or corrupted replication

Multi-Master replication used by IRP failover is able to cope with downtime of less than 24 hours.
Replication will not be able to continue in case of prolonged downtime or corrupt replications. Recovery
in this case might use either:

• new server: follow configuration steps as detailed in Initial failover configuration.

• same server: follow recovery steps below.

MySQL Multi-Master recovery prerequisites

Before proceeding with recovery of replication the following prerequisites must be met:

• Currently active IRP node is designated as MySQL sync ’origin’. This node currently stores
reference configuration parameters and data. These will be synced to the node being recovered
and we designate it as ’destination’.
• Recovery should be scheduled during non-peak hours.
• Recovery must finish before bgpd.db.timeout.withdraw (default 4h) expires. If recovery can not be
completed in time it is required to start MySQL on the active node.

MySQL Multi-Master recovery procedure

It shall be noted that recovery closely follows actions described in and with the clarification that data
and files from origin node target destination node (instead of IRP failover master files targeting IRP
failover slave). Refer sections , for details about the steps below.
The steps are as follows:

1. destination: stop httpd, irp, mysqld

2. origin: sync /etc/noction/db.* to slave:/etc/noction/
CHAPTER 2. CONFIGURATION 84

3. origin: sync /root/.my.cnf to slave:/root/.my.cnf

4. origin: sync /var/lib/mysql/ to slave:/var/lib/mysql/
exclude files:
master.info relay-log.info -bin.* -relay.*

wait until sync at (4) succeeds and continue with:

5. origin: stop httpd, irp (except bgpd), mysqld
6. origin: delete files master.info relay-log.info -bin.* -relay.*

7. origin: sync /var/lib/mysql/ to slave:/var/lib/mysql/

8. destination: start mysqld and check /var/log/mysqld.log for errors
9. origin: start mysqld and check /var/log/mysqld.log for errors
10. origin: run CHANGE MASTER TO from the /usr/share/doc/irp/changemasterto template

11. destination: run CHANGE MASTER TO from the /usr/share/doc/irp/changemasterto template

12. destination: show slave status \G
13. origin: show slave status \G

14. origin: start IRP (bgpd should be already running), httpd

15. destination: start IRP, httpd

2.7 Frontend Configuration

The system frontend can be accessed over HTTP or HTTPS using the main IP or the FQDN of the IRP
server. If the IRP server is firewalled, the TCP:80 and TCP:443 port must be open for each IP/network
that should have access to the frontend.

The default username and password are “admin”/”admin”. It is strongly recommended to change
the password ASAP. Feel free to use the frontend once it becomes accessible. Alternatively local
account passwords can be set using irp-passwd utility under root account directly on the IRP server,
for example:
#irp-passwd.sh <username> <new-password>

Default “admin” user is created only for the initial IRP launch. Separate user accounts must be created
for each user. A separate administrative account must be created for the Noction support team as well.

Do not use the default “admin” user account for regular system access.

For the frontend to function correctly, server’s time zone should be configured in /etc/php.ini.
Example: date.timezone=America/Chicago.
CHAPTER 2. CONFIGURATION 85

2.8 Administrative Components

There are two additional components in the IRP system: dbcron and irpstatd. For detailed configu-
ration parameters description, please see the Administrative settings section.
Dbcron handles periodic database maintenance tasks and statistics processing.
Irpstatd gathers interface statistics from the edge routers. For this component to function properly, the
peer.X.snmp.ip/peer.X.snmp.ipv6, peer.X.snmp.interface and peer.X.snmp.community need to be set for
each provider.
For more details, see the Providers configuration and Providers settings sections.

2.9 IRP Initial Configuration

The platform initial configuration can be performed by using the Initial Setup wizard (Initial Setup)
which can be accessed via the main platform IP address over HTTP or HTTPS protocol.
Example: https://10.11.12.13

2.10 IRP software management

Software repository
IRP packages (current, new and old versions) are available in Noction’s repositories hosted at repo.noction.com.
Coordinate getting access to the repository with Noction’s representatives.

Software installation and upgrade

Once the IRP packages repository is configured standard CentOS package management tools like yum
are used to install, upgrade or downgrade IRP.
Installation:
yum install irplite

Optional packages irplite-documentation and irplite-api-samples should be installed separately.

Upgrade to latest version:

yum upgrade "irplite*"

Downgrade to a specific version:

yum downgrade "irplite*1.0*"

2.11 Starting, stopping and status of IRP components

In order to determine the status of IRP components or to start, stop and restart them standard CentOS
utilities are used.

Critical errors preventing startup of a particular component are logged to console. Details can
be obtained from logs stored in /var/log/irp/ directory.
CHAPTER 2. CONFIGURATION 86

Managing software components

Starting single component:
systemctl start explorer

Multiple components separated by space can be listed

Stopping single component:

systemctl stop explorer

Multiple components separated by space can be listed

Starting all components:

systemctl start irp.target

This also starts all prerequisites

Stopping all components:

systemctl start irp-shutdown.target

Stopping all components except bgpd:

systemctl start irp-shutdown-except-bgpd.target

Restarting all components:

systemctl start irp-shutdown.target
systemctl start irp.target

Obtaining overall status of all components:

systemctl list-dependencies irp.target

Individual statuses can be checked by executing command systemctl status component_name

Managing software components in CentOS 6

Starting single component:
service explorer start

Stopping single component:

service explorer stop

Starting all components:

service irp start
CHAPTER 2. CONFIGURATION 87

Stopping all components:

service irp stop

Stopping all components except bgpd:

service irp stop nobgpd

Restarting all components:

service irp restart

Obtaining status of all components:

service irp status
Chapter 3

Using IRP

3.1 Getting started

The Frontend is a web interface with a comprehensive set of reports, graphs and diagnostic informa-
tion, which can reflect the current and historical network state, as well as the benefits of IRP network
optimization.

3.1.1 Accessing the system

The system frontend is accessible over HTTPS. IRP’s Frontend can be accessed using any major browser,
via the main IRP server IP address, or FQDN. E.g: http://irp-system.noction.com/. When opening the
Frontend’s URL, a login form will pop up:

Figure 3.1.1: Frontend login form

The username and password provided by the system administrator must be used. The “Remember me”
check-box can be used as well.
See also: Frontend Configuration

3.1.2 System dashboard

After a successful login, the default Dashboard will become available. It contains a standard set of
gadgets - elements, which can contain a reduced version of a report or a graph.

88
CHAPTER 3. USING IRP 89

Figure 3.1.2: Default Dashboard

CHAPTER 3. USING IRP 90

3.2 Frontend structure

The Frontend has several components that operate together and allow a regular user or administrator
to access various reports, graphs, or configuration pages.

1. The menu bar (see Figure 3.2.1) allows quick access to any part of the system via its menu items.

Figure 3.2.1: Menu bar

2. The sidebar (See section 3.2.1) is present on the Dashboard or on any graph or report. Its contents
depend on the current page, and can include filters, export functions, etc., as explained in the next
section.

3. The status bar (see Figure 3.2.2), allows one to access the global search function (Section 3.2.3),
as well as system events and notifications (See the Events section).

Figure 3.2.2: Status bar

3.2.1 Sidebar
All dashboards, reports, graphs and events have a sidebar, located at the left side of the screen. Depending
on the current page, the sidebar has a different set of controls, allowing users to manage the dashboard
components or to filter the report and graph results.
Every sidebar contains Export, Email and Print buttons. Reports can be exported in .xls, .csv, .pdf
formats, while the graphs and dashboard - only as .pdf .

Figure 3.2.3: Common sidebar buttons

The “Email” button allows scheduling email delivery of several reports or graphs, or even an entire
dashboard (Figure 3.2.4). Multiple destination email addresses can be used. Most elements in this form
have informative tool-tips, that can be seen when hovering over a specific control. Note that Email
Reports can be removed from <Username>→Emailing from the top panel.
CHAPTER 3. USING IRP 91

Figure 3.2.4: Email schedule configuration

In the case of a report, custom filters can be applied, so that only specific information will be displayed
in the report.

Figure 3.2.5: Report filters

On graphs, reports and dashboard, the time period which the information is being provided for, can be
selected by using the “Timestamp” button. An intuitive control will open, and the desired time period
can be selected.
CHAPTER 3. USING IRP 92

Figure 3.2.6: Time period selection

The number of results can be adjusted on a report page, using the “Results per page” slider.

Figure 3.2.7: Adjusting the number of results per page

Dashboards have a different sidebar layout.

The Delete button cannot be found in the default dashboard. The default dashboard cannot be
deleted.

System dashboard represents the overall system information which includes:

• Configured maximum number of improvements (Max IPv6 Improvements will be shown if IPv6 is
enabled)

• Number of Routing Policies improvements

• Number of improvements performed today
• The amount of traffic improved today
• Percentage of prefixes improved today

• Number of operating iBGP sessions

• Number of announcements to the edge router
• Announcements to each from configured routers
• Number of BGP sessions with the providers

• Number of operating BGP sessions with the providers

CHAPTER 3. USING IRP 93

Figure 3.2.8: Dashboard buttons

See also: Customizing the dashboard.

3.2.2 Failover status

When failover is configured and operational the status of the components is highlighted below system
dashboard for each master and slave nodes.

A valid failover license is required

Figure 3.2.9: Failover status of master node

Slave node status shall be considered together with component status in the dashboard. During normal
operation components Core, Explorer and Irppushd are stopped.
CHAPTER 3. USING IRP 94

Figure 3.2.10: Failover status of slave node

3.2.3 Global search

The Frontend allows to search for historical information on optimized prefixes, AS Numbers and AS
Names. The search function can be accessed using the Search box in the status bar.

Figure 3.2.11: Search box

Enter the needed prefix, AS number or AS name in the search box, then press Enter or click on the
magnifying glass button.
While entering the search terms, results are dynamically loaded in a modal window (Figure 3.2.12).

Figure 3.2.12: Dynamical global search results

When displaying the complete results, specific prefixes can be manually removed, or scheduled
for re-probing, by using the buttons next to each result (figure 3.2.13).
CHAPTER 3. USING IRP 95

Figure 3.2.13: Global search results

The Show all X results in Y category buttons can also be used, to display all matching results from the
selected category (Improvements or AS Path problems)

3.2.4 Warning bars

The warning bars are shown in case the attention should be drawn to operational/performance issues,
some specific IRP configuration settings or license/payment issues.

Figure 3.2.14: Precedence and improve in a group warning

Figure 3.2.15: Internal Monitor warning

Figure 3.2.16: Explorer performance warning

Figure 3.2.17: License expiration warning

3.3 Customizing the dashboard

The Default dashboard is the first screen that can be seen after a successful login to the system. It contains
a default set of gadgets, that can be customized by adding additional gadgets, removing unnecessary
gadgets, or modifying specific gadget’s settings.
CHAPTER 3. USING IRP 96

3.3.1 Custom dashboard gadgets

3.3.1.1 Platform status

Figure 3.3.1: Platform status

3.3.1.2 Interfaces status

Figure 3.3.2: Interfaces status

3.3.1.3 List of improved prefixes

Figure 3.3.3: List of improved prefixes

CHAPTER 3. USING IRP 97

3.3.1.4 AS-Path problems

Figure 3.3.4: AS-Path problems

CHAPTER 3. USING IRP 98

3.4 Reports
The IRP Frontend comes with a comprehensive set of reports, reflecting the current state of the network
as well as overall statistics on the system performance. As explained in the Sidebar section, the results
can be filtered by specific criteria for each report, and the time period of the reported data can also be
selected.

Figure 3.4.1: Reports menu

3.4.1 Platform overview

The “Platform Overview” report provides information on the overall rate of improvements based upon
latency, loss and cost. This report can be used for a quick overview of the overall system performance
and improvements.
One can see here that almost all the improved routes had a loss drop of 20% or more. For more than 80%
of the improved routes the loss was fully eliminated. There are also 7% of the improved routes, which
were redirected from a complete blackout. The report also provides the amount of the accomplished
route improvements made, based upon latency, and cost (if the Cost optimization is enabled, see Cost
optimization).

Figure 3.4.2: Platform overview

CHAPTER 3. USING IRP 99

3.4.2 Platform performance

The “Platform Performance” report offers overall statistics on improvements made, based upon perfor-
mance reasons, commit control and cost reasons on a daily, weekly and monthly basis.

Figure 3.4.3: Platform performance

3.4.3 Loss Improvements

The Loss Improvements report highlights how much of the "Packet loss problem" IRP detected and was
able to solve during a specific timeperiod.
Loss relative rates - highlights the percentual magnitude of the improvement in loss rates for groups of
destinations before and after IRP assessed and re-routed the traffic. The groups represent:

• All destinations that cover good and problematic destinations that registered traffic flowing to
during the timeperiod;
• Problem destinations that includes only routes with packet loss and improved by IRP;
• Loss eliminated represents those problem destinations that IRP was able to improve to a better
route with zero loss ;

• Loss reduced represents those problem destinations that IRP improved but was not able to reduce
loss altogether.
CHAPTER 3. USING IRP 100

Figure 3.4.4: Loss relative rates

Loss absolute rates - highlights the actual loss rate values before and after improvement for the same
groups of destinations as above. While the previous part of the report shows big numbers one needs to
know how big the packet loss problem actually is. The Loss absolute rates part of this report shows just
that. It displays an actual 10% or 5% or 3% or 1% value depending on how well-behaved the network
is. Values of 5-7% are usual. Values exceeding 10% are a bit worrisome. While it allows IRP to boast
a bigger impact for your network it also means that the routing information you receive is not very
accurate and that your providers might not be making the very best effort to service your traffic.

Figure 3.4.5: Loss absolute rates

CHAPTER 3. USING IRP 101

Problem destinations - highlights destination counts of affected routes in the groups as above. The charts
present the rate of problem destinations identified by IRP and subsequently highlights how well IRP was
able to improve on them. The counts allow you to infer whether there are 20 customers suffering minor
packet loss (say, 5% each) or 2 customers suffering severe packet loss (for example 50% each). If for
the top and middle graphs above IRP was able to reduce packet loss by 80% these pie charts allow you
to infer that there were 20 or 2 customers affected and that for some of them IRP solved the problem
completely and how many others are still suffering.

Figure 3.4.6: Loss-improved destinations

3.4.4 Latency Improvements

The Latency Improvements report presents an overview for the latency values during a specific timeperiod
and consists of the following parts:
Latency values (%) - highlights the percentage latency improvement for groups of destinations observed
before and after IRP assessed and re-routed the traffic. The groups represent:

• All destinations that cover good and problematic destinations that registered traffic flowing to
during selected timeperiod;

• Problematic destinations that cover those flows that IRP assessed to have excessive latency rates
for the default route;
• 20% or more cover those destinations where IRP was able to identify a route with latency improve-
ments of 20% or more;

• 50% or more cover those destinations where IRP was able to identify a route with latency improve-
ments of 50% or more
CHAPTER 3. USING IRP 102

Figure 3.4.7: Latency values (%)

Latency values (ms) - highlights the actual latency values before and after improvement for the same
groups of destinations as above.

Figure 3.4.8: Latency values (ms)

Latency destinations - highlights destination counts in the groups as above.

CHAPTER 3. USING IRP 103

Figure 3.4.9: Latency destinations

3.4.5 Probe conversion rates

Probe conversion rates - highlights the counts of networks/prefixes probed by IRP and their conversion
rate into any of the possible improvements:

• Commit

• Performance
• Cost

Figure 3.4.10: Probe conversion rates

3.4.6 Current improvements

The “Improvements” report provides a list of the currently active improvements as per network. A
drill-down capability to a specific ASN or prefix is also provided. The report shows the improved prefix
and the providers from and to which the traffic was redirected. It also provides the reason, which the
improvement was based upon, along with the performance values before and after the traffic was rerouted.
CHAPTER 3. USING IRP 104

If necessary, specific improvements can be manually deleted by clicking the checkbox next to the needed
prefix, afterward using the “Remove selected” button.

Figure 3.4.11: Current improvements

In some specific cases, the “From” and “To” fields will contain the same provider. This is not a
platform issue and is related to the Outage detection algorithm specifics. For more details see the
Outage detection and VIP Improvements sections.

The “ASN” field can be empty in case there is no information about the AS number (a prefix
belongs to) in the IRP ASN dictionary. As soon as the ASN information is available in the dictionary,
the field will be filled accordingly.

3.4.7 VIP Improvements

The “VIP Improvements” report contains a list of VIP Prefixes/VIP ASNs that have been introduced
by user and have also been translated by IRP. The list can be collapsed or expanded. Furthermore a
single prefix or the whole ASN may be reprobed from this report by clicking the “Reprobe” link. The
VIP Prefixes/ASN entries can be removed if necessary.

Figure 3.4.12: VIP Improvements

New VIP Prefixes/ASNs may be added to the system directly from this report using “Add VIP Prefix”
or “Add VIP ASN” buttons.
CHAPTER 3. USING IRP 105

Figure 3.4.13: Add VIP Prefix/ASN

3.4.8 Improvements to Exchanges

The “Improvements to Exchanges” report contains a complete list of current improvements where the new
provider is an Internet Exchange. The report includes data about improved prefixes and their relative
traffic by means of last minute and average bandwidth usage. Average bandwidth is estimated based on
current hour statistics.

Figure 3.4.14: Improvements to Exchanges

3.4.9 Historical records

The “Historical records” report contains a complete list of active and inactive improvements that were
made by the system. The report can be sorted using any of the columns by simply clicking on the column
header.
CHAPTER 3. USING IRP 106

Figure 3.4.15: Historical records

In some specific cases, the “From” and “To” fields will contain the same provider. This is not
a platform issue, and is related to the Outage detection algorithm specifics. Please see the Outage
detection section for more details

3.4.10 Providers overall

The “Providers Overall” report provides the number of prefixes that were rerouted from and to each of
the providers. In the example below, the traffic exchange between the providers is balanced. However, if
the difference between the outgoing and incoming traffic is significant, then the quality of that provider’s
network should be questioned. The report also shows the number of improved prefixes, based upon
performance and cost, for each of the providers.

Figure 3.4.16: Providers overall

3.4.11 Provider performance

During the day IRP makes many probes to determine performance characteristics of alternatives routes.
Provider performance report presents an aggregated view of this measurements for providers. The data
on the report highlights Packet loss and Average latency per provider putting the provider with best
packet loss data at the top.
CHAPTER 3. USING IRP 107

Figure 3.4.17: Provider performance

3.4.12 Cost overview

This report is available only if the system is running in Cost optimization mode (see Cost opti-
mization).

The “Cost Overview” report provides details regarding the cost savings, resulted from running IRP for
each provider. It displays the costs that would be normally paid to the providers and the costs incurred
after IRP has optimized the routing. The system calculates the total transit cost while running in Cost
improvement mode and compares it with the total transit cost that would be incurred without running
IRP. This difference is shown on the right side of the table as Cost Savings. The total cost savings
include the ones made by the Commit Control as well. If the provider’s billing periods differ, then it is
possible to choose a billing period for each provider separately.

Figure 3.4.18: Cost overview

3.4.13 AS pattern problems

This report is available only if Outage detection is enabled.
The “AS pattern problems” report contains all the as-patterns defined as problematic by the system.
The problem state is shown in the “Status” column, as follows:

• - A problem on the specified AS-Pattern is suspected

• - The problem was not confirmed by additional re-probing

• - The problem was confirmed, and all related prefixes were rerouted
CHAPTER 3. USING IRP 108

• - The problem was confirmed, but no better-performing route has been found

Figure 3.4.19: AS-pattern problems

3.4.14 Top volume prefixes

The “Top volume prefixes” displays a list of prefixes sorted by the total volume transferred to these
remote networks.

Figure 3.4.20: Top volume prefixes

3.4.15 Top volume AS

The “Top Volume AS” reports show the Autonomous Systems with the highest volume of traffic coming
from the monitored network. It can help you understand your traffic flow directions. If according to
this report a specific AS has significantly more traffic than the others, then the network operator may
consider getting a direct connection to this AS, to reduce traffic costs.
CHAPTER 3. USING IRP 109

Figure 3.4.21: Top volume AS

3.4.16 Top problem AS

The “Top Problem AS” report reveals the most problematic Autonomous Systems that the monitored
networks are sending traffic to. It does that by showing how many times the traffic going to that AS
was rerouted by the IRP.

Figure 3.4.22: Top problem AS

3.4.17 Prefix statistics

This report lists the total number of improvements as related to specific prefixes.
CHAPTER 3. USING IRP 110

Figure 3.4.23: Prefix statistics

3.4.18 AS statistics
The “AS statistics” report lists the total number of improvements that were enforced by the IRP system
for each AS.

Figure 3.4.24: AS statistics

3.4.19 Current inbound improvements

The “Improvements” report provides a list of the currently active improvements as per network. A
drill-down capability to a specific ASN or prefix is also provided. The report shows the improved prefix
and the providers from and to which the traffic was redirected. It also provides the reason, which the
improvement was based upon, along with the performance values before and after the traffic was rerouted.
If necessary, specific improvements can be manually deleted by clicking the check-box next to the needed
prefix, afterward using the “Remove selected” button.
CHAPTER 3. USING IRP 111

Figure 3.4.25: Current inbound improvements

Refer Inbound (beta) for details.

CHAPTER 3. USING IRP 112

3.5 Graphs
The IRP graphs are visual representations of data found in the reports. They provide an opportunity to
quickly identify problems in the network, as well as forecast periodic changes in the traffic patterns and
network behavior. For each graph you can adjust the time period which you want to get the results for,
and export them as PDF (see the Sidebar section).

Unless otherwise noted, all graphs are plotted using 5 minute intervals.

Figure 3.5.1: Graphs menu

3.5.1 Improvements by type

The “Improvements by type” graph shows the improvements made by IRP based upon: performance,
cost, commit, and outage detection. Various problem patterns can be detected in compliance with the
IRP activity shown in this graph. The graph provides information on the types of improvements that
are more frequent for the network in the current state. Different patterns can be noticed depending on
the improvement mode that is currently running.

Figure 3.5.2: Improvements by type

3.5.2 Performance improvements

The “Performance improvements” graph displays the prefixes improved based upon a performance reason.
By following this graph, the solved loss and latency issues, occurring in the network, can be monitored.
CHAPTER 3. USING IRP 113

The graph also provides the average, maximum and minimum number of improvements for each of the
improvement reasons during the selected time frame. The maximum peaks are indicators of a loss or
latency problem in the network.

Figure 3.5.3: Performance improvements

3.5.3 Probed prefixes and Improvements

The "Probed prefixes and Improvements " graph shows the number of improvements (Loss/Latency) per
time unit as reported to the total probed prefixes.

Figure 3.5.4: Probed prefixes and Improvements

3.5.4 New and retry improvements

After a particular improvement was made, IRP analyzes the path periodically. If the improvement is still
valid it leaves it as it is. The “New and retry improvements” graph provides the number of improvements
made for the first time as related to those, which were simply confirmed after re-probing.
CHAPTER 3. USING IRP 114

Figure 3.5.5: New and retry improvements

3.5.5 Improvements by probing source

The current graph displays the percentage of improvements that are differentiated by the probing source
including Commit Control, Outage Detection, VIP Probing, Regular and Retry Probing.

Figure 3.5.6: Improvements by probing source

3.5.6 Prefixes rerouted from provider

The current graph displays the number of prefixes that were rerouted from a specific provider in order
to resolve performance or cost issues.
CHAPTER 3. USING IRP 115

Figure 3.5.7: Prefixes rerouted from provider

3.5.7 Prefixes rerouted to provider

The graph displays the number of prefixes that were rerouted to a specific provider in order to resolve
performance or cost issues.

Figure 3.5.8: Prefixes rerouted to provider

3.5.8 Improvements by provider

The “Improvements by provider” graph shows all the improvements made by IRP for each of the providers.
You can monitor the issues occurring in the network, which triggered IRP to reroute the traffic. You
can compare the providers based on how they perform in the context of performance and cost. It also
provides the average, maximum and minimum number of improvements per provider for each of the
improvement reasons.
CHAPTER 3. USING IRP 116

Improvements by provider graph categorizes improvements by performance and cost. If cost

mode is disabled this graph shows the same data as Prefixes rerouted to provider graph.

Figure 3.5.9: Improvements by provider

3.5.9 Performance improvements by provider

Similar to the previous graph, the “Performance improvements by provider” displays a graphical repre-
sentation of historical values for all latency and loss-based improvements for each provider. The charts
are displayed next to each other for easier comparison and identification of regularities. Keep in mind
that specific provider lines can be removed from the graph. This way we are able to focus only on the
interesting data.
CHAPTER 3. USING IRP 117

Figure 3.5.10: Performance improvements by provider

3.5.10 Commit improvements by provider

If the Commit Control algorithm is enabled (see Commit Control), this graph will display an overview
of all the commit improvements that were enforced by IRP for a specific time period.

Figure 3.5.11: Commit improvements by provider

3.5.11 Total and Improved traffic

The Total and Improved Traffic graph allows you to see the positive impact of IRP on the network. It
provides the amount of improved traffic, shown in blue color, as related to the total traffic, shown in
green. Thus, you can monitor how much of the traffic is affected by IRP.
CHAPTER 3. USING IRP 118

Figure 3.5.12: Total and improved traffic

3.5.12 Bandwidth usage by provider

The Bandwidth Usage graph shows the total usage for each of the providers. It allows you to compare the
current traffic volume to the current 95th percentile and the commit 95th. The graph provides average,
maximum and minimum traffic usage values for each of the providers during the selected time period.

Figure 3.5.13: Providers bandwidth usage

3.5.13 Bandwidth usage and improvements

Bandwidth usage and improvements chart superimposes two charts for easy analysis and comparison of
how improvement counts correlate with bandwidth.
CHAPTER 3. USING IRP 119

Figure 3.5.14: Bandwidth usage and improvements

3.5.14 Total bandwidth usage

Figure 3.5.15: Total bandwidth usage

3.5.15 Inbound traffic distribution

The graph highlights distribution of inbound traffic across inbound prefixes and providers.
CHAPTER 3. USING IRP 120

Figure 3.5.16: Inbound traffic distribution

Inbound traffic distribution report includes a feature to cross-compare current values with a saved in-
bound traffic distribution from recent past. To cross-compare saved inbound traffic distribution with
freshly retrieved statistics, save and compare traffic shape changes and if they conform to expectations.
Refer Inbound (beta) for details.

3.6 Routing Policies

To see the full list of routing policies configured in the system, select the Routing Policies option from
the main menu and click on the Routing Policies option in the submenu. The list displays all the enabled
and disabled routing policies configured in IRP.

Figure 3.6.1: Routing Policies

As shown in the screenshot above, the list provides the following details:

• The prefix/ASN to which the policy was applied.

• A note to describe the policy.
• The ASN of the prefix to which the policy was applied.
CHAPTER 3. USING IRP 121

• The status of the VIP reprobing (enabled/disabled).

• The type of the policy ( allow/deny/static).
• Providers which are affected by the policy.
• Time passed after the last probe was performed towards this prefix.
• The time left before the next scheduled probe towards this prefix
• The available actions for the policy. These include:

– On-demand probing

– Editing the routing policy

– Enabling the routing policy

– Disabling the routing policy

You can collapse all the policies by clicking the "Collapse All" button and then expand them back by
clicking the "Expand all" button. You can use the "Remove all " button to delete all the policies. You
can also select a set of policies and use the "Remove selected" button to delete them.
A new routing policy can be added by pressing the "Add routing policy" button. The following window
will open up:

Figure 3.6.2: Routing policy window

The following parameters should be configured to add a routing policy:

• A flat to set if policy is VIP

• The policy status (enabled/disabled)
• Prefix/ASN: The prefix/ASN to which the policy is applied
• Note: A note to describe the policy VIP Reprobing: Enable/disable the VIP reprobing mechanism
• Policy Type: The type of the policy (allow/deny/static)
• The Providers for which the policy should be respected
CHAPTER 3. USING IRP 122

• Cascade flag for ASN policies that indicate if the policy should be enforced only for this AS or also
for downstream AS. If cascading is enabled it ensures that the policy is enforced for all traffic that
will eventually be routhed through this AS.

To see a list of all the Static Route policies, select Static Routes in the Routing Policies submenu. A list
like the one below will be displayed.

Figure 3.6.3: Static Routes

The list structure and the actions available are similar to those found in the Routing Policies list. Here
you can also add a new Static Route by using the "Add static route" button.
You can select the VIP option in the Routing Policies submenu to access a list of all the policies with
VIP reprobing enabled.

Figure 3.6.4: VIP policies

Here you can also enable VIP reprobing for a new prefix/ASN by using the "Add VIP" button.

3.7 Flowspec policies

To review Flowspec policies, select the Routing Policies option from the main menu and click on the
Flowspec Policies option in the submenu.

Flowspec must be enabled for BGPd to announce them.

The list displays all the enabled and disabled Flowspec policies configured in IRP.

Figure 3.7.1: Flowspec policies

As shown in the screenshot above, the list highlights:

CHAPTER 3. USING IRP 123

• A note to describe the policy.

• A source ASN/prefix and port(s) for matching packets.
• A destination prefix and port(s) for matching packets.

• The policy action (Throttle, Drop, Redirect or Redirect IP).

• Redirect IP or Provider for redirect policies.
• Protocols of matching packets, for example TCP, UDP or ICMP.

• Rate limits in Mbps for Throttle policies.

Flowspec policies like routing policies can be edited, enabled, disabled or deleted altogether.
A Flowspec policy is added by clicking on the designated button and specifying the following:

Figure 3.7.2: Flowspec policy popup

The following parameters should be configured to add a routing policy:

• Notes to describe the policy

• Status to set if policy is Enabled or Disabled

• Source ASN/Prefix/Port(s): The source ASN/prefix andport(s) of the IP packets that match. A
prefix in CIDR notation or a single IP address should be provided. Multiple valid TCP/UDP ports
can be provided as well as port ranges.

One Flowspec rule is created for each of the prefixes in the designated AS. ASN Flowspec policies
can be setup only on the source of IP packets.
CHAPTER 3. USING IRP 124

Ensure that routers are capable of processing a large number of Flowspec rules before setting
policies for an AS consisting of a very large number of network segments (prefixes).

• Destination Prefix/Port: The destination prefix/port attribute of the IP packets that match. Same
rules as for Source Prefix/Port(s) apply.

Prefixes MUST not equal or include probing IP addresses. Refer to for example
global.master_probing_interface, peer.X.ipv4.master_probingpeer.X.ipv6.master_probing.

• Protocols: Packet protocols that match the policy. Can be any combination of TCP, UDP and
ICMP.
• Policy Type: The type of the policy (Throttle, Drop or Redirect)
• Provider specifies one of the provider identifiers where traffic will be redirected. The provider is
set only for Redirect policies.

In order for Redirect policies to be implemented on routers a community value must be assigned
to them and VRF must be configured on the router itself. Consult Noction support for details.

• Rate limits the allowed bandwidth usage for matching traffic. The value is set only for Throttling
policies. The rate specifies a number in the range 1-4200 Mbps.

In case the provided Flowspec policy attributes are incomplete or invalid on attempting to submit
it a warning will be raised similarly as depicted in the popup.

For further details refer Flowspec configuration parameters for example: global.flowspec, core.flowspec.max,
core.flowspec.max_ipv6, bgpd.peer.X.flowspec, peer.X.flowspec.ipv4.redirect_community, peer.X.flowspec.ipv6.redirect_

3.8 Throttling excessive bandwidth use with Flowspec

Throttling excessive bandwidth use with Flowspec policies use the same Flowspec policies reports. Select
the Routing Policies option from the main menu and click on the Flowspec Policies option in the submenu.

Flowspec must be enabled for BGPd to announce them.

The list displays besides all the other Flowspec policies the automatic throttling rules marked with
Throttle at. These rules cannot be edited.

Figure 3.8.1: Throttling excessive bandwidth use with Flowspec policies

CHAPTER 3. USING IRP 125

Each of the rules sets a specific destination prefix to control outbound bandwidth use and sets an Mbps
rate based on past average bandwidth use.
Refer Core configuration for configuration details.

3.9 Maintenance windows

To review Maintenance windows, navigate to Policies option in main menu and click on Maintenance
Window option. The list displays all the current and future maintenance windows configured in IRP.
Refer Maintenance windows for details about maintenance windows.

Figure 3.9.1: Maintenance windows

As shown in the screen-shot above, the list highlights:

• Current color-coded status of the maintenance window where Red and Orange depict unloading
and actual maintenance phases while Blue and Green depict future planned windows and past
finished windows correspondingly.
• Begin and End date-times of maintenance windows.
• Provider under maintenance.

• Withdraw Improvement option specifies withdraw existing Outbound Improvements to that provider.
• Seconds to Unload defines time interval in seconds prior to beginning of the maintenance window
when IRP starts to unload outbound traffic from provider.

If Seconds to Unload/Seconds to Prepend time intervals set to zero IRP does not perform corre-
sponding action.

• Seconds to Prepend defines time interval in seconds prior to beginning of the maintenance window
when IRP announces all inbound prefixes with maximum prepends to provider in order to deflect
inbound traffic towards other providers.

Maintenance windows can be added, edited or deleted using the designated action buttons.
A maintenance window is added by specifying the following:
CHAPTER 3. USING IRP 126

Figure 3.9.2: Maintenance window popup

The parameters mentioned above except for Maintenance window are provided.
Maintenance windows are also highlighted on IRP Frontend sidebar with the title of the sidebar box
highlighting the status of the next maintenance window so that it is visible even if the contents of the
sidebar is collapsed.

Figure 3.9.3: Maintenance window sidebar

3.10 Events
The system events are diagnostic messages that were sent by the IRP Components (See also: IRP Com-
ponents), as well as notifications caused by the BGP monitoring algorithm (See also: BGP Monitoring).
The events are displayed in the status bar located at the bottom of the system frontend (See Frontend
structure). Several or all the events can be marked as “Read”, so they will no longer be displayed in the
status bar.
CHAPTER 3. USING IRP 127

Figure 3.10.1: System events

CHAPTER 3. USING IRP 128

3.11 Troubleshooting
The IRP Frontend provides several troubleshooting tools, that can offer you quick information on the
specific remote networks.

3.11.1 Traceroute
A traceroute to a specific IP address or hostname can be run from the Frontend. Results will be displayed
in the graphical and detailed formats, as in the examples below. The “Allow automatic improvement”
checkbox can also be used for automatic improvement of the provided IP address.

Figure 3.11.1: Traceroute

Figure 3.11.2: Traceroute results

CHAPTER 3. USING IRP 129

Figure 3.11.3: Traceroute exploring details results

3.11.2 Looking glass

A ping, traceroute or MTR can be run from the Frontend via the default route or via a specific provider.
CHAPTER 3. USING IRP 130

Figure 3.11.4: Looking glass

3.11.3 Whois
The “Whois” tool can query IP addresses, hostnames, network prefixes, or AS Numbers.
The following valid values can be entered in the search box:

• Hostname: fully qualified domain name, e.g: “domain.com”

• IP: valid IP address, e.g: “10.20.30.40”
• IP Block: any valid prefix, in CIDR format, e.g: “10.20.30.40/24”
• ASN: valid ASN, using the following format: “AS1234”

Figure 3.11.5: Whois

3.11.4 Prefix probing

IRP features a manual prefix probing function. One can manually submit a specific IP address for
probing and optimization by the system. Valid hostnames and IP addresses can be entered. The probing
CHAPTER 3. USING IRP 131

results will be displayed on the same page. Please note that IRP probes the exact entered IP address.
In case the IP address doesn’t respond, IRP runs the indirect probing process and optimizes the whole
prefix based on the indirect probing results.
Prefix probing allows to automatically or manually choose the best path for a particular IP address or
prefix.
In order to check what are the possible paths for a particular prefix, enter the IP address or prefix into the
field. If you prefer the prefix to be improved automatically, tick the check box “Improve automatically”
and press the “Submit for probing” button. Otherwise, just press the “Submit for probing” button and
wait until the system returns the probing results.

Figure 3.11.6: Prefix Probing

The system returns the probing results by displaying the loss, latency and cost values for each of the
providers.

Figure 3.11.7: Prefix probing results

After the probing results are received, choose the path you consider the best one. Note, that IRP
highlights the current and the best path. Use the “Reroute” button to redirect the traffic through the
selected provider. You can also set the selected provider as static for the probed prefix by pressing the
“Add static route” button or apply a specific routing policy to the probed prefix by clicking the “Add a
custom policy” button.
CHAPTER 3. USING IRP 132

3.12 Configuration editor

Starting with IRP version 1.7, all system parameters can be modified from the Frontend, using the
“Configuration” menu.
Depending on the parameter type, several control types are available:

• Text boxes, which can hold IP addresses, provider names and numerical values

• Auto-complete boxes, used for parameters that can take a limited set of values

• Auto-complete list boxes, usually containing a list of IPs / prefixes (in CIDR format) / ASNs

• Buttons

• Sliders, used to select a value from a predefined range

• Drop-down lists

• Toggle buttons (used for specific algorithms and BGP peers)

• Provider control buttons, can be used to suspend, resume, stop, or remove a provider
CHAPTER 3. USING IRP 133

3.12.1 Global configuration

Global settings can be adjusted from the Frontend, by using the “Configuration” menu.
Please refer to Global and Core Configuration for a detailed parameters description.
Several global parameters can be configured by selecting their desired values and clicking on the “Submit”
button.
Available parameters:
• Improvement mode (global.improve_mode)
• Management interface (global.master_management_interface)
• BGP mode (Intrusive / Non-intrusive, see global.nonintrusive_bgp)
• Probing interface(s). See global.master_probing_interface
• Prefix aggregation. If this parameter is enabled, the system operates at aggregate level, up to the
Maximum aggregate mask (see: global.aggregate, global.agg_ipv4_max, global.agg_ipv6_max)

Figure 3.12.1: Configuration editor: Global settings

The list of ignored networks (global.ignorednets) and ASNs (global.ignored.asn) can be modified on the
same page, either manually, or by importing a text file containing one IP/network in CIDR format or
ASN per line.

Figure 3.12.2: Configuration editor: Ignored prefixes

CHAPTER 3. USING IRP 134

Failover functionality is switched on or off (global.failover) as part of Global configuration too.

Failover license is required for Failover tab to be available.

Failover configuration covers the following parameters:

• Failover timer - time period in seconds during which master is considered alive and before slave
becomes active if there are no announcements from master (global.failover_timer_fail).
• Failback timer - time period before slave node will release control back to master (global.failover_timer_failback).
This is to protect from failover flapping between master and slave in case of master instability.
• Slave IPv4 address - indicates to master what is the slave node (global.failover_slave.ip). Master
uses this address to setup SSH connections and sync its own configuration files.
• Slave SSH port - indicates the TCP port number for SSH on the slave node (global.failover_slave.port).
Master uses this port to setup SSH connections and sync its own configuration files.

Figure 3.12.3: Configuration editor: Global failover

CHAPTER 3. USING IRP 135

3.12.2 Core configuration

All Core parameters affecting the decision-making algorithms can be modified on the next page (“Configuration→Core”).
See also: Global and Core Configuration, Core settings.
The following configuration parameters can be adjusted:

• Max IPv4 improvements (core.improvements.max, core.improvements.max_ipv6)

• Standard reprobing period (core.improvements.ttl.retry_probe) - the time period after which

a specific improvement is being scheduled for re-probing.
• VIP reprobing period (core.vip.interval.probe) - defines the VIP prefixes probing interval,
in seconds.

• Minimal probe lifetime (core.probes.ttl.min) - Ordinary probing will not be performed for a
specific prefix if its probe age is lower than this value.
• Allowed latency worsening (core.cost.worst_ms)
• Exploring queue slots (core.eventqueuelimit)

• Top-N relevant volume prefixes (core.improvements.retry_probe.volume_top_n)

• Relevant loss for loss-based improvements (core.performance.loss_pct) - a prefix will be improved
based on a loss cause only if the packet loss can be decreased by this value (in %)
• Relevant RTT difference (core.performance.rtt.diff_ms)

• Relevant RTT difference (%) (core.performance.rtt.diff_pct)

• Maximum probe lifetime (core.probes.ttl.max)

Figure 3.12.4: Configuration editor: Core configuration

Outage detection algorithm can be enabled and configured on the same page. Outage detection algorithm
can be enabled or disabled using the toggle On/Off buttons at the top of each form.
CHAPTER 3. USING IRP 136

Figure 3.12.5: Configuration editor: Outage detection

Throttling excessive bandwidth use with Flowspec can be enabled and configured on the same page. The
feature can be enabled or disabled using the toggle On/Off buttons at the top of each form.

Figure 3.12.6: Configuration editor: Overusage

The following configuration parameters can be adjusted:

• Overusage interval (core.overusage.check_interval) sets the frequency in seconds of checking for

excessive bandwidth use.
• Overusage rule retention (core.overusage.hold_timer) sets how much time a rule is kept after
bandwidth use returns to normal.
• Prefix BW average time (core.overusage.out.average.period) sets the number of hours used to
determine the average bandwidth use of a prefix.
• Prefix relevant BW (core.overusage.out.average.relevant_min) sets the relevant bandwidth use
in Mbps by a prefix before considering any rules for it.
CHAPTER 3. USING IRP 137

• Overusage throttle multiplier (core.overusage.out.threshold.throttle) sets the multiplier to ap-

ply to average prefix bandwidth use when setting a rule.
• Overusage threshold multiplier (core.overusage.out.threshold.trigger) sets the threshold mul-
tiplier used to determine excessive bandwidth use.

Prefix relevant BW and Overusage threshold multiplier parameters control the number of
Flowspec rules that will be generated automatically. Adjust these values to control the number
of Throttling Flowspec policies.
CHAPTER 3. USING IRP 138

3.12.3 Providers and Peers configuration

Providers can be added and modified via the “Configuration→Providers and Peers” menu.
See also: Providers settings

Figure 3.12.7: Configuration editor: Providers configuration

Using the provider control buttons, a specific provider can be temporarily suspended (e.g for a short
maintenance in the monitored network), stopped (for long maintenance), or completely deleted. See
also: peer.X.shutdown. Commit Control can also be disabled or enabled for each provider specifically.
(peer.X.cc_disable).

The sections below describe the most common groups of provider configuration parameters. Some
parameters are repeated on more than one of these groups for convenience.

3.12.3.1 Providers configuration

To add a new provider, the “Add provider” button must be used. Several parameters should be configured
for the new provider:

• General settings:

– Provider short name (peer.X.shortname)

– Router (peer.X.bgp_peer)
– Probing IPv4 address (peer.X.ipv4.master_probing) or Probing IPv6 address (peer.X.ipv6.master_probing)
– Provider’s routing domain (peer.X.rd)
– Provider cost per Mbps (peer.X.cost)
– Maximum load per interface (peer.X.limit_load)
– Provider description (peer.X.description)
– Flow agents with outgoing interfaces (peer.X.flow_agents)
CHAPTER 3. USING IRP 139

Figure 3.12.8: Configuration editor: Adding a new provider, General settings

• Commit Control configuration:

– Provider cost per Mbps (peer.X.cost)

– Maximum load per interface (peer.X.limit_load)
– Provider 95th percentile (peer.X.95th)
– Provider inbound 95th percentile (peer.X.95th.in)
– Provider 95th calculation mode for inbound traffic (95th calculation modes)
– Commit Control status for this provider (peer.X.cc_disable)
– CC provider precedence - used for Commit Control and grouping (peer.X.precedence)
– Performance/Cost improvements within provider group flag in case Provider load balancing
is configured, (peer.X.improve_in_group, peer.X.precedence)
– Provider billing day (peer.X.95th.bill_day)
– Centile value (peer.X.95th.centile)
CHAPTER 3. USING IRP 140

Figure 3.12.9: Configuration editor: Adding a new provider, Commit Control

• SNMP-related settings:

– Flag whether IPv4 or/and IPv6 is used

– Provider SNMP IPv4/IPv6 address (peer.X.snmp.ip, peer.X.snmp.ipv6)
– Provider SNMP community (peer.X.snmp.community)
– Provider SNMP interface name or index (peer.X.snmp.interface); the list of SNMP interfaces
is automatically retrieved after Provider SNMP IPv4/IPv6 address field is filled
– SNMP version (peer.X.snmp.version)

Figure 3.12.10: Configuration editor: Adding a new provider, SNMP settings

• SNMP v3 uses additional parameters depending on security services used for statistics collection:
– SNMP security services selects if Authentication and/or Privacy services are used (peer.X.snmp.seclevel)
– SNMP authentication password if authentication is used (peer.X.snmp.auth_password)
– SNMP authentication protocol if authentication is used (peer.X.snmp.auth_protocol)
CHAPTER 3. USING IRP 141

– SNMP encryption password if privacy is used (peer.X.snmp.priv_password)

– SNMP encryption protocol if privacy is used (peer.X.snmp.priv_protocol)
– SNMP Username if authentication is used (peer.X.snmp.auth_username)
– SNMP version (peer.X.snmp.version)
• External Monitor settings:

– External monitor status flags for IPv4 / IPv6 (peer.X.mon.enabled, peer.X.mon.ipv6.external.enabled)

– ICMP/UDP ping monitored IPv4 / IPv6 addresses (peer.X.ipv4.mon, peer.X.ipv6.mon)

Figure 3.12.11: Configuration editor: Adding a new provider, External Monitor

• Internal Monitor settings:

– Internal monitor status flags for IPv4 / IPv6 (peer.X.mon.enabled, peer.X.mon.ipv6.internal.enabled)

– BGP session monitoring IPv4 / IPv6 addresses (peer.X.mon.ipv4.bgp_peer, peer.X.mon.ipv6.bgp_peer)
– BGP MIBs for IPv4 / IPv6 (peer.X.mon.ipv4.internal.mode, peer.X.mon.ipv6.internal.mode)
– BGP Internal Monitor IPv4 / IPv6 addresses (peer.X.mon.snmp.ip, peer.X.mon.snmp.ipv6)
– BGP Internal Monitor SNMP Community (peer.X.mon.snmp.community)
– SNMP version used for monitoring (peer.X.mon.snmp.version)
CHAPTER 3. USING IRP 142

Figure 3.12.12: Configuration editor: Adding a new provider, Monitoring

• SNMP v3 uses additional parameters depending on security services used for monitoring:
– SNMP security services selects if Authentication and/or Privacy services are used (peer.X.mon.snmp.seclevel)
– SNMP authentication password if authentication is used (peer.X.mon.snmp.auth_password)
– SNMP authentication protocol if authentication is used (peer.X.mon.snmp.auth_protocol)
– SNMP encryption password if privacy is used (peer.X.mon.snmp.priv_password)
– SNMP encryption protocol if privacy is used (peer.X.mon.snmp.priv_protocol)
– SNMP Username if authentication is used (peer.X.mon.snmp.auth_username)
– SNMP version (peer.X.mon.snmp.version)
• IPv4 / IPv6 settings:
– IPv4 / IPv6 diagnostic hops (peer.X.ipv4.diag_hop, peer.X.ipv6.diag_hop)
– Probing IPv4 / IPv6 address (peer.X.ipv4.master_probing, peer.X.ipv6.master_probing)
– Remote provider ASN (peer.X.ipv4.next_hop_as), used by the AS-Path restoration algorithm
(bgpd.as_path)
– Router next-hop address (peer.X.ipv4.next_hop), that sets the next-hop value for injected
routes related to this provider

Figure 3.12.13: Configuration editor: Adding a new provider, Commit Control

CHAPTER 3. USING IRP 143

The same parameters can be adjusted for the already configured providers.

3.12.3.2 Internet Exchanges configuration

An Exchange is configured as a special type of provider. The important things to consider when setting
up an Exchange are listed below.

It is recommended that Noction systems engineers guide you through Exchange configuration
process.

IRP needs one ACL and one PBR rule for each Peering Partner on an Exchange. Internet
Exchanges can have hundreds of Peering Partners. Ensure that the number of Access Lists, Access
List entries and PBR entries will not exceed router hardware limitations.

• Once the Exchange is setup IRP will retrieve the routing table on the router in order to setup
Exchange peering partners. IRP will require access to the router in order to do so.

When configuring an Exchange its Diag Hop parameter must be provided. Diag Hop for an
Exchange represents the list of direct-connected networks configured on the router’s interface towards
the Exchange network. This parameter is labeled "IPv4 diagnostic hop" on IRP Frontend.

• Probing IPs for Exchanges no longer require one IP address per peering partner since this number
might be impractically big. Instead a combination of probing IP and DSCP value is used to
configure the PBRs. A single probing IP in conjunction with DSCP can cover up to 64 peering
partners. A sufficient number of probing IPs will be required in order to cover all peering partners
on your exchange.
CHAPTER 3. USING IRP 144

Figure 3.12.14: An Internet Exchange is similar to a provider and it requires configuration of the Router,
the Probing IPs and the other usual attributes

• Once the Exchange is configured, IRP will need to reset its BGP session towards the router in-
terconnecting with the Exchange in order to retrieve the required routing table information about
all peering partners. Once the BGP session is restarted IRP will start fetching Exchange routing
tables.

Give IRP sufficient time (~10 minutes) to fetch Exchange routing tables before continuing con-
figuration.

• IRP offers peering partner autoconfiguration. It retrieves the list of Next Hops (peering partners)
on the Exchange with the corresponding list of prefixes individual peers accept.
CHAPTER 3. USING IRP 145

Figure 3.12.15: The list of peering partners for the Exchange shows details about each of them and offers
access to IRP Autoconfiguration feature and PBR ruleset generator for the router

• Use the Autoconfiguration feature to create an initial configuration for IRP. Review Exchange
peering partners before starting the Exchange. Consider enabling periodic auto re-configurations
for selected IX in order to update periodically the value of BGP session monitoring IPv4 address
from the Exchange and to add new peering partners. Refer global.exchanges.auto_config_interval
and peer.X.auto_config.

Keep in mind that Autoconfiguration might tamper with all the changes you’ve made directly
within IRP config files for peering partners. So, use Autoconfiguration sparingly after you’ve applied
manual changes or try avoiding direct configuration file changes altogether.

• Once the Exchange is configured correctly within IRP you will need to apply the PBR rules on the
router(s). IRP includes the functionality to generate PBR rules for different router vendors. You
will need to review the generated ruleset and apply it on the router manually.
CHAPTER 3. USING IRP 146

Figure 3.12.16: PBR Generator

• It is possible that the Autoconfiguration feature on the Exchange has been run with incomplete
routing tables or that new peering partners have been connected. This is especially true for very
big Exchanges. In this case it is possible that some peering partners are neither in IRP nor router
configurations. When IRP detects such a case it raises a warning about newly discovered peering
partners like in the image below. At this stage you will need to both re-autoconfigure IRP and
extract the PBRs for the router.

Figure 3.12.17: Warning about new peering partners that have been added to the Exchange

• After its creation and up to its complete configuration the Exchange is kept in a Stopped state.
When both IRP and the Router PBRs are configured IRP is ready to start optimizing Exchange
traffic too. Keep in mind that starting the Exchange will require a BGP session restart.

We must mention that before applying changes to IRP configuration the changes are validated.
If errors are detected these will be flagged and the previous good configuration will be kept intact
so you will have the option to review the erroneous data and correct.

3.12.3.3 Switch a provider from one router to another

Switching a provider from one router to another is possible via manual IRP reconfiguration only.
Use the following steps to switch the provider from one router to another:

1. Suspend the provider using IRP Frontend “Providers and Peers” configuration section. IRP with-
draws all existing and stops new improvements towards shutdown provider(s).
CHAPTER 3. USING IRP 147

2. You can remove traffic from the provider (by denying incoming/outgoing announces in the BGP
configuration) and then physically re-cable the provider from one router to another. Then configure
BGP session towards the provider on new router.
3. The PBR rule for the provider should be configured on new router (move provider’s PBR settings
from the old router to the new one).
4. Change IRP box local PBR rules if any.
5. Check if PBR works properly using traceroute tool.
6. Modify (/etc/noction/irp.conf) assigned router for the provider (refer to peer.X.bgp_peer).

7. Modify (/etc/noction/irp.conf) SNMP interface/IP/community for the provider (refer to peer.X.snmp.ip,

peer.X.snmp.ipv6, peer.X.snmp.interface, peer.X.snmp.community, peer.X.mon.snmp.ip, peer.X.mon.snmp.ipv6,
peer.X.mon.snmp.community, peer.X.snmp.version, peer.X.snmp.seclevel).
8. Reload BGPd configuration (affected BGP sessions could be reset).

9. Re-activate the provider using IRP Frontend “Providers and Peers” configuration section
10. Check IRP BGP Internal and External monitor states. They must be UP.

3.12.4 Commit Control configuration

Commit Control algorithm can be enabled and configured on the Commit Control group of pages. It
can be enabled or disabled using the toggle On/Off buttons at the top of the form.

If NetFlow is used to collect statistics Routers MUST be configured to export statistics every
minute (or as often as possible). Some router models have default export intervals for either inactive
or active flows of up to 1800 seconds. Big delays cause IRP to react very slowly to increased load
and reduce the effectiveness of Commit Control feature.
If NetFlow is used to collect statistics Routers MUST be configured to export statistics every minute
(or as often as possible). Some router models have default export intervals for either inactive or
active flows of up to 1800 seconds. Big delays cause IRP to react very slowly to increased load and
reduce the effectiveness of Commit Control feature.

The most important parameters for Commit Control are, as follows:

• Commit Control probing queue slots (core.commit_control.probing_queue_size)

• Minimal prefix bandwidth in Mbps (core.commit_control.agg_bw_min)
CHAPTER 3. USING IRP 148

Figure 3.12.18: Configuration editor: Commit Control

Providers Overall block displays Commit Control configuration for all providers including their prece-
dence. See details in the figure below.
Individual provider parameters can be adjusted as well as detail regarding current Commit Control
changes applied by IRP can be accessed by following the provided links.

Figure 3.12.19: Providers Overall

CHAPTER 3. USING IRP 149

3.12.5 BGP and Routers configuration

BGP daemon-related parameters can be configured in the “Configuration→BGP” section.

• BGPd parameters (See also: BGPd Configuration, BGPd settings):

– BGP mode intrusive/non-intrusive (global.nonintrusive_bgp)

– BGP monitor enabled/disabled, this option is useful during DoS/DDoS attacks. BGPd ICM-
P/SNMP monitors can be disabled on the fly, without restarting the BGPd daemon.
– AS-Path attribute a restore priority (bgpd.as_path)
– Remove prefixes on next-hop update (bgpd.improvements.remove.next_hop_eq)
– Remove prefixes on aggregate withdraw (bgpd.improvements.remove.withdrawn)

• BGP monitoring settings (See also: BGP Monitoring):

– Guard time (bgpd.mon.guardtime)

– Holdtime (bgpd.mon.holdtime)
– Keepalive (bgpd.mon.keepalive)

Figure 3.12.20: Configuration editor: BGP settings

BGP routers can be added, removed or modified on the same page.

A new BGP neighbor can be added using the “Add Router” button. Several parameters must be config-
ured:

• General settings:

– Router name
– ASN to be used for the iBGP session (bgpd.peer.X.as)
– Announced local-pref value (bgpd.peer.X.master_localpref)
– Local IP address (bgpd.peer.X.master_our_ip, bgpd.peer.X.master_our_ipv6)
– Neighbor IP address, usually the router’s IP (bgpd.peer.X.peer_ip, bgpd.peer.X.peer_ipv6)
CHAPTER 3. USING IRP 150

Figure 3.12.21: Configuration editor: New Router, general settings

• Advanced settings:

– Announced communities to be appended to the Community attribute (bgpd.peer.X.master_communities)

– Keepalive (bgpd.peer.X.keepalive)
– Announced MED value (bgpd.peer.X.med)
– Router ID, mandatory for IPv6 only (bgpd.peer.X.slave_router_id)
– Update split feature (More details: bgpd.updates.split)

Figure 3.12.22: Configuration editor: New Router, advanced settings

See also: BGPd Configuration, BGPd settings, BGP sessions settings

• Failover settings - the same set of parameters shall be setup for both master and slave nodes. Use
the toggle to switch between them.

– Announced communities to be appended to Community attribute (bgpd.peer.X.slave_communities,

bgpd.peer.X.slave_communities)
– Announced LocalPref value (bgpd.peer.X.master_localpref, bgpd.peer.X.slave_localpref)
CHAPTER 3. USING IRP 151

– Local IPv4 address (bgpd.peer.X.master_our_ip, bgpd.peer.X.slave_our_ip)

– BGP session password (bgpd.peer.X.master_password, bgpd.peer.X.slave_password)

Figure 3.12.23: Configuration editor: New Router, advanced settings

See also: BGPd Configuration, BGPd settings, BGP sessions settings

3.12.6 Collector configuration

Global collector parameters, as well as Span and Flow-specific settings can be adjusted by using the
“Configuration→Collector” menu item.
See also: Collector Configuration, Collector settings.

• Global collector settings:

– Top volume prefixes per cycle (collector.export.volume.high.top_n)

– Minimal traffic volume (collector.export.volume.min)
– Minimal traffic volume (%) (collector.export.volume.min_pct)

• Flow collector settings:

– UDP port to listen for NetFlow or sFlow packets (collector.flow.listen.nf, collector.flow.listen.sf)

– Flow sources (collector.flow.sources)

• SPAN collector settings:

– Capture interfaces (collector.span.interfaces)

– Mindelay algorithm enable/disable (collector.span.min_delay)
– Mindelay probing queue slots (collector.span.min_delay.probing_queue_size)
CHAPTER 3. USING IRP 152

Figure 3.12.24: Configuration editor: Collector settings

The list of prefixes announced by the monitored network can be edited in the same form. Its contents
can be also imported from a text file.
See also: collector.ournets.
CHAPTER 3. USING IRP 153

3.12.7 Explorer configuration

To adjust the Explorer-related parameters, the “Configuration→Explorer” menu item must be accessed.
The Explorer Configuration and Explorer settings sections should be consulted for detailed Explorer
configuration and parameters description.
Explorer parameters:

• Infrastructure IPs (explorer.infra_ips)

• Explorer worker threads (explorer.maxthreads)
• Probing algorithms and Traceroute algorithms (explorer.probe.algorithm, explorer.trace.algorithms)

• High volume task precedence (explorer.high_vol_precedence)

• Process max collected IPs (explorer.max_collector_ips)
• First probing packets amount (explorer.probing.sendpkts.min)

• Adaptive probing packets count (explorer.probing.sendpkts.adaptive_max)

• ICMP timeout (explorer.timeout)
• Traceroute retry packets and packets per hop (explorer.traceroute.sendpkts, explorer.traceroute.retrypkts)
• Traceroute minimum and maximum ttl (explorer.traceroute.ttl.min, explorer.traceroute.ttl.max)

Figure 3.12.25: Configuration editor: Explorer settings

CHAPTER 3. USING IRP 154

3.12.8 Notifications and Events

Notifications are messages sent to alert about some events registered by IRP. There are two prerequisites
in order to start using notifications:

1. Configure event parameters and channels

2. Subscribe for event notifications

IRP can send notifications as SMS, email and SNMP Trap.

3.12.8.1 Configure notification and events

Events configuration changes default threshold values when an event is raised. The most relevant events
are presented in the following figure. For example Overloaded by (%) indicates that this event will fire
when the aggregated outbound bandwidth usage for all providers exceeds by 10% the configured limits.
Refer section 4.1.9 for details.

Figure 3.12.26: Configuration editor: Events configuration

IRP uses a local email server to send emails. Still, email sent by this server might trigger filtering policies
enforced by some third parties that might block some important event notifications. It is possible to
provide the details of another email server on your network that is protected for this. The following
figure highlights the available email channel configuration parameters. Besides specifying the server
email configuration sets the sender of email messages that will show in receiver’s inbox.

Only SMTP servers without authentication are currently supported.

Figure 3.12.27: Configuration editor: Email configuration

SMS configuration is mandatory for sending SMS notifications.

IRP supports the following SMS gateways:

• Twilio
• Plivo
CHAPTER 3. USING IRP 155

A valid account with a supported SMS gateway is required in order to finish configuration. Check the
following figure for details:

Figure 3.12.28: Configuration editor: SMS configuration

The settings are:

• SMS gateway - choose one of the supported gateways.

• Account ID - the public identifier assigned to your account by the SMS gateway. The following
figure highlights this value for Plivo.
• Max message size - the maximum length of the SMS text. The SMS will be trimmed by IRP before
sending. Subsequently the SMS gateway will split the SMS into multiple parts if required.
• Secret - the secret identifier assigned to your account by the SMS gateway. The following figure
highlights this value for Plivo.
• From phone number - the phone number that shows as sender on the receiver’s mobile device. Use
a phone number supported by the SMS gateway.

Refer section 4.1.9 for complete details about configuration parameters.

Some SMS gateways enforce the From phone number and will reject numbers that do not comply
with their policy.

Figure 3.12.29: Plivo account ID and secret

SNMP Traps configuration requires a list of destination IP addresses and an SNMP community parameter
to deliver SNMP traps. The following figure highlights the basic configuration. There is a series of
advanced configuration parameters like the SNMP Trap port and rate of packets. Refer section 4.1.9 for
details.
CHAPTER 3. USING IRP 156

Figure 3.12.30: Configuration editor: SNMP Traps configuration

Web Hooks configuration requires only a Webhook URL to be provided. There is a series of advanced
configuration parameters in case their default settings are not satisfactory. Refer section 4.1.9 for details.

Figure 3.12.31: Configuration editor: Web Hooks configuration

The settings are:

• Webhook URL - the unique URL the team is assigned by Web hook provider. An example for
Slack.com is shown.
• Avatar icon URL - optional parameter that designates an icon (usually PNG or JPEG image are
supported) assigned to the Webhook bot in the channel.

• Avatar emoji - an alternative avatar specified in the form of an emoji in “:robot-face:” format.
• Bot name - optional bot name assigned to Webhook bot.

Refer section 4.1.9 for complete details about configuration parameters.

Web hooks support has been tested and verified to work for Slack.com API.

3.12.8.2 Subscriptions
Notifications are sent only if a valid subscription has been created. Subscriptions, similar to regular
emailing of reports can be found under <Username> → Notifications menu.
Notifications page provides an overview of existing subscriptions with features to create, edit, delete
them. Check the following figure for a preview.
CHAPTER 3. USING IRP 157

Figure 3.12.32: Notifications

Use Add new subscription or Events drop-down to create a new subscription or edit, delete existing
subscriptions by clicking corresponding icons. Creating or editing a subscription will bring up a pop-up
like the one in the following figure.

Figure 3.12.33: Subscribe notifications

Subscription details include:

• Topic - the title given to subscription in Notification page and also a textual part of email and
SMS notifications.
CHAPTER 3. USING IRP 158

If SMS notifications are used it is advised to keep the topic short so that there is enough space
left to include details about the event.

• Interval between notifications - sets a rate limit of how often an email and/or SMS notification is
sent. Value ’No limit’ for the interval depicts no rate limits and all events will trigger a notification.

SNMP Traps notification are not constrained by the interval between notifications. SNMP Traps
are sent immediately as they are raised.

The rate limit is enforced per subscription so it is still possible to receive multiple notifications if
the subscribed events are part of different subscriptions.

• Destinations - IP address of a trap receiver for SNMP Trap notifications, email addresses, phone
numbers and webhook channels. Multiple destinations of same type can be provided. At least one
valid destination must be provided per subscription.

IRP parses and recognizes whether a provided destination is a valid IP address, email address,
phone number or web hook channel. Web hook channels for example a recognized by detecting that
first character of a channel name is the “#” mark.

• Event filters - allows filtering of events by textual description or by IRP component. Remove the
filters to see all the subscribed events.
• Subscribed events - the full list of events supported by IRP and tick marks for the events in this
subscription. At least one event is required for a valid subscription.

3.12.8.3 Notification text

When a subscribed event is fired IRP will send notifications. A sample notification text (email) looks
like the following figure:

Figure 3.12.34: Subscribe notifications

The email sample above identifies the different parts of the message:

1. IRP server name

2. Subscription topic as specified in the subscription
3. From email address - as configured in email channel
4. Time - date-time of the last event that caused the notification. In case of rate limitation this might
be older than the time of the email.
CHAPTER 3. USING IRP 159

5. Textual description of the event and any relating details.

6. Older events in this subscription that have been retained due to rate limitations. Keep in mind
that all past events are aggregated into a single email or SMS message.

3.12.9 Security Configuration

Security configuration covers restrictions on the ranges of IP addresses that can access IRP’s Frontend
and also user’s allowed to access IRP as well as their roles.

3.12.9.1 Allowed ranges of IP addresses

IRP Frontend includes a feature to restrict access by IP address and this feature can be turned on or off
as needed. When this feature is used allowed IP addresses are maintained as a list of prefixes in CIDR
notation.

Figure 3.12.35: Configuration editor: Security

3.12.9.2 Users and user directories

Users who can access IRP are managed in User Directories. User directory is a generic name for a
external user management provider for example LDAP (either POSIX or Active Directory). IRP can
interface with as many user directories as needed in a specific environment.
IRP recognizes Admin and User privileges that can be granted to users. Once a user logs into the system
IRP creates a profile that maintains his personal configurations like dashboards or notifications.

Figure 3.12.36: Configuration editor: Users and Directories

3.12.9.3 LDAP and Active Directory

LDAP or AD user directories can be added, updated and removed from IRP by accessing “Configuration→Security”
menu item in the User Directories tab. Each user directory takes a series of parameters specific for the
protocol.
CHAPTER 3. USING IRP 160

All operations with DNs (initial bind DN, group DNs, user names) are case insensitive and also
strip redundant whitespace.

Refer individual protocol documentation for how to correctly configure one or another user directory.
The example below offer a generic set of parameters required to configure IRP to use Active Directory
for access management.

Figure 3.12.37: Configuration editor: LDAP User Directory configuration

The general tab covers:

• User directory name - the name assigned to the directory within IRP,
• User directory type - one of the supported User Directories,
• Enabling or disabling a user directory,

• Order specifies when this user directory will be examined by IRP compared to other user directories,
• User directory hostname in the form of either IP address or domain name
• User directory port
CHAPTER 3. USING IRP 161

Figure 3.12.38: Configuration editor: Users and Directories

Advanced configuration of a user directory covers:

• Initial binding user name that IRP uses to authenticate itself,

• Initial bind password assigned to IRP,

Usually bind passwords are not required to access directories. If a password is required and
configured via IRP Frontend, it will be scrambled in IRP configuration files. If the password is
specified directly in IRP configuration it is provided in clear-text with the condition that it does not
start/end with scrambled password encapsulation characters.

• Timeout before failing a connection to this user directory,

• TLS use,
• Certificate verification and

• CA certificate used to verify server’s certificate.

CHAPTER 3. USING IRP 162

Figure 3.12.39: Configuration editor: Users and Directories

Bindings maps User Directory attributes to IRP specific attributes, for example:

• Base DN and User DN specifies the root distinguished name and user subtree

IRP recognizes BOTH short and full user identifiers. Examples below are both valid directory
entries that will match user “chris” with long name “Mr. Chris Smith”:
cn: ops
uniqueMember: chris

and
cn: ops
uniqueMember: cn=Mr. Chris Smith,ou=employees,ou=People,dc=ops,dc=org

• Admin role groups are user directory objects that list users with Administrator privileges within
IRP as compared to all users. Both short and full paths are accepted for Admin role groups.
• User role, Username, Email and Common name fields map User Directory attributes to IRP user
attributes,
• User filters can specify any valid LDAP search criteria (without outside parenthesis).

3.12.9.4 TACACS+ directory

TACACS+ user directories can be added, updated and removed from IRP by accessing “Configuration→Security”
menu item in the User Directories tab. Each user directory takes a series of parameters specific for the
protocol.
CHAPTER 3. USING IRP 163

The example below highlights the parameters required to configure IRP to authenticate against a
TACACS+ directory/host.

Figure 3.12.40: Configuration editor: TACACS+ configuration

The parameters include:

• User directory name - the name assigned to the directory within IRP,
• User directory type - one of the supported User Directories in our case TACACS+,
• Enabling or disabling a user directory,
• Order specifies when this user directory will be examined by IRP compared to other user directories,
• User directory hostname in the form of either IP address or domain name
• User directory port
• A shared secret used by IRP and TACACS+ host to secure communications.

3.12.9.5 Internal user directory

IRP includes an Internal user directory that allows multiple user accounts to be setup. User accounts
management is done by accessing “Configuration→Security” menu item.

User management for external user directories is performed outside IRP using designated tools.
The Internal directory is used when external user directories are not available or their use is not
desirable.

To edit a specific user’s profile, press the Edit button next to it. To add a new user, click on the “Add”
button. Specific users may be given Admin privileges. In such case they can also manage all system
users.
The timezone specified in the user profile will be considered while displaying system Reports and Graphs.
CHAPTER 3. USING IRP 164

Figure 3.12.41: Configuration: Adding a new user account

3.12.10 Frontend configuration

Figure 3.12.42: Configuration editor: Frontend

The currency used in various cost-related reports as well as the system logo (shown in the login form
and on the menu bar) can be changed on the same page.
Currencies can be added/deleted manually by using the “Manage currencies” feature.
CHAPTER 3. USING IRP 165

Figure 3.12.43: Configuration Editor: Manage currencies

Only a Super admin user can change the currency setting. The currency is a global system value,
and cannot be adjusted per-user.
The system logo is a global setting and can be changed by the Super administrator only.

The default currency is “USD - U.S. Dollar”.

A custom logo image can be uploaded by using the Upload button. Afterward, the image can be cropped
to fit the standard logo size. The new settings are applied after clicking the “Submit” button.

3.12.11 Inbound (beta) configuration

Refer Inbound (beta) for an overview of Inbound feature.

Flow (sFlow, NetFlow) is a prerequisite for Inbound optimization. IRP collects and uses statistics
about upstream provider incoming traffic targeting inbound prefixes. This statistics should be made
available by properly setting up Flow and provider specific Flow agents (refer peer.X.flow_agents).

SPAN does not carry details about the specific interface packets enter your network and as such
it is impossible to distribute inbound traffic targeting Inbound prefixes by upstream provider.

During the preview of the Inbound beta all Inbound related features, including configuration, moderation,
reports are listed under main menu Inbound:
CHAPTER 3. USING IRP 166

Figure 3.12.44: Main menu: Inbound

3.12.11.1 Inbound prefixes

Inbound optimization manipulates how much traffic targets YOUR prefixes through different providers.
In order to do this you need to configure IRP and let it know what are the Inbound prefixes. You
can specify a subset of your network prefixes and you can also split larger prefixes into smaller ones in
order to have a more fine-grained control over the Inbound traffic. These prefixes will be announced
back to your originating routers marked with the community designated to prepends count to use when
announcing to individual upstream providers.

Splitting larger prefixes into subprefixes offers IRP the opportunity to better control the granular-
ity of traffic shaping changes enabling improvement of smaller chunks of overall traffic. At the same
time original prefixes can be announced as before and they will guarantee continuity of the prefix
for cases when improvements announced by IRP become unavailable for some reason (for example
if smaller prefixes are filtered in some networks).

Verify during Inbound configuration that the split inbound prefixes are not filtered by your
providers and propagate as expected on the Internet.

IRP Frontend includes facilities that let you define the list of YOUR prefixes covered by Inbound traffic
optimization. Each of the Inbound prefixes must specify the router(s) where announcements are sent.
Optimization of some prefixes can be disabled if needed.

Inbound improvements must be advertised by IRP to core routers that originate the prefix.
This way the improvements will propagate consistently through your network to edge routers and
beyound.
CHAPTER 3. USING IRP 167

Figure 3.12.45: Configuration editor: Inbound prefixes

Inbound prefixes can be found both via Inbound (beta) main menu or Configuration →Collectors

3.12.11.2 Inbound prepends

Inbound optimization in IRP relies on BGP priority rules that generally elect best routes based on AS
PATH length. By instructing your network to add or remove prepends for specific inbound prefixes over
different providers IRP is able to influence how your inbound traffic flows. If your providers or other
networks on the Internet use other policies for electing best routes then this makes Inbound optimization
in IRP less effective for those regions of the Internet. For example, Internet Exchange routes always take
precedence for peers. This makes impossible prepend based inbound traffic optimization for Internet
Exchanges. Often partial providers employ similar policies and inbound traffic flowing through them
cannot be optimized.
Prepends are not added directly by IRP. Instead IRP inbound improvements are marked with designated
communities that are recognized and acted upon by network routers.

Base communities Inbound optimization improvements advertise larger or smaller counts of prepends
to be announced to different providers. IRP uses BGP session to communicate to routers and relies on
BGP communities to communicate the improvements. Each provider is assigned a base community
which represents zero prepends. Additional prepends are represented by incrementing accordingly the
right-side part of the provider’s community.

IRP uses 8 prepend levels. This means that base community should be chosen to allow additional
7 values without intersecting with other community values.

Figure 3.12.46: Configuration editor: Base communities

CHAPTER 3. USING IRP 168

Inbound base communities are assigned for each individual provider Configuration →Providers
and Peers

Additionally routemaps are applied so that routers depict the designated communities in IRP’s improve-
ments and change the announcement of the inbound prefix towards that provider appropriately. Different
routemaps are produced and applied on each router in your network. See a few examples below. Request
assistance from Noction’s support team to prepare exact routemaps for your network configuration and
router models.

3.12.11.3 Routers configuration for Inbound

Complete sample configurations for Cisco, Juniper, Brocade and Vyatta will be provided by
Noction Support on request.

3.12.11.4 Inbound Commit Control

Inbound bandwidth control is part of the Commit Control feature of IRP. You need to enable Com-
mit Control and further to enable Inbound Commit control to make use of this feature. Besides the
above mentioned settings for Inbound Commit Control separate low and high limits are configured to
instruct IRP when to start and when to stop improving and also how to estimate the effects of inbound
improvements.

Figure 3.12.47: Configuration editor: Inbound commit control

Inbound commit control can be found both via Inbound (beta) main menu or Configuration
→Commit Control
CHAPTER 3. USING IRP 169

Commit control for inbound as can be seen above screencapture highlights the relevant configuration
parameters:

• whether inbound commit control is on or off

• whether review and moderation feature is enabled or disabled

• what is the bandwidth estimation algorithm for inbound and

• high and low limits for inbound in case the 95th mode warrants independent inbound vs outbound
optimization.

Refer also to peer.X.95th.mode, core.commit_control.inbound.enabled, core.commit_control.inbound.moderated.

Estimating inbound traffic Internet traffic has high variability and adjacent measurements can differ
significantly between them. In these conditions a leveling function helps mitigate the risk of generating
excessive numbers of Inbound improvements due to higher traffic variability of some networks. This
estimation can be fine tuned by the Inbound bandwidth estimation algorithm parameter that tells IRP
what value to use as the basis of the calculation. See the possible base values below:

Figure 3.12.48: Configuration editor: Inbound BW estimation algorithms

Moderated Inbound Improvements IRP includes a moderation feature that allows review and ap-
proval of Inbound improvements. If this feature is enabled Inbound improvements are not automatically
announced but instead they are queued for review. If an inbound improvement has not been approved
in time for a subsequent cycle of improvements IRP will review the proposal itself and will adjust as
needed thus making another recommendation.
See the capture below.

Figure 3.12.49: Configuration editor: Inbound BW estimation algorithms

Inbound Improvements moderation highlights the number of additional prepends proposed by IRP and
allows submitting or rejecting individual improvements as well as a batch of selected improvements.

3.12.12 Wizards
Wizards can be accessed from Configuration→ Setup Wizards or from Configuration → Frontend →
Configuration Wizards.
CHAPTER 3. USING IRP 170

3.12.12.1 Initial Setup

Figure 3.12.50: Configuration editor: Initial Setup

• Basic Setup

– Setup Infrastructure IP addresses (explorer.infra_ips)

– Setup Analyzed networks (collector.ournets)

Figure 3.12.51: Configuration editor: Setup Infrastructure IP addresses and Analyzed networks

– Configure Collector:
∗ Irpflowd
· Irpflowd enable/disable(collector.flow.enabled)
· NetFlow UDP port(collector.flow.listen.nf)
· sFlow UDP port(collector.flow.listen.sf)
· Flow Sources(collector.flow.sources)
∗ Irpspand
· Irpspand enable/disable (collector.span.enabled)
· Irpspand interfaces (collector.span.interfaces)
CHAPTER 3. USING IRP 171

· Mindelay status (collector.span.min_delay)

· Mindelay probing queue slots (collector.span.min_delay.probing_queue_size)

Figure 3.12.52: Configuration editor: Configure Collector

– Improvement mode (global.improve_mode)

Figure 3.12.53: Configuration editor: Improvement mode

– Set the managements interface (global.master_management_interface)

CHAPTER 3. USING IRP 172

Figure 3.12.54: Configuration editor: Management Interface

– Set the probing interface (global.master_probing_interface)

Figure 3.12.55: Configuration editor: Probing Interface

• Providers Setup

– Add a router, see Add Router

– Add two providers, see Add Provider

3.12.12.2 Add Router

• Router name and AS (bgpd.peer.X.as)

Figure 3.12.56: Configuration editor: Router name and AS

• Router IPv4 addresses

CHAPTER 3. USING IRP 173

– Local router IPv4 address (bgpd.peer.X.master_our_ip)

– Router IPv4 address (bgpd.peer.X.peer_ip)

Figure 3.12.57: Configuration editor: Router IPv4 addresses

• Router IPv6 addresses

– Local router IPv6 address (bgpd.peer.X.master_our_ipv6)

– Router IPv6 address (bgpd.peer.X.peer_ipv6)
– Router ID (bgpd.peer.X.slave_router_id)

Figure 3.12.58: Configuration editor: Router IPv6 addresses

• Router announcing

– Announced LocalPref value (bgpd.peer.X.master_localpref)

– Announced communities(bgpd.peer.X.master_communities)
– Announced MED value (bgpd.peer.X.med)
CHAPTER 3. USING IRP 174

Figure 3.12.59: Configuration editor: Router announcing

3.12.12.3 Add Provider

• Choose a Router (peer.X.bgp_peer)

Figure 3.12.60: Configuration editor: Choose a Router

• Provider name

– Provider short name (peer.X.shortname)

– Provider description (peer.X.description)

Figure 3.12.61: Configuration editor: Provider name

• Provider IPv4 addresses

– Probing IPv4 address (peer.X.ipv4.master_probing)

– IPv4 diagnostic hop (peer.X.ipv4.diag_hop)
– Router next-hop address (peer.X.ipv4.next_hop)
CHAPTER 3. USING IRP 175

– Router ASN (peer.X.ipv4.next_hop_as)

Figure 3.12.62: Configuration editor: Provider IPv4 addresses

• Provider IPv6 addresses

– Provider probing IPv6 address (peer.X.ipv6.master_probing)

– IPv6 diagnostic hop (peer.X.ipv6.diag_hop)
– Router nex-hop IPv6 address (peer.X.ipv6.next_hop)
– Router ASN (peer.X.ipv6.next_hop_as)

Figure 3.12.63: Configuration editor: Provider IPv6 addresses

• Provider Commit Control

– Provider 95th percentile (peer.X.95th)

– Provider cost per Mbps (peer.X.cost)
– Commit Control provider precedence (peer.X.precedence)
– Maximum allowed load per interface (peer.X.limit_load)
– Commit Control status for this provider (peer.X.cc_disable)
– Allow Performance/Cost Improvements within provider group members (peer.X.improve_in_group)
CHAPTER 3. USING IRP 176

Figure 3.12.64: Configuration editor: Provider Commit Control

• Provider Monitoring setup IPv4

– Provider SNMP interface (peer.X.snmp.interface)

– Provider SNMP IPv4 address (peer.X.snmp.ip)
– Provider SNMP community (peer.X.snmp.community)

Figure 3.12.65: Configuration editor: Provider Monitoring setup IPv4

• Provider Monitoring setup IPv6

– Provider SNMP interface (peer.X.snmp.interface)

– Provider SNMP IPv6 address (peer.X.snmp.ipv6)
– SNMP community (peer.X.snmp.community)

Figure 3.12.66: Configuration editor: Provider Monitoring setup IPv6

CHAPTER 3. USING IRP 177

• External Monitor setup IPv4

– External Monitor enable/disable (peer.X.mon.enabled)

– Router monitoring IPv4 address(peer.X.ipv4.mon)

Figure 3.12.67: Configuration editor: External Monitor setup IPv4

• External Monitor setup IPv6

– External Monitor enabled/disabled (peer.X.mon.enabled)

– Router monitoring IPv6 addresspeer.X.ipv6.mon)

Figure 3.12.68: Configuration editor: External Monitor setup IPv6

• Internal Monitor setup

– Internal Monitor enable/disable (peer.X.mon.enabled)

– Monitoring IPv4 address (peer.X.mon.ipv4.bgp_peer)

Figure 3.12.69: Configuration editor: Internal Monitor setup

CHAPTER 3. USING IRP 178

• Provider pre-checks

Figure 3.12.70: Configuration editor: Provider pre-checks

3.12.12.4 Setup Commit Control

Step 1: Enable Commit Control Providers that should maintain a target 95th limit are identified
and the exact limits are set. Link upper limit represents 80-90% of interface capacity and will be pre-
populated for you.
• Commit control enables/disables the feature for a specific provider(peer.X.cc_disable)
• 95th target specifies the contracted bandwidth usage target in Mbps (peer.X.95th)
• Link upper limit specifies the maximum allowed bandwidth on the link in Mbps (peer.X.limit_load).
This represents ~80-90% of interface capacity and will be pre-populated for you. Change it as ap-
propriate.

Figure 3.12.71: Configuration editor: Commit Control basic setup

Step 2: Provider precedence Precedence indicates how important it is to keep the commit levels
for different providers. Traffic will be unloaded to the provider with least precedence in situations when
all other providers are overloaded. If two or more providers have equal precedence they will form groups.
Move the slider or use drag and drop to specify the order of providers from most important to least
important.

Figure 3.12.72: Configuration editor: Commit Control provider precedence setup

CHAPTER 3. USING IRP 179

Step 3: Groups configuration Groups are formed by a set of providers (as configured in IRP) that
share some common characteristics, for example a redundant link or a virtual link consisting of multiple
physical links towards the same upstream provider. Depending on customer’s needs, groups can be
configured to permit or forbid performance rerouting from one provider in the group towards another.
If providers are grouped they will be assigned the same precedence.
Groups are optional and can be skipped.

Figure 3.12.73: Configuration editor: Commit Control groups setup

Step 4: Commit Control Overview The last step of the wizard is there to review the configuration.
Last minute changes can be made to some parameters. When submitted the configuration changes will
be validated and applied if correct.

• Provider name (peer.X.shortname)

• Commit control status for a specific provider (peer.X.cc_disable)
• Configured 95th (peer.X.95th)

• Upper limit (peer.X.limit_load)

• Load balancing for groups
• Improve in group (peer.X.improve_in_group)
• Precedence (peer.X.precedence)

Figure 3.12.74: Configuration editor: Commit Control Provider overall

3.12.12.5 Setup Failover wizard

Step 1: Enable failover Failover configuration has a series of mandatory global settings before
enabling it.

• Failover status (peer.X.cc_disable)

• Slave IP address (peer.X.95th)

CHAPTER 3. USING IRP 180

• Slave SSH port (global.failover_slave.port)

• Failover timer in seconds (global.failover_timer_fail)
• Failback timer in seconds (global.failover_timer_failback)

Figure 3.12.75: Configuration editor: Failover setup

Step 2: Probing IP configuration Probing IPs must be assigned to all configured providers for
both master and slave nodes (peer.X.ipv4.master_probing, peer.X.ipv4.slave_probing).

Figure 3.12.76: Configuration editor: Failover probing IPs for providers

Step 3: Router BGP session settings Master and slave nodes of a failover configuration establish
BGP sessions with all configured routers. The following parameters are required:

• Local IP address for both master and slave (bgpd.peer.X.master_our_ip, bgpd.peer.X.slave_our_ip)

• BGP session password for master and/or slave if any (bgpd.peer.X.master_password, bgpd.peer.X.slave_password)

Figure 3.12.77: Configuration editor: Failover BGP session settings

CHAPTER 3. USING IRP 181

Step 4: BGP LocalPref and Communities Master and slave nodes apply BGP attributes to
announcements:

• LocalPref for master and slave (bgpd.peer.X.master_localpref, bgpd.peer.X.slave_localpref)

• Communities for master and slave (bgpd.peer.X.master_communities, bgpd.peer.X.slave_communities)

Master’s LocalPref value must be greater than slave’s LocalPref.

Figure 3.12.78: Configuration editor: BGP announcement attributes

Step 6: Probing interfaces Failover requires configuration of Management and Probing interfaces
for both master and slave nodes (global.master_probing_interface, global.slave_probing_interface).
Toggle between master and slave settings to specify values for both nodes.

Figure 3.12.79: Configuration editor: Probing and management and interfaces

Chapter 4

Configuration parameters reference

4.1 Complete parameters list

4.1.1 Common database credentials

db.port - TCP port number used for my MySQL database.
db.host - Database host. MySQL server host name or IPv4/IPv6 address.
db.ourhost - IRP host. IRP box host name or IPv4/IPv6 address.

db.username - Database login. Credentials for IRP database.

db.password - Password for database access.
db.dbname - Name of database that holds the IRP tables.

182
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 183

4.1.2 Global parameters

4.1.2.1 global.aggregate
This parameter instructs IRP to operate with the smallest possible prefixes (/24 for IPv4 and /48 IPv6)
if disabled, or with the largest possible aggregates (up to /16 for IPv4 and /32 for IPv6, configurable
using global.agg_ipv4_max & global.agg_ipv6_max parameters).
If global.aggregate is ON, the whole aggregate is improved and only the aggregate will be announced by
BGPd (refer to bgpd.updates.split, bgpd.peer.X.updates.limit.max).
The aggregate dictionary is periodically populated by refresh_asn or BGPd (if BGPd has sufficient
information, formerly at least one full-view BGP peering session).

Possible values: 0 (OFF), 1 (ON)

Default value: 1

4.1.2.2 global.bw_overusage
Enables or disables automatic Flowspec throttling policies for prefixes with excessive bandwidth spikes.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0

4.1.2.3 global.agg_ipv4_max
Defines the maximum IPv4 aggregate mask for the improved prefixes advertised by BGPd. If global.aggregate
is enabled, IRP will not announce any prefix with the mask that’s less than the mask defined in
global.agg_ipv4_max.

Possible values: 12-24

Default value: 16

4.1.2.4 global.agg_ipv6_max
Defines the maximum IPv6 aggregate mask for the improved prefixes advertised by BGPd. If global.aggregate
is enabled, IRP will not announce any prefix with the mask that’s less than the mask defined in
global.agg_ipv6_max.

Possible values: 32-48

Default value: 32

4.1.2.5 global.exchanges
Defines the path to the Exchanges configuration file.

Default value: /etc/noction/exchanges.conf

Recommended value: /etc/noction/exchanges.conf

4.1.2.6 global.exchanges.auto_config_interval
Defines the Internet Exchanges auto re-configuration interval in seconds. Auto re-configuration is applied
to providers of type Internet Exchange with enabled auto re-configuration. Refer peer.X.auto_config.

Possible values: 3600-2678400

Default value: 86400
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 184

4.1.2.7 global.failover
Enables and disables failover feature.

Enabling failover requires extensive preparatory activities. Refer IRP Failover, Failover configu-
ration, Setup Failover wizard for details.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0

4.1.2.8 global.failover.log
Path to failover log file.

Default value: /var/log/irp/failover.log

4.1.2.9 global.failover_identity_file
Defines the path to failover master identity file. Set this parameter only when a SSH key file other than
~/.ssh/id_rsa is used. Refer global.failover.

Possible values: Path to identity file

4.1.2.10 global.failover_role
Defines the role of the IRP instance in a failover configuration. Known roles are Master and Slave. Used
only when failover is enabled. Refer global.failover.

Possible values: 0 (Master), 1 (Slave)

Default value: 0

4.1.2.11 global.failover_slave.ip
Defines the IPv4 address of the slave IRP instance used for configuration sync in a failover configuration.
Used only when failover is enabled. Refer global.failover.

Possible values: Valid IPv4 address

Example value: 10.10.0.2

4.1.2.12 global.failover_slave.ipv6
Defines the IPv6 address of the slave IRP instance used for configuration sync in a failover configuration.
Used only when failover is enabled. Refer global.failover.

Possible values: Valid IPv6 address

Example value: 2001:db8::ff00:42:8329

4.1.2.13 global.failover_slave.port
Defines the SSH port of the slave node in a failover configuration. Used only when failover is enabled.
See also global.failover.

Possible values: 1-65535 (Valid port number)

Default value: 22
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 185

4.1.2.14 global.failover_timer_fail
Defines the period of time in seconds during which master is considered alive before the slave becomes
active in a failover configuration. Used only when failover is enabled.
See also global.failover.
Possible values: 30-3600
Default value: 300

4.1.2.15 global.failover_timer_failback
Defines the period of time in seconds for slave to switch back to standby after it detects master became
alive in a failover configuration. Used only when failover is enabled.
See also global.failover.
Possible values: 30-3600
Default value: 300

4.1.2.16 global.flowspec
Enables/disables FlowSpec capability globally.
Possible values: 0 (Disabled), 1 (Enabled)
Default value: 0

4.1.2.17 global.frontend_acl
Defines whether access to the IRP Frontend must be restricted and controlled by an ACL. See also
global.frontend_acl_ips.
Possible values: 0 (OFF), 1 (ON)
Default value: 0

4.1.2.18 global.frontend_acl_ips
Defines the IPs or networks that should be allowed to access the IRP Frontend.
Possible values: Valid IPv4, IPv6 address or subnet definition in CIDR format
Default value: 0/0 ::/0

4.1.2.19 global.ignored.asn
Defines the list of ASNs to be ignored by IRP.
Format:
1. Space-separated list of AS numbers that must be ignored by IRP.
2. Absolute path to a newline-separated file containing a list of AS numbers that must be ignored by
IRP.
This parameter should list all AS numbers that must be ignored by Irpspand, Irpflowd, Explorer and the
Core. No improvements will be performed by the Core for prefixes that are announced by ASNs listed
within this parameter. No data will be gathered by Irpspand/Irpflowd for any source or destination
IPv4/IPv6 address that are announced by ASNs that are listed within this parameter. No probes will be
sent by Explorer to any destination IPv4/IPv6 address that are announced by ASNs listed within this
parameter.
Possible values: 1 - 4294967295.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 186

4.1.2.20 global.ignorednets
Defines the list of networks to be ignored by IRP.
Format:

1. Space-separated list of local IPv4/IPv6 prefixes that should be ignored by IRP.

2. Absolute path to a newline-separated file containing a list of IPv4/IPv6 prefixes that should be
ignored by IRP.

If netmask is not clearly specified, the system assumes /32 for IPv4 addresses, and /128 for IPv6 ad-
dresses.

224.0.0.0/3 is always ignored.

- IPv6 probing is performed only to 2000::/3 address range (see: IPV6 Address Space)
- All IPv4/IPv6 addresses assigned to IRP server are automatically added to ignored networks

This parameter lists all IPv4/IPv6 addresses that should be ignored by Irpspand, Irpflowd, Explorer and
the Core. No improvements will be performed by the Core for /24 subnets listed within this parameter
as /24 or a less specific network. No data will be gathered by Irpspand/Irpflowd for any source or
destination IPv4/IPv6 address listed within this parameter. No probes will be sent by the Explorer to
any destination IPv4/IPv6 address listed within this parameter.

Possible values: See above.

Default value: 10.0.0.0/8 169.254.0.0/16 172.16.0.0/12 192.168.0.0/16 2001:db8::/32

Recommended value: See default value.

Recommended networks to ignore are:

- networks from: Section 3 of RFC5735, as listed in the default value above
- networks from: Section 2 of RFC3849

4.1.2.21 global.improve_mode
Defines the IRP operating mode (see IRP Optimization modes).
This parameter adjusts the priorities and rules for prefix improvements. Prefixes can be improved in
three different ways:

1. Performance optimization: Decrease loss, then decrease latency;

2. Cost optimization: Decrease loss, then decrease cost while keeping the latency within the precon-
figured level;

Loss, cost and latency are improved in strict order, depending on the selected operating mode.

Possible values: 1, 2 (see above).

Default value: 1

See also: Commit Control, core.commit_control, peer.X.cc_disable

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 187

4.1.2.22 global.inbound_conf
Defines path to file with configured inbound prefixes.

Possible values: path to file

Default value: /etc/noction/inbound.conf
Recommended value: /etc/noction/inbound.conf

4.1.2.23 global.ipv6_enabled
Defines whether IPv6 is enabled in the system. Currently used for the Frontend only. Even if IPv6 is
enabled via this parameter, other components configuration must be adjusted for IPv6 as well.

IRP prior to 3.3-2 allows configuration of both IPv4 and IPv6 sessions on a single router, while
IRP 3.3-2 requires definition of two separate BGP sessions. It is recommended to split BGP sessions
in the form of two different routers before upgrading to 3.3-2, otherwise IRP configuration will not
be valid. Make sure the newly added router is linked to corresponding providers to ensure IPv6
optimization works properly. InternalMon for IPv6 session monitoring requires correct configuration
of BGP MIB (IPv6) mode (see peer.X.mon.ipv6.internal.mode parameter). Currently support for
Brocade, Cisco and Juniper MIBs is available.

Possible values: 0, 1
Default value: 0

4.1.2.24 global.master_management_interface
Defines the management network interface. In most cases it is the same as the probing interface. When
failover is enabled (global.failover) slave’s management interface (global.slave_management_interface)
must be configured too.

Possible values: any valid system network interface name

Default value: eth0

4.1.2.25 global.slave_management_interface
Defines the management network interface for slave node in a failover configuration. In most cases it is
the same as the probing interface. When failover is disabled (global.failover) the parameter is not used.
See also global.master_management_interface.

Possible values: any valid system network interface name

Default value: eth0

4.1.2.26 global.nonintrusive_bgp
Instructs the system to run in a non-intrusive BGP mode (see IRP Operating modes).
All improvements made in a non-intrusive mode, will not be automatically injected into the routers.

Possible values: 0 (OFF), 1 (ON)

Default value: 1
Recommended value: 1 at first start, 0 after manual tests are performed and the system is ready to
go into intrusive mode
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 188

4.1.2.27 global.outbound
Enables or disables outbound optimization.

Outbound optimization is disabled only for standalone Inbound optimization is used.

Possible values: 0 (OFF), 1 (ON)

Default value: 1
Recommended value: 1

4.1.2.28 global.png.datadir
Defines the file-system directory path for storing image files (Graphs).

Default value: /usr/share/irp/web/RRD

Recommended value: /usr/share/irp/web/RRD

4.1.2.29 global.policies
Defines the path to the Routing Policies (1.2.9) configuration file.

Default value: /etc/noction/policies.conf

Recommended value: /etc/noction/policies.conf

4.1.2.30 global.master_probing_interface
Defines the probing network interface. In most cases it is the same as the management interface. When
failover is enabled (global.failover) slave’s probing interface (global.slave_probing_interface) must be
configured too.

This parameter is used only for displaying operating system interface(s) status in Frontend. It
does not actually configure Explorer behavior, which depends on Providers settings.

Possible values: any valid system network interface name

Default value: eth0

4.1.2.31 global.slave_probing_interface
Defines the probing network interface for the slave node in a failover configuration. In most cases it is
the same as the management interface. When failover is disabled (global.failover) the parameter is not
used.
See also global.master_probing_interface.

Possible values: any valid system network interface name

Default value: eth0

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 189

4.1.2.32 global.rd_rtt
Defines the latency distances between routing domains in the format rda:rdb:rtt where rda is the id
assigned to one routing domain, rdb is the id assigned to the second routing domain and rtt represents
the round trip time in miliseconds between them. rda and rdb must be different and assigned to providers.
IRP assumes that distance from rda to rdb is equal to distance from rdb to rda.
The parameter takes a collection of such triplets that will define all the available inter-datacenter links
between routing domains.

It is important that the RTT value between routing domains is accurate. In case the value
differs significantly from the correct value IRP will make improvement decision based on incorrect
information and it will make unnecessary global improvements that will reroute more traffic via
inter-datacenter links.

Refer peer.X.rd, peer.X.flow_agents, bgpd.rd_local_mark

4.1.2.33 global.master_rd
Specifies the Routing Domain that hosts the master node of IRP in a failover configuration. By default
RD=1 hosts IRP nodes.

It is recommended that master node of IRP is hosted in RD=1 at all times.

Possible values: 1-100

Default value: 1
Recommended value: 1

4.1.2.34 global.slave_rd
Specifies the Routing Domain that hosts the slave node of IRP in a failover configuration. By default
RD=1 hosts IRP nodes.

Possible values: 1-100

Default value: 1
Recommended value: 1

4.1.2.35 global.rrd.age_max
Defines the maximum trusted interface load data age (seconds)
Data older than this interval will not be trusted by IRP. If interface rate for each provider link has not
been updated for a specified amount of time, then IRP behavior will be changed as follows:

• If provider’s limit_load is set, no further Cost/Performance improvements will be performed to

that provider
• Commit Control will not perform further in/out improvements for this provider

Possible values: 120-240

Default value: 120
Recommended value: 120. Should be increased only if frequent SNMP timeouts occur.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 190

4.1.2.36 global.rrd.datadir
Defines the file system directory path for storing RRD database files.

Possible values: valid directory

Default value: /var/spool/irp

Recommended value: /var/spool/irp

4.1.2.37 global.user_directories_conf
Defines the path to the User Directories configuration file.

Default value: /etc/noction/user_directories.conf

Recommended value: /etc/noction/user_directories.conf

4.1.3 API daemon settings

4.1.3.1 apid.log
Defines the file-system path to the iprapid log file.

Default value: /var/log/irp/irpapid.log

4.1.3.2 apid.log.level
Defines the logging level for the irpapid service.

Possible values: emerg, fatal, alert, crit, error, warn, notice, info, debug
Default value: info

Recommended value: info

4.1.3.3 apid.listen.master_ip
Defines the IPv4/IPv6 address of the API. Allows IRP API calls to target another node on the network.
If no value provided then localhost is used. When failover is enabled (global.failover) slave’s listen IP
(apid.listen.slave_ip) must be configured too.

Possible values: IPv4/IPv6 address

Default value: 127.0.0.1 ::1

4.1.3.4 apid.listen.slave_ip
Defines the IPv4/IPv6 address of the API of the slave node in a failover configuration. Allows IRP API
calls to target another node on the network. If no value provided then localhost is used. When failover
is disabled (global.failover) slave’s listen IP is not used.
See also apid.listen.master_ip.

Possible values: IPv4/IPv6 address

Default value: 127.0.0.1 ::1
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 191

4.1.3.5 apid.listen.port
Defines the TCP port on which the irpapid is listening. If no value provided port 10443 is used.

Possible values: 1-65535

Default value: 10443

4.1.3.6 apid.maxthreads
Defines the number of threads that irpapid is allowed to run. If no value provided 50 threads are used.

Possible values: 1-300

Default value: 50

4.1.3.7 apid.path.mtr
System path to MTR utility.

Possible values: /valid/path

Default value: /usr/sbin/mtr

4.1.3.8 apid.path.ping
System path to ping utility.

Possible values: /valid/path

Default value: /bin/ping

4.1.3.9 apid.path.ping6
System path to ping for IPv6 utility.

Possible values: /valid/path

Default value: /bin/ping6

4.1.3.10 apid.path.traceroute
System path to traceroute utility.

Possible values: /valid/path

Default value: /bin/traceroute

4.1.4 BGPd settings

4.1.4.1 bgpd.log
Defines the complete path to the BGPd log file

Possible values: full path to log file

Default value: /var/log/irp/bgpd.log
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 192

4.1.4.2 bgpd.log.level
Defines the logging level for the BGPd service.

Possible values: emerg, fatal, alert, crit, error, warn, notice, info, debug
Default value: info
Recommended value: info

4.1.4.3 bgpd.as_path
Defines the way BGPd handles the AS-PATH attribute in the outgoing announcements.
Keeping a correct AS-PATH can be a requirement for some NetFlow/sFlow processing since the provider
AS or the destination AS for the corresponding flow record is taken from the BGP routing table.
Some installations use outgoing filters that allow empty AS-Path while redistributing iBGP routes to
upstreams. In such cases, BGPd must be configured not to advertise the improvements with an empty
AS-PATH to prevent further redistribution of the improvements to upstream routers.
This only works when your provider is sending aggregate prefixes, which contain corresponding AS-PATH
attributes, otherwise the method will not succeed and an empty AS-Path will be produced.
Furthermore, during a BGP session initialization, no aggregates are received from peering routers, and
AS-Path remains empty until consequent information is received.
The reconstructed AS-path does not always correspond to the actual BGP AS-path.
Note that the Internet Exchange router’s AS number will not be prepended to AS Path (refer to
peer.X.ipv4.next_hop_as, peer.X.ipv6.next_hop_as).
Examples:
bgpd.as_path=2 3 - this will instruct BGPd to take first path from the database (reconstructed AS-
Path). If that path is empty, then an AS-path with the provider’s AS number and the prefix AS number
should be composed.
bgpd.as_path=3 0 - this will instruct BGPd to compose an AS-path with the provider’s AS number
and the prefix AS number. If AS-Path remains empty, the improvements with an empty AS-Path are
announced.

Possible values: Space separated list of options in the order of preference

• 0 - Allow empty AS-PATH

• 2 - Use non-empty reconstructed AS-PATH (Announce AS-path reconstructed from
route trace or empty path, if no information is available)
• 3 - Reconstruct AS path with provider ASN and prefix origin ASN

Third algorithm should be enabled in bgpd.as_path when explorer.trace.all is disabled

Default value: 2 3

4.1.4.4 bgpd.db.timeout.withdraw
Defines the time period (in seconds) before prefixes are withdrawn from the routing tables, after a
database failure. This allows BGP daemon to function independently for a period of time after the IRP
database becomes inaccessible due to a failure or manual intervention.

Possible values: 600 - 21600(6 hours)

Default value: 14400
Recommended values: 3600-14400
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 193

4.1.4.5 bgpd.improvements.remove.hold_time
Defines the time interval (in seconds) for deleting (from the IRP database) the improvements affected
by "Remove on next-hop eq" or "Remove on aggregate withdraw" conditions
(see bgpd.improvements.remove.next_hop_eq and bgpd.improvements.remove.withdrawn).
This reduces the effects of route flapping to improvement cleanup.
Verification is performed on each BGP Scan, so that the values that are lower than the value of the
bgpd.scaninterval parameter are not taken into account. The higher the values - the longer the time
interval needed for improvements, which are still valid after aggregate route is withdrawn or on equal
next-hop.
BGPd does improvements check against the routers received and sent via iBGP in periodic time intervals.
The process is referred to as the BGP Scan process.
Time interval between BGP Scannings can be configured in bgpd.scaninterval.

Possible values: 1 - 1000 sec

Default value: 60
Recommended values: 30 - 120

4.1.4.6 bgpd.improvements.remove.next_hop_eq
Instructs BGPd to remove a prefix from improvements when aggregate route next hop is changed to
improve the following hop.

This parameter shouldn’t be enabled when multiple BGP peers are configured.

Possible values: 0 (OFF), 1 (ON)

Default value: 0

Recommended value: 0

4.1.4.7 bgpd.improvements.remove.withdrawn
Instructs BGPd to remove prefix from improvements when aggregate is being withdrawn from the
router.

Possible values: 0 (OFF), 1 (ON)

Default value: 1
Recommended value: 1

4.1.4.8 bgpd.policy.cascade.amount
Defines the maximum number of downstream AS for cascading policies. If IRP identifies more down-
stream AS from the designated AS the policy for those AS will not be enforced.

Possible values: 1-1000000

Default value: 1000
Recommended value: 1000
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 194

4.1.4.9 bgpd.mon.guardtime
Defines the BGPd Monitoring guard time. All the controlled IP addresses must respond within the
specified amount of time, for the provider to be restored from the FAIL state. In FAIL state, all
improvements are withdrawn from that provider.
It is recommended that bgpd.mon.guardtime is set as a double value of the bgpd.mon.holdtime.

Possible values: 10-600

Default value: 60
Recommended value: 60

4.1.4.10 bgpd.mon.holdtime
Defines the BGPd Monitoring holdtime.
If any controlled IP address does not respond to all the requests during the holdtime, then corresponding
provider enters the FAIL state. In FAIL state, all the improvements are withdrawn from that provider.

Possible values: 10-60

Default value: 30
Recommended value: 30

4.1.4.11 bgpd.mon.keepalive
Defines the BGPd Monitoring keepalive interval (in seconds) between consequent ICMP Echo Requests
to single controlled IP address.

Possible values: 1-10

Default value: 10

Recommended value: 5

4.1.4.12 bgpd.mon.longholdtime
Defines the BGPd Monitoring long holdtime.
If any controlled IP address does not respond to all the requests during the long holdtime, then corre-
sponding provider enters the FAIL state. In FAIL state, all the improvements are withdrawn from that
provider.

Possible values: 60-3600

Default value: 1800

Recommended value: 1800

4.1.4.13 bgpd.rd_local_mark
Specifies a marker to distinguish local improvements from global improvements in the case of multiple
routing domains optimization. Parameter represents a valid value for BGP community attribute of the
form X:Y. Value in bgpd.rd_local_mark is APPENDED to communities attribute. Refer global.rd_rtt,
peer.X.rd, peer.X.flow_agents.

Possible values: X:Y

Default value: 65535:1
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 195

4.1.4.14 bgpd.scaninterval
The interval in seconds between the execution of the BGP Scan process.
BGP scan is used for:

• checking the improvements belonging to aggregated routes

• checking the improvements for aggregate withdrawn(4.1.4.7) and for aggregate next-hop
equal to improvements next-hop(4.1.4.6) conditions

• checking the improvements for changed BGP attributes

• checking for changes for the improvements to be announced to/withdrawn from iBGP neighbors

Possible values: 10-100

Default value: 20
Recommended value: 60

4.1.4.15 bgpd.snmp.simultaneous
The number of the simultaneous PDUs that will be contained in a SNMP request.

Possible values: 1-300

Default value: 10

Recommended value: 10

4.1.4.16 bgpd.updates.split
Instructs BGPd to split advertised prefixes in two equal parts (e.g /24 is split into two /25 prefixes).
This parameter should be enabled in order to preserve the original BGP UPDATE attributes received
from the corresponding aggregates.
Refer to global.aggregate, global.agg_ipv4_max, global.agg_ipv6_max parameters.

If this option is enabled, the number of announced prefixes will be twice the
core.improvements.max.

If the bgpd.peer.X.updates.limit.max parameter value is established, then the limitation is set on the
total amount of announced prefixes, AFTER split. For example, if the value of core.improvements.max
is set to 10000 and bgpd.peer.X.updates.limit.max is set to 5000, then the amount of the improvements
towards this particular provider is no more than 2500, split onto 5000.

Possible values: 0 (OFF), 1 (ON)

Default value: 1
Recommended value: 1

4.1.5 BGP sessions settings

4.1.5.1 bgpd.peer.X.as
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 196

Mandatory for each iBGP session definition.

Parameter changes cause reset of BGP session.

Defines the Autonomous System Number for the iBGP session.

Possible values: 1-4294967295

4.1.5.2 bgpd.peer.X.cap_4byte_as

Parameter changes cause reset of BGP session.

Defines usage of 16 or 32-bit autonomous system numbers. Capability can be negotiated during session
setup or forced to either 16 or 32 bits.

• 1 - Negotiated on OPEN
• 2 - Always 16-bit AS path
• 3 - Always 32-bit AS path

When the router is operating in legacy mode and does not negotiate capabilities but still sends 32-bit
AS Path then BGPd considers this a malformed AS_PATH attribute and disconnects the session. To
avoid this the parameter must be set to force use of 32bit AS numbers.
For example: A BGP session with IRP with “disable-capability-negotiation” option configured on Vyatta
router.
As a result, the BGP session is established and then teared down with log messages:

Listing 4.1: Error log

Jan 29 10:20:40.777965 WARN: BGP session RTR/IPv4 (10.0.0.1 AS 65530)
Incoming UPDATE error: Invalid elements ignored. Malformed AS_PATH
Jan 29 10:20:40.778021 ERROR: BGP session RTR/IPv4 (10.0.0.1 AS 65530)
Incoming UPDATE error: malformed AS_PATH xxxxxxx

The solution is to set bgpd.peer.RTR.cap_4byte_as = 3, then the BGP session succeeds.

Possible values: 1 (negotiate), 2 (force 16bit), 3 (force 32bit)

Default value: 1

4.1.5.3 bgpd.peer.X.flowspec
Enables/disables FlowSpec capability for BGP session.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0

4.1.5.4 bgpd.peer.X.master_communities
Defines the BGP Community that will be appended by BGPd to all advertised prefixes. The format is:
“X:Y”.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 197

Avoid colisions of communities values assigned to IRP both within its configuration and/or on
customer’s network.

When failover is enabled (global.failover) slave’s BGP Community (bgpd.peer.X.slave_communities)

must be configured too.
In case BGPd receives the full or partial RIB (Routing Information Base), values for: MED, Origin,
LocalPref and Communities are taken from a less-specific Aggregate route. If values for MED,
Origin, LocalPref are set in config, it will override any value from Aggregate route. If value for
Communities is set in config, it will be appended to communities from Aggregate route.

4.1.5.5 bgpd.peer.X.slave_communities
Defines the BGP Community that will be appended by BGPd on the slave node of a failover configura-
tion to all advertised prefixes. The format is: “X:Y”.

See also bgpd.peer.X.master_communities.

4.1.5.6 bgpd.peer.X.inbound.ipv4.next_hop
Defines IPv4 address which will be used as next_hop in BGP UPDATE for inbound improvements/an-
nouncements towards this router.
The next-hop address should be known by the router (refer Routers configuration for Inbound).
An inbound rule can be configured with a rule-specific next-hop. Refer to inbound.rule.X.next_hop.
Possible values: IPv4 address

4.1.5.7 bgpd.peer.X.inbound.ipv6.next_hop
Defines IPv6 address which will be used as next_hop in BGP UPDATE for inbound improvements/an-
nouncements towards this router.
The next-hop address should be known by the router (refer Routers configuration for Inbound).
An inbound rule can be configured with a rule-specific next-hop. Refer to inbound.rule.X.next_hop.
Possible values: IPv6 address

4.1.5.8 bgpd.peer.X.inbound.localpref
Defines localpref value used by IRP for inbound improvements for this router.
Assigned localpref value should allow Inbound Improvement to become best route.

Avoid colisions of localpref values assigned to IRP both within its configuration and/or on cus-
tomer’s network.

Possible values: 0 - 4294967295

4.1.5.9 bgpd.peer.X.keepalive
Defines the session keepalive time (sec). See RFC1771 for details. Hold time is calculated as keepalive *
3.
Possible values: 1-100
Default value: 60
Recommended value: 1/3 holdtime
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 198

4.1.5.10 bgpd.peer.X.listen
Instructs the BGP daemon to listen for incoming sessions. It must be set to 1 to be RFC1771 compliant.
It can be set to 0 to resolve specific issues.

Possible values: 0 (OFF), 1 (ON)

Default value: 1
Recommended value: 1

4.1.5.11 bgpd.peer.X.master_localpref
Defines the local-preference value for prefixes announced by BGPd.

Avoid colisions of localpref values assigned to IRP both within its configuration and/or on cus-
tomer’s network.

When failover is enabled (global.failover) slave’s BGP LocalPref (bgpd.peer.X.slave_localpref) must be

configured too.

If failover is enabled, master’s LocalPref value must be greater than slave’s LocalPref value.

In case BGPd received the full or partial RIB (Routing Information Base), values for MED, Origin,
LocalPref and Communities will be taken from less-specific Aggregate route. If values for MED,
Origin, LocalPref are set in config, it will override any value from Aggregate route. If value for
Communities is set in config, it will be appended to communities from Aggregate route.

Possible values: 0 - 4294967295

Default value:

4.1.5.12 bgpd.peer.X.slave_localpref
Defines the local-preference value for prefixes announced by BGPd.
See also bgpd.peer.X.master_localpref.

Possible values: 0 - 4294967295

Default value:

4.1.5.13 bgpd.peer.X.med
Defines the Multi-Exit Discriminator (MED) value for prefixes announced by BGPd.
In case BGPd received the full or partial RIB (Routing Information Base), values for MED, Origin,
LocalPref and Communities will be taken from less-specific Aggregate route. If values for MED,
Origin, LocalPref are set in config, it will override any value from Aggregate route. If value for
Communities is set in config, it will be appended to communities from Aggregate route.

Possible values: 0 - 4294967295

4.1.5.14 bgpd.peer.X.origin
Defines the Origin value for prefixes announced by BGPd.

Possible values: 0 (IGP), 1 (EGP), 2 (INCOMPLETE)

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 199

4.1.5.15 bgpd.peer.X.master_our_ip

Mandatory for IPv4 BGP session.

Parameter changes cause reset of BGP session.

Defines iBGP session’s local IPv4 address. When failover is enabled (bgpd.peer.X.slave_our_ip) slave’s
local IP address (bgpd.peer.X.slave_localpref) must be configured too.

Possible values: IPv4 address

4.1.5.16 bgpd.peer.X.slave_our_ip

Mandatory for IPv4 BGP session.

Parameter changes cause reset of BGP session. Affects only BGP session of slave node in a
failover configuration.

Defines iBGP session’s local IPv4 address of slave node in a failover configuration.
See also bgpd.peer.X.master_our_ip.

Possible values: IPv4 address

4.1.5.17 bgpd.peer.X.master_our_ipv6

Mandatory for IPv6 BGP session.

Parameter changes cause reset of BGP session.

Defines iBGP session’s local IPv6 address. When failover is enabled (bgpd.peer.X.slave_our_ip) slave’s
local IPv6 address (bgpd.peer.X.slave_our_ipv6) must be configured too.

Possible values: IPv6 address

4.1.5.18 bgpd.peer.X.slave_our_ipv6

Mandatory for IPv6 BGP session in failover configuration.

Parameter changes cause reset of BGP session. Affects only BGP session of slave node in a
failover configuration.

Defines iBGP session’s local IPv6 address of slave node in a failover configuration.
See also bgpd.peer.X.master_our_ipv6.

Possible values: IPv6 address

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 200

4.1.5.19 bgpd.peer.X.peer_ip

Mandatory for IPv4 BGP session.

Parameter changes cause reset of BGP session.

Defines iBGP session’s peer IPv4 address.

Possible values: IPv4 address

4.1.5.20 bgpd.peer.X.peer_ipv6

Mandatory for IPv6 BGP session.

Parameter changes cause reset of BGP session.

Defines iBGP session’s peer IPv6 address.

Possible values: IPv6 address

4.1.5.21 bgpd.peer.X.master_router_idbgpd.peer.X.slave_router_id

Mandatory for IPv6 BGP session either as standalone master or when failover is enabled and the
value should be different from bgpd.peer.X.slave_router_id.

Parameter changes cause reset of BGP session.

Defines IRP server’s router ID. The BGP router ID is used in the BGP algorithm for determining the
best path to a destination where the preference is given to the BGP router with the lowest router ID.

Possible values: 4-byte value in the IPv4 address format. Any valid IPv4 address can be used.

4.1.5.22 bgpd.peer.X.slave_router_id

Mandatory for IPv6 BGP session either as standalone slave or when failover is enabled and the
value should be different from bgpd.peer.X.master_router_idbgpd.peer.X.slave_router_id.

Parameter changes cause reset of BGP session. Affects only BGP session of slave node in a
failover configuration.

Defines IRP server’s router ID. The BGP router ID is used in the BGP algorithm for determining the
best path to a destination where the preference is given to the BGP router with the lowest router ID.

Possible values: 4-byte value in the IPv4 address format. Any valid IPv4 address can be used.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 201

4.1.5.23 bgpd.peer.X.shutdown

Parameter changes cause reset of BGP session.

Defines whether the corresponding iBGP session is active or shutdown.

Possible values: 0 (Active), 1 (Shutdown)

Default value: 0

4.1.5.24 bgpd.peer.X.updates.limit.max
Represents a maximum number of prefixes that can be announced simultaneously in one session.
If bgpd.updates.split is ON, the number of announced prefixes is twice the number of the improvements.
If the BGP neighbor (typically - the edge router) has any hardware/software limitation for the number
of routes in active routing table, then BGPd can be instructed not to announce more than a specified
amount of prefixes. Value 0 means no limit for current iBGP session.
Values less than maximum allowed improvements, can cause not all the improved prefixes to be injected
into such peer.
Higher values can be incompatible with the router (please consult the router’s vendor regarding the
maximum amount of entries in routing table as well as the BGP table).

Possible values: 0-100000

Default value: 0
Recommended value: 0

4.1.5.25 bgpd.peer.X.updates.limit.ps
Defines the maximum number of updates per second that will be sent to the current BGP neighbor. Low
values will slow down the improvements injection. High values can cause router to drop the improvement
without installing it into the routing database.

Possible values: 1-1000000

Default value: 500

Recommended values: 100-1000

4.1.5.26 bgpd.peer.X.master_password
Defines iBGP session’s password. When failover is enabled (bgpd.peer.X.slave_our_ip) slave’s local
IPv6 address (bgpd.peer.X.slave_password) must be configured too.

Possible values: up to 80 alphanumeric characters

4.1.5.27 bgpd.peer.X.slave_password
Defines iBGP session’s password for slave node in a failover configuration.
See also bgpd.peer.X.master_password.

Possible values: up to 80 alphanumeric characters

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 202

4.1.6 Collector settings

4.1.6.1 collector.detect.explorer_ips
Instructs the collector to ignore the Explorer-generated traffic, which can initiate false IRP reaction to
the network events (Excessive loss, blackout).

This feature is used only by the SPAN collector, instructing it to ignore the IPv4 traffic when
the network address translation is used to masquerade IP addresses.

Possible values: 0 (OFF), 1 (ON)

Default value: 0

Recommended value: 0

4.1.6.2 collector.export.interval
Defines the time interval (in seconds) between data exports in Collectors (flow and span). Lower val-
ues result in more accurate traffic data, but higher storage usage. High values cause slow statistics
accumulation and less accurate calculation of prefix data.

Possible values: 10-3600

Default value: 60

Recommended value: 60-300

4.1.6.3 collector.export.ttl
Defines the collector-gathered data lifetime (in seconds). Larger values lead to excessive database size.
Aggregated data is kept for one year, disregarding this parameter.

Possible values: 1-’unlimited’

Default value: 86400
Recommended value: 1-30days

4.1.6.4 collector.export.volume.high.top_n
Defines the number of top volume prefixes.
Top N prefixes will be marked for priority probing in descending order of volume. Lower values lead to
small number of events for priority probing of high volume prefixes. Higher values lead to overflow of
probing queue with jobs for unimportant prefixes.

Possible values: 0-’unlimited’

Default value: 50
Recommended value: 10-50
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 203

4.1.6.5 collector.export.volume.min
Defines the minimum collected prefix volume (bytes). The prefix will not be exported into the IRP
database if its traffic volume is less than the value of the current parameter (collector.export.volume.min)
and the percentage of the traffic volume less the collector.export.volume.min_pct parameter.
Lower values will lead to a higher number of prefixes exported into the IRP database.

Possible values: 1-’unlimited’

Default value: 100000
Recommended value: 10000-1000000

4.1.6.6 collector.export.volume.min_pct
Defines the minimum prefix volume (%) for a prefix to be exported into the IRP database. The pre-
fix will not be exported into the IRP database if its percentage of the traffic volume is less than the
value of the current parameter (collector.export.volume.min_pct) and the traffic volume less than col-
lector.export.volume.min. The percentage of the traffic volume is calculated according to the export
interval (collector.export.interval). Lower values will lead to a higher number of prefixes exported into
the IRP database.

Possible values: 0.0001-100

Default value: 0.01
Recommended value: 0.01-0.1

4.1.6.7 collector.flow.buffer.size
Defines the buffer size (in packets) for the Flow collector (irpflowd). Higher values lead to extra memory
used by the collector. Slightly increase this value if the network has traffic spikes that cause buffer
overflows and packet drop in the collector.

Possible values: 10000-1000000

Default value: 10000
Recommended value: 10000-1000000

4.1.6.8 collector.flow.enabled
Enables the Flow collector. If the parameter is enabled, Flow Collector will be used to gather network
prefix data, but it is still possible to use SPAN Collector to acquire network events.

Possible values: 0 (OFF), 1 (ON)

Default value: 0
Recommended value: 0 - for SPAN collector (see collector.span.enabled), 1 - for Flow collector.

4.1.6.9 collector.flow.listen.nf
Defines the UDP port the Flow collector will listen to, for NetFlow traffic.

Possible values: 1-65535

Default value: 2055
Recommended value: 2055
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 204

4.1.6.10 collector.flow.listen.sf
Defines the UDP port the Flow collector will listen to, for sFlow traffic.

Possible values: 1-65535

Default value: 6343
Recommended value: 6343

4.1.6.11 collector.flow.log
Defines the file-system path to the irpflowd log file.

Default value: /var/log/irp/irpflowd.log

4.1.6.12 collector.flow.log.level
Defines the logging level for the irpflowd service.

Possible values: emerg, fatal, alert, crit, error, warn, notice, info, debug
Default value: info
Recommended value: info

4.1.6.13 collector.flow.sources
Defines the valid NetFlow/sFlow/jFlow exporters IP addresses. Flow data coming in from other IP
addresses that are not listed in this parameter will be ignored.
It is recommended to specify only trusted Flow exporters IP addresses.

Possible values: list of valid IPv4/IPv6 addresses

Default value: 0/0 ::/0
Recommended value: Trusted IPv4/IPv6 flow exporter addresses

4.1.6.14 collector.ournets

Mandatory parameter

Defines the list of networks to be analyzed. Typically this is the list of prefixes advertised by your AS.
Format:

1. Space-separated list of local IPv4/IPv6 prefixes that should be analyzed and optimized by the IRP.
2. Absolute path to a newline-separated file containing a list of local IPv4/IPv6 prefixes that should
be analyzed and optimized by the IRP.

IPv4 prefixes are treated as /24 subnets and IPv6 prefixes as /48 subnets, if netmask is not clearly
specified.

The downstream clients’ networks can be indicated as well.

Possible values: See above.

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 205

4.1.6.15 collector.sessions.max
Defines the maximum number of TCP sessions inside the collector process. It can be used to minimize
memory usage.

Both Flow and Span collectors use this parameter.

Estimated memory usage can be calculated as follows: usage = 80MB + (collector.sessions.max*1464)

Possible values: 10000-10000000

Default value: 2000000
Recommended value: Depends on available server memory and the maximum number of simultaneous
sessions during peak hours.

4.1.6.16 collector.span.buffer.size
Defines the buffer size (in packets) for the Span collector (irpspand). Higher values lead to extra memory
used by the collector. Slightly increase this value if the network has traffic spikes that cause buffer
overflows and packet drop in the collector.

Possible values: 10000-5000000

Default value: 100000
Recommended value: 100000-5000000

4.1.6.17 collector.span.enabled
Enables the Span collector. Span collector will be used to gather network prefix data, if the parameter
is enabled.

Possible values: 0 (OFF), 1 (ON)

Default value: 0
Recommended value: 0 - if no traffic analysis should be performed by the Span collector, 1 - if the
Span collector should analyze mirrored traffic for network events and/or prefix statistics.

See also: collector.flow.enabled

4.1.6.18 collector.span.interfaces

Mandatory parameter

Defines a space-separated network interfaces list for passive packet analysis by the Span collector.
Example:
collector.span.interfaces = eth0 eth1 eth2

4.1.6.19 collector.span.log
Defines the file-system path to the irpspand log file.

Default value: /var/log/irp/irpspand.log

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 206

4.1.6.20 collector.span.log.level
Defines the logging level for the irpspand service.

Possible values: emerg, fatal, alert, crit, error, warn, notice, info, debug
Default value: info
Recommended value: info

4.1.6.21 collector.span.min_delay
Enables fast probing tasks queuing for prefixes with one of the following issues:

• Blackouts
• Congestion
• Excessive packet delays.

Possible values: 0 (OFF), 1 (ON)

Default value: 0
Recommended value: 0

See also: collector.span.min_delay.probing_queue_size

4.1.6.22 collector.span.min_delay.probing_queue_size
Defines the number of slots in the probing queue that can be used by the min_delay algorithm
(see: collector.span.min_delay)

Possible values: 1-200

Default value: 50
Recommended value: 30-200

See also: collector.span.min_delay

4.1.6.23 collector.span.threshold.blackout
Defines the percentage of retransmitted packets in regards to the total packets number. If retransmit is
higher than this value, this prefix is considered to have a blackout.

Possible values: 1-100

Default value: 70
Recommended value: 70-90

4.1.6.24 collector.span.threshold.congestion
Defines the percentage of retransmitted packets in regards to the total packets number. If retransmit is
higher than this value, this prefix is considered to have a congestion.

Possible values: 1-100

Default value: 50
Recommended value: 30-70
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 207

4.1.6.25 collector.span.threshold.delay
Percentage of excessive delayed packets by the total packets number (see collector.span.threshold.excessive).
If this percentage is higher than this value, we consider this prefix to have a delay.

Possible values: 1-100

Default value: 20

Recommended value: 10-30

4.1.6.26 collector.span.threshold.excessive
Defines the excessive RTT (%). The value is compared to the average round trip time. If RTT is higher
by collector.span.threshold.excessive in % than the average RTT, then the counter of excessive delay
packets is incremented.

Possible values: > 0

Default value: 200

Recommended value: 100-500

4.1.6.27 collector.speaking_ips
Defines the number of speaking IPs to be stored in the database.

Possible values: 0-1000

Default value: 100
Recommended value: 100
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 208

4.1.7 Core settings

4.1.7.1 core.log
Defines the file-system path to the core log file.

Default value: /var/log/irp/core.log

4.1.7.2 core.log.level
Defines the logging level for the core service.

Possible values: emerg, fatal, alert, crit, error, warn, notice, info, debug
Default value: info

Recommended value: info

4.1.7.3 core.commit_control
Enables or disables Commit Control (see the Commit Control section)

The following parameters must be set in the configuration file to ensure proper functionality of
the Commit Control feature:

1. SNMP parameters must be set for each provider:

(a) peer.X.snmp.ip or peer.X.snmp.ipv6

(b) peer.X.snmp.interface
(c) peer.X.snmp.community
(d) irpstatd needs to be started (see section 2.8)

2. Each provider needs to have peer.X.95th set to the desired Commit level
3. peer.X.precedence must be set for each provider

When Commit Control is disabled, all existing Commit Control improvements are removed from current
improvements. The improvements are preserved temporarily until core.commit_control.probe_ttl after
Core service restart. If Commit Control is re-enabled before expiration these improvements will be
restored into current improvements.
See also:
Commit Control
core.commit_control.rate.group
core.commit_control.rate.high
core.commit_control.rate.low

Possible values: 0 (OFF), 1 (ON)

Default value: 0

4.1.7.4 core.commit_control.agg_bw_min
Defines the minimum bandwidth for a single prefix. Prefixes, whose bandwidth is less than the specified
value, are ignored by Commit Control and are not being used in the Commit Control algorithm.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 209

It is not recommended to set high values for this parameter since it limits the number of prefixes
that can be rerouted by the Commit Control mechanism.

This parameter is considered only during initial improvement. During retry probing of existing
Commit Control improvements IRP might detect that the current bandwidth usage for some prefixes
is below this limit. IRP will preserve these improvements as relevant if there are no other reasons
to withdraw them.

Possible values: 0.1-50

Default value: 1

Recommended value: 0.1 - if summary throughput is lower than 200-500Mbps,

1 - if summary throughput is higher than 500Mbps.

4.1.7.5 core.commit_control.del_irrelevant
If enabled, Commit Control algorithm deletes improvements that have traffic volume less than value in
the4.1.7.4 parameter.

Possible values: 0 (OFF), 1 (ON)

Default value: 1

Recommended value: 1

4.1.7.6 core.commit_control.inbound.enabled
Enables or disables Inbound bandwidth control.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0

4.1.7.7 core.commit_control.inbound.moderated
Enables or disables review and moderate feature of inbound bandwidth control.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0

4.1.7.8 core.commit_control.inbound.rate.high
Defines provider’s high load rate limit (%). Refer core.commit_control.rate.high for details.
This limit is used when inbound and outbound commit control operate independently. Otherwise
core.commit_control.rate.high value overrides this. Refer also peer.X.95th.mode.

Possible values: 50 - 99

Default value: 90
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 210

4.1.7.9 core.commit_control.inbound.rate.low
Defines provider’s low load rate limit (%). Refer core.commit_control.rate.low for details.
This limit is used when inbound and outbound commit control operate independently. Otherwise
core.commit_control.rate.low value overrides this. Refer also peer.X.95th.mode.

Possible values: 30 - 99
Default value: 70

4.1.7.10 core.commit_control.inbound.volume_estimation
Defines method of estimating inbound bandwidth. The available options are:

• 0: Last minute data

• 1: Largest of last minute and current hour average

• 2: Largest of last minute and daily average

Possible values: 0 - 2
Default value: 1

4.1.7.11 core.commit_control.probe_ttl
Defines the TTL (time-to-live) for a specific probe. If the probe results are older than the value specified
here, the system will disregard it and the Commit Control algorithm will schedule it for expedited
re-probing.

Possible values: 600-86400

Default value: 7200

Recommended value: 7200

4.1.7.12 core.commit_control.probing_queue_size
Defines the number of slots in the probing queue that can be used by the Commit Control algorithm.

Possible values: 1-500

Default value: 100
Recommended value: 10-100

4.1.7.13 core.commit_control.rate.group
If using provider load balancing in group, this parameter defines allowed deviation of a provider’s current
bandwidth(in %) compared with other providers in the same group.
Example 1. There are three grouped providers with 95th set to 1 Gbps, 2 Gbps and 3 Gbps with a total
current bandwidth of 600 Mbps. In this case, load balancing expects that each provider’s bandwidth
usage will be 100 Mbps, 200 Mbps and 300 Mbps accordingly. These values are proportional to individual
provider’s 95th settings in the group. Let’s assume core.commit_control.rate.group is set to 5%. If a
provider’s bandwidth exceeds by more than the 5% limit the expected value (105 Mbps, 210 Mbps and
315 Mbps accordingly and the total for the group did not change), IRP will start active rerouting of
excessive bandwidth from that provider to providers within or outside this group.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 211

Load balancing in a provider group is performed even when the 95th is not exceeded.

Possible values: 1-30

Default value: 5

Recommended value: 5

See also: Provider load balancing

4.1.7.14 core.commit_control.rate.high
Defines provider’s high load rate limit (%).
IRP will stop Latency/Cost improvement if provider bandwidth is over core.commit_control.rate.high
% of load (percents of 95th). The improvements will start happening again after provider’s bandwidth
drops below core.commit_control.rate.low % of load.
These parameters are used for passive 95th overload prevention as well as an additional method for
Commit Control to be less aggressive.

Provider’s Latency/Cost improvement ability does not change when current bandwidth rate varies
between core.commit_control.rate.low and core.commit_control.rate.high.

Example 2. if provider’s current load is 1100 Mbps, 95th is set to 1000 Mbps. Commit Control will try
to unload 200Mbps (to reduce current load to 90% of the 95th level) in order to prevent 95th overload
as well as allow provider’s bandwidth natural growth during peak hours.

Possible values: 50-99, but larger or equal to core.commit_control.rate.low

Default value: 90
Recommended value: 90

See also: Provider load balancing

4.1.7.15 core.commit_control.rate.low
Defines provider’s high load rate limit (%).
IRP will start Latency/Cost improvements again after provider’s bandwidth drops below core.commit_control.rate.low
% of load. Latency/Cost improvements will be stopped if provider bandwidth is over core.commit_control.rate.high
% of load (percents of 95th).
These parameters are used for passive 95th overload prevention as well as an additional method for
Commit Control to be less aggressive.

Provider’s Latency/Cost improvement ability does not change when current bandwidth rate varies
between core.commit_control.rate.low and core.commit_control.rate.high.

Possible values: 30-99, but lower or equal to core.commit_control.rate.high

Default value: 80

Recommended value: 80

See also: Provider load balancing

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 212

4.1.7.16 core.commit_control.react_on_collector
Defines if Commit Control algorithm should react immediately on collector overload values.

Enabling this feature represents a tradeoff. Take into consideration the benefits of a faster react
time vs reliance on older probes and statistics when making Commit Control improvements.

Possible values: 0 (OFF), 1 (ON)

Default value: 1
Recommended value: 1

4.1.7.17 core.cost.worst_ms
Latency worsening.
Defines the allowed latency worsening for an improved prefix while running in Cost optimization mode
(see global.improve_mode) and the Commit Control feature is enabled. While operating in these modes
IRP will consider as alternatives candidates with latency degradation not exceeding this limit. Of
course, the candidate with least latency degradation or even with better latency will be chosen as an
improvement.
When IRP detects that an existing Commit Control improvement has alternatives with latencies better
than specified value it will replace it with a new performance (latency) improvement. Over-usage of the
alternative route will be avoided.

This parameter is applicable for local improvements within a Routing Domain. Global improve-
ments will also take into account core.global.worst_ms values.

Possible values: 1-unlimited

Default value: 10
Recommended value: 10

4.1.7.18 core.eventqueuelimit
Defines the exploring events queue size. Lower values may result in IRP missing some important network
issues.

Possible values: 1-10000

Default value: 1000

Recommended value: 100-1000

4.1.7.19 core.eventqueuelimit.retry_probe_pct
Defines a percentage of the total exploring queue length (see core.eventqueuelimit) to be used for retry
probing.

Possible values: 1-100

Default value: 40

Recommended value: 20-60

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 213

4.1.7.20 core.global.worst_ms
Defines acceptable latency worsening for cost and commit for global improvements. Refer core.cost.worst_ms

Possible values: 1-unlimited

Default value: 30

Recommended value: 10

4.1.7.21 core.global.allow_commit
Enables or disables global commit control Improvements in a multiple routing domain configuration. By
default these Improvements are enabled.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 1

4.1.7.22 core.global.allow_latency_cost
Enables or disables latency and cost global Improvements in a multiple routing domain configuration.
By default these Improvements are enabled.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 1

4.1.7.23 core.improvements.clearslots
Specifies the number of outdated improvements that will be discarded when slots are needed for new
improvements. Outdated are improvements as defined by core.improvements.clearslots.days_max. In
case there are no outdated improvements usual improvement replacement mechanisms are employed
relying on Improvements weight.

Possible values: 0 - 100

Default value: 10
Recommended value: 10

4.1.7.24 core.improvements.clearslots.days_max
Sets the number of days after which an improvement is considered outdated. Outdated improve-
ments have not been re-confirmed for a long time. The oldest of them will be discarded by the slot
clearing process for outdated improvements when new slots are needed for new improvements. Refer
core.improvements.clearslots.

Possible values: 1 - 60

Default value: 7
Recommended value: 7
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 214

4.1.7.25 core.improvements.defer
Enables or disables Improvements confirmation feature. Improvements confirmation schedules a potential
improvement for confirmation after a short period of time instead of immediately announcing it. This
allows IRP to verify that a detected problem is short-lived and there is no need to make the improvement.
Only if IRP gets the same best route during confirmation, an improvement is considered relevant and
will be announced.
Improvements confirmation represents a tradeoff since all improvements are deferred for some time and
problems persist a little longer.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0
Recommended value: 0

4.1.7.26 core.improvements.defer.hold
Defines the period an improvement is held before re-scheduling it for confirmation. Refer core.improvements.defer.

Possible values: 10-1800

Default value: 120
Recommended value: 120

4.1.7.27 core.improvements.max
Defines the maximum number of active IPv4 improvements.

The number of announced prefixes can be twice this value if bgpd.updates.split is enabled.

Possible values: 1-100000

Default value: 10000
Recommended value: 10000

4.1.7.28 core.flowspec.max
Defines the maximum number of IPv4 Flowspec rules that IRP is allowed to advertise.

Possible values: 1-10000

Default value: 100

4.1.7.29 core.improvements.max_ipv6
Defines the maximum number of active IPv6 improvements.

Possible values: 1-100000

Default value: 2000
Recommended value: 2000
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 215

4.1.7.30 core.flowspec.max_ipv6
Defines the maximum number of IPv6 Flowspec rules that IRP is allowed to advertise.

Possible values: 1-10000

Default value: 100

4.1.7.31 core.improvements.retry_probe.volume_top_n
Traffic volume top-N prefixes
The improvements are checked for relevance when they are improved the first time, with the relevancy
flag set accordingly. All prefixes (starting from current month’s first day) are sorted by traffic volume.
The top-N prefixes from this list are treated as "relevant" prefixes.
In case an empty slot is required for a new improvement, the old irrelevant prefix(es) will be deleted first
to free-up a slot.
Volume-relevant prefixes will be queued for retry probing before non-relevant prefixes.

Possible values: 1-20000

Default value: 5000
Recommended value: chosen for each network individually

4.1.7.32 core.improvements.ttl.retry_probe
Defines the retry probing interval (in seconds). Once an improvement is older than this value, it will be
sent for re-probing.

Possible values: 600-86400

Default value: 14400

Recommended value: 7200-14400

See also: core.improvements.retry_probe.volume_top_n

4.1.7.33 core.outage_detection
Enables the Outage Detection algorithm.

Possible values: 0 (OFF), 1 (ON)

Default value: 0

Recommended value: 1

4.1.7.34 core.outage_detection.limit_pct
Defines the problem prefixes threshold (in percent) over which the system confirms an outage
(if core.outage_detection is enabled).
See also: Outage detection

Possible values: 1-100

Default value: 60
Recommended value: 60
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 216

4.1.7.35 core.overusage.check_interval
Defines the interval to check overusage for Flowspec policies prefixes in order to apply automatic throt-
tling Flowspec policies.

Possible values: 60-600

Default value: 60

Recommended value: 60

4.1.7.36 core.overusage.hold_timer
Defines the retention time for throttling rules after prefix average bandwidth returns to normal for
automatic throttling Flowspec policies.

Possible values: 60-600

Default value: 60

Recommended value: 300

4.1.7.37 core.overusage.out.average.period
Defines the duration of the time interval in hours used to determine average prefix usage for automatic
throttling Flowspec policies.

Possible values: 1-24

Default value: 1
Recommended value: 1

4.1.7.38 core.overusage.out.average.relevant_min
Defines minimum prefix bandwidth average (in Mbps) that is relevant for automatic throttling Flowspec
policies.

Possible values: 1-100000

Default value: 50
Recommended value: 50

4.1.7.39 core.overusage.out.threshold.throttle
Defines the multiplier of prefix average bandwidth applied on outbound traffic for automatic throttling
Flowspec policies.

Possible values: 2-10000

Default value: 2
Recommended value: 2
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 217

4.1.7.40 core.overusage.out.threshold.trigger
Defines the multiplier of prefix average bandwidth that sets the overusage threshold of a prefix. An
automatic throttling Flowspec policy is created when current prefix bandwidth exceeds the average
multipled by this multiplier.

Possible values: 2-10000

Default value: 10
Recommended value: 10

4.1.7.41 core.performance.loss_pct
Defines the relevant packet loss difference (in percent) after which a loss improvement is made.
For current route loss >= 50%: Do not perform rerouting, if packet loss difference between routes is less
than 15%.
For current route loss < 50%: Do not perform rerouting, if packet loss difference between routes is less
than core.performance.loss_pct.

Possible values: 1-15

Default value: 3
Recommended value: 3-5

4.1.7.42 core.performance.rtt.diff_ms
Defines the relevant RTT difference (ms) after which a latency improvement is made. If the RTT
difference between the current route and another provider is less than core.performance.rtt.diff_ms,
then the system ignores this prefix as an improvement candidate.

core.performance.rtt.diff_ms and core.performance.rtt.diff_pct are grouped together at decision

making, using a logical AND condition.

Possible values: 1-’unlimited’

Default value: 10
Recommended value: 2-20

4.1.7.43 core.performance.rtt.diff_pct
Defines the relevant RTT difference (in percent) after which a latency improvement is made. If the RTT
difference between the current route and another provider is less than core.performance.rtt.diff_pct, then
the system ignores this prefix as an improvement candidate.

core.performance.rtt.diff_ms and core.performance.rtt.diff_pct are grouped together at decision

making, using a logical AND condition.

Possible values: 1-100

Default value: 10
Recommended value: 5-20
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 218

4.1.7.44 core.performance.rtt.ix_diff_ms
Defines the relevant RTT difference (ms) to make latency improvements across Internet Exchanges and
transit providers. A latency improvement from an IX to a transit provider is allowed only when the
latency is improved by at least this threshold. Inversely, a latency improvement from a transit provider
to an IX is allowed even if latency degradation is less thanthis threshold.

A default value of zerofor this parameter disables this feature. A value smaller than or
equal to core.performance.rtt.diff_ms isn’t allowed. When the feature is disabled then the
core.performance.rtt.diff_ms and core.performance.rtt.diff_pct parameters are considered to deter-
mine relevancy of latency improvements.

core.performance.rtt.ix_diff_ms and core.performance.rtt.ix_diff_pct are grouped together at

decision making, using a logical AND condition.

Possible values: 1-1000000

Default value: 0
Recommended value: 20-50

4.1.7.45 core.performance.rtt.ix_diff_pct
Defines the relevant RTT difference (in percent) to make latency improvements across Internet Exchanges
and transit providers. Refer core.performance.rtt.ix_diff_ms for details.

core.performance.rtt.ix_diff_ms and core.performance.rtt.ix_diff_pct are grouped together at

decision making, using a logical AND condition.

Possible values: 1-100

Default value: 10
Recommended value: 5-20

4.1.7.46 core.probes.ttl.failed
Defines the failed probe lifetime (in seconds).
Regular and Retry probing will not be performed until the age of the failed probe is less than core.probes.ttl.failed

Possible values: 120-14400

Default value: 300
Recommended value: 300

4.1.7.47 core.probes.ttl.max
Defines the probe lifetime (in seconds).
Probed prefixes data is kept in the IRP database for the specified amount of time. Automatic cleanup
of outdated data is performed periodically by Dbcron.
See also: Administrative Components
High values will lead to database size growth.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 219

Possible values: 86400-604800

Default value: 86400
Recommended value: 86400

4.1.7.48 core.probes.ttl.min
Defines the relevant probe lifetime (in seconds)
Ordinary probing will not be performed for a specific prefix if its probe age is less than core.probes.ttl.min.

Possible values: 300-43200

Default value: 7200
Recommended value: 1800-14400

4.1.7.49 core.problem.outage_timeout
Outage detection timeout (in seconds).
The IRP will disregard a possible outage, if it was not confirmed during last core.problem.outage_timeout.

Possible values: 0-3600

Default value: 600
Recommended value: 600

See also: core.outage_detection

4.1.7.50 core.problem.rtt.diff_pct
Defines the RTT difference (in percent) between the current route and the new route, for Outage detection
algorithm. IRP will choose a new route for problematic prefixes only if the RTT difference (in percent)
is greater than this parameter.

Possible values: 1-100

Default value: 50
Recommended value: 30-60

4.1.7.51 core.vip.interval.probe
Defines the VIP prefixes probing interval, in seconds. All networks and ASNs specified in /etc/noction/policies.con
and will be queued for priority probing every core.vip.interval.probe seconds.
See also: VIP Improvements

Possible values: 300-14400

Default value: 3600

4.1.8 Explorer settings

4.1.8.1 explorer.log
Defines the file-system path to the explorer log file.

Default value: /var/log/irp/explorer.log

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 220

4.1.8.2 explorer.log.level
Defines the logging level for the explorer service.

Possible values: emerg, fatal, alert, crit, error, warn, notice, info, debug
Default value: info

Recommended value: info

4.1.8.3 explorer.aipi
If enabled, AIPI (Adaptive Inter-Packet Interval) algorithm is executed using inter-packet intervals con-
figured for each provider.

Possible values: 0 (OFF), 1(ON)

Default value: 0
Recommended value: 0

See also: peer.X.diag_hop.interval_min, peer.X.diag_hop.interval_max

4.1.8.4 explorer.high_vol_precedence
High volume tasks have priority over the tasks with variable cardinality.
The parameter establishes the balance between the tasks allocated for processing high volume prefixes
(high priority for the Customer network) and the network events tasks (such as blackout, congestion,
etc).

Possible values: 10-90

Default value: 50

See also: explorer.maxthreads, collector.export.volume.high.top_n

4.1.8.5 explorer.infra_ips

Mandatory parameter

Defines the IPv4/IPv6 addressees, used in internal infrastructure to determine the current route for a
specific prefix. If netmask is not specified, then /32 is assumed for IPv4 address and /128 for IPv6
address (host addresses).
Format:

1. Space-separated list of infrastructure IPv4/IPv6 addresses.

2. Absolute path to a newline-separated file containing a list of local IPv4/IPv6 prefixes that should
be analyzed and optimized by the IRP.

Possible values: See above.

Recommended value: all IPv4/IPv6 addresses used in the internal infrastructure.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 221

4.1.8.6 explorer.interval.infra
Defines the time interval (in milliseconds) between the packets sent to infrastructure hops (4.1.8.5). By
decreasing this value the Explorer performance enhances.
Possible values: 1-200
Default value: 5
Recommended value: 5

4.1.8.7 explorer.interval.other
Defines the time interval (in milliseconds) between the packets sent to destination hops (4.1.8.5).
Possible values: 1-1000
Default value: 200
Recommended value: 200

4.1.8.8 explorer.interval.other.trace
Defines the time interval (in milliseconds) between traceroute packets sent to non-infrastructure hops.
Possible values: 1-1000
Default value: 20
Recommended value: 20

4.1.8.9 explorer.ipv4_test
Defines public IPv4 address that will be used for ensuring that PBR is operational.
Possible values: IPv4 address
Default value: 8.8.8.8
Recommended value: 8.8.8.8

4.1.8.10 explorer.ipv6_test
Defines public IPv6 address that will be used to verify that PBR is operational.
Possible values: IPv6 address
Default value: 2620:0:ccc::2
Recommended value: 2620:0:ccc::2

4.1.8.11 explorer.maxthreads
Defines the number of simultaneously processed exploring tasks.

If this value is excessively large Explorer tasks can take very long to finish. Hardware/router
diagnostic packet rate limitations kick in causing all threads to wait for exploring tasks to finish.

Possible values: 10-2000

Default value: 200
Recommended value: 200-300
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 222

4.1.8.12 explorer.max_collector_ips
Process max collected IPs. Maximum number of IPs to probe for a specific prefix.

Possible values: 1-255

Default value: 10
Recommended value: 10

4.1.8.13 explorer.algorithms
Enables or disables the use of new scanning, probing and tracing Explorer algorithms.

Possible values: 0 (Disabled), 1(Enabled)

Default value: 0

4.1.8.14 explorer.probe.algorithm
Defines the probing algorithm(s) used by explorer, in the preferred order.
The following algorithms can be used:

• UDP - udp requests (destination port: 33434)

• ICMP - regular ICMP ping sweep
• TCP_SYN - tcp syn scan (destination port: 33434)

Possible values: ICMP UDP TCP_SYN

Default value: ICMP UDP TCP_SYN
Recommended value: ICMP UDP TCP_SYN

4.1.8.15 explorer.probe.indirect_priority
Defines the execution priority between direct and indirect probing algorithms. Parameter value of 1
means that indirect probing will be executed prior to direct probing algorithm. Parameter value of 2
disables indirect probing.

This parameter conflicts with explorer.trace.all configuration option.

Possible values: 0 (Direct then Indirect), 1 (Indirect then Direct), 2 (Direct only)
Default value: 0
Recommended value: 0

4.1.8.16 explorer.probing.sendpkts.min
Defines the default ping packets count for each probe. If any loss is detected, additional packets (up to
explorer.probing.sendpkts.adaptive_max) are sent, for a more accurate packet loss detection.

Possible values: positive Integer value

Default value: 5
Recommended value: 5
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 223

4.1.8.17 explorer.probing.sendpkts.adaptive_max
Defines the maximum number of ping packets for adaptive prefix probing.

Possible values: positive Integer value, higher or equal to explorer.probing.sendpkts.min

Default value: 100

Recommended value: 100

4.1.8.18 explorer.probing.simultaneous
Defines the number of scanned IP addresses.

Possible values: 1-100

Default value: 50
Recommended value: 50

4.1.8.19 explorer.scanning.confirm_ips
Defines the number of scanned speaking IP addresses for a provider with loss.

Possible values: 1-10

Default value: 5

Recommended value: 5

4.1.8.20 explorer.scanning.replypkts.min
Defines the number of response packets from a scanned speaking IP address to qualify as a candidate.

Possible values: 1-10

Default value: 5
Recommended value: 5

4.1.8.21 explorer.scanning.rtt.dispersion_ms
Defines RTT maximum dispersion in miliseconds to qualify a speaking IP as candidate.

Possible values: 1-1000

Default value: 50
Recommended value: 50

4.1.8.22 explorer.scanning.sendpkts.factor
Defines the number of attempts to send packets to all selected speaking IPs.

Possible values: 1-10

Default value: 5

Recommended value: 5
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 224

4.1.8.23 explorer.timeout
Defines the probes ICMP timeout (in ms)

Default value: 2000

Recommended value: 2000

4.1.8.24 explorer.timeout.infra
Defines the waiting time (in milliseconds) for infrastructure hops’ (4.1.8.5) responses expected during
trace execution.

Possible values: 50-20000

Default value: 10000
Recommended value: 10000

4.1.8.25 explorer.trace.algorithms
Defines the traceroute algorithm(s) to be used, arranged by priority. See explorer.probe.algorithm for
algorithms description.

Possible values: UDP ICMP TCP_SYN

Default value: UDP ICMP

Recommended value: UDP ICMP

4.1.8.26 explorer.trace.all
Defines tracing behaviour. If traces are forced, each probed prefix unconditionally runs traces through
all configured providers. Regular tracing is needed for Outage Detection and for AS Path reconstruction.
Traces can be disabled altogether by setting explorer.trace.all to 2.

Third algorithm should be enabled in bgpd.as_path when traces are fully disabled.

Forcing ALL traces (explorer.trace.all = 1) can significantly slow down probing for networks.
Review probing times before and after changing this parameter and if probing times become unac-
ceptable revert to previous setting.

When all traces are disablled (explorer.trace.all = 2) IRP is missing essential trace data and is
not able to take action on some types of problems. This effectively cuts off those features.

Possible values: 0 (Regular), 1 (Force all), 2 (Disable all)

Default value: 0
Recommended value: 0
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 225

4.1.8.27 explorer.traceroute.retrypkts
Defines the number of additional traceroute packets to be sent to the intermediate hop in case it has not
replied and might be an infrastructure boundary hop.

Possible values: 0-50

Default value: 10
Recommended value: 10

4.1.8.28 explorer.traceroute.sendpkts
Defines the number of traceroute packets per hop to be sent for each probe.

Possible values: 3-5

Default value: 3
Recommended value: 3

4.1.8.29 explorer.traceroute.simultaneous
Defines the number of simultaneously probed hops during trace execution initiated from the hop defined
by (4.1.8.31).

Possible values: 1-30

Default value: 5
Recommended value: 5

4.1.8.30 explorer.traceroute.simultaneous.infra
Defines the number of simultaneously probed infrastructure hops during trace execution.

Possible values: 1-30

Default value: 3
Recommended value: 3

4.1.8.31 explorer.traceroute.ttl.min
Defines the minimum traceroute TTL. Basically this defines the first hop after which explorer analyzes
the trace.

Possible values: 1-255

Default value: 2
Recommended value: 2-5

4.1.8.32 explorer.traceroute.ttl.max
Defines the maximum traceroute TTL. Essentially this defines the last hop at which explorer stops the
trace.

Possible values: 1-255

Default value: 30
Recommended value: 30
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 226

4.1.9 Notification and events parameters

4.1.9.1 pushd.log
Defines the complete path to the irppushd log file

Possible values: full path to log file

Default value: /var/log/irp/irppushd.log

4.1.9.2 pushd.log.level
Defines the logging level for the irppushd service.

Possible values: emerg, fatal, alert, crit, error, warn, notice, info, debug
Default value: info
Recommended value: info

4.1.9.3 pushd.listen.port
Defines the TCP listen port of irppushd service.

Possible values: 1-65535

Default value: 10499

4.1.9.4 pushd.email.from
Defines the From email address used for sending email notifications.

Possible values: valid email address

Default value: root@localhost

4.1.9.5 pushd.email.host
Defines the IPv4, IPv6 address or email server host name used to send emails.

Possible values: Hostname or IPv4/IPv6 address

Default value: 127.0.0.1

4.1.9.6 pushd.email.port
Defines the TCP port used for sending emails.

Possible values: 1 - 65535

Default value: 25
Recommended value: 25

4.1.9.7 pushd.sms.account_sid
Defines the public account identifier issued to user by the SMS gateway. Usually this is a random string.

Possible values: random string

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 227

4.1.9.8 pushd.sms.auth_token
Defines the secret authentication token issued by the SMS gateway to the account holder. Usually this
is a longer random string.

Possible values: random string

4.1.9.9 pushd.sms.gateway
Defines the SMS gateway preferred by the user. A valid account with this SMS gateway is required to
send SMS.

Possible values: twilio, plivo

Default value: none

4.1.9.10 pushd.sms.message_size
Defines the maximum size of SMS texts. Texts that are longer than this limit will be trimmed and a ...
will be added. The SMS gateway will split the SMS text into multiple messages if the request includes
a longer than supported SMS message.

The SMS gateway enforces its own text size limits. In case the notification exceeds SMS gateway
limits the SMS message can be either trimmed or rejected.

Possible values: 150-6000

Default value: 150

4.1.9.11 pushd.sms.phone_number
Defines the From phone number used for sending SMS notifications.

SMS gateways might have policies regarding the valid phone numbers that they will accept.
Check with your SMS gateways if there are any restrictions and if yes what phone numbers can be
provided in this configuration parameter.

Possible values: valid phone number

4.1.9.12 pushd.sms.uri.twilio
Defines the URL of the Twilio SMS API.

Do not change this parameter unless Twilio SMS API URL has changed.

Possible values: valid URL

Default value: https://api.twilio.com/2010-04-01/Accounts/%1%/Messages.json
Recommended value: https://api.twilio.com/2010-04-01/Accounts/%1%/Messages.json

4.1.9.13 pushd.sms.uri.plivo
Defines the URL of the Plivo SMS API.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 228

Do not change this parameter unless Plivo SMS API URL has changed.

Possible values: valid URL

Default value: https://api.plivo.com/v1/Account/%1%/Message/

Recommended value: https://api.plivo.com/v1/Account/%1%/Message/

4.1.9.14 pushd.templates.datadir
Defines the directory for notification templates used by irppushd service to format email and sms mes-
sages.

Possible values: full path to notification templates

Default value: /etc/noction/templates

4.1.9.15 pushd.webhook.avatar_emoji
Defines an emoji to be used as the IRP bot’s avatar image. The emoji is expected to be in the “:robot-
face:” notation.

The avatar emoji is used if an avatar icon in pushd.webhook.avatar_urlis not specified.

Possible values: valid emoji in :colon: notation

Default value: :robot-face:

4.1.9.16 pushd.webhook.avatar_url
Defines the URL of the IRP bot’s avatar image. URL should be a publicly accessible PNG or JPEG
image that the webhook provider is able to retrieve and use.

This is the default avatar image. The avatar icon will be used instead of an emoji if both are
specified. Refer to pushd.webhook.avatar_emoji.

Possible values: valid URL

Default value: http://www.noction.com/round-logo.png

Recommended value: http://www.noction.com/round-logo.png

4.1.9.17 pushd.webhook.botname
Defines the name assigned to IRP bot.

Possible values: text

Default value: IRP
Recommended value: IRP
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 229

4.1.9.18 pushd.webhook.url
Defines the URL assigned to Web Hooks user/team. Web Hooks work by POST-ing JSON content to
this designated URL for example
POST https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX

This parameter is required in order for any subscription to web hooks channels to work. Currently
only the Slack.com web hooks API has been tested and confirmed to work.

Possible values: valid URL

4.1.9.19 trap.destination.auth_password
SNMP user’s password for IRP generated traps. Applicable only when SNMP v3 with authentication is
used. Refer to trap.destination.version, trap.destination.seclevel, trap.destination.auth_username.

Possible values: text

4.1.9.20 trap.destination.auth_protocol
SNMP authentication protocol for IRP generated traps. Applicable only when SNMP v3 with authen-
tication is used. Refer to trap.destination.version, trap.destination.seclevel.

• 0 - MD5
• 1 - SHA

Possible values: 0, 1

4.1.9.21 trap.destination.auth_username
SNMP user name for IRP generated traps. Applicable only when SNMP v3 with authentication is used.
Refer to trap.destination.version, trap.destination.seclevel.

Possible values: text

4.1.9.22 trap.destination.community
Defines the SNMP community used for sending SNMP traps.

Possible values: textual community name

4.1.9.23 trap.destination.port
Defines the UDP port used for sending SNMP traps.

Possible values: 1 - 65535

Default value: 162
Recommended value: 162

4.1.9.24 trap.destination.priv_password
SNMP password for IRP generated traps. Applicable only when SNMP v3 with privacy is used. Refer
to trap.destination.version, trap.destination.seclevel.

Possible values: text

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 230

4.1.9.25 trap.destination.priv_protocol
SNMP encryption protocol for IRP generated traps. Applicable only when SNMP v3 with privacy is
used. Refer to trap.destination.version, trap.destination.seclevel.

• 0 - DES
• 1 - AES

Possible values: 0, 1

4.1.9.26 trap.destination.seclevel
SNMP security services for IRP generated traps. IRP supports either No security, Authentication only or
both Authentication and Privacy. Applicable only when SNMP v3 is used (trap.destination.version). De-
pending on security services used further parameters should be configured, for example: trap.destination.auth_username,
trap.destination.auth_password, trap.destination.auth_protocol, trap.destination.priv_password, trap.destination.priv_

• 0 - Neither Authentication nor Privacy

• 1 - Authentication only
• 2 - Authentication and Privacy

Possible values: 0, 1, 2
Default value: 0

4.1.9.27 trap.destination.version
SNMP version for IRP generated traps.

• 2 - SNMP v2c
• 3 - SNMP v3

Possible values: 2, 3
Default value: 2

4.1.9.28 trap.bgpd_announced_rate_low.limit_pct
Defines the announcements rate limit percentage.

Possible values: 1 - 100

Default value: 60
Recommended value: 60

4.1.9.29 trap.core_cc_overload.limit_mbps
Defines the overall overload limit in Mbps for the Commit Control overload by X Mbps event. The event
is raised if the overall outbound traffic of the network exceeds the configured value (sum of peer.X.95th
for all providers) by this limit.

Possible values: 1 - 10000

Default value: 100
Recommended value: 100
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 231

4.1.9.30 trap.core_cc_overload.limit_pct
Defines the overall overload limit in percent for the Commit Control overload by X% event. The event
is raised if the overall outbound traffic of the network exceeds the configured value (sum of peer.X.95th
for all providers) by this limit.

Possible values: 1 - 1000

Default value: 10
Recommended value: 10

4.1.9.31 trap.core_cc_provider_overload.limit_mbps
Defines the per provider overload limit in Mbps for the Commit Control provider X overloaded by Y
Mbps event. The event is raised if the outbound traffic for a provider exceeds the configured value (refer
peer.X.95th) by more than this limit.

Possible values: 1 - 10000

Default value: 100
Recommended value: 100

4.1.9.32 trap.core_cc_provider_overload.limit_pct
Defines the per provider overload limit in percent for the Commit Control provider X overloaded by
Y% event. The event is raised if the outbound traffic for a provider exceeds the configured value (refer
peer.X.95th) by more than this limit.

Possible values: 1 - 1000

Default value: 10
Recommended value: 10

4.1.9.33 trap.core_improvement_latency.limit_ms
Defines the packet latency limit in milliseconds.

Possible values: 1 - 10000

Default value: 300
Recommended value: 300

4.1.9.34 trap.core_improvement_loss.limit_pct
Defines the packet loss percentage limit.

Possible values: 1 - 100

Default value: 50
Recommended value: 50

4.1.10 Administrative settings

4.1.10.1 dbcron.api.log
Defines file-system path to the dbcron API log file.

Default value: /var/log/irp/api.mlog

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 232

4.1.10.2 dbcron.api.socket
Defines dbcron API listen socket.
Default value: /var/spool/irp/api

4.1.10.3 dbcron.log
Defines the file-system path to the dbcron log file.
Default value: /var/log/irp/dbcron.log

4.1.10.4 dbcron.log.level
Defines the logging level for the dbcron service.
Possible values: emerg, fatal, alert, crit, error, warn, notice, info, debug
Default value: info
Recommended value: info

4.1.10.5 irpstatd.log
Defines the file-system path to the irpstatd log file.
Default value: /var/log/irp/irpstatd.log

4.1.10.6 irpstatd.log.level
Defines the logging level for the irpstatd service.
Possible values: emerg, fatal, alert, crit, error, warn, notice, info, debug
Default value: info
Recommended value: info

4.1.11 Providers settings

4.1.11.1 peer.X.95th

Mandatory if core.commit_control is enabled.

Defines current provider’s desired 95th value (also known as Committed Interface Rate).
The value of peer.X.limit_load(if set to positive value) must be greater or equal to value of peer.X.95th.
Please refer to core.commit_control and Commit Control for Commit Control description.
Depending on the peer.X.95th.bill_day parameter value, the behavior of the 95th level calculation has
two different states.
If peer.X.95th.bill_day is set to -1, then this parameter is treated as "committed interface rate limit"
and actual 95th calculation is not performed. In this case, IRP will actively reroute traffic from an
overloaded provider to keep it below the peer.X.95th level.
If peer.X.95th.bill_day is set to a positive value, then the system calculates the actual 95th value based
on the historical traffic information from peer.X.95th.bill_day to the current date of this month. Then,
the result is compared to the peer.X.95th value, and the maximum of these two is chosen as the target
95th. Thus, if the 95th for the current month has already exceeded the peer.X.95th value, IRP will keep
the traffic usage from increasing above current 95th.
Possible values: 1-9999999
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 233

4.1.11.2 peer.X.95th.bill_day
Defines the first billing period day. If current month has less days than the specified value, then last
day of the month will be taken as the start of the new billing period. The parameter is used for 95th
percentile calculation.
If -1 is specified, then peer.X.95th is treated as "Committed Interface Rate".

Possible values: -1, 1-31

Default value: 1
Recommended value: First day of the billing period.

4.1.11.3 peer.X.95th.centile
Defines the actual percentage for the 95th percentile, as some providers may have non-standard traffic
billing policies.

Possible values: 1-99

Default value: 95

Recommended value: please consult your agreements with this specific provider

4.1.11.4 peer.X.95th.in
Defines the 95th level (in Mbps) for inbound when inbound and outbound commit control operate
independently. Refer peer.X.95th, peer.X.95th.mode.

Possible values: 1-100000

4.1.11.5 peer.X.95th.mode
Defines the way how 95th value is determined. The following modes are supported:

• 0: Separate 95th for in/out

• 1: 95th of combined in+out
• 2: 95th from greater of in or out

• 3: Largest of the two 95th for in/out

Only mode with separate 95th for each inbound and outbound traffic keeps inbound and outbound
commit control independent of each other.

Refer also peer.X.95th, peer.X.95th.in.

Possible values: 0-3

Default value: 2

4.1.11.6 peer.X.auto_config
Enables or disables periodic execution of Internet Exchanges auto re-configuration process once per
4.1.2.6 interval.
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 234

Explorer will be restarted after the process is completed.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0

4.1.11.7 peer.X.bgp_peer

Mandatory

Defines the iBGP session(s) over which an improvement made for this provider is advertised. This
parameter should be set to a valid BGP neighbor name as defined in BGP sessions settings. Typically,
this is the session with the Edge router, which this provider is connected to. Multiple sessions should be
set only if there are some additional (backup) links to this provider on multiple Edge routers.

Possible values: One or a list of valid BGP neighbor names, as defined in BGP sessions settings.

4.1.11.8 peer.X.cc_disable
Instructs IRP to exclude this provider from the Commit Control algorithm.

Has effect only if core.commit_control is enabled.

Possible values: 0, 1

Default value: 0

4.1.11.9 peer.X.cost
Defines the cost per Mbps for the current provider. All peer.X.cost parameters should be specified in
the same currency.

Parameter’s value should be the same for each provider in a provider’s group (refer to 4.1.11.54)
and cost mode is enabled (4.1.2.21)

Possible values: 1-’unlimited’

Default value: 0

4.1.11.10 peer.X.description
Defines the provider’s description (full provider name)

Possible values: text

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 235

4.1.11.11 peer.X.diag_hop.interval_max
Defines maximum value for Adaptive Inter-Packet Interval. Adaptive Inter-Packet Interval is used while
sending packets to a specific Provider’s router (packets with a specific TTL). Inter-packet interval is
raised up to the maximum value when the router does not respond to the packets and is decreased to
the minimum one in case the router responds. The Adaptive Inter-Packet Interval is changed gradually
to obtain better performance.
A value of the 4.1.8.6 parameter is used in case zero value is specified in this parameter.

Possible values: 0 - 50000000 (50 ms)

Default value: 0

4.1.11.12 peer.X.diag_hop.interval_min
Defines minimum value for Adaptive Inter-Packet Interval. Adaptive Inter-Packet Interval is used while
sending packets to a specific Provider’s router (packets with a specific TTL). Inter-packet interval is
raised up to the maximum value when the router does not respond to the packets and is decreased to
the minimum one in case the router responds. The Adaptive Inter-Packet Interval is changed gradually
to obtain better performance.

Possible values: 10000 (0.01 ms) - 10000000 (10 ms)

Default value: 1000000

4.1.11.13 peer.X.disable_pbr_confirmation
Set to disable periodic transit/partial routing provider’s PBR confirmation check.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0

Recommended value: 0

4.1.11.14 peer.X.flowspec.ipv4.redirect_community
Defines the BGP Community that will be appended by BGPd to advertised Flowspec redirect policies
for IPv4 sessions. The format is: “X:Y”.

Avoid colisions of communities values assigned to IRP both within its configuration and/or on
customer’s network.

4.1.11.15 peer.X.flowspec.ipv6.redirect_community
Defines the BGP Community that will be appended by BGPd to advertised Flowspec redirect policies
for IPv6 sessions. The format is: “X:Y”.

Avoid colisions of communities values assigned to IRP both within its configuration and/or on
customer’s network.

4.1.11.16 peer.X.shortname
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 236

Mandatory

Defines the provider’s short, abbreviated name (3-20 characters). This parameter’s value is used for
reports and graphs.

Possible values: text

4.1.11.17 peer.X.flow_agents
Specifies a collection of matching pairs for the flow agents and the physical interface that carry traffic.
Specified in the form IPv4/interfaceIdentifier. Refer global.rd_rtt, peer.X.rd, bgpd.rd_local_mark.

Possible values: -

4.1.11.18 peer.X.group_loadbalance
Defines whether commit control algorithm load balances the group of providers or optimizes it on ag-
gregated bandwidth usage. For details refer Provider load balancing and Commit control of aggregated
groups.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 1

4.1.11.19 peer.X.improve_in_group
Enables or disables improvements within a provider group. This parameter must be equal for all providers
in a provider group.
If disabled, IRP will not make any Cost or Performance improvements inside the group.

Possible values: 0 (OFF), 1 (ON)

Default value: 0
Recommended value: depends on network infrastructure and policies

4.1.11.20 peer.X.inbound.announce_type
Defines how IRP announces inbound improvements to network.

Possible values: 0 (BGP)

Default value: 0

4.1.11.21 peer.X.inbound.community_base
Defines the base community of consecutive range of eight community values which IRP uses to announce
inbound improvements for this provider over BGP. Base community instructs an announcement of a
route without any additional prepend via a particular provider. Next communities in the range instructs
for additional prepends from one up to seven.

For example: when peer.1.inbound.community_base is set to 65530:50100, the value 65530:50102

represents “prepend 2 times while advertizing to provider 1”.

Possible values: valid BGP community attribute

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 237

4.1.11.22 peer.X.ipv4.diag_hop

Mandatory

Defines the diagnostic hop or subnet in CIDR format for the current provider. Usually, it is the IP
address of the first hop of the specific provider. The parameter is used to make sure that a route actually
passes through this provider.
It is also used for Internet Exchanges autoconfiguration process. The Peering Partners next-hops will be
gathered based on the provided subnet(s) in CIDR format.

Any parameter changes (via Frontend) will caused the BGPd component restart if the provider
type (peer.X.type) is Internet Exchanges

Possible values: valid IPv4 address (usually equal to peer.X.ipv4.next_hop) or subnet definition in
CIDR format

4.1.11.23 peer.X.ipv4.master_probing

Mandatory

Defines the local IPv4 address, assigned for probing via the current provider. When failover is enabled
(global.failover) slave’s probing IP (peer.X.ipv4.slave_probing) must be configured too.
See also: Specific PBR configuration scenarios

Possible values: valid local IPv4 address

4.1.11.24 peer.X.ipv4.mon
Defines the List of IPv4 addresses to be monitored by BGPd to ensure that the immediate upstream
path through this provider is operational. Each specified IP address is probed every bgpd.mon.keepalive
seconds to verify accessibility from BGPd (ICMP/UDP pings are used). Highly available IP addresses
that reliably answer to ICMP/UDP pings should be used. It is remommended to setup at least two IP
addresses belonging to different networks.
If all monitored IP addresses do not respond for bgpd.mon.holdtime seconds, then any improvements
designated for this provider will be withdrawn from edge router(s). Improvements will be re-announced
after all IP addresses respond within bgpd.mon.guardtime seconds.

Possible values: space-separated list of valid IPv4 addresses

Default value: 208.67.222.222 8.8.8.8

4.1.11.25 peer.X.ipv4.next_hop

Mandatory

Defines the next-hop IPv4 address for BGP route injection. Usually, it is the IPv4 address of the BGP
partner from the provider.

Possible values: valid IPv4 address

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 238

4.1.11.26 peer.X.ipv4.next_hop_as
Defines the provider autonomous system number for IPv4. This parameter must be set in order for the
3rd algorithm of BGPd AS-PATH to work properly (refer to bgpd.as_path).
If this parameter is set, then it’s value will be used as part of AS-PATH for outgoing improvements if
the 3rd algorithm is enabled in bgpd.as_path.
In case of Internet Exchanges, the AS-Path begins with peering partner’s AS number, instead of AS
number of route server.
The first AS number will be stripped from AS-Path when advertising improvement towards Exchange in
case the first AS number is equal to value set in this parameter.

Possible values: 0-4294967295

Default value: 0

4.1.11.27 peer.X.ipv4.route_server
Defines the Internet Exchange Route Server(s) IP address(es) used to properly auto-configure provider.X.rule.Y.bgp_peer
parameter.
See also peer.X.mon.ipv4.internal.mode, peer.X.mon.enabled,

Possible values: valid IPv4 address(es)

4.1.11.28 peer.X.ipv4.slave_probing
Defines the local IPv4 address, assigned for probing via the current provider for the slave node in a
failover configuration.
See also: Specific PBR configuration scenarios

4.1.11.29 peer.X.ipv6.diag_hop

Mandatory

Defines the IPv6 diagnostic hop for the current provider. Usually, it is the IP address of the first hop of
the specific provider. The parameter is used to verify that a route actually passes through this provider.

Possible values: valid IPv6 address, usually equals to peer.X.ipv6.next_hop

4.1.11.30 peer.X.ipv6.master_probing

Mandatory for IPv6

Defines the local IPv6 address assigned for probing via the current provider. When failover is enabled
(global.failover) slave’s probing IPv6 address (peer.X.ipv4.slave_probing) must be configured too.
See also: Specific PBR configuration scenarios

Possible values: valid local IPv6 address

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 239

4.1.11.31 peer.X.ipv6.mon
Defines the List of IPv6 addresses to be monitored by BGPd to ensure that the immediate upstream
path through this provider is operational. Each specified IP address is probed every bgpd.mon.keepalive
seconds to verify accessibility from BGPd (ICMP/UDP pings are used). Highly available IP addresses
that reliably answer to ICMP/UDP pings should be used. It is remommended to setup at least two IP
addresses belonging to different networks.
If all IP addresses do not respond for bgpd.mon.holdtime seconds, then any improvements designated
for this provider will be withdrawn from the edge router(s). Improvements will be announced again after
all IP addresses respond within bgpd.mon.guardtime seconds.

Possible values: space-separated list of valid IPv6 addresses

Default value: 2620:0:ccc::2 2001:4860:4860::8888

4.1.11.32 peer.X.ipv6.next_hop

Mandatory for IPv6

Defines the next-hop IPv6 address for BGP route injection. Usually, it is the IPv6 address of the BGP
partner from the provider.

Possible values: valid IPv6 address

4.1.11.33 peer.X.ipv6.next_hop_as
Defines the provider autonomous system number for IPv6. This parameter must be set in order for the
3rd algorithm of BGPd AS-PATH to work properly (refer to bgpd.as_path).
If this parameter is set, then it’s value will be used as part of an AS-PATH for outgoing improvements
if the 3rd algorithm is enabled in bgpd.as_path.

Possible values: 0-4294967295

Default value: 0

4.1.11.34 peer.X.ipv6.route_server
Defines the Internet Exchange Route Server(s) IP address(es) used to properly auto-configure provider.X.rule.Y.bgp_peer
parameter.
See also peer.X.mon.ipv6.internal.mode, peer.X.mon.enabled,

Possible values: valid IPv6 address(es)

4.1.11.35 peer.X.ipv6.slave_probing
Defines the local IPv6 address assigned for probing via the current provider for the slave node in a failover
configuration.
See also peer.X.ipv6.master_probing.

Possible values: valid local IPv6 address

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 240

4.1.11.36 peer.X.limit_load
Defines the load limit for current provider. IRP will improve routes to this provider as long as its load
is less than the specified value in megabits per second. SNMP must be properly configured in order for
this feature to work properly. If this limit is configured, but it is impossible to retrieve interface data,
no new improvements will take place.
The value of peer.X.limit_load(if set to positive value) must be greater or equal to value of peer.X.95th.
Possible values: -1(unlimited), 1-9999999
Default value: -1
Recommended value: it is recommended to set it to not more than ~60-80% of the physical interface
rate
See also:
peer.X.snmp.ip
peer.X.snmp.ipv6
peer.X.snmp.interface
peer.X.snmp.community

4.1.11.37 peer.X.mon.enabled
Defines the BGP Monitoring behavior for this provider. Several values are possible:
• 0 - BGP Monitoring is disabled for this provider
• 1 - ICMP BGP monitoring is enabled
• 2 - SNMP BGP monitoring is enabled
• 3 - Both SNMP and ICMP BGP monitoring is enabled
See also: BGP Monitoring
Possible values: 0, 1, 2, 3
Default value: 0
Recommended value: 3

4.1.11.38 peer.X.mon.ipv4.bgp_peer
Defines the current provider’s IPv4 address that must be monitored by BGPd, usually equal to
peer.X.ipv4.next_hop
Possible values: valid IPv4 address
See also: peer.X.mon.snmp.ip, peer.X.mon.snmp.community

4.1.11.39 peer.X.mon.ipv4.internal.mode
Specifies router supported BGP MIB to monitor this provider IPv4 BGP sessions.
• 0 - Generic (BGP4-MIB)
• 1 - Cisco (CISCO-BGP4-MIB)
• 2 - Juniper (BGP4-V2-MIB-JUNIPER)
• 3 - Brocade (draft-ietf-idr-bgp4-mibv2-12)

Possible values: 0, 1, 2, 3
Default value: 0
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 241

4.1.11.40 peer.X.mon.ipv6.bgp_peer
Defines the current provider’s IPv6 address that must be monitored by BGPd, usually equal to
peer.X.ipv6.next_hop

Possible values: valid IPv6 address

See also: peer.X.mon.snmp.ipv6, peer.X.mon.snmp.community

4.1.11.41 peer.X.mon.ipv6.external.enabled
Enables or disables external monitors for IPv6.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0

4.1.11.42 peer.X.mon.ipv6.internal.enabled
Enables or disables internal monitors for IPv6.

Possible values: 0 (Disabled), 1 (Enabled)

Default value: 0

4.1.11.43 peer.X.mon.ipv6.internal.mode
Specifies router supported BGP MIB to monitor this provider IPv6 BGP sessions.

• 1 - Cisco (CISCO-BGP4-MIB)
• 2 - Juniper (BGP4-V2-MIB-JUNIPER)
• 3 - Brocade (draft-ietf-idr-bgp4-mibv2-12)

Possible values: 1, 2, 3

4.1.11.44 peer.X.mon.snmp.auth_password
SNMP user’s password for monitoring provider. Applicable only when SNMP v3 with authentication is
used. Refer to peer.X.mon.snmp.version, peer.X.mon.snmp.seclevel, peer.X.mon.snmp.auth_username.

Possible values: text

4.1.11.45 peer.X.mon.snmp.auth_protocol
SNMP authentication protocol for monitoring provider. Applicable only when SNMP v3 with authenti-
cation is used. Refer to peer.X.mon.snmp.version, peer.X.mon.snmp.seclevel.

• 0 - MD5

• 1 - SHA

Possible values: 0, 1
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 242

4.1.11.46 peer.X.mon.snmp.auth_username
SNMP user name for monitoring provider. Applicable only when SNMP v3 with authentication is used.
Refer to peer.X.mon.snmp.version, peer.X.mon.snmp.seclevel.
Possible values: text

4.1.11.47 peer.X.mon.snmp.community
Defines the SNMP community for retrieving monitoring statistics.
Possible values: textual community name

4.1.11.48 peer.X.mon.snmp.ip
Defines the IPv4 address for retrieving SNMP monitoring statistics (use either peer.X.mon.snmp.ip or
peer.X.mon.snmp.ipv6)
Possible values: IPv4 address

4.1.11.49 peer.X.mon.snmp.ipv6
Defines the IPv6 address for retrieving SNMP monitoring statistics (use either peer.X.mon.snmp.ip or
peer.X.mon.snmp.ipv6)
Possible values: IPv6 address

4.1.11.50 peer.X.mon.snmp.priv_password
SNMP password for monitoring provider. Applicable only when SNMP v3 with privacy is used. Refer
to peer.X.mon.snmp.version, peer.X.mon.snmp.seclevel.
Possible values: text

4.1.11.51 peer.X.mon.snmp.priv_protocol
SNMP encryption protocol for monitoring provider. Applicable only when SNMP v3 with privacy is
used. Refer to peer.X.mon.snmp.version, peer.X.mon.snmp.seclevel.
• 0 - DES
• 1 - AES

Possible values: 0, 1

4.1.11.52 peer.X.mon.snmp.seclevel
SNMP security services for monitoring provider. IRP supports either No security, Authentication only
or both Authentication and Privacy. Applicable only when SNMP v3 is used (peer.X.mon.snmp.version).
Depending on security services used further parameters should be configured, for example: peer.X.mon.snmp.auth_usernam
peer.X.mon.snmp.auth_password, peer.X.mon.snmp.auth_protocol, peer.X.mon.snmp.priv_password,
peer.X.mon.snmp.priv_protocol.
• 0 - Neither Authentication nor Privacy
• 1 - Authentication only
• 2 - Authentication and Privacy

Possible values: 0, 1, 2
Default value: 0
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 243

4.1.11.53 peer.X.mon.snmp.version
SNMP version for monitoring provider.

• 2 - SNMP v2c
• 3 - SNMP v3

Possible values: 2, 3
Default value: 2

4.1.11.54 peer.X.precedence

Mandatory if core.commit_control is enabled.

Defines this provider’s precedence, used for provider grouping and defining commit priorities.
See core.commit_control and Commit Control for Commit Control functionality.

All providers belonging to one group must have equal cost (peer.X.cost)

If two different providers have the same peer.X.precedence value, then these providers are considered to
be a group, and traffic is balanced between them.
The provider or a group with the lowest precedence value are also used by Commit Control as last resort
destination (if all the providers are exceeding their 95th, then traffic is rerouted to the upstream with
the smallest precedence - usually this upstream has either the highest throughput, or the lowest cost).
For example:
peer.1.precedence = 20
peer.2.precedence = 10
peer.3.precedence = 30

If the peer.X.95th is exceeded for all configured providers, then the excessive traffic from providers 1 and
3 will be rerouted to the 2nd provider.
If provider groups are used:
peer.1.precedence = 50
peer.2.precedence = 40
peer.3.precedence = 40
peer.4.precedence = 40
peer.5.precedence = 30

Traffic between providers 2, 3 and 4 is evenly distributed. If all providers are already overloaded, then
the excessive traffic will be rerouted to the 5th provider (since it has the lowest precedence value).

Possible values: 0-100

4.1.11.55 peer.X.rd
Assigns a routing domain identifier to the provider. Providers in the same routing domains should be
assigned the same identifier.
Providers with identifier 1 (one) are assumed to be in the same routing domain that hosts IRP instance.
Refer global.rd_rtt, peer.X.flow_agents, bgpd.rd_local_mark.

Possible values: 1-100

Default value: 1
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 244

4.1.11.56 peer.X.routes_config
Defines whether Internet Exchanges auto configuration is enabled or not. In case it is enabled, IRP
BGPd gathers Peering Partners (next-hops) with their announced routes and stores the data in IRP
database. Otherwise, manual configuration is required.

Possible values: 0(OFF), 1(ON)

Default value: 1
Recommended value: 1

4.1.11.57 peer.X.shutdown
Defines whether provider is Active (0), Suspended (1) or Shutdown (2).
After the provider suspend action, all the improvements will be stored for short period of time (1h). In
case of short maintenance windows, use provider suspend.
After the provider shutdown action, all the improvements will be removed. In case of long maintenance
windows, use provider shutdown.
After the provider reactivation (after a previous suspend), all the improvements will be sent to retry
probing.
The provider state can be updated in Frontend or using the irpPeerSuspendShutdown.pl script.
In order to get the provider suspended, shutdown or reactivated, use the irpPeerSuspendShutdown.pl
helper script.
Usage: /usr/bin/irpPeerSuspendShutdown.pl [-a] [-p] [-h]

-a|--action - Perform an action under a peer.

Possible values:
0 or ’activate’
1 or ’suspend’
2 or ’shutdown’
-p|--peerId - An ID of the peer that should be Activated, Shutdown or
Suspended
-h|--help - Print this help

Possible values: 0, 1, 2
Default value: 0

4.1.11.58 peer.X.snmp.auth_password
SNMP user’s password for collecting statistics for provider. Applicable only when SNMP v3 with authen-
tication is used. Refer to peer.X.mon.snmp.version, peer.X.mon.snmp.seclevel, peer.X.snmp.auth_username.

Possible values: text

4.1.11.59 peer.X.snmp.auth_protocol
SNMP authentication protocol for collecting statistics for provider. Applicable only when SNMP v3 with
authentication is used. Refer to peer.X.mon.snmp.version, peer.X.mon.snmp.seclevel.

• 0 - MD5
• 1 - SHA

Possible values: 0, 1
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 245

4.1.11.60 peer.X.snmp.auth_username
SNMP user name for collecting statistics for provider. Applicable only when SNMP v3 with authentica-
tion is used. Refer to peer.X.mon.snmp.version, peer.X.mon.snmp.seclevel.

Possible values: text

4.1.11.61 peer.X.snmp.community

Mandatory parameter

Defines the SNMP community for retrieving interface counters.

Possible values: textual community name

4.1.11.62 peer.X.snmp.interface

Mandatory parameter

Defines the interface used for the interconnection to the current provider. It can be either an integer
value specifying the interface ifIndex, or a textual interface description.

Possible values: Integer value, or textual interface description

Recommended value: textual interface description

4.1.11.63 peer.X.snmp.ip
Defines the IPv4 address that should be polled for SNMP interface counters.

peer.X.snmp.ip and peer.X.snmp.ipv6 are mutually exclusive.

Possible values: valid IPv4 address

4.1.11.64 peer.X.snmp.ipv6
Defines the IPv6 address that should be polled for SNMP interface counters.

peer.X.snmp.ip and peer.X.snmp.ipv6 are mutually exclusive.

Possible values: valid IPv6 address

4.1.11.65 peer.X.snmp.priv_password
SNMP password for collecting statistics for provider. Applicable only when SNMP v3 with privacy is
used. Refer to peer.X.snmp.version, peer.X.snmp.seclevel.

Possible values: text

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 246

4.1.11.66 peer.X.snmp.priv_protocol
SNMP encryption protocol for collecting statistics for provider. Applicable only when SNMP v3 with
privacy is used. Refer to peer.X.snmp.version, peer.X.snmp.seclevel.

• 0 - DES
• 1 - AES

Possible values: 0, 1

4.1.11.67 peer.X.snmp.seclevel
SNMP security services for collecting statistics for provider. IRP supports either No security, Authentica-
tion only or both Authentication and Privacy. Applicable only when SNMP v3 is used (peer.X.snmp.version).
Depending on security services used further parameters should be configured, for example: peer.X.snmp.auth_username,
peer.X.snmp.auth_password, peer.X.snmp.auth_protocol, peer.X.snmp.priv_password, peer.X.snmp.priv_protocol.

• 0 - None - neither Authentication nor Privacy is used

• 1 - Authentication only
• 2 - Authentication and Privacy

Possible values: 0, 1, 2
Default value: 0

4.1.11.68 peer.X.snmp.version
SNMP version for collecting statistics for provider.

• 2 - SNMP v2c
• 3 - SNMP v3

Possible values: 2, 3
Default value: 2

4.1.11.69 peer.X.type
Defines the type of a provider. The following types are available:

• 0 - Transit provider
• 1 - Partial routes provider
• 2 - Exchanges provider

Possible values: 0, 1, 2
Default value: 0

4.1.12 Inbound parameters

4.1.12.1 inbound.rule.X.bgp_peer
Defines the router(s) where IRP announces improvements of this inbound prefix.

Possible values: a router identifier(s) separated by space

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 247

4.1.12.2 inbound.rule.X.enabled
Enables or disables Inbound Optimization for this prefix.

Possible values: 0 (Disabled), 1(Enabled)

Default value: 1

4.1.12.3 inbound.rule.X.next_hop
Defines a next_hop specific for the inbound rule. When the next_hop is unset, then a value from
bgpd.peer.X.inbound.ipv4.next_hop/bgpd.peer.X.inbound.ipv6.next_hop is taken.
Next-hop should point to a router that routes/terminates traffic. Otherwise null route should be config-
ured for the Next-Hop.

next_hop value should be of the same IP address family as the inbound prefix it is assigned to.
Refer inbound.rule.X.prefix.

Possible values: IP address

4.1.12.4 inbound.rule.X.prefix
Defines an inbound prefix. Inbound prefixes should belong to your network.

Inbound Optimization of transit networks will be supported in future releases.

Each inbound prefix logically extends the list of ournets when it is processed by IRP Collector.
Refer collector.ournets.

Possible values: prefix in CIDR format

4.1.13 Routing Policies settings

4.1.13.1 asn/prefix
Defines the ASN or the prefix to which the routing policy is applied.

Possible values: 1-65535 for an ASN or a valid IPv4/IPv6 prefix.

4.1.13.2 cascade
Defines the ASN policies that cascade to downstream AS from designated AS. The parameter applies to
ASN policies (and not to prefixes).

Possible values: 0(Disable), 1(Enable)

Default value: 0

4.1.13.3 enabled
Defines whether the policy is enabled or not

Possible values: 0(OFF), 1(ON)

Default value: 1
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 248

4.1.13.4 notes
Defines a description of the routing policy

Possible values: text

Routing Policy working example:

asn=1122
vip=0
policy=deny
providers=1 3 5
enabled=1
notes=AN1122 deny via Level3, Cogent and nLayer

4.1.13.5 policy
Defines the type of policy

Possible values: allow, deny, static

Default value: allow

4.1.13.6 providers
Defines the list of providers (IDs) affected by the policy. If the ’providers’ property is not specified, then
all the providers will be included in the policy by default.

Possible values: 1-unlimited

4.1.13.7 vip
Defines whether the VIP probing is enabled for the specified ASN/prefix

Possible values: 0(OFF), 1(ON)

Default value: 0

4.1.14 Exchanges settings

4.1.14.1 provider.X.rule.Y.bgp_peer
Defines the BGP Router-ID used for BGP session state monitoring, which is performed by the Internal
BGP Monitor (BGP Monitoring).
Depending on whether a Route Server or Peering sessions are used, the parameter value should be set
to Route Server Router-ID or Peering Partner Router-ID accordingly.

Possible values: IPv4 address

4.1.14.2 provider.X.rule.Y.enabled
Defines whether the rule is enabled or not.

Possible values: 0(OFF), 1(ON)

Default value: 1
CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 249

4.1.14.3 provider.X.rule.Y.next_hop
Defines the Peering Partner’s IPv4/IPv6 next-hop address.

Possible values: IPv4/IPv6 address

4.1.14.4 provider.X.rule.Y.probing_dscp
Defines the DSCP (Differentiated services code point) used with provider.X.rule.Y.probing_ip for a
specific Peering Partner.

Possible values: 0-63

4.1.14.5 provider.X.rule.Y.probing_ip
Defines the probing IPv4/IPv6 address used for a specific Peering Partner.

Possible values: IPv4/IPv6 address

4.1.14.6 provider.X.rule.Y.shortname
Defines a short name for a configured Peering Partner.

Possible values: text

4.1.14.7 provider.X.rule.Y.pbr_check
Defines if PBR check for a configured Peering Partner is enabled or not.

Possible values: 0(OFF), 1(ON)

Default value: 1

4.1.15 User Directory settings

4.1.15.1 directory.X.admin_role_groups
Defines the specific objects/groups that list IRP users with Admin privileges. Also refer directory.X.user_role_attr.

Possible values: text

4.1.15.2 directory.X.base_dn
Defines the base DN (distinguished name) of users in the directory. Base DN is used in conjunction
with directory.X.username and directory.X.user_dn to uniquily identify and query the directory during
authentication.

Possible values: text

4.1.15.3 directory.X.email
Defines the directory attribute that stores user’s email address.

Possible values: text

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 250

4.1.15.4 directory.X.fullname
Defines the directory attribute that stores user’s full name.

Possible values: text

4.1.15.5 directory.X.hostname
Defines the hostname or the IP address of the User Directory.

Possible values: domain name or IP address

4.1.15.6 directory.X.initial_bind_dn
Defines User DN assigned to IRP to bind to directory. This user’s password is configured in direc-
tory.X.initial_bind_password.

Possible values: text

4.1.15.7 directory.X.initial_bind_password
Defines the password assigned to IRP to bind to directory. Refer directory.X.initial_bind_dn.

Possible values: text

4.1.15.8 directory.X.name
Assigns a name to user directory within IRP.

Possible values: text

4.1.15.9 directory.X.order
Defines the sequence this user directory is queried when multiple user directories are configured. Smaller
values take precedence.

The value must be unique across configured user directories.

Possible values: 0-255

Default value: 255

4.1.15.10 directory.X.port
Defines the port used by the directory, for example 389 for LDAP.

Possible values: 0-65535

4.1.15.11 directory.X.secret
Defines the shared secret for TACACS+ host and IRP to secure communications.

Possible values: text

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 251

4.1.15.12 directory.X.state
Enables or disables use of this directory within IRP.

Possible values: 0(Disabled), 1(Enabled)

Default value: 1

4.1.15.13 directory.X.timeout
Defines the timeout (in seconds) after which IRP stops trying to communicate with the directory.

Possible values: 0-300

Default value: 60

4.1.15.14 directory.X.tls
Enables or disables use of TLS for this directory.

Possible values: 0(Disabled), 1(Enabled)

Default value: 0

4.1.15.15 directory.X.tls_cacertfile
Defines the path to a CA certificate file used when server certificate validation is enabled and CA
certificate cannot be included in trusted server certificates store, for example a single purpose self-signed
CA certificate is used. Refer directory.X.verify_cert.

Possible values: text (path)

4.1.15.16 directory.X.type
Defines user directory type for example LDAP, Active Directory, Internal etc.

Possible values: internal, ldap_posix, active_directory...

Default value: internal

4.1.15.17 directory.X.user_dn
Defines the DN of users. User DN is used in conjunction with directory.X.username and directory.X.base_dn
to uniquely identify and query the directory during authentication.

Possible values: text

4.1.15.18 directory.X.user_role_attr
Defines the attribute that stores the list of users in a role/group. Refer directory.X.admin_role_groups.

Possible values: text

4.1.15.19 directory.X.username
Defines the directory attribute that represents the username of users in the directory. This attribute
together with directory.X.user_dn and directory.X.base_dn are used to uniquely identify and query the
directory during authentication.

Possible values: text

CHAPTER 4. CONFIGURATION PARAMETERS REFERENCE 252

4.1.15.20 directory.X.verify_cert
Enables or disables verification of directory server certificate when TLS is enabled. When verification is
enabled a path to CA certificate file must be provided too. Refer directory.X.tls, directory.X.tls_cacertfile.

Possible values: 0(Disabled), 1(Enabled)

Default value: 0
Chapter 5

Appendixes

5.1 Frontend labels for configuration parameters

Account ID pushd.sms.account_sid
Adaptive packets count explorer.probing.sendpkts.adaptive_max
Admin role groups directory.X.admin_role_groups
Allowed IP addresses global.frontend_acl_ips
Allowed latency worsening (ms) core.cost.worst_ms
Analyzed prefixes collector.ournets
Announced communities bgpd.peer.X.master_communities, bgpd.peer.X.slave_communities
Announced localpref value bgpd.peer.X.master_localpref, bgpd.peer.X.slave_localpref
Announced MED value bgpd.peer.X.med
AS_PATH capability bgpd.peer.X.cap_4byte_as
AS-PATH restore priority bgpd.as_path
Autonomous System bgpd.peer.X.as
Avatar emoji pushd.webhook.avatar_emoji
Avatar icon URL pushd.webhook.avatar_url
Base DN directory.X.base_dn
BGP Internal Monitor SNMP Community peer.X.mon.snmp.community
BGP Internal Monitor SNMP IPv4 address peer.X.mon.snmp.ip
BGP Internal Monitor SNMP IPv6 address peer.X.mon.snmp.ipv6
BGP mode global.nonintrusive_bgp
BGP session monitoring IPv4 address peer.X.mon.ipv4.bgp_peer
BGP session password bgpd.peer.X.master_password, bgpd.peer.X.slave_password
BGPd monitoring guard time (sec) bgpd.mon.guardtime
BGPd monitoring holdtime (sec) bgpd.mon.holdtime
BGPd monitoring keepalive (sec) bgpd.mon.keepalive
BGPd monitoring long holdtime (sec) bgpd.mon.longholdtime
Bot name pushd.webhook.botname
CC probing queue slots core.commit_control.probing_queue_size
CC probing TTL (sec) core.commit_control.probe_ttl
CC provider precedence peer.X.precedence
Cascading policy max AS bgpd.policy.cascade.amount
Centile value peer.X.95th.centile
Certificate verification directory.X.verify_cert
Commit Control core.commit_control

253
CHAPTER 5. APPENDIXES 254

Commit Control for provider peer.X.cc_disable

Common name attribute directory.X.fullname
Community mark for local improvements bgpd.rd_local_mark
Delete irrelevant CC improvements core.commit_control.del_irrelevant
Email attribute directory.X.email
Email server pushd.email.host
Explorer new algorithmsexplorer.algorithms
Explorer worker threads explorer.maxthreads
Exploring queue slots core.eventqueuelimit
Failed probe lifetime (sec) core.probes.ttl.failed
Failback timer (s) global.failover_timer_failback
Failover global.failover
Failover timer (s) global.failover_timer_fail
Flow agents with outgoing interfaces peer.X.flow_agents
Flow Collector collector.flow.enabled
Flow sources collector.flow.sources
From email pushd.email.from
From phone number pushd.sms.phone_number
Frontend access restriction global.frontend_acl
Global commit improvements core.global.allow_commit
Global latency/cost improvements core.global.allow_latency_cost
High volume precedence explorer.high_vol_precedence
Holdtime of BGP updates (sec) bgpd.db.timeout.withdraw
Hostname directory.X.hostname
ICMP & SNMP monitoring status peer.X.mon.enabled
ICMP timeout (ms) explorer.timeout
ICMP/UDP ping monitored IPv4 addresses peer.X.ipv4.mon
ICMP/UDP ping monitored IPv6 addresses peer.X.ipv6.mon
Ignored ASNs global.ignored.asn
Ignored prefixes global.ignorednets
Improvements confirmation core.improvements.defer
Improvement confirmation delay core.improvements.defer.hold
Improvement mode global.improve_mode
Indirect probing precedence explorer.probe.indirect_priority
Infrastructure IPs explorer.infra_ips
Initial bind DN directory.X.initial_bind_dn
Initial bind Password directory.X.initial_bind_password
IX auto re-configuration peer.X.auto_config
IX auto re-configuration interval (s) global.exchanges.auto_config_interval
IPv4 aggregate size global.agg_ipv4_max
IPv4 diagnostic hop peer.X.ipv4.diag_hop
IPv4 Redirect community peer.X.flowspec.ipv4.redirect_community
IPv6 aggregate size global.agg_ipv6_max
IPv6 diagnostic hop peer.X.ipv6.diag_hop
IPv6 External monitor peer.X.mon.ipv6.external.enabled
IPv6 Internal monitor peer.X.mon.ipv6.internal.enabled
IPv6 Redirect community peer.X.flowspec.ipv6.redirect_community
CHAPTER 5. APPENDIXES 255

IPv6 support global.ipv6_enabled

Irpspand interfaces collector.span.interfaces
Keepalive interval (sec) bgpd.peer.X.keepalive
Load balancing peer.X.group_loadbalance
Local IPv4 address bgpd.peer.X.master_our_ip, bgpd.peer.X.slave_our_ip
Local router IPv6 address bgpd.peer.X.master_our_ipv6
Management interface global.master_management_interface
Master routing domain global.master_rd
Max IPv4 Flowspec improves core.flowspec.max
Max collected IPs explorer.max_collector_ips
Max IPv6 Flowspec improves core.flowspec.max_ipv6
Max IPv4 Improvements core.improvements.max
Max IPv6 Improvements core.improvements.max_ipv6
Max message size pushd.sms.message_size
Maximum load per interface peer.X.limit_load
Maximum probe lifetime (sec) core.probes.ttl.max
Min loss detection packets explorer.probing.sendpkts.min
min_delay status collector.span.min_delay
Mindelay probing queue slots collector.span.min_delay.probing_queue_size
Minimal prefix bandwidth (Mbps) core.commit_control.agg_bw_min
Minimal probe lifetime (sec) core.probes.ttl.min
Minimal traffic volume (%) collector.export.volume.min_pct
Minimal traffic volume (bytes) collector.export.volume.min
NetFlow UDP port collector.flow.listen.nf
Next hop inbound.rule.X.next_hop.
Order directory.X.order
Outage confirmation timeout (sec) core.problem.outage_timeout
Outage detection core.outage_detection
Outage prefix confirmation rate core.outage_detection.limit_pct
Outage round trip rate (%) core.problem.rtt.diff_pct
Overload by (%) trap.core_cc_overload.limit_pct
Overload by (Mbps) trap.core_cc_overload.limit_mbps
Overusage interval (sec)core.overusage.check_interval
Overusage policies global.bw_overusage
Overusage rule retention (sec) core.overusage.hold_timer
Overusage threshold multiplier core.overusage.out.threshold.trigger
Overusage throttle multiplier core.overusage.out.threshold.throttle
Peering Partner enable trace provider.X.rule.Y.pbr_check
Peering Partner name provider.X.rule.Y.shortname
Peering Partner next-hop provider.X.rule.Y.next_hop
Peering Partner status provider.X.rule.Y.enabled
Performance/Cost improvements within peer group peer.X.improve_in_group
Port directory.X.port
Prefix aggregation global.aggregate
Prefix announcement rate (%) trap.bgpd_announced_rate_low.limit_pct
Prefix BW average time (hours)core.overusage.out.average.period
Prefix latency limit (ms) trap.core_improvement_latency.limit_ms
CHAPTER 5. APPENDIXES 256

Prefix loss limit (%) trap.core_improvement_loss.limit_pct

Prefix relevant BW (Mbps)core.overusage.out.average.relevant_min
Probing algorithm explorer.probe.algorithm
Probing DSCP provider.X.rule.Y.probing_dscp
Probing interface(s) global.master_probing_interface
Probing IP address provider.X.rule.Y.probing_ip
Probing IPv4 address peer.X.ipv4.master_probing, peer.X.ipv4.slave_probing
Provider 95th percentile peer.X.95th, peer.X.95th.in
Provider billing day peer.X.95th.bill_day
Provider cost per Mbps (USD) peer.X.cost
Provider description peer.X.description
Provider overload by (%) trap.core_cc_provider_overload.limit_pct
Provider overload by (Mbps) trap.core_cc_provider_overload.limit_mbps
Provider probing IPv6 address peer.X.ipv6.master_probing
Provider short name peer.X.shortname
Provider Shutdown peer.X.shutdown
Provider SNMP community peer.X.snmp.community
Provider SNMP interface peer.X.snmp.interface
Provider SNMP IPv4 address peer.X.snmp.ip
Provider SNMP IPv6 address peer.X.snmp.ipv6
Provider type peer.X.type
Provider’s bandwidth max deviation (%) core.commit_control.rate.group
Provider’s high load bandwidth limit (%) core.commit_control.rate.high
Provider’s low load bandwidth limit (%) core.commit_control.rate.low
Provider’s routing domain peer.X.rd
Public IPv4 address for PBR tests explorer.ipv4_test
Public IPv6 address for PBR explorer.ipv6_test
Relevant loss (%) core.performance.loss_pct
Relevant RTT difference (%) core.performance.rtt.diff_pct
Relevant RTT difference (ms) core.performance.rtt.diff_ms
Relevant RTT: IX vs transit (ms) core.performance.rtt.ix_diff_ms
Relevant RTT: IX vs transit (%) core.performance.rtt.ix_diff_pct
Remote provider ASN peer.X.ipv4.next_hop_as
Remove next hop update bgpd.improvements.remove.next_hop_eq
Remove on aggregate withdrawal bgpd.improvements.remove.withdrawn
Role global.failover_role
Route Server or Peering session BGP Router-ID provider.X.rule.Y.bgp_peer
Router peer.X.bgp_peer
Router ASN peer.X.ipv6.next_hop_as
Router ID bgpd.peer.X.master_router_idbgpd.peer.X.slave_router_id bgpd.peer.X.slave_router_id
Router IPv4 address bgpd.peer.X.peer_ip
Router IPv6 address bgpd.peer.X.peer_ipv6
Router next-hop address peer.X.ipv4.next_hop
Router next-hop IPv6 address peer.X.ipv6.next_hop
Routes configuration mode peer.X.routes_config
RTT between RDs global.rd_rtt
Scanning attemptsexplorer.scanning.sendpkts.factor
CHAPTER 5. APPENDIXES 257

Secret pushd.sms.auth_token
Server port pushd.email.port
sFlow UDP port collector.flow.listen.sf
Shutdown bgpd.peer.X.shutdown
Slave IPv4 address global.failover_slave.ip
Slave routing domain global.slave_rd
Slave SSH port global.failover_slave.port
SMS gateway pushd.sms.gateway
SNMP authentication password (monitor) peer.X.mon.snmp.auth_password
SNMP authentication password (stats) peer.X.snmp.auth_password
SNMP authentication password (traps) trap.destination.auth_password
SNMP authentication (monitor) peer.X.mon.snmp.auth_protocol
SNMP authentication (stats) peer.X.snmp.auth_protocol
SNMP authentication (traps) trap.destination.auth_protocol
SNMP encryption password (monitor) peer.X.mon.snmp.priv_password
SNMP encryption password (stats) peer.X.snmp.priv_password
SNMP encryption password (traps) trap.destination.priv_password
SNMP encryption (monitor) peer.X.mon.snmp.priv_protocol
SNMP encryption (stats) peer.X.snmp.priv_protocol
SNMP encryption (traps) trap.destination.priv_protocol
SNMP security (monitor) peer.X.mon.snmp.seclevel
SNMP security (stats) peer.X.snmp.seclevel
SNMP security (traps) trap.destination.seclevel
SNMP Username (monitor) peer.X.mon.snmp.auth_username
SNMP Username (stats) peer.X.snmp.auth_username
SNMP Username (traps) trap.destination.auth_username
SNMP version (monitor) peer.X.mon.snmp.version
SNMP version (stats) peer.X.snmp.version
SNMP version (traps) trap.destination.version
SNMP Traps community trap.destination.community
SNMP Traps destination port trap.destination.port
SPAN Collector collector.span.enabled
Speaking IP responses for candidates explorer.scanning.replypkts.min
Speaking IPs RTT dispersion (ms) explorer.scanning.rtt.dispersion_ms
Speaking IPs to scan on loss explorer.scanning.confirm_ips
Split announcements to preserve original route attributes bgpd.updates.split
Standard reprobing period (sec) core.improvements.ttl.retry_probe State directory.X.state
Timeout directory.X.timeout
TLS directory.X.tls
TLS CA Certificate file directory.X.tls_cacertfile
Top volume prefixes per cycle collector.export.volume.high.top_n
Top-N relevant volume prefixes core.improvements.retry_probe.volume_top_n
Trace all providers explorer.trace.all
Traceroute algorithm order explorer.trace.algorithms
Traceroute max hops explorer.traceroute.ttl.max
Traceroute min hops explorer.traceroute.ttl.min
Traceroute packets per hop explorer.traceroute.sendpkts
GLOBAL SETTINGS 258

Traceroute retry packets explorer.traceroute.retrypkts

Traps SNMP community trap.destination.community
Type directory.X.type
User directory name directory.X.name
User DN directory.X.user_dn
User role attribute directory.X.user_role_attr
Username attribute directory.X.username
VIP reprobing period (sec) core.vip.interval.probe
Webhook URL pushd.webhook.url

5.2 Configuration parameter index

Global settings
bgpd.peer.X.flowspec, 124, 196

global.agg_ipv4_max, 133, 183, 195, 254

global.agg_ipv6_max, 133, 183, 195, 254
global.aggregate, 48, 133, 183, 195, 255
global.bw_overusage, 183, 255
global.exchanges, 183
global.exchanges.auto_config_interval, 145, 183, 254
global.failover, 134, 184, 185, 187, 188, 190, 197, 198, 237, 238, 254
global.failover.log, 184
global.failover_identity_file, 74, 184
global.failover_role, 184, 256
global.failover_slave.ip, 134, 184, 257
global.failover_slave.ipv6, 184
global.failover_slave.port, 134, 180, 184, 257
global.failover_timer_fail, 35, 134, 180, 185, 254
global.failover_timer_failback, 35, 134, 180, 185, 254
global.flowspec, 124, 185
global.frontend_acl, 185, 254
global.frontend_acl_ips, 185, 253
global.ignored.asn, 133, 185, 254
global.ignorednets, 73, 133, 186, 254
global.improve_mode, 47, 133, 171, 186, 212, 254
global.inbound_conf, 187
global.ipv6_enabled, 187, 255
global.master_management_interface, 133, 171, 187, 255
global.master_probing_interface, 124, 133, 172, 181, 188, 256
global.master_rd, 34, 189, 255
global.nonintrusive_bgp, 47, 133, 149, 187, 253
global.outbound, 188
global.png.datadir, 188
global.policies, 188
global.rd_rtt, 26, 28, 189, 194, 236, 243, 256
global.rrd.age_max, 189
global.rrd.datadir, 190
global.slave_management_interface, 187
global.slave_probing_interface, 181, 188
global.slave_rd, 34, 189, 257
global.user_directories_conf, 190
COMMON DATABASE CREDENTIALS 259

Common database credentials

db.dbname, 182
db.host, 182
db.ourhost, 182
db.password, 182
db.port, 182
db.username, 182

BGPd settings
bgpd.as_path, 73, 142, 149, 192, 224, 238, 239, 253
bgpd.db.timeout.withdraw, 79, 83, 192, 254
bgpd.improvements.remove.hold_time, 193
bgpd.improvements.remove.next_hop_eq, 149, 193, 256
bgpd.improvements.remove.withdrawn, 149, 193, 256
bgpd.log, 191
bgpd.log.level, 192
bgpd.mon.guardtime, 16, 149, 194, 237, 239, 253
bgpd.mon.holdtime, 15, 17, 149, 194, 237, 239, 253
bgpd.mon.keepalive, 15, 16, 149, 194, 237, 239, 253
bgpd.mon.longholdtime, 15, 17, 194, 253
bgpd.policy.cascade.amount, 193, 253
bgpd.rd_local_mark, 27, 189, 194, 236, 243, 254
bgpd.scaninterval, 193, 195
bgpd.snmp.simultaneous, 195
bgpd.updates.split, 150, 183, 195, 201, 214, 257

BGP sessions Settings

bgpd.peer.X.as, 149, 172, 195, 253
bgpd.peer.X.cap_4byte_as, 196, 253
bgpd.peer.X.inbound.ipv4.next_hop, 197, 247
bgpd.peer.X.inbound.ipv6.next_hop, 197, 247
bgpd.peer.X.inbound.localpref, 197
bgpd.peer.X.keepalive, 150, 197, 255
bgpd.peer.X.listen, 198
bgpd.peer.X.master_communities, 150, 173, 181, 196, 197, 253
bgpd.peer.X.master_localpref, 149, 150, 173, 181, 198, 253
bgpd.peer.X.master_our_ip, 149, 151, 173, 180, 199, 255
bgpd.peer.X.master_our_ipv6, 149, 173, 199, 255
bgpd.peer.X.master_password, 151, 180, 201, 253
bgpd.peer.X.master_router_id, 200, 256
bgpd.peer.X.med, 150, 173, 198, 253
bgpd.peer.X.origin, 198
bgpd.peer.X.peer_ip, 149, 173, 200, 256
bgpd.peer.X.peer_ipv6, 149, 173, 200, 256
bgpd.peer.X.shutdown, 201, 257
bgpd.peer.X.slave_communities, 150, 181, 197, 253
bgpd.peer.X.slave_localpref, 150, 181, 198, 199, 253
bgpd.peer.X.slave_our_ip, 151, 180, 199, 201, 255
bgpd.peer.X.slave_our_ipv6, 199
bgpd.peer.X.slave_password, 151, 180, 201, 253
bgpd.peer.X.slave_router_id, 150, 173, 200, 256
bgpd.peer.X.updates.limit.max, 183, 195, 201
bgpd.peer.X.updates.limit.ps, 201
COLLECTOR SETTINGS 260

Collector settings
collector.detect.explorer_ips, 202
collector.export.interval, 202, 203
collector.export.ttl, 202
collector.export.volume.high.top_n, 151, 202, 220, 257
collector.export.volume.min, 151, 203, 255
collector.export.volume.min_pct, 151, 203, 255
collector.flow.buffer.size, 203
collector.flow.enabled, 49, 170, 203, 205, 254
collector.flow.listen.nf, 49, 151, 170, 203, 255
collector.flow.listen.sf, 49, 151, 170, 204, 257
collector.flow.log, 204
collector.flow.log.level, 204
collector.flow.sources, 49, 151, 170, 204, 254
collector.ournets, 49, 54, 152, 170, 204, 247, 253
collector.sessions.max, 205
collector.span.buffer.size, 205
collector.span.enabled, 53, 170, 203, 205, 257
collector.span.interfaces, 53, 54, 151, 170, 205, 255
collector.span.log, 205
collector.span.log.level, 206
collector.span.min_delay, 54, 151, 171, 206, 255
collector.span.min_delay.probing_queue_size, 151, 171, 206, 255
collector.span.threshold.blackout, 206
collector.span.threshold.congestion, 206
collector.span.threshold.delay, 207
collector.span.threshold.excessive, 207
collector.speaking_ips, 207

Core settings
core.commit_control, 43, 44, 48, 186, 208, 232, 234, 243, 253
core.commit_control.agg_bw_min, 18, 46, 147, 208, 255
core.commit_control.del_irrelevant, 209, 254
core.commit_control.inbound.enabled, 169, 209
core.commit_control.inbound.moderated, 169, 209
core.commit_control.inbound.rate.high, 209
core.commit_control.inbound.rate.low, 210
core.commit_control.inbound.volume_estimation, 210
core.commit_control.probe_ttl, 208, 210, 253
core.commit_control.probing_queue_size, 147, 210, 253
core.commit_control.rate.group, 208, 210, 256
core.commit_control.rate.high, 208, 209, 211, 256
core.commit_control.rate.low, 43, 208, 210, 211, 256
core.commit_control.react_on_collector, 44, 212
core.cost.worst_ms, 135, 212, 213, 253
core.eventqueuelimit, 135, 212, 254
core.eventqueuelimit.retry_probe_pct, 18, 212
core.flowspec.max, 124, 214, 255
core.flowspec.max_ipv6, 124, 215, 255
core.global.allow_commit, 213, 254
core.global.allow_latency_cost, 213, 254
core.global.worst_ms, 28, 212, 213
core.improvements.clearslots, 213
core.improvements.clearslots.days_max, 213
core.improvements.defer, 214, 254
EXPLORER SETTINGS 261

core.improvements.defer.hold, 214, 254

core.improvements.max, 18, 135, 195, 214, 255
core.improvements.max_ipv6, 18, 135, 214, 255
core.improvements.retry_probe.volume_top_n, 18, 135, 215, 257
core.improvements.ttl.retry_probe, 18, 44, 135, 215, 257
core.log, 18, 208
core.log.level, 208
core.outage_detection, 48, 215, 219, 255
core.outage_detection.limit_pct, 215, 255
core.overusage.check_interval, 136, 216, 255
core.overusage.hold_timer, 136, 216, 255
core.overusage.out.average.period, 136, 216, 255
core.overusage.out.average.relevant_min, 136, 216, 256
core.overusage.out.threshold.throttle, 137, 216, 255
core.overusage.out.threshold.trigger, 137, 217, 255
core.performance.loss_pct, 135, 217, 256
core.performance.rtt.diff_ms, 135, 217, 218, 256
core.performance.rtt.diff_pct, 135, 217, 218, 256
core.performance.rtt.ix_diff_ms, 218, 256
core.performance.rtt.ix_diff_pct, 218, 256
core.probes.ttl.failed, 218, 254
core.probes.ttl.max, 135, 218, 255
core.probes.ttl.min, 135, 219, 255
core.problem.outage_timeout, 219, 255
core.problem.rtt.diff_pct, 219, 255
core.vip.interval.probe, 17, 135, 219, 258

Explorer settings
explorer.aipi, 220
explorer.algorithms, 222, 254
explorer.high_vol_precedence, 153, 220, 254
explorer.infra_ips, 66, 153, 170, 220, 254
explorer.interval.infra, 221
explorer.interval.other, 221
explorer.interval.other.trace, 221
explorer.ipv4_test, 221, 256
explorer.ipv6_test, 221, 256
explorer.log, 219
explorer.log.level, 220
explorer.max_collector_ips, 153, 222, 255
explorer.maxthreads, 153, 220, 221, 254
explorer.probe.algorithm, 153, 222, 224, 256
explorer.probe.indirect_priority, 222, 254
explorer.probing.sendpkts.adaptive_max, 153, 222, 223, 253
explorer.probing.sendpkts.min, 153, 222, 223, 255
explorer.probing.simultaneous, 223
explorer.scanning.confirm_ips, 223, 257
explorer.scanning.replypkts.min, 223, 257
explorer.scanning.rtt.dispersion_ms, 223, 257
explorer.scanning.sendpkts.factor, 223, 256
explorer.timeout, 153, 224, 254
explorer.timeout.infra, 224
explorer.trace.algorithms, 153, 224, 257
explorer.trace.all, 192, 222, 224, 257
explorer.traceroute.retrypkts, 153, 225, 258
explorer.traceroute.sendpkts, 153, 225, 257
ADMINISTRATIVE SETTINGS 262

explorer.traceroute.simultaneous, 225
explorer.traceroute.simultaneous.infra, 225
explorer.traceroute.ttl.max, 153, 225, 257
explorer.traceroute.ttl.min, 153, 225, 257

Administrative settings
dbcron.api.log, 231
dbcron.api.socket, 232
dbcron.log, 232
dbcron.log.level, 232

irpstatd.log, 232
irpstatd.log.level, 232

Upstream provider settings

peer.X.95th, 139, 175, 178, 179, 208, 230–233, 240, 243, 256
peer.X.95th.bill_day, 139, 232, 233, 256
peer.X.95th.centile, 139, 233, 253
peer.X.95th.in, 139, 233, 256
peer.X.95th.mode, 169, 209, 210, 233
peer.X.auto_config, 145, 183, 233, 254
peer.X.bgp_peer, 138, 147, 174, 234, 256
peer.X.cc_disable, 44, 138, 139, 175, 178, 179, 186, 234, 254
peer.X.cost, 138, 139, 175, 234, 243, 256
peer.X.description, 138, 174, 234, 256
peer.X.diag_hop.interval_max, 220, 235
peer.X.diag_hop.interval_min, 220, 235
peer.X.disable_pbr_confirmation, 235
peer.X.flow_agents, 27, 138, 165, 189, 194, 236, 243, 254
peer.X.flowspec.ipv4.redirect_community, 124, 235, 254
peer.X.flowspec.ipv6.redirect_community, 124, 235, 254
peer.X.group_loadbalance, 44, 46, 236, 255
peer.X.improve_in_group, 139, 175, 179, 236, 255
peer.X.inbound.announce_type, 236
peer.X.inbound.community_base, 236
peer.X.ipv4.diag_hop, 142, 174, 237, 254
peer.X.ipv4.master_probing, 124, 138, 142, 174, 180, 237, 256
peer.X.ipv4.mon, 15, 17, 141, 177, 237, 254
peer.X.ipv4.next_hop, 15, 142, 174, 237, 240, 256
peer.X.ipv4.next_hop_as, 142, 175, 192, 238, 256
peer.X.ipv4.route_server, 238
peer.X.ipv4.slave_probing, 180, 237, 238, 256
peer.X.ipv6.diag_hop, 142, 175, 238, 254
peer.X.ipv6.master_probing, 124, 138, 142, 175, 238, 239, 256
peer.X.ipv6.mon, 15, 17, 141, 177, 239, 254
peer.X.ipv6.next_hop, 15, 175, 238, 239, 241, 256
peer.X.ipv6.next_hop_as, 175, 192, 239, 256
peer.X.ipv6.route_server, 239
peer.X.ipv6.slave_probing, 239
peer.X.limit_load, 138, 139, 175, 178, 179, 232, 240, 255
peer.X.mon.enabled, 16, 141, 177, 238–240, 254
peer.X.mon.ipv4.bgp_peer, 17, 141, 177, 240, 253
peer.X.mon.ipv4.internal.mode, 141, 238, 240
peer.X.mon.ipv6.bgp_peer, 17, 141, 241
peer.X.mon.ipv6.external.enabled, 141, 241, 254
ROUTING POLICIES SETTINGS 263

peer.X.mon.ipv6.internal.enabled, 141, 241, 254

peer.X.mon.ipv6.internal.mode, 141, 187, 239, 241
peer.X.mon.snmp.auth_password, 142, 241, 242, 257
peer.X.mon.snmp.auth_protocol, 142, 241, 242, 257
peer.X.mon.snmp.auth_username, 142, 241, 242, 257
peer.X.mon.snmp.community, 16, 141, 147, 240–242, 253
peer.X.mon.snmp.ip, 16, 141, 147, 240, 242, 253
peer.X.mon.snmp.ipv6, 16, 141, 147, 241, 242, 253
peer.X.mon.snmp.priv_password, 142, 242, 257
peer.X.mon.snmp.priv_protocol, 142, 242, 257
peer.X.mon.snmp.seclevel, 142, 241, 242, 244, 245, 257
peer.X.mon.snmp.version, 141, 142, 241–245, 257
peer.X.precedence, 42, 44, 139, 175, 179, 208, 243, 253
peer.X.rd, 26, 138, 189, 194, 236, 243, 256
peer.X.routes_config, 244, 256
peer.X.shortname, 138, 174, 179, 235, 256
peer.X.shutdown, 138, 244, 256
peer.X.snmp.auth_password, 140, 244, 246, 257
peer.X.snmp.auth_protocol, 140, 244, 246, 257
peer.X.snmp.auth_username, 141, 244–246, 257
peer.X.snmp.community, 85, 140, 147, 176, 208, 240, 245, 256
peer.X.snmp.interface, 85, 140, 147, 176, 208, 240, 245, 256
peer.X.snmp.ip, 85, 140, 147, 176, 208, 240, 245, 256
peer.X.snmp.ipv6, 85, 140, 147, 176, 208, 240, 245, 256
peer.X.snmp.priv_password, 141, 245, 246, 257
peer.X.snmp.priv_protocol, 141, 246, 257
peer.X.snmp.seclevel, 140, 147, 245, 246, 257
peer.X.snmp.version, 140, 141, 147, 245, 246, 257
peer.X.type, 237, 246, 256

Routing Policies settings

asn/prefix, 247

cascade, 247

enabled, 247

notes, 248

policy, 248
providers, 248

vip, 248

Exchanges
provider.X.rule.Y.bgp_peer, 238, 239, 248, 256
provider.X.rule.Y.enabled, 248, 255
provider.X.rule.Y.next_hop, 249, 255
provider.X.rule.Y.pbr_check, 249, 255
provider.X.rule.Y.probing_dscp, 249, 256
provider.X.rule.Y.probing_ip, 249, 256
provider.X.rule.Y.shortname, 249, 255
EVENTS PARAMETERS 264

Events parameters
pushd.email.from, 226, 254
pushd.email.host, 226, 254
pushd.email.port, 226, 257
pushd.listen.port, 226
pushd.log, 226
pushd.log.level, 226
pushd.sms.account_sid, 226, 253
pushd.sms.auth_token, 227, 257
pushd.sms.gateway, 227, 257
pushd.sms.message_size, 227, 255
pushd.sms.phone_number, 227, 254
pushd.sms.uri.plivo, 227
pushd.sms.uri.twilio, 227
pushd.templates.datadir, 228
pushd.webhook.avatar_emoji, 228, 253
pushd.webhook.avatar_url, 228, 253
pushd.webhook.botname, 228, 253
pushd.webhook.url, 229, 258

trap.bgpd_announced_rate_low.limit_pct, 230, 255

trap.core_cc_overload.limit_mbps, 230, 255
trap.core_cc_overload.limit_pct, 231, 255
trap.core_cc_provider_overload.limit_mbps, 231, 256
trap.core_cc_provider_overload.limit_pct, 231, 256
trap.core_improvement_latency.limit_ms, 231, 255
trap.core_improvement_loss.limit_pct, 231, 256
trap.destination.auth_password, 229, 230, 257
trap.destination.auth_protocol, 229, 230, 257
trap.destination.auth_username, 229, 230, 257
trap.destination.community, 229, 257, 258
trap.destination.port, 229, 257
trap.destination.priv_password, 229, 230, 257
trap.destination.priv_protocol, 230, 257
trap.destination.seclevel, 229, 230, 257
trap.destination.version, 229, 230, 257

API daemon settings

apid.listen.master_ip, 190
apid.listen.port, 191
apid.listen.slave_ip, 190
apid.log, 190
apid.log.level, 190
apid.maxthreads, 191
apid.path.mtr, 191
apid.path.ping, 191
apid.path.ping6, 191
apid.path.traceroute, 191

Inbound settings
bgpd.peer.X.inbound.ipv4.next_hop, 197, 247
bgpd.peer.X.inbound.ipv6.next_hop, 197, 247
bgpd.peer.X.inbound.localpref, 197
USER DIRECTORIES 265

core.commit_control.inbound.enabled, 169, 209

core.commit_control.inbound.moderated, 169, 209
core.commit_control.inbound.rate.high, 209
core.commit_control.inbound.rate.low, 210
core.commit_control.inbound.volume_estimation, 210

global.inbound_conf, 187

inbound.rule.X.bgp_peer, 246
inbound.rule.X.enabled, 247
inbound.rule.X.next_hop, 197, 247, 255
inbound.rule.X.prefix, 247

peer.X.95th.in, 139, 233, 256

peer.X.95th.mode, 169, 209, 210, 233
peer.X.inbound.announce_type, 236
peer.X.inbound.community_base, 236

User Directories
directory.X.admin_role_groups, 249, 251, 253
directory.X.base_dn, 249, 251, 253
directory.X.email, 249, 254
directory.X.fullname, 250, 254
directory.X.hostname, 250, 254
directory.X.initial_bind_dn, 250, 254
directory.X.initial_bind_password, 250, 254
directory.X.name, 250, 258
directory.X.order, 250, 255
directory.X.port, 250, 255
directory.X.secret, 250
directory.X.state, 251, 257
directory.X.timeout, 251, 257
directory.X.tls, 251, 252, 257
directory.X.tls_cacertfile, 251, 252, 257
directory.X.type, 251, 258
directory.X.user_dn, 249, 251, 258
directory.X.user_role_attr, 249, 251, 258
directory.X.username, 249, 251, 258
directory.X.verify_cert, 251–253
List of Figures

1.2.1 IRP Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.2.2 System Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.2.3 Error: Internal BGP Monitor not configured . . . . . . . . . . . . . . . . . . . . . . . . 16
1.2.4 Support for Centralized Route Reflectors . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.2.5 IRP configuration in a multi-homed network connected to transit providers as well as
and Internet Exchange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.2.6 City wide network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.2.7 Regional network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.2.8 Intercontinental network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.2.9 Multiple routing domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.2.10 Failover high level overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
1.2.11 Unbalanced inbound/outbound peering . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.2.12 Fluctuating inbound traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.3.1 Performance optimization algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
1.3.2 Cost optimization algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
1.3.3 More aggressive when actual exceeds schedule . . . . . . . . . . . . . . . . . . . . . . . 43
1.3.4 Provider load balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
1.3.5 Commit control of aggregated groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
1.3.6 Sorted Commit Control improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.3.1 Flow export configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.3.2 Span port configuration (on the router) . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.3.3 Span port configuration (on the switch) . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.4.1 PBR configuration: single router and separate probing Vlan . . . . . . . . . . . . . . . 56
2.4.2 PBR configuration: two routers and separate probing Vlan . . . . . . . . . . . . . . . . 59
2.4.3 PBR configuration via GRE tunnels . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.4.4 PBR configuration: single router and separate probing Vlan . . . . . . . . . . . . . . . 67

3.1.1 Frontend login form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

3.1.2 Default Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.2.1 Menu bar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.2.2 Status bar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.2.3 Common sidebar buttons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.2.4 Email schedule configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.2.5 Report filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3.2.6 Time period selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.2.7 Adjusting the number of results per page . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.2.8 Dashboard buttons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.2.9 Failover status of master node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.2.10 Failover status of slave node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.2.11 Search box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.2.12 Dynamical global search results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.2.13 Global search results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.2.14 Precedence and improve in a group warning . . . . . . . . . . . . . . . . . . . . . . . . 95
3.2.15 Internal Monitor warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.2.16 Explorer performance warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.2.17 License expiration warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

266
LIST OF FIGURES 267

3.3.1 Platform status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

3.3.2 Interfaces status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.3.3 List of improved prefixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.3.4 AS-Path problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
3.4.1 Reports menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.4.2 Platform overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
3.4.3 Platform performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.4.4 Loss relative rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.4.5 Loss absolute rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.4.6 Loss-improved destinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.4.7 Latency values (%) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.4.8 Latency values (ms) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.4.9 Latency destinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.4.10 Probe conversion rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.4.11 Current improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.4.12 VIP Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3.4.13 Add VIP Prefix/ASN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.4.14 Improvements to Exchanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.4.15 Historical records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.4.16 Providers overall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.4.17 Provider performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.4.18 Cost overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.4.19 AS-pattern problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.4.20 Top volume prefixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.4.21 Top volume AS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.4.22 Top problem AS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.4.23 Prefix statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.4.24 AS statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.4.25 Current inbound improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.5.1 Graphs menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.5.2 Improvements by type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.5.3 Performance improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.5.4 Probed prefixes and Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.5.5 New and retry improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.5.6 Improvements by probing source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
3.5.7 Prefixes rerouted from provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.5.8 Prefixes rerouted to provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.5.9 Improvements by provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.5.10 Performance improvements by provider . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.5.11 Commit improvements by provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.5.12 Total and improved traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.5.13 Providers bandwidth usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.5.14 Bandwidth usage and improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.5.15 Total bandwidth usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.5.16 Inbound traffic distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
3.6.1 Routing Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
3.6.2 Routing policy window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
3.6.3 Static Routes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.6.4 VIP policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.7.1 Flowspec policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
3.7.2 Flowspec policy popup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
3.8.1 Throttling excessive bandwidth use with Flowspec policies . . . . . . . . . . . . . . . . 124
3.9.1 Maintenance windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
3.9.2 Maintenance window popup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
3.9.3 Maintenance window sidebar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
3.10.1 System events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
3.11.1 Traceroute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
3.11.2 Traceroute results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
LIST OF FIGURES 268

3.11.3 Traceroute exploring details results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

3.11.4 Looking glass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
3.11.5 Whois . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
3.11.6 Prefix Probing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
3.11.7 Prefix probing results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
3.12.1 Configuration editor: Global settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
3.12.2 Configuration editor: Ignored prefixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
3.12.3 Configuration editor: Global failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
3.12.4 Configuration editor: Core configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 135
3.12.5 Configuration editor: Outage detection . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
3.12.6 Configuration editor: Overusage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
3.12.7 Configuration editor: Providers configuration . . . . . . . . . . . . . . . . . . . . . . . . 138
3.12.8 Configuration editor: Adding a new provider, General settings . . . . . . . . . . . . . . 139
3.12.9 Configuration editor: Adding a new provider, Commit Control . . . . . . . . . . . . . . 140
3.12.10 Configuration editor: Adding a new provider, SNMP settings . . . . . . . . . . . . . . . 140
3.12.11 Configuration editor: Adding a new provider, External Monitor . . . . . . . . . . . . . 141
3.12.12 Configuration editor: Adding a new provider, Monitoring . . . . . . . . . . . . . . . . . 142
3.12.13 Configuration editor: Adding a new provider, Commit Control . . . . . . . . . . . . . . 142
3.12.14 An Internet Exchange is similar to a provider and it requires configuration of the Router,
the Probing IPs and the other usual attributes . . . . . . . . . . . . . . . . . . . . . . . 144
3.12.15 The list of peering partners for the Exchange shows details about each of them and
offers access to IRP Autoconfiguration feature and PBR ruleset generator for the router 145
3.12.16 PBR Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
3.12.17 Warning about new peering partners that have been added to the Exchange . . . . . . 146
3.12.18 Configuration editor: Commit Control . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
3.12.19 Providers Overall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
3.12.20 Configuration editor: BGP settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
3.12.21 Configuration editor: New Router, general settings . . . . . . . . . . . . . . . . . . . . 150
3.12.22 Configuration editor: New Router, advanced settings . . . . . . . . . . . . . . . . . . . 150
3.12.23 Configuration editor: New Router, advanced settings . . . . . . . . . . . . . . . . . . . 151
3.12.24 Configuration editor: Collector settings . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
3.12.25 Configuration editor: Explorer settings . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
3.12.26 Configuration editor: Events configuration . . . . . . . . . . . . . . . . . . . . . . . . . 154
3.12.27 Configuration editor: Email configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 154
3.12.28 Configuration editor: SMS configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 155
3.12.29 Plivo account ID and secret . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
3.12.30 Configuration editor: SNMP Traps configuration . . . . . . . . . . . . . . . . . . . . . . 156
3.12.31 Configuration editor: Web Hooks configuration . . . . . . . . . . . . . . . . . . . . . . 156
3.12.32 Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
3.12.33 Subscribe notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
3.12.34 Subscribe notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
3.12.35 Configuration editor: Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
3.12.36 Configuration editor: Users and Directories . . . . . . . . . . . . . . . . . . . . . . . . 159
3.12.37 Configuration editor: LDAP User Directory configuration . . . . . . . . . . . . . . . . 160
3.12.38 Configuration editor: Users and Directories . . . . . . . . . . . . . . . . . . . . . . . . 161
3.12.39 Configuration editor: Users and Directories . . . . . . . . . . . . . . . . . . . . . . . . 162
3.12.40 Configuration editor: TACACS+ configuration . . . . . . . . . . . . . . . . . . . . . . 163
3.12.41 Configuration: Adding a new user account . . . . . . . . . . . . . . . . . . . . . . . . . 164
3.12.42 Configuration editor: Frontend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
3.12.43 Configuration Editor: Manage currencies . . . . . . . . . . . . . . . . . . . . . . . . . . 165
3.12.44 Main menu: Inbound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
3.12.45 Configuration editor: Inbound prefixes . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
3.12.46 Configuration editor: Base communities . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
3.12.47 Configuration editor: Inbound commit control . . . . . . . . . . . . . . . . . . . . . . . 168
3.12.48 Configuration editor: Inbound BW estimation algorithms . . . . . . . . . . . . . . . . . 169
3.12.49 Configuration editor: Inbound BW estimation algorithms . . . . . . . . . . . . . . . . . 169
3.12.50 Configuration editor: Initial Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
3.12.51 Configuration editor: Setup Infrastructure IP addresses and Analyzed networks . . . . 170
 3.12.52 Configuration editor: Configure Collector . . . . . . . . . . . . . . . . . . . . . . . . . 171
3.12.53 Configuration editor: Improvement mode . . . . . . . . . . . . . . . . . . . . . . . . . 171
3.12.54 Configuration editor: Management Interface . . . . . . . . . . . . . . . . . . . . . . . . 172
3.12.55 Configuration editor: Probing Interface . . . . . . . . . . . . . . . . . . . . . . . . . . 172
3.12.56 Configuration editor: Router name and AS . . . . . . . . . . . . . . . . . . . . . . . . 172
3.12.57 Configuration editor: Router IPv4 addresses . . . . . . . . . . . . . . . . . . . . . . . . 173
3.12.58 Configuration editor: Router IPv6 addresses . . . . . . . . . . . . . . . . . . . . . . . . 173
3.12.59 Configuration editor: Router announcing . . . . . . . . . . . . . . . . . . . . . . . . . 174
3.12.60 Configuration editor: Choose a Router . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
3.12.61 Configuration editor: Provider name . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
3.12.62 Configuration editor: Provider IPv4 addresses . . . . . . . . . . . . . . . . . . . . . . . 175
3.12.63 Configuration editor: Provider IPv6 addresses . . . . . . . . . . . . . . . . . . . . . . . 175
3.12.64 Configuration editor: Provider Commit Control . . . . . . . . . . . . . . . . . . . . . . 176
3.12.65 Configuration editor: Provider Monitoring setup IPv4 . . . . . . . . . . . . . . . . . . 176
3.12.66 Configuration editor: Provider Monitoring setup IPv6 . . . . . . . . . . . . . . . . . . 176
3.12.67 Configuration editor: External Monitor setup IPv4 . . . . . . . . . . . . . . . . . . . . 177
3.12.68 Configuration editor: External Monitor setup IPv6 . . . . . . . . . . . . . . . . . . . . 177
3.12.69 Configuration editor: Internal Monitor setup . . . . . . . . . . . . . . . . . . . . . . . 177
3.12.70 Configuration editor: Provider pre-checks . . . . . . . . . . . . . . . . . . . . . . . . . 178
3.12.71 Configuration editor: Commit Control basic setup . . . . . . . . . . . . . . . . . . . . 178
3.12.72 Configuration editor: Commit Control provider precedence setup . . . . . . . . . . . . 178
3.12.73 Configuration editor: Commit Control groups setup . . . . . . . . . . . . . . . . . . . 179
3.12.74 Configuration editor: Commit Control Provider overall . . . . . . . . . . . . . . . . . . 179
3.12.75 Configuration editor: Failover setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
3.12.76 Configuration editor: Failover probing IPs for providers . . . . . . . . . . . . . . . . . . 180
3.12.77 Configuration editor: Failover BGP session settings . . . . . . . . . . . . . . . . . . . . 180
3.12.78 Configuration editor: BGP announcement attributes . . . . . . . . . . . . . . . . . . . 181
3.12.79 Configuration editor: Probing and management and interfaces . . . . . . . . . . . . . . 181

List of configuration examples

1.1 Stop IRP Core and purge existing improvements . . . . . . . . . . . . . . . . . . . . . . . 12

1.2 Switch to Intrusive Mode and adjust IRP Core and BGPd parameters . . . . . . . . . . . 12
1.3 Inserting test improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.4 Remove next-hop-self route-map (RM-NHS) example . . . . . . . . . . . . . . . . . . . . . 13
1.5 Restart IRP BGPd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.6 Show BGP information for specified IP address or prefix . . . . . . . . . . . . . . . . . . . 13
1.7 Traceroute destination networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.8 Delete test improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.9 Configure the maximum improvements limit and revert BGPd configuration . . . . . . . . 14
1.10 Restart IRP Core and BGPd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.11 Delete test improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.12 Switch the system to non-intrusive mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.13 Restart IRP Core and BGPd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1 Global MLS settings configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.2 Global NetFlow settings and export configuration . . . . . . . . . . . . . . . . . . . . . . . 50
2.3 Per-interface NetFlow settings configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.4 Flexible NetFlow monitor configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.5 Flexible NetFlow exporter configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.6 Flexible NetFlow sampler configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

269
LIST OF CONFIGURATION EXAMPLES 270

2.7 Per-interface Flexible NetFlow settings configuration . . . . . . . . . . . . . . . . . . . . . 51

2.8 NetFlow configuration on Cisco 7200/3600 series routers . . . . . . . . . . . . . . . . . . . 51
2.9 Ingress/egress flow export configuration on peering interfaces . . . . . . . . . . . . . . . . 51
2.10 Ingress flow export configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.11 Configuring NetFlow export on Vyatta . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.12 Configuration of an interface for the flow accounting . . . . . . . . . . . . . . . . . . . . . 52
2.13 Juniper flow export configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.14 Per-interface NetFlow sampling configuration . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.15 Cisco IPv4 PBR configuration: 1 router, 2 providers . . . . . . . . . . . . . . . . . . . . . 56
2.16 Cisco IPv4 PBR configuration for IOS XR: 1 router, 2 providers . . . . . . . . . . . . . . 56
2.17 Juniper IPv4 PBR configuration: 1 router, 2 providers . . . . . . . . . . . . . . . . . . . . 57
2.18 Vyatta IPv4 PBR configuration example . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.19 Vyatta (VC6.5 and up) IPv4 PBR configuration example . . . . . . . . . . . . . . . . . . . 58
2.20 IRP server source-based routing using the ‘ip‘ command . . . . . . . . . . . . . . . . . . 59
2.21 IRP server source-based routing using standard CentOS configuration files . . . . . . . . . 59
2.22 Cisco IPv4 PBR configuration: 2 routers, 2 providers, separate Vlan . . . . . . . . . . . . 59
2.23 IRP-side IPv4 GRE tunnel CLI configuration . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.24 IRP-side IPv4 GRE tunnel configuration using standard CentOS configs . . . . . . . . . . 60
2.25 Router-side IPv4 GRE tunnel configuration (Vyatta equipment) . . . . . . . . . . . . . . . 61
2.26 Router-side IPv4 GRE tunnel configuration (Cisco equipment) . . . . . . . . . . . . . . . 61
2.27 Router-side IPv4 GRE tunnel configuration (Juniper equipment) . . . . . . . . . . . . . . 61
2.28 IRP server IPv4 source-based routing via GRE using the ‘ip‘ command . . . . . . . . . . 61
2.29 IRP server IPv4 source-based routing via GRE using standard CentOS configuration files 61
2.30 Cisco IPv4 PBR configuration: 2 routers, 2 providers, separate Vlan . . . . . . . . . . . . 62
2.31 Cisco IPv4 PBR configuration template . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
2.32 Cisco IPv4 PBR configuration template . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.33 Juniper IPv4 PBR configuration template . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.34 PBR validation using ‘traceroute‘ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.35 PBR validation using Explorer self check . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.36 PBR validation using ‘iptables‘ and ‘traceroute‘ . . . . . . . . . . . . . . . . . . . . . . . . 66
2.37 Sample provider configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.38 SNMP parameters validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.39 iBGP session configuration example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
2.40 Vyatta IPv4 iBGP session configuration example . . . . . . . . . . . . . . . . . . . . . . . 70
2.41 Vyatta IPv6 iBGP session configuration example . . . . . . . . . . . . . . . . . . . . . . . 70
2.42 Vyatta IPv4 route-map for setting local-pref on the router . . . . . . . . . . . . . . . . . . 70
2.43 Vyatta IPv6 route-map for setting local-pref on the router . . . . . . . . . . . . . . . . . . 70
2.44 Cisco IPv4 iBGP session configuration example . . . . . . . . . . . . . . . . . . . . . . . . 70
2.45 Cisco IPv6 iBGP session configuration example . . . . . . . . . . . . . . . . . . . . . . . . 71
2.46 Cisco IPv4 route-map for setting local-pref on the router . . . . . . . . . . . . . . . . . . . 71
2.47 Cisco IPv6 route-map for setting local-pref on the router . . . . . . . . . . . . . . . . . . . 71
2.48 Limiting the number of received prefixes for an IPv4 neighbor on Cisco . . . . . . . . . . . 71
2.49 Limiting the number of received prefixes for an IPv6 neighbor on Cisco . . . . . . . . . . . 71
2.50 Juniper IPv4 iBGP session configuration example . . . . . . . . . . . . . . . . . . . . . . . 71
2.51 Juniper IPv6 iBGP session configuration example . . . . . . . . . . . . . . . . . . . . . . . 72
2.52 Juniper IPv4 route-map for setting local-pref on the router . . . . . . . . . . . . . . . . . 72
2.53 Limiting the number of received prefixes for an IPv4 neighbor on Juniper . . . . . . . . . 72
2.54 Reload IRP BGPd configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.55 Generate keys on $IRPMASTER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.56 Install public key on $IRPSLAVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.57 Check SSH certificate-based authentication works . . . . . . . . . . . . . . . . . . . . . . . 75
2.58 Generate CA and certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
2.59 Verify certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
2.60 Install certificates in designated directories . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
2.61 Cross copy client key and certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
2.62 Set file permissions for keys and certificates . . . . . . . . . . . . . . . . . . . . . . . . . . 76
2.63 Example $IRPSLAVE configuration from template . . . . . . . . . . . . . . . . . . . . . . 77
2.64 Check MySQL on $IRPSLAVE works correctly . . . . . . . . . . . . . . . . . . . . . . . . 77
LIST OF CONFIGURATION EXAMPLES 271

2.65 Set $IRPMASTER as a first node for Multi-Master replication . . . . . . . . . . . . . . . 77

2.66 Check MySQL on $IRPMASTER works correctly . . . . . . . . . . . . . . . . . . . . . . . 77
2.67 Replication user and grants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2.68 Copy database root user configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2.69 Copy database configuration files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2.70 Copy database data files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2.71 Copy differences of database files (CentOS 7) . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.72 Copy differences of database files (CentOS 6) . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.73 Set $IRPMASTER as replication slave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
2.74 Starting replication slave on $IRPMASTER . . . . . . . . . . . . . . . . . . . . . . . . . . 80
2.75 Set $IRPSLAVE as replication slave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
2.76 Starting replication slave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
2.77 Starting IRP services and Frontend (CentOS 7) . . . . . . . . . . . . . . . . . . . . . . . . 81
2.78 Starting IRP services and Frontend (CentOS 6) . . . . . . . . . . . . . . . . . . . . . . . . 81
2.79 Synchronize RRD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.1 Error log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196