Trees
Trees
Trees
Copyright © 2007
http://eprints.gla.ac.uk/3659/
Abstract—GMPLS is viewed as an attractive intelligent control Graceful restart mechanisms have been dened for many
plane for different network technologies and graceful restart is a routing protocols, exploiting the fact that most modern routers
key technique to ensure this control plane is resilient and able to separate the routing and forwarding processes. Therefore,
recover adequately from faults. This paper analyses the graceful
restart mechanism proposed for a key GMPLS protocol, RSVP- since it is possible for the routing process to fail indepen-
TE. A novel analytical model, which may be readily adapted to dently of the forwarding process, it is desirable for trafc
study other protocols, is developed. This model allows the efcacy forwarding to continue in the presence of a routing process
of graceful restart to be evaluated in a number of scenarios. It is fault and for the routing process to be restored as quickly
found that, unsurprisingly, increasing control message loss and and efciently as possible. Graceful restart techniques dene
increasing the number of data plane connections both increased
the time to complete recovery. It was also discovered that a the procedures that allows these goals to be met. Naturally,
threshold exists beyond which a relatively small change in the the exact graceful restart procedures are protocol-dependent
control message loss probability causes a disproportionately large but have many common aspects, including the requirements
increase in the time to complete recovery. The interesting ndings for the restarting router to inform its neighbours whether it
in this work suggest that the performance of graceful restart is supports graceful restart, for the neighbours to detect when
worthy of further investigation, with emphasis being placed on
exploring procedures to optimise the performance of graceful it has failed and restarted, for the neighbours to attempt to
restart. minimise data plane disruption and for the restarting router to
resynchronise its database with those of the neighbours after
the restart.
I. I NTRODUCTION Since RSVP-TE is a key GMPLS protocol and its graceful
It is widely accepted that Generalized Multi-Protocol Label restart techniques are as yet not completely standardised, this
Switching (GMPLS) is an attractive automated, intelligent paper studies the graceful restart mechanisms which have been
control plane for different network technologies. GMPLS proposed for RSVP-TE to date, using an analytic method that
comprises of IP-based protocols to support routing, signalling may be readily adapted to other protocols. Section II describes
and link management that, when properly orchestrated, will the graceful restart mechanisms proposed for RSVP-TE and
simplify network operation and offer the possibility of po- a novel analytical model for evaluating its performance is
tentially lucrative, novel, on-demand services. Consequently, developed in Section III. Numerical results are presented in
there have been signicant developments in the standardisation Section IV and Section V concludes the paper and suggests
of GMPLS and the salient parts of the GMPLS architecture avenues for further work.
have been well-dened [12], [4]. GMPLS typically comprises
of the use of Open Shortest Path First - Trafc Engineering ex- II. RSVP-TE G RACEFUL R ESTART
tensions (OSPF-TE) [9] for routing, the Resource Reservation The RSVP-TE hello extension, dened in RFC 3209 [1],
Protocol - Trafc Engineering extensions (RSVP-TE) [1], [3] forms the basis of RSVP-TE graceful restart since it provides
for signalling and the Link Management Protocol (LMP) [10] a means for an RSVP-TE node to detect when a neighbour
for link management. is unreachable or when it has restarted. The hello extension
As with any network architecture, resiliency is a key require- requires Hello messages, shown in Figure 1, to be exchanged
ment of GMPLS controlled networks. Since GMPLS may be at regular intervals (5ms suggested [1], although 3s is a more
used for a range of different network technologies, most of practical default) and the failure to receive a Hello message
the work on data plane resiliency is applicable. However, the within a certain interval (default is 3 12 times the hello inter-
fact that GMPLS typically necessitates a separation of the data val [1]) means a node presumes it can no longer communicate
plane and control plane means it is often necessary to consider with its neighbour. In order to detect when a neighbour
control plane resilience independently. Consequently, control has restarted, Hello messages contain a Src Instance eld
plane reliability and resilience is a topic gaining increasing and a Dst Instance eld, as shown in Figure 1. Each node
attention [11] . While approaches that seek to minimise the lls the Src Instance eld with a value representing its per
possibility of control plane failures have been proposed [7], the neighbour instance and lls the Dst Instance eld with the
consensus within the community is that mechanisms to ensure Src Instance value most recently received from the neighbour.
the GMPLS control plane recovers adequately from failure are Since whenever a node restarts it changes its Src Instance
more likely to be deployed and, hence, such mechanisms are value, restarts are easily detectable.
attracting ever-increasing attention [3], [14], [6]. RFC 3473 [3] enhances the RSVP-TE hello extension to
Approaches to ensure that the control plane recovers ad- specify procedures for detecting and handling control chan-
equately from faults are mostly based on ”graceful restart”. nel faults and nodal faults. A node that supports RSVP-TE
1-4244-0353-7/07/$25.00 ©2007 IEEE
2324
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2007 proceedings.
Message Type
Common
Hello Object
(RFC 3209)
(22) Type
Src Instance
in Path messages from its downstream neighbour. A newly
dened RecoveryPath message conveys the information con-
Dst Instance
tained in the most recently received Path message to the
Restart Cap Object
Length
Class Number Class
restarting node. An ability to send and receive RecoveryPath
(RFC 3473)
(131) Type
Restart Time
messages is indicated by appropriate elds in the newly
dened Capability object included in the Hello messages. The
Recovery Time
downstream neighbour sends a RecoveryPath message for each
rsvp-restart-ext-05)
Capability Object
(draft-ietf-ccamp-
Length Class Number
Class
Type
data plane connection associated with the restarting node for
Reserved T R S
which it had previously sent a Resv message, spacing the
messages over half of the recovery period to avoid inundating
the restarting node. Upon the receipt of a RecoveryPath
message, the restarting node checks its forwarding table for
Fig. 1. Format of RSVP-TE Hello Message the corresponding entry and responds with a Path message,
as shown in Figure 2. The downstream neighbour responds to
this Path message with a Resv message which the restarting
node processes and duly forwards to the upstream neighbour.
graceful restart adds a new Restart Cap object to its Hello The receipt of a Path message containing the Recovery La-
messages. This object includes a Restart Time eld and a bel object from the upstream neighbour and the RecoveryPath
Recovery Time eld. The Restart Time is the time the sender and Resv messages from the downstream neighbour means
takes to restart. If the data plane is unaffected by control plane the restarting node is able to reconstruct its RSVP state. It is
failure so that the restart may occur over an indeterminate desirable to complete graceful restart as quickly as possible.
time, the sender indicates an innite Restart Time with the Consequently, the total time to reconstruct and resynchronise
value 0xffffffff. However, if a nite restart time is specied, the appropriate RSVP states for all the relevant connections,
after detecting that communication with the sender is lost, allowing normal control plane operations to resume, is pivotal
a neighbour waits for at least the duration specied by the since any state that is not resynchronised is typically cleared
Restart Time, behaving as if it were receiving the relevant at the end of the recovery period and the corresponding data
RSVP-TE refresh messages from the sender for established plane connections removed. An analytical model which may
data plane connections and sending only Hello messages be used to estimate the time to complete graceful restart, as
with the Dst Instance set to 0 to the sender, as shown in a function of the number of connections, durations of the
Figure 2. Eventually, the neighbour receives a Hello message different constituent stages and message loss probability is
from the sender. If the Src Instance value is unchanged, developed in Section III.
then there has been a control channel failure and the nodes
refresh all shared state. If, on the other hand, the Src Instance
value has changed, the sender must have failed and restarted. III. A NALYTICAL M ODEL
The Recovery Time eld in the received Hello message also The key stages in RSVP-TE graceful restart, described
indicates whether the restarting node was able to preserve its in Section II, may be modelled using an absorbing Markov
forwarding state and, if so, the duration for which it is willing chain. The analytical model used in this paper exploits known
to participate in the recovery process. The behaviour of a results about absorbing Markov chains [5] and is derived from
neighbour thereafter is dependent upon whether it is upstream a model previously used to study the stability of routing
or downstream of the restarting node. protocols [15]. The model focuses on exchanges between
An upstream neighbour refreshes all Path state that it shares the restarting node and its downstream neighbour and allows
with the restarting node. Each Path message contains a Recov- the time to complete recovery to be calculated in different
ery Label object, corresponding to the label value in the most circumstances.
recently received Resv message from the restarting node. Since The probabilities of a packet being lost on links towards
there may be a large number of Path messages produced and it the failed node and on links f rom the failed node are pt and
is essential not to overwhelm the restarting node with a deluge pf respectively. These different values are used because of the
of messages in a short time interval, it is recommended that likelihood that the system will behave as if there is a greater
the Path messages be distributed evenly over half the recovery likelihood of message loss on links towards the restarting node
time. Upon receipt of a Path message containing a Recovery due, for example, to the restarting node being highly loaded
Label object from its upstream neighbour, the restarting node because of a deluge of messages being sent to it. Figure 3
searches its forwarding table for the corresponding entry. If shows a state diagram focusing on the exchanges between
found, the appropriate RSVP state is created and the entry the restarting node and the downstream neighbour during the
bound to the associated data plane connection [3]. recovery period. In addition to the transition probabilities,
2325
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2007 proceedings.
Node
fails Src Instance = F …
Hello Dst Instance = G …
Src Instance = B …
Hello Dst Instane = A …
Restart Period, P
Src Instance = F …
Src Instance = B … Hello Dst Instance = 0 …
Hello Dst Instance = 0 …
Node
restarts
=P Src Instance = H
Src Instance = C Restart Time Hello Restart Time = P
Hello Time = Q ≠ 0 Dst Instance = F
Dst Instance = B Recovery Recovery Time = Q ≠
0
Src Instance = B … Src Instance = F …
Hello Dst Instance Hello
=C… Dst Instance = H …
Connection Connection
Path
1
Resv
Resv
2
Resv
Recovery Period, Q
Resv
Connection
Path Recovery Label RecoveryPath
Recovery
complete Path
N
Resv
Resv
1
c=X
c = TRetxInterval c = TRetxInterval c = TRetxInterval c=0
pt pf c = TRetxInterval pt pf 1
Figure 3 shows the cost, c, associated with each transition. other hand, if this Hello message is lost, another Hello message
This cost is an estimate of the time taken for the corresponding will be produced after the hello interval, THelloInterval . Hence,
transition. there is a transition from State S0 to itself with a transition
probability of pf and a cost of THelloInterval . In State S1, the
The system begins in State S0, where the restarting node downstream neighbour has received the Hello message from
sends an Hello message to the downstream neighbour. Given the restarting node. The downstream neighbour subsequently
that the message loss probability on links from the restarting processes the Hello message, taking a time of TP rocessHello
node is pf , this Hello message reaches the downstream neigh- to do so, generates and sends a response Hello message,
bour with a probability of 1 − pf , the transition probability taking TGenerateHello , and creates the RecoveryPath message
from State S0 to S1. The cost of this transition is the cor- for the rst connection, taking TGenerateRecoveryP ath . The
responding time, the propagation delay, TP ropagation . On the
2326
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2007 proceedings.
summation of these times is the cost of the transition from Two of the main approximations necessary to reduce com-
State S1 to State S2-1 and the transition probability is 1 since plexity and ensure the model remains tractable are:
it is assumed that, after receiving the Hello message, the • RFC 2961 [2], which denes the Ack message and the
downstream neighbour always eventually sends a Recovery- retransmission algorithm for RSVP messages, suggests
Path message. the use of an exponential back off algorithm for un-
The transitions between State S2-1 and State S9-1 are acknowledged trigger messages. The suggested defaults
repeated once for each data plane connection; i.e. States 2- were an initial default retransmission interval of 0.5s
2 to 9-2 refer to the second connection and so on, up to (or the round-trip time, if known), doubling this in-
States 2-N to 9-N which correspond to the N th connection. terval between successive retransmissions and limiting
In State S2-1, a hello adjacency has been formed between the number of retransmissions to three. In the system
the downstream neighbour and restarting node and so the depicted in Figure 3, it is evident that the retransmission
downstream neighbour sends a RecoveryPath message for the interval, TRetxInterval remains xed and the number of
rst connection. This message may be lost with a probability retransmissions is unrestricted. Furthermore, the impact
of pt , the loss probability on links towards the restarting node. of sending or losing Ack messages is not incorporated
If lost, a retransmission occurs after an appropriate interval, in the model. These approximations, essential to make
TRetxInterval , and so there is a corresponding transition from the model tractable, mean that the model will slightly
State S2-1 to itself. On the other hand, the RecoveryPath outperform an implementation adhering to the suggested
message reaches the restarting node with a probability of default values.
1 − pt . Therefore, the transition probability from State S2-1 to • Pipelining the recovery of the connections will incur
State S3-1 is 1−pt and the associated cost is TP ropagation , the some additional overhead (e.g. due to resource contention
propagation delay. Upon receiving the RecoveryPath message, and sharing) which may result in an increase in the
the restarting node processes the message and generates an recovery time as the number of connection being simul-
appropriate Path message, taking TP rocessRecoveryP ath and taneously recovered rises. However, for simplicity, the
TGenerateP ath respectively. Summing these values gives the model assumes that the time to recover each connection
costs associated with the transition from State S3-1 to State is independent of the total number of connections being
S4-1; the transition probability is 1 since it is assumed the simultaneously recovered. This assumption means the
restarting node always processes the received RecoveryPath model will likely outperform a real-life implementation.
message and generates a Path message. The remaining tran- Given the Markov chain illustrated in Figure 3, known
sitions up to State S9-1 may be readily understood from the properties of absorbing Markov chains [5] may be used to
message exchanges depicted in Figure 2. compute the average time to absorption in State S10. This
In State S9-1, the upstream neighbour has received the Resv time, T , is given in Equation 1.
sent by the restarting node and so the recovery is complete
T = TGenerateHello + TP rocessHello
for the rst connection. Recovery for the second connection
commences with the transition from State S9-1 to State S2-2 +TGenerateRecoveryP ath
with a probability of 1. The cost of this transition, the interval +(N + 1)TP rocessResv + (4N + 1)TP ropagation
between the end of recovery of the rst connection and start pf THelloIntveral
of recovery of the second connection, is X. In the ”worst case” +(N − 1)X +
1 − pf
scenario, the message exchanges pertaining to each connection 2pf TRetxIntveral 2pt TRetxIntveral
are carried out serially, hence X is the time taken for the +N [ +
1 − pf 1 − pt
upstream neighbour to process the Resv it received from the
+TP rocessRecoveryP ath + TGenerateP ath
restarting node for the last connection and for the downstream
neighbour to generate the RecoveryPath message for the next +TP rocessP ath + 2TGenerateResv ] (1)
connection, i.e. X = TP rocessResv + TGenerateRecoveryP ath . Equation 1 means that the time for graceful restart to
On the other hand, it is likely that some pipelining will occur. be completed may be readily computed for a given number
In an attempt to expedite the recovery process, neighbours of connections and packet loss probabilities, provided the
may send recovery messages for subsequent connections with- duration of the key constituent processes are known. Obtaining
out waiting for the previous connections to be completely Equation 1 is a key contribution of this work since it sug-
recovered. If it is assumed that the time to recover each gests that an appropriate analytical model may be applied to
connection is independent of the total number of connections investigating the performance of graceful restart. This model
being simultaneously recovered, pipelining may be modelled may be readily adapted to different implementations of RSVP-
by having X < 0. Since X refers to the time taken for the TE graceful restart or to other protocols. Section IV assigns
transition, it may appear strange for X to have a a negative reasonable exemplar values to parameters in Equation 1,
value, however, X < 0 simply means an attempt is made allowing the performance of RSVP-TE graceful restart to be
to recover the subsequent connection before the previous evaluated in a number of scenarios.
connection is completely recovered. The Markov property
means the value of X will not affect the behaviour of the rest IV. E XEMPLAR N UMERICAL R ESULTS
of the system. Hence, by setting X appropriately, the impact Some of the parameters in Equation 1 have default values
of pipelining the recovery of connections can be investigated. suggested which are initially used in this section; the hello
2327
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2007 proceedings.
80
decrease in the time to complete recovery for any given 350 100
500
300
packet loss probability and, furthermore, the impact of the 1000
250
number of connections is diminished. The hello interval is 200
5ms default value in RFC 3209 [1] to the more realistic 100
2328
This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the ICC 2007 proceedings.
500 50
1 connection
450 45 1 connection
20
20
40
400 40 40
Time to Complete Recovery (s)
60
250 25
200 20
150 15
100 10
50 5
0 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Packet Loss Probability Packet Loss Probability
Fig. 5. Impact of packet loss on time to complete recovery when connections Fig. 7. Impact of packet loss on time to complete recovery when connections
recovered serially (TRetxInterval = 50ms, THelloInterval = 5ms & recovery pipelined (TRetxInterval = 500ms, THelloInterval = 5ms &
pt = pf ) pt = pf )
500
450 1 connection
20
measurement), to study graceful restart thoroughly. Interesting
40
400
open issues include quantifying the impact of the heavy load
Time to Complete Recovery (s)
60
80
350
100
500
on the restarting node, identifying the optimal spacing of
300
250
1000
the recovery process for different connections, determining
200 the most expedient recovery period in different circumstances
150 and studying the impact of multiple nodal/link faults on the
100
graceful restart.
50
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Packet Loss Probability Towards Restarting Node
0.8 0.9 VI. ACKNOWLEDGMENTS
The authors wish to thank the UK Engineering and Physical
Fig. 6. Impact of packet loss on time to complete recovery when connections Sciences Research Council for their support of this research
recovered serially (TRetxInterval = 500ms, THelloInterval = 5ms & through grant EP/C004442/1. The authors also thank Adrian
pt = 1000pf ) Farrel for helpful comments on the paper.
R EFERENCES
existence of these trends suggest that studying the effect of [1] D. Awduche, L. Berger, D. Gan, T. Li, V. Srinivasan, G. Swallow, “RSVP-
non-ideal behaviour during graceful restart is worthwhile since TE: Extensions to RSVP for LSP Tunnels’, RFC 3209, Dec. 2001.
they suggest that the effect of additional errors on the already [2] L. Berger, D. Gan, G. Swallow, P. Pan, F. Tommasi, S. Molendini, ”RSVP
Refresh Overhead Reduction Extensions” RFC 2961, April 2001.
degraded control plane may be severe. [3] L. Berger (Ed.), ”Generalized Multi-Protocol Label Switching (GMPLS)
Signaling Resource Reservation Protocol-Trafc Engineering (RSVP-TE)
V. C ONCLUSIONS AND F UTURE W ORK Extensions” RFC 3473, Jan. 2003.
[4] A. Farrel, I. Bryskin, ”GMPLS Architecture and Applications”, Morgan
This paper has studied the graceful restart mechanism Kaufmann, 2006.
proposed for RSVP-TE, a key GMPLS protocol. A novel [5] C. Grinstead, J. Snell, ”Introduction to Probability”, AMS, 1997.
[6] A. Jajszczyk and P. Rozycki, ”Recovery of the Control Plane after Failures
analytical model, which may be readily adapted to study other in ASON/GMPLS Networks”, IEEE Network, pp.4-10, Jan/Feb 2006.
protocols, has been developed. This model allows the impact [7] Y. Kim, ”Requirements for the Resilience of Control Plane”, draft-kim-
of the duration of the key constituent stages, the loss of ccamp-cpr-reqts-01.txt, Oct. 2005.
[8] K. Kompella (Ed.), Y. Rekhter (Ed.), ”Routing Extensions in Support of
control messages and the number of data plane connections Generalized Multi-Protocol Label Switching (GMPLS)” RFC 4202, Oct.
on the efcacy of graceful restart to be evaluated. It was 2005.
found that, unsurprisingly, increasing control message loss and [9] K. Kompella (Ed.), Y. Rekhter (Ed.), ”OSPF Extensions in Support of
Generalized Multi-Protocol Label Switching (GMPLS)” RFC 4203, Oct.
increasing the number of connections increased the time to 2005.
complete recovery. It was also discovered that a threshold [10] J. Lang (Ed.) ”Link Management Protocol (LMP)” RFC 4204, Oct.
exists beyond which a relatively small change in the message 2005.
[11] G. Li, J. Yates, D. Wang, C. Kalmanek, ”Control Plane Design for
loss probability causes a disproportionately large increase in Reliable Optical Networks”, IEEE Communications Magazine, pp. 90-
the time to complete recovery. 96, February 2002.
The analytical model presented in this paper is viewed as [12] E. Mannie (Ed.), ”Generalized Multi-Protocol Label Switching (GM-
PLS) Architecture”, RFC 3945, Oct. 2004.
a rst step in the evaluation of RSVP-TE graceful restart. [13] A. Neogi, T. Chiueh, P. Stirpe, ”Performance Analysis of an RSVP-
Naturally, simplications had to made when developing the Capable Router”, IEEE Network, pp.56-63, Sept./Oct. 1999.
model, implying the results presented in this paper are likely [14] A. Satyanarayana (Ed.), R. Rahman (Ed.), ”Extensions to GMPLS RSVP
Graceful Restart”, draft-ietf-ccamp-rsvp-restart-ext-05.txt, Oct. 2005.
to be somewhat idealistic. Hence, an interesting avenue of [15] A. Shaikh, A. Varma, L. Kalampoukas, R. Dube, ”Routing Stability
future work is to enhance the analytical model, or to employ in Congested Networks: Experimentation and Analysis”, Proc. ACM
other performance evaluation techniques (e.g. simulation or SIGCOMM 2000.
2329