Stochastic Disturbance Accommodating Control

Using a Kalman Estimator

Jemin George,∗ Puneet Singla,† and John L. Crassidis‡

University at Buffalo, State University of New York, Amherst, NY, 14260-4400

Disturbance accommodating control theory provides a method for designing feedback

controllers which automatically detect and minimize the effect of waveform-structured dis-
turbances. This paper presents a stochastic disturbance accommodating controller which
utilizes a Kalman estimator to determine the necessary corrections to the nominal con-
trol input and thus minimizes the adverse effects of both model uncertainties and external
disturbances on the controlled system. Stochastic stability analysis conducted on the con-
trolled system reveals a lower-bound requirement on the estimator parameters to ensure
the stability of the closed-loop system when the nominal control action on the true plant
is unstable. Validity of the stability analysis is verified by implementing the proposed
technique on a two degree-of-freedom helicopter.

Nomenclature
(Ω, F , P) Complete probability space
E[·] Expectation operator
(A, B) True system state matrix and control distribution matrix, A ∈ Rn×n and B ∈ Rn×r
X(t, ω) ∈ Rn Stochastic state vector, t ∈ [t0 tf ] and ω ∈ Ω, for fixed t, X(t) is a random variable
Y(t, ω) ∈ Rm Stochastic output vector
u(t) ∈ Rr Input vector
W(t, ω) ∈ Rn External stochastic disturbance vector
V(t, ω) ∈ Rm


Measurement noise, assumed to be Gaussian white noise, fV (v) ∼ N 0, Rδ(τ )
(Am , Bm ) Assumed state matrix and assumed control distribution matrix, Am ∈ Rn×n and Bm ∈ Rn×r
Xm (t), Ym (t) State vector and output vector corresponding to the assumed system
C Known output matrix, C ∈ Rm×n
D(t, ω) ∈ Rn True lumped disturbance term
D m (t, ω) ∈ Rn Assumed lumped disturbance term
L1 (·), L2 (·) Linear mappings: Rn → Rn
V(t, ω) ∈ Rn


Zero mean Gaussian white noise process, fV (v) ∼ N 0, Qδ(τ )
x̄(t), ū(t) Nominal system states and control input, x̄(t) ∈ Rn , ū(t) ∈ Rr
Z(t) ∈ R2n , [XT (t) D T (t)]T , True augmented state vector
Zm (t) ∈ R2n , [XTm (t) D Tm (t)]T , Assumed augmented state vector
(F, D) True state matrix and control distribution matrix for the augmented system,
F ∈ R2n×2n and D ∈ R2n×r
ADm Assumed state matrix for disturbance term, ADm ∈ Rn×n
(Fm , Dm ) Assumed state matrix and control distribution matrix for the augmented system,
Fm ∈ R2n×2n and Dm ∈ R2n×r
W(t, ω) ∈ Rn


Zero mean Gaussian white noise process, fW (w) ∼ N 0, Qδ(τ )
(G, H) Known disturbance input matrix and output matrix for the augmented system,
G ∈ R2n×n and H ∈ Rm×2n
K(t) ∈ R2n×m Optimal observer gain or Kalman gain for the assumed system
∗ Graduate Student, Department of Mechanical & Aerospace Engineering, [email protected], Student Member AIAA.
† Assistant Professor, Department of Mechanical & Aerospace Engineering, [email protected], Member AIAA.
‡ Professor, Department of Mechanical & Aerospace Engineering, [email protected], Associate Fellow AIAA.

1 of 22

American Institute of Aeronautics and Astronautics

 P (t) ∈ R2n×2n Assumed estimation error covariance matrix
Km (t) ∈ Rr×n Nominal feedback gain
In×m An n × m identity matrix
0n×m An n × m zero matrix
ˆ Indicate estimated quantities
S Feedback gain for DAC
Z̃(t) , Ẑ(t) − Z(t), Estimation error
µ[·] , E[·]
T T
Z(t) , [Z̃ (t) Ẑ (t)]T , Appended vector
Υ(t) State matrix of Ż(t)
G(t) , [V T (t) VT (t)]T , Appended noise vector
Γ(t) Appended noise distribution matrix
Φ(t, t0 ) Evolution operator generated by Υ(t)
P(t) , E[Z(t)Z T (t)]
Λδ(τ ) , E[G(t)G T (t − τ )]
T T T
PZ̃ , PẐ , PZ̃Ẑ , E[Z̃(t)Z̃ (t)], E[Ẑ(t)Ẑ (t)], and E[Z̃(t)Ẑ (t)], respectively
¯ ¯
Ẑ(t), Z̃(t), Z̄(t) , Ẑ(t), Z̃(t), and Z(t) when there is no model uncertainties,
i.e., F = Fm , D = Dm , and V(t) = W(t)
Ῡ, Ḡ, P̄, Φ̄ , Υ(t), G(t), P(t), and Φ(t, t0 ) when there is no model uncertainties
∆Υ, Φ∆ , Υ(t) − Ῡ(t), and the evolution operator generated by ∆Υ, respectively
inf{·}, sup{·} Infimum, Supremum
σmax (·), σmin (·) Maximum and minimum singular values
k·k Appropriate 2-norm
L, N (t) , [GT 0T2n×n ]T , and [HP (t) − HP (t)]T , respectively
∗ ∗ ∗
Q ,R ,P Appropriate Q & R that would guarantee stability and the corresponding P
C2,1 Family of functions V (x, t) which are twice continuously differentiable in x and once in t.
class-K Family of continuous non-decreasing functions
R∞ κ(x) : R+ → R+ , κ(0) = 0, and κ(x) > 0 ∀x > 0
L1 Family of all functions η(t) such that 0 η(t)dt < ∞
Tr{·} Matrix trace

I. Introduction
System uncertainties and noisy measurements can obscure the development of a viable control law. The
main objective of a feedback controller design is to develop a compensator that will maintain given design
specifications in the presence of realistic ranges of uncertainty. A useful compensator that handles uncertainty
is Disturbance-Accommodating Control (DAC), which was first proposed by Johnson in 1971.1 Though the
traditional DAC only considers disturbance functions which exhibit waveform patterns over short intervals of
time,2 a more general formulation of DAC can accommodate the simultaneous presence of both “noise” type
disturbances and “waveform structured” disturbances.3
The disturbance-accommodating observer approach has shown to be extremely effective for disturbance
attenuation;4–6 however, the performance of the observer can significantly vary for different types of exoge-
nous disturbances, which is due to observer gain sensitivity. This paper presents a robust control approach
based on a significant extension of the disturbance accommodating control concept, which compensates for
both model parameter uncertainties and external disturbances by estimating a model-error vector (through-
out this paper we will use the phrase “disturbance term” to refer to this quantity) in real time that is used as
a signal synthesis adaptive correction to the nominal control input to achieve maximum performance. This
control approach utilizes a Kalman filter in the feedback loop for simultaneously estimating the system states
and the disturbance term from measurements.7 The estimated states are then used to develop a nominal
control law while the estimated disturbance term is used to make necessary corrections to the nominal control
input to minimize the effect of system uncertainties and the external disturbance. Similar developments of
disturbance accommodating controllers using Kalman filter can be found in Refs. 8 and 9. There are several
advantages of implementing the Kalman filter in the DAC approach: 1) tuning of the estimator parame-
ters, such as the process-noise covariance matrix, can be done easily unlike the standard DAC techniques

2 of 22

American Institute of Aeronautics and Astronautics

in which the adaptation involves the entire feedback gain, 2) the estimated disturbance term is a natural
byproduct of state estimation, and 3) the Kalman filter can also be used to filter noisy measurements. A
comparative study of the DAC approach to other adaptive techniques, such as the self-tuning regulator and
model-reference adaptive control is presented in Ref. 10. Although the disturbance accommodating observer
approach using a Kalman filter has been successfully implemented on linear time-invariant (LTI) systems
with both noise type and waveform structured disturbances, to the best knowledge of the present authors,
a comprehensive stochastic stability analysis has not been conducted before.
A detailed formulation of the stochastic disturbance accommodating controller for multi-input-multi-
output (MIMO) systems is given next. Afterwards, a stability analysis is conducted on the proposed control
scheme. The stochastic stability analysis indicates a lower-bound requirement on the assumed disturbance
term process noise matrix and the measurement noise matrix to guarantee exponential stability in the mean
sense when the nominal control action on the true plant would result in an unstable system. The stability
analysis also indicates that the controlled stochastic system is almost surely asymptotically stabile if the
noise distribution matrix satisfies a given decay rate. The results of the stability analysis are then verified
by implementing the proposed control scheme on a two degree-of-freedom helicopter. Finally, conclusions
and plans for future work are presented.

II. Controller Formulation

A detailed formulation of the DAC for LTI-MIMO systems is presented in this section. Throughout this
paper, random vectors are denoted using boldface capital letters and for convenience, the dependency of a
stochastic process on ω is not explicitly shown. Consider an nth -order system of the following form:
Ẋ(t) = AX(t) + Bu(t) + W(t), X(t0 ) = x0
(1)
Y(t) = CX(t) + V(t)
Here, the true state and control distribution matrices are assumed to be unknown. Also, the system is
assumed to be under-actuated, i.e., r < n. The external disturbance dynamics is
Ẇ(t) = L1 (X(t), u(t), W(t)) + V(t), W(t0 ) = 0 (2)
The assumed (known) system model is
Ẋm (t) = Am Xm (t) + Bm u(t), Xm (t0 ) = x0
(3)
Ym (t) = CXm (t) + V(t)
The external disturbance and the model uncertainties can be lumped into a disturbance term, D(t), through
D(t) = ∆AX(t) + ∆Bu(t) + W(t) (4)
where ∆A = (A − Am ) and ∆B = (B − Bm ). Using this disturbance term the true model can be written in
terms of the known system matrices as shown by
Ẋ(t) = Am X(t) + Bm u(t) + D(t)
(5)
Y(t) = CX(t) + V(t)
The control law, u(t), is selected so that the true system will track the reference model:
˙
x̄(t) = Am x̄(t) + Bm ū(t) (6)
The true system tracks the reference model if the following two conditions are satisfied:
x0 = x̄(t0 ) (7a)
Bm u(t) = Bm ū(t) − D(t) (7b)
where convergence is understood in the mean-square sense. The disturbance term is not known, but an
observer can be implemented in the feedback loop to estimate the disturbance term online. For this purpose,
system Eq. (1) is rewritten as the following extended dynamically equivalent system:
Ẋ(t) = Am X(t) + Bm u(t) + D(t)
(8)
Ḋ(t) = ∆AẊ(t) + ∆B u̇(t) + L1 (X(t), u(t), W(t)) + V(t) = L2 (X, D, u) + V

3 of 22

American Institute of Aeronautics and Astronautics

The extended system given in Eq. (8) can be written in state space form as
" # " #" # " # " #
Ẋ Am I(n×n) X Bm 0(n×1)
= + u+ (9)
Ḋ L2 X L2 D D L2 u V

where L"2 X , L# L2 u are partitions

2 D , and " # on L
" 2 (·) #that are acting
" on #X(t), D(t), and u(t), respectively. Let
X(t) Am I(n×n) Bm 0
Z(t) = , F = , D = , and G = n×n . Now the extended system given in
D(t) L2X L2D L2u In×n
Eq. (9) can be written as

Ż(t) = F Z(t) + Du(t) + GV(t), Z(t0 ) = [x0 D 0 ]T (10)

We do not have precise knowledge about the dynamics of the disturbance term. For simplicity, the distur-
bance term dynamics is modeled as

Ḋ m = ADm D m + W(t), Dm (t0 ) = 0 (11)

where ADm is Hurwitz. Equation (11) is used solely in the estimator design
" to estimate
# the true disturbance
Xm (t)
term. Now construct the assumed augmented state vector, Zm (t) = , the assumed model of the
D m (t)
system Eq. (9) can be written as
" # " #" # " # " #
Ẋm Am I(n×n) Xm Bm 0(n×1)
= + u+ (12)
Ḋ m 0(n×n) ADm Dm 0(n×r) W

The zero elements in the disturbance term dynamics are assumed for the sake of simplicity, the control
formulation given here is also valid if non-zero elements are assumed. Equation (12) can be written in terms
of the appended state vector, Zm , as

Żm (t) = Fm Zm (t) + Dm u(t) + GW(t), Zm (t0 ) = [x0 0]T (13)

" # " #
Am I(n×n) Bm
where Fm = and Dm = . Notice that the uncertainty is now only associated with
0(n×n) ADm 0(n×r)
the dynamics of the disturbance term. The assumed output equation can also be written in terms of the
appended state vector, Zm , as
h i
Ym (t) = C 0(m×n) Zm (t) + V(t) (14)

and the measured output equation can be written as

h i
Y(t) = C 0(m×n) Z(t) + V(t) (15)

Let H = [C 0m×n ], then Y = HZ + V and Ym = HZm + V. Though the disturbance term is unknown,
assuming W(t) and V(t) possess certain stochastic properties, an optimal estimator such as a Kalman filter
can be implemented in the feedback loop to estimate the unmeasured system states and the disturbance
term from the noisy measurements. The estimator dynamics can be written as
˙
Ẑ(t) = Fm Ẑ(t) + Dm u(t) + K(t)[Y(t) − Ŷ(t)], Ẑ(t0 ) = Zm (t0 ) (16)

where K(t) is the Kalman gain and Ŷ = H Ẑ. The estimator dynamics can be rewritten as
˙
Ẑ(t) = Fm Ẑ(t) + Dm u(t) + K(t)H[Z(t) − Ẑ(t)] + K(t)V(t) (17)

Notice that the estimator uses the assumed system model given in Eq. (13) for the propagation stage and
the actual measurements from Eq. (15) for the update stage, i.e., Ẑ(t) = E[Zm (t)|{Yt . . . Y0 }]. The Kalman

4 of 22

American Institute of Aeronautics and Astronautics

gain can be calculated as K(t) = P (t)H T R−1 , where P (t) = E (Zm (t) − Ẑ(t))(Zm (t) − Ẑ(t))T can be
 

obtained by solving the continuous-time matrix differential Riccati equation:11

Ṗ (t) = Fm P (t) + P (t)Fm T − P (t)H T R−1 HP (t) + GQGT (18)

The total control law, u(t), consists of a nominal control and necessary corrections to the nominal control
to compensate for the disturbance term as shown in Eq. (7b). The nominal control, ū, is selected so that
it guarantees the desired performance of the assumed system. For the system given in Eq. (5), the nominal
controller is given as
ū(t) = −Km X̂(t) (19)
where Km ∈ Rr×n is the feedback gain. While the nominal controller guarantees the desired performance of
the assumed model, the second term, −D(t), in Eq. (7b) ensures the complete cancelation of the disturbance
term which is compensating for the external disturbance and the model uncertainties. Now the control law
can be written in terms of the estimated system states and the estimated disturbance term as
" #
h i X̂(t)
T −1 T
u(t) = (Bm Bm ) Bm − Bm Km − I(n×n) = S Ẑ(t) (20)
D̂(t)
h i
T
where S = (Bm Bm )−1 BmT
−Bm Km −I . Notice that (Bm T
Bm ) is a nonsingular matrix since Bm is assumed
to have linearly independent columns. A summary of the proposed control scheme is given Table. 1.

Table 1. Summary of Overall Control Process

Plant Ż(t) = F Z(t) + Du(t) + GV(t)

Y(t) = HZ(t) + V(t)

Initialize Ẑ(t0 ), P (t0 )

Observer Gain Ṗ (t) = Fm P (t) + P (t)Fm T − P (t)H T R−1 HP (t) + GQGT

K(t) = P (t)H T R−1

˙
Estimate Ẑ(t) = Fm Ẑ(t) + Dm u(t) + K(t)[Y(t) − Ŷ(t)]
h i
T
Control Synthesis u(t) = (Bm Bm )−1 Bm
T
− Bm Km − I Ẑ(t)

It is important to note that if Q = 0, then D m (t) = D m (t0 ) = 0 and the total control law becomes just
the nominal control. If the nominal control, ū(t), on the true plant would result in an unstable system,
then selecting a small Q would also result in an unstable system. On the other hand, selecting a large Q
value would compel the estimator to completely rely upon the measurement signal and therefore the noise
associated with the measurement signal is directly transmitted into the estimates. This could result in noisy
control signal which could lead to problems, such as chattering and controller saturation. Also note that as
R, the measurement noise covariance, increases, the observer gain decreases and thus the observer fails to
update the propagated disturbance term based on measurements. For a highly uncertain system, selecting
a small Q or a large R will result in an unstable closed-loop system as shown in Ref. 12. A schematic
representation of the proposed controller is given in Fig. 1. In the next section a detailed stability analysis
is given, which investigates the dependency of closed-loop system stability on Q and R.

III. Stability Analysis

Notice that P (t) given in Eq. (18) is not the estimation error covariance, a detailed derivation of the
true error covariance is considered first. Closed-loop system stability based on the formulation of the true

5 of 22

American Institute of Aeronautics and Astronautics

 W(t) V(t)
Ref. Signal
ū(t) u(t) Y(t)
Nominal Controller + Plant +
X̂(t)

D̂(t) T
−(Bm Bm )−1 Bm
T

Estimator

Figure 1. DAC Block Diagram

error covariance is then presented. Finally it is shown that the system stability depends on a lower bound
requirement on Q and R−1 .

A. Estimation Error Covariance

Substituting the control law, Eq. (20), into the plant dynamics, Eq. (10), the true system can be written as

Ż(t) = F Z(t) + DS Ẑ(t) + GV(t)

(21)
Y(t) = HZ(t) + V(t)

The estimator dynamics can be written as

˙
Ẑ(t) = Fm Ẑ(t) + Dm S Ẑ(t) + K(t)H[Z(t) − Ẑ(t)] + K(t)V(t) (22)

From hereon the explicit notation for time varying quantities is omitted when there is no risk of confusion.
Let Z̃ = Z − Ẑ be the estimation error, then the error dynamics can be written as
˙
Z̃˙ = Ż − Ẑ = F Z + DS Ẑ + GV − Fm Ẑ − Dm S Ẑ − KH[Z − Ẑ] − KV
= [Fm − KH + ∆F ]Z̃ + [∆F + ∆DS]Ẑ + GV − KV

where △F = F − Fm and △D = D − Dm . Let µZ̃ = E[Z̃], and µẐ = E[Ẑ], i.e.,

˙ = E (F − KH + ∆F )Z̃ + (∆F + ∆DS)Ẑ + GV − KV

µ̇Z̃ = E[Z̃] m
= (Fm − KH + ∆F )µZ̃ + (∆F + ∆DS)µẐ

Now µ̇Ẑ can be written as

µ̇Ẑ = (Fm + Dm S)µẐ + KHµZ̃

Combining the error dynamics and the estimator dynamics we could write,

Z̃˙
" # " #" # " #" #
(Fm − KH + ∆F ) (∆F + ∆DS) Z̃ G −K V
˙ = KH (Fm + Dm S) Ẑ
+
0 K V
Ẑ

or in a more compact form as

Ż(t) = Υ(t)Z(t) + Γ(t)G(t) (23)

6 of 22

American Institute of Aeronautics and Astronautics

 " # " # " #
Z̃(t) (Fm − K(t)H + ∆F ) (∆F + ∆DS) G −K(t)
where Z(t) = , Υ(t) = , Γ(t) = , and G(t) =
Ẑ(t) K(t)H (Fm + Dm S) 0 K(t)
" #
ϕ(t)
. The solution of above equation can be written as
v(t)

Zt
Z(t) = Φ(t, t0 )Z(t0 ) + Φ(t, τ )Γ(τ )G(τ )dτ (24)
t0

Let P(t) ≡ E[Z(t)Z T (t)], i.e.,

h Zt
P(t) = E Φ(t, t0 )Z(t0 )Z (t0 )Φ (t, t0 ) + Φ(t, t0 )Z(t0 )G T (τ )ΓT (τ )ΦT (t, τ )dτ +
T T

t0
Zt Zt Zt i
T T
Φ(t, τ )Γ(τ )G(τ )Z (t0 )Φ (t, t0 )dτ + Φ(t, τ1 )Γ(τ1 )G(τ1 )G T (τ2 )ΓT (τ2 )ΦT (t, τ2 )dτ1 dτ2
t0 t0 t0

Assuming G(t) and Z(t0 ) are uncorrelated we have E[G(t)Z T (t0 )] = E[Z(t0 )G T (t)] = 0. The initial P is
P(t0 ) = E[Z(t0 )Z T (t0 )]. Since V(t) and V(t) are uncorrelated, the expectation of G(τ1 )G T (τ2 ) is
" #
T Q 0
E[G(τ1 )G (τ2 )] = δ(τ1 − τ2 )
0 R
" #
Q 0
Let Λ = , now P(t) can be rewritten as
0 R

Zt
T
P(t) = Φ(t, t0 )P(t0 )Φ (t, t0 ) + Φ(t, τ )Γ(τ )ΛΓT (τ )ΦT (t, τ )dτ (25)
t0

Taking the time derivative of the above equation results in

∂Φ(t, t0 ) ∂ΦT (t, t0 )

Ṗ(t) = P(t0 )ΦT (t, t0 ) + Φ(t, t0 )P(t0 ) + Φ(t, t)Γ(t)ΛΓT (t)ΦT (t, t)+
∂t ∂t
Zt Zt
∂Φ(t, τ ) ∂ΦT (t, τ )
Γ(τ )ΛΓ (τ )Φ (t, τ )dτ + Φ(t, τ )Γ(τ )ΛΓT (τ )
T T
dτ
∂t ∂t
t0 t0

Utilizing the fundamental properties of the evolution operator, the above equation can be rewritten as

Ṗ(t) =Υ(t)Φ(t, t0 )P(t0 )ΦT (t, t0 ) + Φ(t, t0 )P(t0 )ΦT (t, t0 )ΥT (t) + Γ(t)ΛΓT (t)+
Zt Zt
Υ(t) Φ(t, τ )Γ(τ )ΛΓ (τ )Φ (t, τ )dτ + Φ(t, τ )Γ(τ )ΛΓT (τ )ΦT (t, τ ))dτ ΥT (t)
T T

t0 t0

Therefore

Ṗ(t) = Υ(t)P(t) + P(t)ΥT (t) + Γ(t)ΛΓT (t) (26)

Let
T T
" # " #
E[Z̃Z̃ ] E[Z̃Ẑ ]
PZ̃ PZ̃Ẑ
P(t) = T T =
E[ẐZ̃ ] E[ẐẐ ] PZ̃Ẑ PẐ

7 of 22

American Institute of Aeronautics and Astronautics

Now Ṗ(t) can be rewritten as
" # " #" #
ṖZ̃ ṖZ̃Ẑ (Fm − KH + ∆F ) (∆F + ∆DS) PZ̃ PZ̃Ẑ
= +
ṖZ̃Ẑ ṖẐ KH (Fm + Dm S) PZ̃Ẑ PẐ
" #" # " #
PZ̃ PZ̃Ẑ (Fm − KH + ∆F )T (KH)T (GQGT + KRK T ) −KRK T
+
PZ̃Ẑ PẐ (∆F + ∆DS)T (Fm + Dm S)T −KRK T KRK T

From the above equation, ṖZ̃ , ṖZ̃Ẑ , and ṖẐ can be written as

ṖZ̃ =(Fm − KH + ∆F )PZ̃ + (∆F + ∆DS)PZ̃Ẑ + PZ̃ (Fm − KH + ∆F )T +

(27)
PZ̃Ẑ (∆F + ∆DS)T + GQGT + KRK T
ṖZ̃Ẑ =(Fm − KH + ∆F )PZ̃Ẑ + (∆F + ∆DS)PẐ + PZ̃ (KH)T + PZ̃Ẑ (Fm + Dm S)T − KRK T (28)
T T T
ṖẐ =(KH)PZ̃Ẑ + (Fm + Dm S)PẐ + PZ̃Ẑ (KH) + PẐ (Fm + Dm S) + KRK (29)

Thus the true estimation error covariance is

h i
E (Z̃(t) − µZ̃ (t))(Z̃(t) − µZ̃ (t))T = PZ̃ (t) − µZ̃ (t)µTZ̃ (t) (30)

Since the model errors are unknown, the above equation cannot be utilized in the filter implementation.

B. Closed-Loop Stability and Transient Response Bound for Systems with No Uncertainties
A detailed analysis of the closed-loop system’s asymptotic stability in the mean when there are no uncer-
tainties is now given. As shown here, a transient bound on the system response mean can be obtained in
terms of the time varying correlation matrix. Most of the definitions and formulations given in this section
are similar to the ones given in Ref. 13 for deterministic systems.

Consider a case where there is no model error, i.e., F = Fm , D = Dm , and V(t) = W(t). If there is no
¯ = µ = 0. Now we could write
model error, then the estimator is unbiased, i.e., E[Z̃] ¯
Z̃
  "
¯˙ ¯
#" # " #" #
Z̃
 = (Fm − KH) 0 Z̃ G −K W
˙¯ KH (F + D S) ¯ + 0 K V
Ẑ m m Ẑ

or in a more compact form as

˙
Z̄(t) = Ῡ(t)Z̄(t) + Γ(t)Ḡ(t) (31)

Before discussing the stability analysis, a few definitions regarding the stability of stochastic processes are
presented.
˙
Definition 1. Given M ≥ 1 and β ∈ R, the system Z̄(t) = Ῡ(t)Z̄(t) + Γ(t)Ḡ(t) is said to be (M, β)-stable
in the mean if

k Φ̄(t, t0 )µZ̄ (t0 ) k≤ M eβ(t−t0 ) k µZ̄ (t0 ) k (32)

where Φ̄(t, t0 ) is the evolution operator generated by Ῡ(t) and µZ̄ (t) = E[Z̄(t)].
Since most applications involve the case where β ≤ 0, (M, β)-stability guarantees both a specific decay
rate of the mean (given by β) and a specific bound on the transient behavior of the mean (given by M ).
˙
Definition 2. If a stochastic system, Z̄(t) = Ῡ(t)Z̄(t) + Γ(t)Ḡ(t), is (M, β)-stable in the mean, then the
transient bound of the system mean response for the exponential rate β is defined to be
n o
Mβ = inf M ∈ R; ∀t ≥ t0 :k Φ̄(t, t0 ) k≤ M eβ(t−t0 ) (33)

8 of 22

American Institute of Aeronautics and Astronautics

 The optimal transient bound Mβ = 1 can be achieved by choosing a sufficiently large β, i.e.,
Zt R t
kῩ(τ )kdτ
β(t − t0 ) ≥ k Ῡ(τ ) k dτ =⇒ k Φ̄(t, t0 ) k≤ e t0
≤ eβ(t−t0 ) , t ≥ t0
t0

Therefore it is of interest to know the smallest β ∈ R such that k Φ̄(t, t0 ) k≤ eβ(t−t0 ) , t ≥ t0 . Given the
˙
system, Z̄(t) = Ῡ(t)Z̄(t) + Γ(t)Ḡ(t), which is (M, β)-stable in the mean, the transient bound Mβ of the
system mean can be readily obtained based on the premises of the following theorem.

Theorem 1. Suppose the system Z̄(t)˙ = Ῡ(t)Z̄(t) + Γ(t)Ḡ(t) is (M, β)-stable in the mean, then there exists
T
a continuously differentiable positive definite matrix function P̄(t) (P̄ = E[Z̄ Z̄ ]) satisfying the matrix
Lyapunov differential equation
˙
P̄(t) = Ῡ(t)P̄(t) + P̄(t)ῩT (t) + Γ(t)Λ̄ΓT (t) (34)

such that

Mβ2 ≤ sup σmax (P̄(t))/σmin (P̄(t0 )) (35)

t≥t0

Proof. Solution to Eq. (34) can be written as

Zt
T
P̄(t) = Φ̄(t, t0 )P̄(t0 )Φ̄ (t, t0 ) + Φ̄(t, τ )Γ(τ )Λ̄ΓT (τ )Φ̄T (t, τ )dτ
t0

Notice P̄(t) ≥ Φ̄(t, t0 )P̄(t0 )Φ̄T (t, t0 ) ≥ σmin (P̄(t0 ))Φ̄(t, t0 )Φ̄T (t, t0 ), i.e.,

σmax (P̄(t)) ≥k Φ̄(t, t0 )P̄(t0 )Φ̄T (t, t0 ) k≥ σmin (P̄(t0 )) k Φ̄(t, t0 ) k2

Now Eq. (35) follows from

σmax (P̄(t))/σmin (P̄(t0 )) ≥k Φ̄(t, t0 ) k2

C. Closed-Loop Stability and Transient Response Bound for Uncertain Systems

Consider a scenario where model error is present, i.e.,

Z̃˙
" # " #" # " #" #
(Fm − KH + ∆F ) (∆F + ∆DS) Z̃ G −K V
˙ = KH (Fm + Dm S) Ẑ
+
0 K V
Ẑ

or in a more compact form as

Ż(t) = Ῡ(t)Z(t) + ∆Υ(t)Z(t) + Γ(t)G(t) (36)

where
" #
(∆F ) (∆F + ∆DS)
∆Υ(t) =
0 0

˙
In the previous section we analyzed the stability of the unperturbed system Z̄(t) = Ῡ(t)Z̄(t) + Γ(t)Ḡ(t).
Here we will analyze the stability of the perturbed system, Ż(t) = Ῡ(t)Z(t) + ∆Υ(t)Z(t) + Γ(t)G(t).

The correlation matrix P(t) = E[Z(t)Z T (t)] satisfies the following matrix Lyapunov differential equation

Ṗ(t) = (Ῡ(t) + ∆Υ(t))P(t) + P(t)(Ῡ(t) + ∆Υ(t))T + Γ(t)ΛΓT (t) (37)

9 of 22

American Institute of Aeronautics and Astronautics

Note that Γ(t)ΛΓT (t) can be factored as shown below:
" # " # " #
T (GQGT + KRK T ) −KRK T (GQGT ) −0 (KRK T ) −KRK T
Γ(t)ΛΓ (t) = = +
−KRK T KRK T 0 0 −KRK T KRK T
" # " #
G h i P HT −1
h i
= Q GT 0 + R HP −HP = LQLT + N (t)R−1 N T (t)
0 −P H T

Theorem 2. The uncertain system, Ż(t) = Ῡ(t)Z(t) + ∆Υ(t)Z(t) + Γ(t)G(t), is (M, β)-stable in the mean
if

k ∆Υ(t)P̄(t) k2 ≤ σmin (Q)σmin (R−1 ) k L k2 k N (t) k2 (38)

where P̄(t) satisfying

˙
P̄(t) = Ῡ(t)P̄(t) + P̄(t)ῩT (t) + LQLT + N (t)R−1 N (t)T (39)

Proof. In order to show the asymptotic stability of the mean we consider the following equation:

µ̇Z (t) = Ῡ(t)µZ (t) + ∆Υ(t)µZ (t)

Construct the following Lyapunov candidate function:

V [µZ (t)] = µTZ (t)P̄ −1 (t)µZ (t) (40)

The matrix P̄(t) is required to be a positive definite matrix, therefore P̄ −1 (t) exists and V [µZ (t)] > 0 for
all µZ (t) 6= 0. Since P̄(t)P̄ −1 (t) = I, the time derivative of P̄(t)P̄ −1 (t) is 0:
dh i
˙ P̄ −1 (t) + P̄(t)P̄˙ −1 (t) = 0
P̄(t)P̄ −1 (t) = P̄(t)
dt

Solving the above equation for P̄˙ −1 (t) and substituting Eq. (39) gives

P̄˙ −1 (t) = −P̄ −1 (t)P̄(t)

˙ P̄ −1 (t)
(41)
= −P̄ −1 (t)Ῡ(t) − ῩT (t)P̄ −1 (t) − P̄ −1 (t)LQLT P̄ −1 (t) − P̄ −1 (t)N (t)R−1 N (t)T P̄ −1 (t)

Now the time derivative of Eq. (40) can be written as

V̇ [µZ (t)] =µ̇TZ P̄ −1 µZ + µTZ P̄˙ −1 µZ + µTZ P̄ −1 µ̇Z

=[ῩµZ + ∆ΥµZ ]T P̄ −1 µZ − µTZ P̄ −1 ῩµZ − µTZ ῩT P̄ −1 µZ −
µTZ P̄ −1 LQLT P̄ −1 µZ − µTZ P̄ −1 N R−1 N T P̄ −1 µZ + µTZ P̄ −1 [ῩµZ + ∆ΥµZ ]
=µTZ ∆ΥT P̄ −1 µZ + µTZ P̄ −1 ∆ΥµZ − µTZ P̄ −1 LQLT P̄ −1 µZ − µTZ P̄ −1 N R−1 N T P̄ −1 µZ
n o
=µTZ ∆ΥT P̄ −1 + P̄ −1 ∆Υ − P̄ −1 LQLT P̄ −1 − P̄ −1 N R−1 N T P̄ −1 µZ

We have asymptotic stability in the mean if

n o
− P̄∆ΥT − ∆ΥP̄ + LQLT + N R−1 N T > 0

Note
−∆ΥP̄ − P̄∆ΥT +LQLT + N R−1 N T = LQLT − P̄∆ΥT (N R−1 N T )−1 ∆ΥP̄+
h i h iT
P̄∆ΥT (N R−1 N T )−1 − I (N R−1 N T ) P̄∆ΥT (N R−1 N T )−1 − I

Now we need to show

LQLT ≥ P̄∆ΥT (N R−1 N T )−1 ∆ΥP̄

10 of 22

American Institute of Aeronautics and Astronautics

Note the following:

σmin (Q)LLT ≤ LQLT

σmin (R−1 )N N T ≤ N R−1 N T ⇒ P̄∆ΥT (σmin (R−1 )N N T )−1 ∆ΥP̄ ≥ P̄∆ΥT (N R−1 N T )−1 ∆ΥP̄

Thus now we need to show

LQLT ≥ σmin (Q)LLT ≥ P̄∆ΥT (σmin (R−1 )N N T )−1 ∆ΥP̄ ≥ P̄∆ΥT (N R−1 N T )−1 ∆ΥP̄

σmin (Q)σmin (R−1 ) k LLT k≥k P̄∆ΥT (N N T )−1 ∆ΥP̄ k

k P̄∆ΥT kk (N N T ) k−1 k ∆ΥP̄ k≥k P̄∆ΥT (N N T )−1 ∆ΥP̄ k

Hence we have

σmin (Q)σmin (R−1 ) k L k2 k N k2 ≥k ∆ΥP̄ k2 (42)

Therefore (M, β)-stability in the mean is guaranteed if the inequality Eq. (38) is satisfied. Let Q∗ and
R is chosen so that the above inequality is satisfied. Now substituting Q∗ and R∗ into Eq. (37) we have
∗

P˙∗ (t) = (Ῡ(t) + ∆Υ(t))P ∗ (t) + P ∗ (t)(Ῡ(t) + ∆Υ(t))T + LQ∗ LT + N (t)R∗ −1 N T (t) (43)

The solution of the above equation is

P ∗ (t) =[Φ̄(t, t0 ) + Φ∆ (t, t0 )]P ∗ (t0 )[Φ̄(t, t0 ) + Φ∆ (t, t0 )]T +

Zt
(44)
[Φ̄(t, τ ) + Φ∆ (t, τ )]{LQ∗ LT + N (τ )R∗ −1 N T (τ )}[Φ̄(t, τ ) + Φ∆ (t, τ )]T dτ
t0

Corollary 1. If the system given in Eq. (36) is (M, β)-stable in the mean, then there exists a continuously
differentiable positive definite symmetric matrix function P ∗ (t) given by Eq. (44) such that

Mβ2 ≤ sup σmax (P ∗ (t))/σmin (P ∗ (t0 )) (45)

t≥t0

where Mβ represents the transient bound of the perturbed system’s mean response.
Proof. If P ∗ (t) satisfies Eq. (44), then

P ∗ (t) ≥ [Φ̄(t, t0 )+Φ∆ (t, t0 )]P ∗ (t0 )[Φ̄(t, t0 )+Φ∆ (t, t0 )]T ≥ σmin (P ∗ (t0 ))[Φ̄(t, t0 )+Φ∆ (t, t0 )][Φ̄(t, t0 )+Φ∆ (t, t0 )]T

i.e.,

σmax (P ∗ (t)) ≥k [Φ̄(t, t0 ) + Φ∆ (t, t0 )]P ∗ (t0 )[Φ̄(t, t0 ) + Φ∆ (t, t0 )]T k≥ σmin (P ∗ (t0 )) k [Φ̄(t, t0 ) + Φ∆ (t, t0 )] k2

Now we have

σmax (P ∗ (t))/σmin (P ∗ (t0 )) ≥k Φ̄(t, t0 ) + Φ∆ (t, t0 ) k2

Therefore the transient bound, Mβ2 , of the perturbed system can be obtained from

Mβ2 ≤ sup σmax (P ∗ (t))/σmin (P ∗ (t0 ))

t≥t0

11 of 22

American Institute of Aeronautics and Astronautics

D. Mean Square Stability
Previously we analyzed stability in the mean. Here, it is shown that the (M, β)-stability in the mean implies
mean square stability. More details on mean square stability can be found in Refs. 14 and 15.
Definition 3. A stochastic system of the following form Ż(t) = Υ(t)Z(t) + Γ(t)G(t) is mean square stable
if
lim E[Z(t)Z T (t)] < C (46)
t→∞

where C is a constant square matrix whose elements are finite.

Note that
d
E[Z(t)Z T (t)] = Ṗ(t) = Υ(t)P(t) + P(t)ΥT (t) + Γ(t)ΛΓT (t)
dt
and the solution to the above equation can be written as
Zt
P(t) = Φ(t, τ )Γ(τ )ΛΓT (τ )ΦT (t, τ )dτ
−∞

The exponentially stable in the mean implies the system matrix, Υ(t) = Ῡ(t) + ∆Υ(t), generates a stable
evolution operator, therefore P(t) has a bounded solution.16

E. Almost Sure Asymptotic Stability

Solution to the stochastic system given in Eq. (36) cannot be based on the ordinary mean square calculus
because the integral involved in the solution depends on G(t), which is of unbounded variation. For the
treatment of this class of problems, the stochastic differential equation can be rewritten in Itô form as17
dZ(t) = [Ῡ(t)Z(t) + ∆Υ(t)Z(t)]dt + Γ(t)Λ1/2 dB(t)
or simply as
dZ(t) = Υ(t)Z(t)dt + Γ(t)Λ1/2 dB(t) (47)
where dB(t) is an increment of Brownian motion process with zero-mean, Gaussian distribution and covari-
ance
E[dB(t)dB T (t)] = Idt (48)
The solution Z(t) of Eq. (47) is a semimartingale process that is also Markov.18
Definition 4. The linear stochastic system given in Eq. (47) is asymptotically stable with probability 1, or
almost surely asymptotically stable, if


P Z(t) → 0 as t → ∞ = 1 (49)
Given below is the well-known classical result on the global asymptotic stability for stochastic sys-
tems:14, 19
Theorem 3. Assume that there are functions V (z, t) ∈ C2,1 and κ1 , κ2 , κ3 ∈ class-K such that
κ1 (k z k) ≤ V (z, t) ≤ κ2 (k z k) (50a)
LV (z, t) ≤ −κ3 (k z k) (50b)
for all z, t ∈ R4n × R+ , where z indicate a sample path of Z(t, ω), i.e., z(t) = Z(t, ωı ) |ωı ∈Ω . Then, for

every initial valuse Z 0 , the solution of Eq. (47) has the property that
Z(t) → 0 almost surely as t → ∞ (51)
The operator L{·} acting on V (z, t) is given by
1  
LV (z, t) = lim E dV (Z(t), t)|Z(t) = z (52)
dt→0 dt
where dV (Z(t), t) can be calculated using the Itô Formula.

12 of 22

American Institute of Aeronautics and Astronautics

 If is often very difficult to show the negative definiteness of LV (z, t). One way to get around this problem
is to replace the condition given in Eq. (50b) with two weaker conditions.
Theorem 4. Assume that there are functions V (z, t) ∈ C2,1 , κ1 , κ2 , κ3 ∈ class-K, and η(t) ∈ L1 such that

κ1 (k z k) ≤ V (z, t) ≤ κ2 (k z k) (53a)
LV (z, t) ≤ η(t) and (53b)
LV (z, t) ≤ η(t)+ k VzT (z, t)Γ(t) k2 −κ3 (k z k) (53c)
∂V
where Vz = ∂z . Then the conclusion of Theorem 3 still holds.
More detailed derivation and proof of this theorem can be found in Ref. 20. Notice that if the inequality
Eq. (38) is satisfied, then there exists a P(t) which satisfies the following equation:

Ṗ(t) = Υ(t)P(t) + P(t)ΥT (t) + Γ(t)ΛΓT (t)

n o
z T (t)P −1 (t)z(t)
Consider the function V (z, P −1 ) = Msup , where Msup = sup∞>τ ≥t0 Tr P −1 (τ )Γ(τ )ΛΓT (τ ) . Now
dV (Z(t), P −1 (t)) can be written as

dV (Z(t), P −1 (t)) = V (Z(t) + dZ(t), P −1 (t) + dP −1 (t)) − V (Z(t), P −1 (t))

Using a Taylor series up to second order, we have

n  ∂V T o n  ∂V o
V (Z(t) + dZ(t), P −1 (t) + dP −1 (t)) ≈ V (Z(t), P −1 (t)) + Tr dZ +Tr dP −1 +
∂Z ∂P −1
2
1 n  ∂ V o
Tr dZdZ T
2 ∂Z∂Z T
here dZ is given in Eq.(47) and dP −1 is
n o
dP −1 = −P −1 Υ − ΥT P −1 − P −1 ΓΛΓT P̄ −1 dt

The partials are

∂V 2 ∂V 1 ∂2V 2
= P −1 Z, = ZZ T , and T
= P −1
∂Z Msup ∂P −1 Msup ∂Z∂Z M sup

Now we have
2 1 1 n o
dV (Z(t), P −1 (t)) ≈ Z T P −1 dZ + Z T dP −1 Z + Tr P −1 dZdZ T
Msup Msup Msup
2 2 1 n o
≈ Z T P −1 ΥZdt + Z T P −1 ΓΛ1/2 dB − Z T P −1 Υ + ΥT P −1 + P −1 ΓΛΓT P −1 Zdt+
Msup Msup Msup
1 n  o
Tr P −1 ΥZZ ΥT dt2 + ΓΛ1/2 dBZ T ΥT dt + ΥZdB T Λ1/2 ΓT dt + ΓΛ1/2 dBdB T Λ1/2 ΓT
T
Msup
Now taking the conditional expectation, we obtain
2 1 n  o
E dV (Z(t), P −1 (t))|Z(t) = z = z T P −1 Υzdt + Tr P −1 Υzz T ΥT dt2 + ΓΛΓT dt −
 
Msup Msup
1 n o
z T P −1 Υ + ΥT P −1 + P −1 ΓΛΓT P −1 zdt
Msup
Now we can calculate
2 1 n o 1 n o
LV (z, P −1 ) = z T P −1 Υz − z T P −1 Υ + ΥT P −1 + P −1 ΓΛΓT P −1 z + Tr P −1 ΓΛΓT
Msup Msup Msup
1 1 n o
=− z T P −1 ΓΛΓT P −1 z + Tr P −1 ΓΛΓT
Msup Msup

13 of 22

American Institute of Aeronautics and Astronautics

 n o
Notice Msup ≥ Tr P −1 (t)Γ(t)ΛΓT (t) , ∀t ≥ t0 , i.e.,

σmin {P −1 ΓΛΓT P −1 }
LV (z, P −1 ) ≤ 1 − k z k2
Msup

σmin {P −1 (t)Γ(t)ΛΓT (t)P −1 (t)}

Let k(t) = Msup , thus LV (z, P −1 ) ≤ 1 − k(t) k z(t) k2 .

Notice that
1
lim k z(t) k2 ≤ =⇒ 1 − k k z k2  η(t)
t→∞ k(t)
Thus we do not have almost sure asymptotic stability for the stochastic system given in Eq. (47). In fact,
given a Υ(t) that generates an asymptotically stable evolution for the linear system in Eq. (47), the necessary
and sufficent condition for the almost sure asymptotic stability is

lim k Γ(t) k2 log(t) = 0 (54)

t→∞

Detailed proof of this argument can be found in Ref. 21. Equation (54) constitutes the sufficent condition
for the almost sure asymptotic stability of a linear stochastic system gievn (M, β)-stability in the mean.

IV. Results
A detailed investigation of the above Lyapunov stability analysis through numerical simulations is given
in this section. For simulation purposes, we consider a two degree of freedom helicopter that pivots about
the pitch axis by angle θ and about the yaw axis by angle ψ. As shown in Fig. 2, the pitch is defined positive
when the nose of the helicopter goes up and the yaw is defined positive for a counterclockwise rotation. Also
in Fig. 2, there is a thrust force Fp acting on the pitch axis that is normal to the plane of the front propeller
and a thrust force Fy acting on the yaw axis that is normal to the rear propeller. Therefore a pitch torque
is being applied at a distance rp from the pitch axis and a yaw torque is applied at a distance ry from the
yaw axis. The gravitational force, Fg , generates a torque at the helicopter center of mass that pulls down
on the helicopter nose. As shown in Fig. 2, the center of mass is a distance of lcm from the pitch axis along
the helicopter body length.

Figure 2. Two Degree of Freedom Helicopter

The helicopter equations of motion can be written as

2 2
(Jeq,p + mheli lcm )θ̈ = Kpp Vm,p + Kpy Vm,y − Bp θ̇ − mheli glcm cos(θ) − mheli lcm cos(θ)sin(θ)ψ̇ 2 (55a)
2
(Jeq,y + mheli lcm cos(θ)2 )ψ̈ = Kyy Vm,y + Kyp Vm,p − By ψ̇ + 2
2mheli lcm cos(θ)sin(θ)ψ̇ θ̇ (55b)

14 of 22

American Institute of Aeronautics and Astronautics

After linearizing about θ0 = ψ0 = θ̇0 = ψ̇0 = 0, the helicopter equations of motion can be written as
2
(Jeq,p + mheli lcm )θ̈ = Kpp Vm,p + Kpy Vm,y − Bp θ̇ − mheli glcm (56a)
2
(Jeq,y + mheli lcm )ψ̈ = Kyy Vm,y + Kyp Vm,p − By ψ̇ (56b)
A detailed description of system parameters and assumed values are given in Table 2. Note that the negative
viscous damping about the yaw axis is purposefully selected to ensure that the nominal control on the true
plant is unstable.

Table 2. Two Degree-of-Freedom Helicopter Model Parameters

System Assumed True

Parameter Description Values Values Unit
Bp Equivalent viscous damping about pitch axis 0.8 1 N/V
By Equivalent viscous damping about yaw axis 0.318 -0.3021 N/V
Jeq,p Total moment of inertia about yaw pivot 0.0384 0.0288 Kg·m2
Jeq,y Total moment of inertia about pitch pivot 0.0432 0.0496 Kg·m2
Kpp Trust torque constant acting on pitch axis
from pitch motor/propeller 0.204 0.2552 N ·m/V
Kpy Trust torque constant acting on pitch axis
from yaw motor/propeller 0.0068 0.0051 N ·m/V
Kyp Trust torque constant acting on yaw axis
from pitch motor/propeller 0.0219 0.0252 N ·m/V
Kyy Trust torque constant acting on yaw axis
from yaw motor/propeller 0.072 0.0684 N ·m/V
mheli Total mass of the helicopter 1.3872 1.3872 Kg
lcm Location of center-of-mass along helicopter body 0.186 0.176 m

The control input to the system are the input voltages of the pitch and yaw motors, Vm,p and Vm,y ,
respectively. Let u = [u1 u2 ]T = [Vm,p Vm,y ]T . Now the linearized equations can be rewritten as
θ̈ = a1 θ̇ + b1 u1 + b2 u2 − mheli glcm (57a)
ψ̈ = a2 ψ̇ + b3 u1 + b4 u2 (57b)
where
−Bp −By
a1 = 2 )
a2 = 2 )
(Jeq,p + mheli lcm (Jeq,y + mheli lcm
Kpp Kpy
b1 = 2 )
b2 = 2 )
(Jeq,p + mheli lcm (Jeq,p + mheli lcm
Kyp Kyy
b3 = 2 )
b4 = 2 )
(Jeq,y + mheli lcm (Jeq,y + mheli lcm

Let x = [θ ψ θ̇ ψ̇]T . Now the state-space representation of the above system is

ẋ = Ax + Bu + w (58)
     
0 0 1 0 0 0 0
0 0 0 1 0 0 0
     
where A =  , B =  , and w =  . The state-space representation of the
 
 0 0 a1 0  b1 b2  −mheli glcm 
0 0 0 a2 b3 b4 0
assumed system model is
ẋm = Am xm + Bm u (59)

15 of 22

American Institute of Aeronautics and Astronautics

    
0 0 1 0 0 0
0 0 0 1  0 0 
   
where Am =   and Bm = . The measured output and the assumed output

0 0 a1m 0  b1m b2m 
0 0 0 a2m b3m b4m
equations are given as

Y = Cx + V (60)
Ym = Cxm + V (61)
" #
1 0 0 0
where C = . Notice that the disturbance term, d = [0 0 dθ̇ dψ̇ ]T , can be written as
0 1 0 0

dθ̇ = (a1 − a1m )θ̇ + (b1 − b1m )u1 + (b2 − b2m )u2 − mheli glcm = △a1 θ̇ + △b1 u1 + △b2 u2 − mheli glcm (62a)
dψ̇ = (a2 − a2m )ψ̇ + (b3 − b3m )u1 + (b4 − b4m )u2 = △a2 ψ̇ + △b3 u1 + △b4 u2 (62b)

The first two zero elements in the disturbance term indicate the perfect knowledge of the system kinematics.
The disturbance term in vector notation can be written as

d = △Ax + △Bu + w (63)

   
0 0 0 0 0 0
0 0 0 0   0 0 
   
where △A =   and △B =  . Using the disturbance term the true model can
0 0 △a1 0  △b1 △b2 
0 0 0 △a2 △b3 △b4
be written in terms of the assumed parameters as shown below:
        
θ̇ 0 0 1 0 θ 0 0 " # 0
0
ψ̇  0 0 0 1  ψ   0 0  u1
      
 =   +  +  (64)
 
 dθ̇ 

 θ̈  0 0 a1m 0   θ̇  b1m b2m  u2
ψ̈ 0 0 0 a2m ψ̇ b3m b4m dψ̇

or in vector notation:

ẋ = Am x + Bm u + d (65)

The disturbance term dynamics is modeled as

d˙θ̇m = −dθ̇m + W1 (t) (66a)

d˙ψ̇m = −3dψ̇m + W2 (t) (66b)

Since the model uncertainty is only associated with the dynamics, only the nonzero elements of the dis-
turbance term need to be appended to the system states. Let the extended assumed state vector, Zm =
[xTm dθ̇m dψ̇m ]T . Now the assumed extended state-space equation can be written as

Żm = Fm Zm + Dm u + GW (67)
 
0 0 1 0 0 0
0 0 0 1 0 0
 
  " # " # " #
0 0 a1m 0 1 0 B m 0 W (t)
where Fm =  , Dm = , G = 4×2 , and W = 1
. The assumed
0
 0 0 a2m 0 1 02×2 I2×2 W2 (t)
0 0 0 0 −1 0 
 

0 0 0 0 0 −3
output equation can be written in terms of the appended state vector, zm , as

Ym = HZm + V (68)

16 of 22

American Institute of Aeronautics and Astronautics

where H = [C 02×2 ]. A Kalman filter is implemented in the feedback loop to estimate the system rates
and the disturbance term. The filter dynamics is
˙
Ẑ = Fm Ẑ + Dm u + KH[Z − Ẑ] + KZ (69)

The reference model that is of interest is

x̄˙ = Am x̄ + Bm ū (70)

where the nominal controller is a linear quadratic regulator which minimizes the cost function
Z∞
1
(x(t) − xd )T Qx (x(t) − xd ) + uT (t)Ru u(t) dt


J= (71)
2
0

where xTd = [θd ψd 0 0], θd and ψd are some desired final values of θ and ψ, respectively, and Qx and Ru
are two symmetric positive definite matrices. The nominal control that minimizes the above cost function is

ū(t) = −Km (x(t) − xd ) (72)

where Km is the feedback gain that minimizes the cost Eq. (71). Now the total control law can be written
in terms of the estimated states and the estimated disturbance term as
 
h i x̂ − xd
u = (Bm T
Bm )−1 Bm
T
− Bm Km − I2×2  dˆθ̇  = S Ẑ + Km xd (73)
 
ˆ
dψ̇

Since Eq. (65) does not contain any noise-like external disturbances, after substituting the above control law
into Eq. (63), the true disturbance-term dynamics can be written as

ḋ = (△AA + △BSKC)x + (△AB + △BSDm )(S Ẑ + Km xd ) + △BS(Fm − KH)Ẑ + △BSKv (74)

Equation (74) indicates that selecting a large Q or small R would amplify the measurement noise effect on
the disturbance term dynamics. This is clearly shown in the simulation results given next.

Table 3. Nominal Controller/Estimator Matrices

LQR Weighting
" Matrices
# " Covariance
# " Matrices #
10 0 q1 0 1 × 10−5 0
Ru = Q= R=
0 10 0 q2 0 1 × 10−5
 
  1 × 10−3 0 0 0 0 0
2000 0 0 0 0 1 × 10−3 0 0 0 0
 

 
 0 2000 0 0  0 0 0.1 0 0 0
  
Qx =   P (t0 ) =  
 0 0 100 0  
 0 0 0 0.1 0 0

0 0 0 100 0 0 0 0 1 0
 

0 0 0 0 0 1

Table 3 shows the nominal controller and estimator matrices. Since the measurement noise covariance,
R, can be obtained from sensor calibration, the process noise matrix, Q, is treated as a tuning parameter.
Based on the weighting matrices given in Table 3, the feedback gain is calculated to be
" #
14.0529 1.5865 2.1762 0.3790
Km =
−1.5865 14.0529 −0.1712 3.6387

For simulation purposes the initial states are selected to be [θ0 ψ0 θ̇0 ψ̇0 ]T = [−45o 0 0 0]T and the
desired states θd and ψd are selected to be 45o and 30o , respectively.

17 of 22

American Institute of Aeronautics and Astronautics

 6 8
x 10 x 10
3
Desired Desired
0
Truth Truth
Estimate 2 Estimate

−5

ψ(deg)
θ(deg)
1

−10
0

−15
−1

0 2 4 6 8 10 0 2 4 6 8 10
Time(sec) Time(sec)
(a) (b)
7 9
x 10 x 10
3 0.5
Desired Desired
2 Truth 0 Truth
Estimate Estimate

ψ̇(deg/sec)
1 −0.5
θ̇(deg/sec)

0 −1

−1 −1.5

−2 −2

−3 −2.5
0 2 4 6 8 10 0 2 4 6 8 10
Time(sec) Time(sec)
(c) (d)

Figure 3. Unstable System Response: q1 = q2 = 0.10

7 6
x 10 x 10
2 12
u1 dst1
u2 10 dst2
dθ̇ and dψ̇ (deg/sec)

0 8
u1 and u2

6
−2 4
2
−4 0
−2
−6 −4
0 2 4 6 8 10 0 2 4 6 8 10
Time(sec) Time(sec)
(a) (b)

0
Stability Indicator

−50

−100

−150

0 2 4 6 8 10
Time(sec)
(c)

Figure 4. Control Input, Estimated Disturbance Term, and Stability Indicator: q1 = q2 = 0.10

18 of 22

American Institute of Aeronautics and Astronautics

 Figure 3 shows the unstable system response obtained for the first simulation. The desired response given
in Fig. 3 is the system response to nominal control when there is no model error and external disturbance.
Figure 4 shows the system control input, estimated disturbance term and stability indicator obtained for the
first simulation. The stability indicator is calculated as
Stability Indicator = σmin (Q)σmin (R−1 ) k L k2 k N (t) k2 − k ∆Υ(t)P̄(t) k2
Notice that the negative values in stability indicator reveal that the inequality Eq. (38) is violated for the
selected Q matrix.

50 35

30
Desired
Truth 25 Desired
Estimate Truth

ψ(deg)
θ(deg)

20 Estimate
0
15

10

−50 0
0 2 4 6 8 10 0 2 4 6 8 10
Time(sec) Time(sec)
(a) (b)

70
Desired Desired
150 Truth 60 Truth
Estimate Estimate
50
ψ̇(deg/sec)
θ̇(deg/sec)

100 40
30

50 20
10

0 0
−10
0 2 4 6 8 10 0 2 4 6 8 10
Time(sec) Time(sec)
(c) (d)

Figure 5. Stable System Response: q1 = q2 = 1 × 104

# "
1 × 104 0
A second set of simulations are conducted using Q = . The system response obtained
0 1 × 104
for the second simulation is given in Fig. 5. The system is stable when Q increased because a large Q
satisfies the inequality Eq. (38) as shown in Fig. 6(c). Figure 6 shows the system control input, estimated
disturbance term and stability indicator obtained for the second simulation. Notice that the estimated
system rates, estimated disturbance term and the control input are highly noisy because of the large Q
selected. " #
10 0
A third and final simulation is conducted after tuning Q to Q = . The system response
0 200
obtained for the third simulation is given in Fig. 7. Figure 8 shows the system control input and estimated
disturbance term. Notice that the estimated system rates, estimated disturbance term and the control input
are relatively less noisier after tuning Q.
The simulation results given here explicitly reveal the direct dependency of the proposed control scheme on
the disturbance term process noise matrix, Q. Since the nominal control action on the true plant is unstable,
selecting a very low Q value resulted in an unstable system. Conversely, selecting a large Q stabilized the
system but resulted in a highly noisy control input. The third simulation indicates that there is an optimal
Q value that would minimize the noise in the control input and guarantee stability. Though the closed-loop
stability depends on Q and R, here we only consider the variations in Q only since the measurement noise
covariance can easily be determined from sensor calibration while the process noise covariance is more or
less a tuning parameter.

19 of 22

American Institute of Aeronautics and Astronautics

 25 10
u1 dst1
20 u2 dst2

dθ̇ and dψ̇ (deg/sec)

15
5

u1 and u2
10
5
0
0
−5
−10
−15 −5
0 2 4 6 8 10 0 2 4 6 8 10
Time(sec) Time(sec)
(a) (b)
7
x 10

15
Stability Indicator

10

0
0 2 4 6 8 10
Time(sec)
(c)

Figure 6. Control Input, Estimated Disturbance Term, and Stability Indicator: q1 = q2 = 1 × 104

50 40

Desired
Truth 30
Estimate
ψ(deg)

Desired
θ(deg)

0 20 Truth
Estimate

10

−50 0
0 2 4 6 8 10 0 2 4 6 8 10
Time(sec) Time(sec)
(a) (b)

120
Desired Desired
150 Truth 100 Truth
Estimate Estimate
80
ψ̇(deg/sec)
θ̇(deg/sec)

100 60
40

50 20
0

0 −20
−40
0 2 4 6 8 10 0 2 4 6 8 10
Time(sec) Time(sec)
(c) (d)

Figure 7. Stable System Response: q1 = 10, q2 = 200

20 of 22

American Institute of Aeronautics and Astronautics

 10
u1 dst1
20 u2 dst2
8

dθ̇ and dψ̇ (deg/sec)

6
10

u1 and u2
4

0 2

0
−10
−2

−20 −4
0 2 4 6 8 10 0 2 4 6 8 10
Time(sec) Time(sec)
(a) (b)

Figure 8. Control Input and Estimated Disturbance Term: q1 = 10, q2 = 200

V. Conclusions
This paper presents the formulation of a stochastic disturbance accommodating control with observer ap-
proach for linear time-invariant multi-input-multi-output systems which automatically detects and minimizes
the adverse effects of both model uncertainties and external disturbances on a controlled system. Assuming
all system uncertainties and external disturbances can be lumped in a disturbance term, this control ap-
proach utilizes a Kalman filter in the feedback loop for simultaneously estimating the system states and the
disturbance term from measurements. The estimated states are then used to develop a nominal control law
while the estimated disturbance-term is used to make necessary corrections to the nominal control input to
minimize the effect of system uncertainties and the external disturbance.
The stochastic stability analysis conducted on the controlled system reveals a lower-bound requirement
on the estimator matrices, Q and R−1 , to ensure stability in the mean or the mean-square stability of the
closed-loop system. If the nominal control on the true plant would result in an unstable system, then selecting
a small Q would also result in an unstable system. On the other hand, selecting a large Q value would compel
the estimator to completely rely upon the measurement signal and therefore the noise associated with the
measurement signal is directly transmitted to the estimates. This could result in noisy control signal which
could lead to problems, such as chattering and controller saturation. Also note that as R, the measurement
noise covariance, increases, the observer gain decreases and thus the observer fails to update the propagated
disturbance term based on the measurements. Thus for a highly uncertain systems, selecting a small Q or
a large R will result in an unstable closed-loop system. The stochastic Lyapunov style analysis indicates
that the controlled stochastic system is almost surely asymptotically stable if the noise distribution matrix,
Γ(t), satisfies a specific decay rate. Since the measurement noise covariance can be obtained from sensor
calibration, the process noise matrix Q is treated as a tuning parameter. The simulation results reveal that
if the selected Q is too low, then the system is unstable and if the selected Q is too large, then the resulted
control input is highly noisy. Simulation results also indicate that there is an optimal parameter that would
guarantee stability with minimal control input noise. Future research plans include developing an adaptive
law for Q that would guarantee asymptotic stability in the mean based on the stochastic Lyapunov analysis,
and also extending the current approach to nonlinear systems where the disturbance term also accommodate
for system nonlinearities.

Acknowledgments
This material is based upon work supported in part by U.S. Air Force Research Laboratory, Space Vehicles
Directorate. The authors would like to thank Khanh D. Pham for his contributions.

References
1 Johnson, C., “Accommodation of External Disturbances in Linear Regulator and Servomechanism Problems,” IEEE

Transactions on Automatic Control, Vol. AC-16, No. 6, December 1971, pp. 635–644.
2 Johnson, C. and Kelly, W., “Theory of Disturbance-Utilizing Control: Some Recent Developments,” Proceedings of IEEE

21 of 22

American Institute of Aeronautics and Astronautics

Southeast Conference, Huntsville, AL, 1981, pp. 614–620.
3 Johnson, C., “Disturbance-Utilizing Controllers for Noisy Measurements and Disturbances,” International Journal of

Control , Vol. 39, No. 5, 1984, pp. 859–862.

4 Biglari, H. and Mobasher, A., “Design of Adaptive Disturbance Accommodating Controller with Matlab and Simulink,”

NAECON, Proceedings of the IEEE National Aerospace and Electronics Conference (NAECON , Dayton, OH, 2000, pp. 208–
211.
5 Joseph A. Profeta III, W. G. and Mickle, M. H., “Disturbance Estimation and Compesation in Linear Systems,” IEEE

Transactions on Aerospace and Electronic Systems, Vol. 26, No. 2, March 1990, pp. 225–231.
6 Kim, J.-H. and Oh, J.-H., “Disturbance Estimation using Sliding Mode for Discrete Kalman Filter,” Proceedings of the

37th IEEE Conference on Dicision and Control , Tampa, FL, 1998, pp. 1918–1919.
7 Sorrells, J. E., “Design of Disturbance Accommodating Controllers for Stochastic Environments,” Proceedings of the 1982

American Control Conference, Arlington, VA, 1982, pp. 2603–2608.

8 Sorrells, J., “Comparison of contemporary adaptive control design techniques,” System Theory, 1989. Proceedings.,

Twenty-First Southeastern Symposium on, March 1989, pp. 636–640.

9 Davari, A. and Chandramohan, R., “Design of linear adaptive controller for nonlinear systems,” System Theory, 2003.

Proceedings of the 35th Southeastern Symposium on, March 2003, pp. 222–226.
10 Sorrells, J., “Adaptive Control Law Design for Model Uncertainty Compensation,” Tech. rep., Dynetics, Inc, Huntsville,

AL, 1989.
11 Crassidis, J. L. and Junkins, J. L., Optimal Estimation of Dynamic System, Chapman & Hall/CRC, Boca Raton, FL,

2004.
12 George, J. and Crassidis, J. L., “Sensitivity Analysis of Disturbance Accommodating Control with Kalman Filter Esti-

mation,” Proceedings of the AIAA Guidance, Navigation and Control Conference and Exhibit, Hilton Head, SC, 2007.
13 Hinrichsen, D. and Pritchard, A. J., Mathematical Systems Theory I; Modeling, State Space Analysis, Stability, and

Robustness, Vol. 48 of Text in Applied Mathematics, Springer, Heidelberg, Germany, 2005.

14 Kushner, H. J., Stochastic Stability and Control, Academic Press, New York, NY, 1967.
15 Soong, T. T., Random Differential Equations in Science and Engineering, Academic Press, New York, NY, 1973.
16 Abou-Kandil, H., Freiling, G., Ionescu, V., and Jank, G., Matrix Riccati Equations in Control and Systems Theory,

Systems & Control: Foundations & Applications, Birkhäuser, Basel, Switzerland, 1st ed., 2003.
17 Soong, T. T. and Grigoriu, M., Random Vibration of Mechanical and Structural Systems, Prentice Hall, Englewood

Cliffs, NJ, 1993.

18 Grigoriu, M., Stochastic Calculus, Birkhäuser, Boston, MA, 2002.
19 Arnold, L., Stochastic Differential Equations: Theory and Applications, Wiley, New York, NY, 1972.
20 Mao, X., “Some Contributions to Stochastic Asymptotic Stability and Boundedness via Multiple Lyapunov Functions,”

Journal of Mathematical Analysis and Applications, Vol. 260, No. 2, August 2001, pp. 325–340.
21 Appleby, J. A. D., “Almost Sure Stability of Linear Itô-Volterra Equations with Damped Stochastic Perturbations,”

Electronic Communications in Probability, Vol. 7, August 2002, pp. 223–234.

22 of 22

American Institute of Aeronautics and Astronautics