On Smooth Transition Autoregressive Family of Models and Their Applications-Asif
On Smooth Transition Autoregressive Family of Models and Their Applications-Asif
On Smooth Transition Autoregressive Family of Models and Their Applications-Asif
Abstract: One of the most important family of nonlinear time-series models, capable of
exhibiting limit cycle behaviour, is Self-exciting threshold autoregressive (SETAR)
models. However, one limitation of this family is that transitions between various regimes
take place in a discontinuous and sudden manner. For a more realistic modeling, these
transitions should be smooth. Here, a flexible family of nonlinear time-series models, viz.
Smooth transition autoregressive (STAR) family of nonlinear time-series models is
thoroughly studied. Various models of this family, viz. Exponential autoregressive (EAR)
model, Exponential smooth transition autoregressive (ESTAR) model, Logistic smooth
transition autoregressive (LSTAR) model, along with their estimation procedures are
described in detail. Finally, two examples of fitting these models to real data are discussed.
1. Introduction
Over the last one decade or so, interest in applications of nonlinear time-series models has
been steadily increasing. Models which allow for state-dependent or regime-switching
behaviour have been most popular. For example, changes in government policy may
instigate a change in regime. With a view to modelling this type of time-series data, the
family of Smooth transition autoregressive (STAR) models has been proposed by
Terasvirta (1994). A survey of recent developments related to one of these regime-
switching models, viz. the STAR nonlinear time-series models are presented here. The
data-generating process to be modeled is viewed as a linear process that switches between
a number of regimes according to some rule. It has been assumed that there is a continuum
of switches, that is, there is a smooth transition from one extreme regime to the other. It
consists of specification, estimation and evaluation stages and, thus, is similar to the
modelling cycle for Box-Jenkins linear models. Some models of STAR family are briefly
considered below.
( )
where {ε t } is a sequence of normal 0, σ 2 independent errors, φ = (φ0 , φ1 , . . ., φp ) and
′
( )
consisting of an intercept and the first p lags of y t and G Lk γ, c; y t − d is known as transition
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
function. Depending upon the forms of transition function, different forms of STAR
models are defined.
The function G Lk is symmetric around zero, where it obtains the value one, and
G Lk (γ, c; y t − d ) → 0 as y t −1 → ∞ . In case of EAR model, parameter vectors do not contain
intercept terms, i. e. φ0 = θ0 = 0 . Equation (2.1) indicates that the model can be interpreted
as a linear autoregressive model with stochastic time-varying coefficients φ +θ G Lk (γ, y t − d ) .
(
When γ → 0 or γ → ∞ , the model becomes linear. In the latter case, G Lk γ, c; y t − d = 0 , )
except when y t −1 = 0 . A heartening feature of this model is that it is capable of generating
limit cycles.
{ }
G Lk (γ, c, y t − d ) =1− exp − γ (y t − d − c ) 2 , γ > 0 …(2.3)
Terasvirta (1994) called this model as Exponential smooth transition autoregressive
(ESTAR) model and discussed procedure for estimation of parameters. It has the property
that the minimum value of the transition function equals zero (Figure 2.1). It has been
successfully used to model macroeconomic series, such as strongly fluctuating inflation
series (Baharumshah and Liew, 2006).
Figure 2.1: Graph of exponential transition function (2.3) for γ = 0.01, 3, 20, and 50 .
2
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
The graph corresponding to the lowest value of γ lies closest to the horizontal axis G Lk = 0
Figure 2.2: Graphs of the logistic function (2.4) with k = 1 for γ = 0.01, 3, 20, and 50 . The
graph corresponds to the lowest value of γ lying closest to the line G Lk = 1 .
2
3
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
Figure 2.3: Graphs of the logistic function (4) with k = 2 for γ = 0.01, 3, 20, and 50 . The
graph corresponds to the lowest value of γ lying closest to the line G Lk = 1 .
2
4
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
It is based on the relationship between the parameters in (3.1) and (2.1) with either (2.3) or
(2.4). For the ESTAR model π 3 j = 0 , j = 1,...., p, but π2 j = 0 for at least one j if θ j ≠ 0 . For
the LSTAR model π1j ≠ 0 for at least one j if θ j ≠ 0 . If H 03 is rejected, a LSTAR model is
selected. If H 03 is accepted and H 02 is rejected then an ESTAR model is selected. If H 03
and H 02 are accepted but H 01 is rejected a LSTAR model is selected. No clear-cut
conclusion is obtained when H 02 and H 01 are rejected. In this case we test:
H ′02 = π 2 j = 0 π1 j = π 3 j = 0 , j = 1,...,p.
However, if H 02 is rejected, then H ′02 should be rejected even more strongly. In any case,
the decision is based on whether H 03 , H 02 or H 01 is rejected more strongly. Teräsvirta
(1994) found that the selection procedure works very well when the true model is LSTAR
or ESTAR; in the latter case the observations do not have to be symmetrically distributed
around c. The procedure finds it difficult to distinguish between the two types of models
when only a small number of observations are located at one of the tails of the transition
function.
Under some regularity conditions the estimates are consistent and asymptotically normal,
that is T (ψˆ − ψ * ) → N (0, C ) where ψ * is the true parameter vector and C is the
covariance matrix of estimates. The parameters of the STAR model will hence be
estimated by nonlinear least squares, and to do that, a suitable iterative optimization
algorithm is needed. The optimization problem in nonlinear least squares is conditional on
the starting values y 0 , y −1 , . . ., y − p +1 and consists of finding the minimum of the criteria
′
q (θ ) = (y − f (y; θ )) (y − f (y; θ )) , …(4.1)
with respect to the parameter vector θ . To find a starting-point for the iteration procedure,
we approximate Equation (4.1) with a second-order Taylor expansion about θ n which
yields
5
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
( n )+ h′n (θ − θ n )+ 12 (θ − θ n )′ H n (θ − θ n )
q (θ ) ≅ q θ
…(4.2)
∂q (θ n )
where the gradient evaluated at θ n , h n = h (θ n ) = and the Hessian evaluated at θ n ,
∂θ
∂ 2 q (θ n ) ∂q (θ)
H n = H (θ n ) = . The first-order conditions for the minimum, i. e. = 0 will be
∂θ∂θ ′ ∂θ
obtained by differentiating Equation (4.2) with respect to θ and the basis for iteration
yields:
h n + H n (θ − θ n ) = 0 …(4.3)
If we have fixed θ n and know how to compute H n and h n , Equation (4.3) yields the
search direction for the next value of θ : θ = θ n +1 which is obtained from θ = θ n + k n H n−1 h n
where k n is the step-length. The value forms the starting-point of the next iteration. This is
the Newton-Raphson method. In general the optimization algorithm is very sensitive to the
choice of the starting values of the parameters. The implemented program is estimating all
the models according to the Levenberg-Marquardt algorithm with cubic interpolation linear
search. Concerning the selection of the starting values, the following algorithm is used:
⎛ w 1′ ~′ ⎞
w
⎜ 1
⎟
⎜ ... ... ⎟
⎜ ⎟
W = ⎜ ... ... ⎟
⎜ ⎟
⎜
⎜ ... . . . ⎟⎟
⎜ ~′ ⎟
⎝ w ′T w T ⎠
Once γ , d and c have been determined, the parameter vector λ can be estimated by
λˆ = (W ′ W ) W y .
−1
5. Some Applications
(a) Example 5.1: Fitting of ESTAR Model
In order to explain “Swedish business cycles”, Skalin and Terasvirta (1999) applied
ESTAR models to yearly time-series data in respect of nine macroeconomic variables, like
Employment, Industrial production, Consumption, Export, and Import for Sweden during
the period 1870 to 1988. However, here, we shall confine ourselves to discussing the
results for “Employment” (hours worked in manufacturing and mining). The most distinct
features of this time-series are a large drop in employment in the years following the First
World War and another one followed by a rapid recovery in 1930s.
6
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
Testing linearity against ESTAR constitutes the first step of the model specification stage.
In order to do that, first a linear autoregressive model for the time-series with no
autocorrelation in residual is selected. This is done by applying an appropriate model
selection criterion such as Akaike information criterion (AIC). Here all linearity tests are
carried out conditionally on d and the results are used to select delay. The value of d
associated with the test with smallest p-value is selected (Table 5.1). If none of the p-value
is sufficiently small, linearity is not rejected.
Table 5.1: Results of linearity tests
Here p is the number of lags in the linear AR model; minimum p-value, min [p(FL)], over
the delays d = 1, 2,..., p; d is corresponding delay; p-values of the test in the model
selection sequence (p(F4), p(F3), p(F2)) for the selected d; the selected model family (E =
ESTAR, L = LSTAR). The ESTAR model is selected if p(F3) is the smallest p-value of the
trio p(F4), p(F3), p(F2); otherwise the choice is the LSTAR.
7
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
Period 3.5
The characteristic polynomial for G = 0 contains a pair of complex roots with a very large
modulus. This pair remains explosive for considerably large values of G. Thus the
recovery of the growth rate from a deep trough has always been quick. Graphs of fitted
ESTAR and AR models along with data are given in Figure 5.1.
Figure 5.1: Graphs of fitted ESTAR and AR models along with Employment data
(first difference of logarithm)
The ESTAR model makes two major contributions. First, it explains the big decrease in
employment in the beginning of 1920s better than the linear AR model. Second, it tracks
the data well from mid 1960s onwards where the linear model fails. Whereas the series up
to the 1960s displays little or no cyclical movements, roughly the last quarter of the
century is characterized by a prominent asymmetric cycle with a rather short period.
where PDIFF = Mean Sea Level Pressure (MSLP) difference between Tahiti and Darwin,
PDIFFAVE = long term average of PDIFF, and SD (PDIFF) = standard deviation of PDIFF. This
definition of SOI is given by Australian Commonwealth Bureau of Meteorology. The
monthly data from January 1876 to May 1998 are used here for modelling purpose. A
8
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
casual inspection of the data indicates possible turbulent periods and asymmetric cyclical
variations in the sense that downturns appear to occur more rapidly than the recovery and,
therefore, STAR models are particularly appropriate to describe such a data (Figure 5.2).
40
30
20
10
Value of SOI
Apr, 1932
Jul, 1958
Apr, 1962
Jan, 1966
Oct, 1969
Jul, 1973
Apr, 1977
Jan, 1981
Oct, 1984
Jul, 1988
Apr, 1992
Jan, 1996
Oct, 1909
Jul, 1913
Apr, 1917
Jan, 1921
Oct, 1924
Jul, 1928
Jan, 1936
Oct, 1939
Jul, 1943
Apr, 1947
Jan, 1951
Oct, 1954
Jan, 1876
Oct, 1879
Jul, 1883
Apr, 1887
Jan, 1891
Oct, 1894
Jul, 1898
Apr, 1902
Jan, 1906
-10
-20
-30
-40
-50
Year
Hall et al. (2001) results of fitting LSTAR model to above data are now discussed. Using
AIC a lag length of 13 is chosen for the linear autoregressive model. A summary of the
results of the test for nonlinearity is presented in Table 5.3. Table 5.3 contains the lag
length, p, of the autoregressive model chosen by AIC, the p-value of the test corresponding
to values of the delay parameter, d, from 1 to 5. Linearity is rejected against STAR models,
particularly with delay length of 1, 3 and 5. For these delay values, the results of the model
selection test sequence for choosing between ESTAR and LSTAR is also shown in Table
5.3 and LSTAR model is the preferred model. A number of LSTAR model with a variety
of delay variables are estimated. The selected model for this time-series is LSTAR (13)
model with d = 1. The intercept in linear part of the selected model is constrained to be
equal to that in nonlinear part of the equation. This imposes a restriction that in normal
times the estimated mean of the SOI will be zero.
9
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
Here p is the number of lags in linear autoregressive model, d is the delay parameter, p(FL)
is the p-value of the linearity test; p(F2), p(F3), p(F4) are the p-values of the tests in the
model selection sequence, and the selected model family is (E = ESTAR, L = LSTAR). E
is selected if p(F3) is the smallest value of the sequence, otherwise L is selected
(Terasvirta, 1994).
Table 5.4: Roots of characteristic polynomials for values of the transition function G
Regime G = 0 Regime G = 1
Root Modulus Period Root Modulus Period
2.62 2.62 0.90 ± 0.13 i 0.91 42.4
− 0.56 ± 1.24i 1.36 3.15 − 0.45 ± 0.71i 0.84 2.94
− 1.01 ± 0.31i 1.05 2.21 0.61 ± 0.53 i 0.81 8.78
− 0.57 ± 0.84i 1.02 2.89 − 0.73 ± 0.33 i 0.80 2.31
0.75 ± 0.67i 1.01 8.62 0.30 ± 0.73 i 0.79 5.32
0.23 ± 0.82i 0.85 4.82 − 0.02 ± 0.77 i 0.77 3.93
10
On Smooth Transition Autoregressive (STAR) Models and their Applications: An Overview
6. CONCLUSIONS
STAR family of nonlinear time-series models are thoroughly studied. These types of
models are of particular importance to describe those data sets that have cyclical variations
along with turbulent periods. Application of these models to cyclical data in fisheries, lac
production/ export, etc is required. There is also a need to apply, in due course of time,
several extensions of above models, viz. Time-varying STAR models, and STAR models
for vector time-series.
References
Arango, L. E. and Gonazalez, A. (2001). Some evidence of smooth transition nonlinearity
in Colombian inflation. Applied Economics, 33. 155-162.
Baharumshah, A. Z. and Liew, V. K. (2006). Forecasting performance of exponential
smooth transition autoregressive exchange rate models. Open Economies Review,
17. 235 – 251.
Fan, J. and Yao, Q. (2003). Nonlinear time series: Nonparametric and parametric
methods. Springer, New York.
Hall, A. D., Skalin, J. and Terasvirta, T. (2001). A nonlinear time series model of El Nino.
Environmental Modelling and Software, 16. 139-146.
Leybourne, S., Newbold, P. and Vougas, D. (1998). Unit root test and smooth transitions.
Journal of Time Series Analysis, 19. 83-97.
Skalin, J. and Terasvirta, T. (1999). Another look at Swedish business cycles, 1861-1988.
Journal of Applied Econometrics, 14. 359-378.
Terasvirta, T. (1994). Specification, estimation, and evaluation of smooth transition
autoregressive models. Journal of the American Statistical Association, 89. 208-
218.
Terasvirta, T., van Dijk, D. and Medeiros, M. C. (2005). Linear models, smooth transition
autoregressions, and neural networks for forecasting macroeconomic time series: A
re-examination. International Journal of Forecasting, 21. 755-774.
11