1

Use of Deep Learning to Study Modelling Deterioration of Pavements a Case

Study in Iowa

Authors: Seyed Amirhossein Hosseini1, Ahmad Alhassan2, and Omar Smadi1

Affiliation: 1Iowa State University – Institute for Transportation, Ames, Iowa

2
Applied Research Associates, Inc

Abstract

This paper describes the process and outcome of deterioration modeling for three

different pavement types in the state of Iowa. Pavement condition data is collected by the Iowa

Department of Transportation (DOT) and stored in a Pavement-Management Information System

(PMIS). Typically, the overall pavement condition is quantified using the Pavement Condition

Index (PCI), which is a weighted average of indices representing different types of distress,

roughness, and deflection. Deterioration models of PCI as a function of time were developed for

the different pavement types using two modeling approaches. The first approach is the

Long/Short Term Memory (LSTM), a subset of a recurrent neural network. The second

approach, used by the Iowa DOT, is developing individual regression models for each section of

the different pavement types. A comparison is made between the two approaches to assess the

accuracy of each model. The results show that while the individual regression models achieved

higher prediction accuracy with respect to asphalt pavements, the LSTM model achieved a

higher prediction accuracy over time for concrete and composite pavement types.

Keywords: Long/Short Term Memory (LSTM) model, deterioration model, regression model,

pavement surface distress, deep learning, prediction accuracy

 2

Introduction

Public agencies use pavement management systems (PMSs) to make objective decisions

and conduct activities for maintaining pavements in acceptable conditions at minimal cost

(AASHTO, 2012). Since the early 1970s, departments of transportation (DOTs) and other

transportation agencies have been implementing and establishing PMSs to match their needs,

achieving significant savings and improvement in network conditions (Vasquez, 2011). The

Arizona DOT, for example, saved $14 million and $101 million during the first year and the first

four years of PMS implementation, respectively (Hassan, 2017). The Colorado Department of

Transportation (CDOT) uses PMS to efficiently spend its $740 million annual budget for

maintaining and preserving more than 9,100 center-line miles (about 23,000 total lane miles)

(Saha, 2017). It appears that there is potential for all such expenses to be more effective if PMS

improvements can be developed and implemented.

A major component of any PMS is evaluation and modeling of pavement conditions at

the network level. Recently, most states have begun to use automated pavement-condition

surveying tools that generate images from remote sensors to collect distress information and

report individual distresses through an overall condition index (Ragnoli, 2018). The concept of

Pavement Condition Index (PCI), was developed by the U.S. Army Corps of Engineers in 1970

based on different types of distresses and severity levels (Shahnazari, 2012). Since then, most

DOTs and related agencies have been using the PCI to evaluate pavement conditions. The PCI

provides important information to pavement engineers by describing overall pavement condition

based on different types of distress, roughness, and deflection (Ceylan 2014, Haas 2015). The

PCI is defined as a numerical rating between 0 and 100, with 0 being the worst condition and

100 the best condition for pavement segments. Based on monitored and modelled PCI values and

other important condition indices, decision-makers can evaluate the functionality of pavement
 3

networks, predict the best time for maintenance and rehabilitation activities, and estimate future

funding needs (Bektas, 2014).

Long and short-term planning of maintenance and rehabilitation activities is the major tool for

maximizing proper network conditions at the lowest possible cost and requires accurate and

robust deterioration models for pavement networks. A Deterioration Model (DM) predicts future

pavement conditions and helps agencies identify the most effective maintenance and

rehabilitation activities (George 2000, Lytton 1987), and such planning and optimization become

more critical when agencies face budget reduction or are otherwise budget-constrained (Hassan,

2017).

Deterministic, probabilistic, neural network-based, and knowledge-based performance

models have been used in pavement management to predict future conditions of pavement

sections (Wolters, 2010). Currently, the Iowa Department of Transportation (Iowa DOT)

forecasts the future conditions of pavement sections based on individual deterministic regression

models for each pavement section. Deterministic models assume that the described process is

nonrandom and that observed differences between predicted and measured values are due to

random noise in the observation process.

A deterministic model will thus always produce the same output from a given starting

condition or initial state. Most deterministic models are based on explicit regression expressions

and are categorized into the following three subsets ( Li, 1996):
 4

1. Empirical Models: An empirical model, solely based on experimental observations,

provides no explanation of the fundamental behavior through constitutive models. These

models require large databases for deriving accurate and representative models. Some

advantages and disadvantages of empirical methods, based on a study reported by Bulleit

and Ylitalo (de Melo, 2000), are:

Advantages:

 The mathematical approach for prediction is not complex

 The relationship between actual and predicted values can be easily described

Disadvantages:

 Model sensitivity

 Restricted to the conditions used to derive the relationships and not useful for

extrapolation.

2. Mechanistic Models: Mechanistic models primarily use laboratory testing data and

idealized models to mathematically describe fundamental pavement responses like stress,

deflection, and strain caused by traffic loading and other surrounding conditions (Mills,

2012). It has been observed over time that sometimes these idealized lab tests and models

do not reflect actual conditions in the field and may therefore fail to accurately predict

pavement performance. The availability and feasibility of more recent pavement

condition assessment tools have resulted in practitioners and agencies avoiding use of

mechanistic models (Haas, et al., 1994).

3. Mechanistic-Empirical (ME): While these models are fundamentally based on

mechanistic models, they are calibrated and coupled with the empirical long-term

observations from pavement sections under real-life operating conditions. ME models are
 5

more representative of actual conditions, because they consider additional parameters like

traffic loading, climate factors, and material properties.

Probabilistic models are another group of pavement performance models, an

alternative to deterministic models that do not provide probabilistic distribution of

existing values. Markov probabilistic modeling uses samples of probabilistic models,

with the transition process represented by a pavement-performance curve ( Li, 1996).

Using information from the pavement’s “before” state, the Markov process predicts the

“after” state (George, 2000). The Markov transition method is useful in network-level

applications where neither historical data or good regression equations are available,

(Shabanpour, 2017). Another advantage of probabilistic models is their use of different

distributions for finding expected values of the dependent variable. Also, uncertainty with

respect to environmental conditions, material properties, and traffic loading can be

captured by these models. The main disadvantage of probabilistic models is that they do

not consider the effects of pavement aging on transition probabilities (Shabanpour, 2017).

In addition to Markov models, there are other types of probabilistic models like

Bayesian decision models, Bayesian regression models, and semi-Markov models, that

generate survivor curves (Golroo, 2012). The greatest advantages of probabilistic models

are their capability for capturing uncertainty in the pavement prediction model, and for

producing more realistic results than deterministic models.

Over the past few years, Neural Network (NN) applications have received greater

attention, and many research studies on the application of NNs in transportation and civil

engineering have been published (Adeli 2001, Dougherty 1995, Flood 2008, Flood &

Kartam, 1994). Because of their capability for interconnecting neurons between layers,
 6

NN applications can often solve complex problems more efficiently than traditional

methods (Basheer, 2000). The capabilities of Neural Network models for solving

problems from several pavement-engineering categories are as follows (Ceylan, 2014):

 Classification: Supervised learning in neural networks can be used to deal with

unknown inputs. Neural network models have been used to investigate the

classification of pavement distresses from digital images (Nallamothu, 1996).

Another research study by Hu (2001) reported using a neural network to detect

pavement cracks.

 Performance Prediction: Neural Networks have been used in various studies as

powerful and versatile computational tools for both determining the performance

of existing pavement systems and predicting future conditions. The Pavement

Distress Index (PDI), based on surface thickness, pavement age, and traffic level,

was predicted using a NN model that outperformed other multiple-linear

regressions (Owusu-Ababio, 1998). A back-propagation neural network model

was developed by (Lin, 2003) for predicting IRI based on pavement distress.

 Optimization and maintenance strategies: Neural networks have been used as

computational tools to determine which maintenance and rehabilitation actions

should be performed on deteriorated pavement sections, using a hybrid NN and

Genetic Algorithm method developed for optimizing maintenance strategy of

flexible pavements (Taha, 1995).

 Distress Prediction: Neural networks can help pavement engineers predict future

distresses, and a multi-layer perceptron back-propagation NN with one hidden

 7

layer has been used to predict future roughness distress in flexible pavements

(Huang, 1997).

NNs could be a powerful alternative to traditional techniques that are always limited by

normality, linearity, and colinearity assumptions. Two major advantages of using NNs are their

ability to model complex and nonlinear large amounts of data, and detect all possible interactions

between predictor variables.

It should be mentioned that, because pavement deterioration happens over time, it is

important to include the dependency of performance measures on historical data (time) in a

prediction model. Accurate time-series prediction is also critical for abnormality detection,

resource allocation, and financial planning (Laptev, 2017). Predicting data time-dependency is

challenging because such prediction depends on external factors like weather and traffic load

(Horne, 2004). Time-series analysis works better with highly-correlated measurements over

time, because explanatory variables may fail to explain the correlation mechanisms. On the other

hand, in regression analysis the explanatory variables should sufficiently explain the trend,

resulting in independent fitting residuals.

A deep-learning method designed for sequential data is the Recurrent Neural Network

(RNN) that has recently received additional attention from researchers primarily because of its

capability in learning sequences (Graves 2010, LeCun 2015, Sutskever 2013). RNNs have been

widely applied to many time-dependent datasets for use in prediction problems like speech

prediction, pattern prediction, economic prediction, and traffic prediction (Busseti 2012, Martens

2011, Wong 2010). Since RNNs are developed to utilize historical data in time-series analysis,

inclusion of a regression model that relies on explanatory variables and historical data of the

response variable improved the model accuracy. These networks are designated as recurrent
 8

because future forecasting depends on both current and previous stages. Several RNN algorithms

such as the Long Short Term Memory (LSTM) network have been developed over the past two

decades. LSTM was introduced to support modeling and forecasting of long-term data series.

The network was developed to overcome the vanishing gradient problem in which algorthims

tend to accumulate errors when a long string of observations are added as predictor variables,

increasing prediction variability and associated total error. Based on the literature, another RNN

network called the Gated Recurrent Unit (GRU) also solved the vanishing gradient problem, but

the LSTM outperformed the GRU in many details.

In this study, the LSTM was used for time-dependent prediction of the pavement

condition index. This network is suitable for pavement applications because the data is presented

in time series with both low observation frequency and high levels of variability. The goal of this

study was to develop a new robust deterioration model suitable for long term forecasting, in

which the model performance can be objectively evaluated. An LSTM network will utilize

historical pavement condition records of the Iowa DOT Pavement Management Information

System (PMIS) in the time span between 1998 and 2018. The new time series algorithm, a deep-

learning approach specifically developed by LSTM networks, was used to predict future

conditions of the three different pavement types. The Keras software package, a high-level

neural network API written in Python, was used for generating the LSTM model with a focus on

enabling fast experimentation. This package uses a deep-learning open-source library based on

the TensorFlow software library. The performance and results of the new algorithm are

compared to the current method used by Iowa DOT for deterioration modeling.
 9

Methodology

Figure 8 shows the steps required to be completed in the proposed method, with the

individual steps described in detail in the following subsections.

Data Collection

Preprocessing

Developing LSTM

Model Training

Model Validation

Comparison

Figure 1: Research Steps

Data

To develop and implement the new framework, historical records of pavement condition

data were acquired from the Iowa DOT Pavement Management Information System (PMIS).

These data were collected for Iowa’s interstate and primary network since 1997, the year in

which the Iowa DOT began collecting automated pavement distress data (Bursanescu, 1997).

The data used in this study were acquired between 1998 and 2018, and include information

regarding highway system classification, construction and reconstruction dates, unique section

identifiers, traffic levels, automated pavement distress data, faulting, and pavement ride quality.
 10

The pavement types included in the study were asphalt concrete (AC), Portland cement concrete

(PCC), and composite (COM) pavements.

The pavement distress information collected includes rutting and cracking data such as

transverse cracking, longitudinal cracking, alligator cracking, wheel-path cracking, and patching,

with low, medium, and high severity levels assigned to cracking data for all pavement types. For

AC and COM pavements, rutting was reported as the average rut depth in both wheel paths, and

for PCC pavements faulting was estimated using the acquired longitudinal profile. The

international roughness index (IRI) was also used to characterize ride quality for all pavement

types. Pavement condition data is collected in two-year cycles in which half the network is

surveyed every other year. The Iowa DOT spends about $1 million annually on collecting

pavement condition data (Bektas, 2014).

In many cases, minor maintenance and rehabilitation records were not available, so the

maintenance impact on pavement condition overtime was not modelled in this study. Moreover,

segments with PCI values increasing over time were discarded from the analysis because they

might be associated with unrecorded maintenance activities. A ten-point PCI increase was

arbitrarily considered to be a normal fluctuation due to measurement errors or seasonal impacts.

Figure 9 shows the number of different sections for each pavement type, with the descriptive

statistics for each pavement type given in Table (3). The total number of data records for all 20

years time frame was comprised of 3,805 AC records, 14,117 COM records, and 13,123 PCC

records.
 11

COM 1830

PCC 1562

AC 477

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Figure 2: Number of sections in each pavement type

Table 1: Summary statistic of pavement sections (Alharbi, 2018)

Pavement Types Average Length Minimum Length Maximum Length

(Miles) (Miles) (Miles)
AC 3.88 0.16 18.61
PCC 2.7 0.05 18.91
COM 2.69 0.05 18.14

Preprocessing

After collecting and arranging the data based on pavement type, condition indices were

estimated using the reported condition data. Pavement condition can be summarized using four

scaled indices with values ranging from 0 to 100, with 0 corresponding to the worst condition

and 100 to the best condition. These indices can then be used to calculate the overall PCI using

the same scale for individual indices, resulting in the definition of a global index for comparing

different pavement types. In this study, the indices were calculated based on definitions provided

in a previous study for the Iowa DOT (Bektas, 2014) and included:

 Riding Index

 Rutting Index (AC and COM Only)

 Cracking Index
 12

 Faulting Index (PCC Only)

In AC and COM pavements, four different cracking sub-indecies were used to calculate

the cracking index; these included transverse, longitudinal, alligator, and longitudinal-wheel-path

cracking. Only two sub-indecies, transverse and longitudinal cracking, were used to characterize

PCC pavements. Three severity levels were used by the Iowa DOT in evaluating pavement

distresses, with 1, 1.5, and 2 coefficient values, used for low, medium, and high aggregated

severities, respectively. All severity levels were then converted into low severity. Since a

maximum value (threshold) corresponds to a deduction of 100 points, a cracking sub-index of 0

was determined for each crack type within pavement type, and all threshold values were

extracted from a previous Iowa DOT study (Bektas, 2014). The cracking index values for all

three pavement types, based on the coefficient values provided by Iowa DOT experts, were as

follows:

Cracking Index ( AC∧COM ) =0.2∗( Transverse Index ) +0.1∗( Longitiudinal Index )+ 0.3∗( Wheel− path Index ) +0.4∗(Allig

Cracking Index ( PCC )=0.6∗( Transverse Index ) +0.4∗( Longitiudinal Index )

The International Roughness Index (IRI) is the most commonly used ride-quality index.

The Riding Index used in this study was based on the IRI acquired by the Iowa DOT and

expressed on a scale of 100. IRI values below 0.5m/km were taken as a perfect 100, while values

above 4.0m/km were taken as 0 on the index scale. Other values between 0.5 and 4 m/km were

calculated using linear interpolation.

Rutting is defined as the permanent total deformation or consolidation accumulated in an

asphalt pavement surface wheel path. The rutting index from this study used rut depths available

in the PMIS database, and, based on previous research, a threshold value of 12 mm corresponded
 13

to 0 on the rutting Index scale of 100, and values below 12 mm were applied as corresponding

deductions.

Faulting is defined as the difference in slab elevation across a joint or crack occuring due

to differential vertical displacement between two sides. Similar to the rutting index for AC

pavements, the faulting index is expressed on a scale of 100, with the faulting value equal to or

greater than 12 mm set to 0 and the faulting value equal to zero set to 100 on the index scale

(Bektas, 2014).

After calculating all cracking, riding, rutting, and faulting indecies for AC, COM, and

PCC pavements, a weighted average formula was used to calculate the PCI values. The current

formulae for calculating the PCI for AC, COM, and PCC pavements are as follows (Bektas,

2014):

PCI ( PCC )=0.4∗( Cracking Index ) +0.4∗( Riding Index )+ 0.2∗( Faulting Index )

PCI ( AC )=0.4∗( Cracking Index ) + 0.4∗ ( Riding Index ) +0.2∗( Rutting Index )

PCI ( COM )=0.4∗( Cracking Index ) +0.4∗ ( Riding Index ) +0.2∗( Rutting Index )

Based on PCI values, the Iowa DOT classifies pavement condition for the interstate

highway system as good, with a PCI value between 76-100, fair, with a PCI value between 51

and 75, and poor, with a PCI value between 0 and 50. Based on these classifications,

approximately 91% and 79% of the interstate highway system and the non-interstate highway

system in the state of Iowa was categorized as good condition pavement up to the end of 2017

(Iowa DOT Transportation Asset Management Plan, 2018).

 14

Developing the Long Short Term Memory (LSTM) Deterioration Model

To predict the future condition of individual pavement sections a modified RNN

algorithm called LSTM was used in this research. While in conventional feed-forward neural

networks, all observations are considered independent, the models in RNN consider the effects

of previous observations and therefore account for the correlation between consecutive

observations. It is worth mentioning that RNNs can work properly only with short term

dependencies, and for making an accurate prediction with an RNN, having information from

previous stages is mandatory. In fact, an RNN fails when too many inputs from historical

observations are used. Observations added as predictor variables will increase variability in the

predictions and the total error, a phenomenon referred to as the vanishing gradient effect.

Generally, in feed-forward neural networks, the multiplication of errors from previous layers,

rate of learning, and input for a layer define the updating weight for the following layer. As a

result of several multiplications of the small value of activation-function derivatives (Sigmoid,

Tanh, ReLU), the gradient approaches zero, increasing training complexity and causing

information loss within the training layers. To overcome this limitation, LSTM was proposed as

a modified version of traditional RNNs while taking advantage of the effectiveness of RNN

methods.

The information in LSTMs flows through a cell states mechanism in which LSTMs can

selectively either forget or remember information based on its impact on model performance

(Chris Olah, 2015). Figure 10 is a schematic of the repeating module in an RNN that goes

through three major steps.

 15

Figure 3: Schematic of Repeating Module in RNN (Chris Olah, 2015)

In the first step, the LSTM passes the output from the previous time step ( t−1 ) to the

forget gate, where it is classified using the sigmoidal function shown in Equation 1 either as

significant information passed to the next step in the training or insignificant information

dropped from the training model.

Ft =σ (Wf [ h ( t−1 ) , Xt ] )+ bf (1)

where Ft represents the forget gate, σ is the Sigmoid function, Wf represents the weight for

the forget gate neurons, h ( t−1 ) is the output of a previous LSTM block at time( t−1 ), Xt

represents the input at the current time step, and bf represents biases for the forget gate.

In the second step, the LSTM decides what new information should be stored in the cell state

by identifying values requiring updating by the Sigmoidal function and the vector of new

candidate values created by the Tanh function that could be added to the next state. These two

functions are shown in Equations 2 and 3:

It=σ ¿ (2)
 16

C ´ t=tanh ¿ (3)

where I t represents the input gate, Wi represents the weight for respective gate neurons, Xt

represents the input at the current time step, and C ´ t represents the candidate for cell state at time step

(t).

By combining information from the previous cell and the input gate from the current time step, the

information for the later step will be updated. Equation 4 represents how information is filtered from the

forget gate layer combined with new information from the current time step. Other Sigmoid and Tanh

functions help the LSTM cell decide what information should be taken as output. Equations 5 and 6

represent the Sigmoidal and Tanh functions in the last step:

Ct =Ft∗C ( t−1 )+ It∗C ´ t (4)

Ot=σ ¿ (5)

ht=Ot∗tanh ⁡(Ct ) (6)

where Ct is a cell state (memory) at time step (t), O t represents the output gate, and h t

represents the output of the LSTM block at time step (t).

Model Training

For the learning process in the LSTM algorithm, the dataset corresponding to PCC and

COM pavements is divided into training (70%) and validation (30%) sets. Because the number

of records in AC pavements was less than that of the two other pavement types, the database was

divided into training (80%) and validation (20%) sets for AC pavements. The training dataset

was used for developing the model and conducting the learning process, while the validation

dataset was used for checking the accuracy of the model.

 17

Validation

Model validation is performed to confirm that the output of the statistical model is

acceptable with respect to the collected data (actual data). In order to evaluate any machine

learning model, it is necessary to test the model with data not used in the training set. In this

study, a Train_Test split approach was used for Cross-Validation (CV), a validation technique

that checks the effectiveness of the machine-learning model. After performing model training on

70% of the database (the training dataset), the validation dataset was used as a test sample to

validate model performance.

Comparison

The LSTM model performance was compared with the sigmoidal and exponential

functions used by Iowa DOT to fit deterioration models for individual sections. The accuracy of

each model with respect to riding, cracking, and rutting in AC and COM pavement types, and

riding, cracking, and faulting in PCC pavement types were compared for both models.

Result and Discussion

In the following sections, the application of each modeling approach in the databases of

the three different pavement types is described and the results are presented and discussed. The

overall results from both models are presented in Table 4, with the actual value of each index

compared with the predicted value of the same index from the LSTM and Iowa DOT regression

models.

The Iowa DOT has an individual regression model for each individual section with

specific factors for predicting the future condition of the pavements based on age. While the

sigmoidal transformation functions were applied to cracking, rutting, and faulting indices, the

exponential function was used to fit the riding index. Based on the actual and predicted values of
 18

each index, the PCI value was calculated for each pavement type. Figures (11-13) present the

comparisons between actual PCI value and predicted PCI value for each pavement type in the

DOT and LSTM models.

 Table 2: Summary statistic of each model on the test dataset
Actual mean Predicted mean Actual standard deviation Predicted standard deviation R-square
PCC DOT PCI 58.06 68.63 23.18 19.14 0.44
Crac 79.62 83.02 23.83 17.56 0.26
k
Fault 61.27 99.74 20.04 0.21 -3.68
Ride 34.89 38.69 39.55 37.82 0.66
LSTM PCI 58.06 54.13 23.18 21.12 0.70
Crac 79.62 67.67 23.83 20.95 -0.26
k
Fault 61.27 62.78 20.04 14.30 0.62
Ride 34.89 36.27 39.55 40.48 0.86
COM DOT PCI 68.71 78.66 19.61 17.9 0.11
Crac 62.91 78.08 19.74 15.75 -0.05
k
Rut 60.44 98.36 17.35 0.57 -4.7
Ride 78.64 74.51 32.41 34.35 -0.02
LSTM PCI 68.71 72.48 19.61 17.55 0.50
Crac 62.91 66.01 19.74 16.88 0.39
k
Rut 60.44 61.92 17.35 15.35 0.19
Ride 78.64 84.23 32.41 29.03 0.43
AC DOT PCI 71.02 82.95 19.58 17.73 0.31
Crac 64.11 80.88 24.52 16.02 0.15
k
Rut 64.05 98.42 15.14 0.47 -5.11
Ride 81.51 77.29 29.83 33.73 0.55
LSTM PCI 71.02 72.89 19.58 17.36 0.61
Crac 64.11 67.08 24.52 21.78 0.35
k
Rut 64.05 63.74 15.14 12.33 0.19
 20

Ride 81.51 83.28 29.83 27.65 0.61

 21

Figure 4: The Actual PCI over Predicted PCI in AC sections for DOT and LSTM models
respectively
 22

Figure 5: The Actual PCI over Predicted PCI in COM sections for DOT and LSTM models
respectively
 23

Figure 6: The Actual PCI over Predicted PCI in PCC sections for DOT and LSTM models
respectively
 24

It should be noted that the evaluations of the regression models are restricted to the

residuals between the fitted functions and the actual readings, although the LSTM evaluation was

based on its ability to predict full performance curves not included during the training stage. For

validating the prediction results of the individual regression models and comparing the results of

the current Iowa DOT method with LSTM models, 50 AC, 80 PCC, and 80 COM sections were

tested. The results were compared with the actual value of each index.

The comparison included models developed for AC, COM, and PCC pavements. R-

square and Standard Error of Estimate (SEE) were considered to evaluate the accuracy of the

models. The R-square and SEE functions are shown in equation 7 and 8:

R2=1− ( SSres
SStot )
=1−¿ (7)

SEE= √ ∑ ( Y −Ẏ ) /N
i i (8)

Where Y i is the actual value, Y^i is the predicted value, Y i is the average of actual values, and N

represents the number of observation.

The results for AC pavements showing that the LSTM model got higher prediction

accuracy, compared to the individual DOT regression models. The R-square values in the LSTM

models were 0.61 for the riding index, 0.19 for the rutting index, 0.35 for the cracking index, and

0.61 for the PCI. This is while the values for the DOT models were 0.55, -5.11, 0.15, and 0.31,

respectively. It is worth mentioning, that, R-square is defined as the proportion of variance

explained by the fit, if the fit is actually worse than just fitting a horizontal line, then R-square is

negative. Also the result of SEEs for both models indicates that the LSTM model got less

standard error of estimate, compared to DOT models. The SEE values in the LSTM models were

18.66 for the riding index, 19.74 for cracking index, 13.58 for rutting index, and 12.18 for the
 25

PCI. This is while the values for the DOT models were 20.08, 22.57, 37.40, and 16.17,

respectively.

Also The results for COM pavements showing that the LSTM model got higher

prediction accuracy. The R-square values were 0.43 for the riding index, 0.19 for the rutting

index, 0.39 for the cracking index, and 0.50 for the PCI in LSTM models, while the

corresponding values for the DOT models were -0.02, -4.7, -0.05, and 0.11, respectively. Also

SEE metrics for both models indicates that the LSTM model got less standard error of estimate,

compared to DOT models. The SEE values in the LSTM models were 24.5 for the riding index,

15.29 for cracking index, 15.57 for rutting index, and 13.78 for the PCI. This is while the values

for the DOT models were 32.7, 19.72, 41.48, and 18.46, respectively.

Also, the LSTM model outperformed DOT’s regression models with respect to PCC

pavements. Fluctuations in the PCC database due to maintenance activities were less than the

two other pavement types. The R-square values were 0.86 for the riding index, 0.62 for the

rutting index, -0.26 for the cracking index, and 0.70 for the PCI in LSTM models, and the

corresponding values in the DOT models were 0.66, -3.68, 0.26, and 0.44, respectively. Also the

result of SEEs for both models indicates that the LSTM model got less standard error of

estimate, compared to DOT models. The SEE values in the LSTM models were 14.71 for the

riding index, 26.83 for cracking index, 12.4 for faulting index, and 12.51 for the PCI. This is

while the values for the DOT models were 22.96, 20.37, 43.35, and 17.21, respectively.

Figures (14-16) also reflect the effect of age on the prediction residuals for each model in

both the short and long term duration. These results show that the errors will more significantly

widen and fluctuate after the first five years of pavement age for all three pavement types.

Residuals can generally be either positive or negative; however consistent differences between
 26

the predicted and observed values to one side of the prediction model is referred to as bias, and

the variability in the mean observed value of these residuals is referred to as variance. Bias can

be formally defined as the expected value of the model residuals, as shown in Equation 9.
N
1
Bias=E [ ^y − y ] ≈ ∑ϵ (9)
N i=1

Where ^y is the predicted value, y is the observed value, and ϵ is the model residual ϵ =^y − y.
As can be seen in Figures (14-16), the DOT regression models show a consistently higher

bias as the average line deviates from the zero value. To check whether the bias of the DOT

regression model was significantly higher or lower than the LSTM model bias a hypothesis test

was performed to calculate the regression and LSTM models average absolute residual values.

To determine the possibly unequal residual variance between the models, the Kolmogorov-

Smirnov test, a non-parametric test that allows for testing with unequal variances, was

performed. Results showed that the regression model had a significantly higher bias with a

negative value, meaning that the regression model will consistently overestimate the index

values and result in less conservative predictions. The means of the residual of the PCI for the

LSTM and DOT models were (3.93, -10.57) for PCC, (-1.87, -11.93) for AC, and (-3.77, -9.94)

for COM pavement types. Even though the variance of the residuals increased in the LSTM over

time, the mean of the residual in the LSTM model was still less than that of the regression

models. The solid black line and dotted blue line in the figures show how the mean errors

changed over time.

 27

Figure 7: PCI Residual vs Age in AC pavements

Figure 8: PCI Residual vs Age in COM pavements

 28

Figure 9: PCI Residual vs Age in PCC pavements

Conclusion

The deterioration models of the historical pavement condition data for the state of Iowa

were developed using an LSTM approach. The proposed model and current method in Iowa

DOT were compared to investigate the model accuracy. The comparison between the developed

model and the individual regression models used by the Iowa DOT from the three different

pavement types indicates that prediction accuracy in the LSTM model is higher than individual

regression models.

The LSTM achieved a higher PCI prediction accuracy than the individual regression

models in all three pavement types. A hypothesis analysis of mean was conducted for the PCI

residual in both techniques and the results exhibit less LSTM bias than that of individual

regression models.

Overall, each of these two methods has its own advantages and disadvantages. The

equation of the individual regression models requires an annual update, and each section will
 29

exhibit a new year-by-year behavior, making the prediction process more complex. The LSTM is

only one more consistent model compatible for all sections using a training process. The LSTM

approach was sensitive to the data fluctuation resulting from unrecorded maintenance activities.

While the evaluation of the regression models was restricted to residuals between the fitted

functions and the actual readings, the evaluation for the LSTM was based on its ability to predict

full performance curves not included during the training stage.

References

AASHTO. (2012). Pavement management guide. 2012. (2nd ed.). AASHTO

Adeli, H. (2001). Neural Networks in Civil Engineering : 1989 − 2000, Computer-Aided Civil
and Infrastructure Engineering, 16(2), 126-142,
https://doi.org/10.1111/0885-9507.00219

Alharbi, F. (2018). Predicting pavement performance utilizing artificial neural network (ANN)
models. Graduate Theses and Dissertations at Iowa State University.
https://lib.dr.iastate.edu/etd/16703/

Basheer, I. A., & Hajmeer, M. (2000). Artificial neural networks: Fundamentals, computing,
design, and application. Journal of Microbiological Methods, 43(1), 3-31.

Bektas, F., Smadi, O. G., & Al-Zoubi, M. (2014). Pavement management performance modeling:
Evaluating the existing PCI equations. Institute for Transportation at Iowa State University
project reports. https://lib.dr.iastate.edu/intrans reports/100/

Bursanescu, L., & Blais, F. (1997, May). Automated pavement distress data collection and
analysis: a 3-D approach. In Proceedings. International Conference on Recent Advances in
3-D Digital Imaging and Modeling (Cat. No. 97TB100134) (pp. 311-317). IEEE

Busseti, E., Osband, I., & Wong, S. (2012). Deep Learning for Time Series Modeling, Final
Project Report,
https://pdfs.semanticscholar.org/a241/a7e26d6baf2c068601813216d3cc09e845ff.pdf

Ceylan, H., Birkan Bayrak, M., Gopalakrishnan, K., & . (2014). Neural Networks Applications
in Pavement Engineering: A Recent Survey. International Journal of Pavement Research
and Technology, 7(6), 434–444. https://doi.org/10.6135/ijprt.org.tw/2014.7(6).434

Chris Olah. (2015). Understanding LSTM Networks -- colah’s blog. Retrieved from
https://colah.github.io/posts/2015-08-Understanding-LSTMs/
 30

Dougherty, M. (1995). A review of neural networks applied to transport. Transportation

Research Part C. https://doi.org/10.1016/0968-090X(95)00009-8

Flood, I. (2008). Towards the next generation of artificial neural networks for civil engineering.
Advanced Engineering Informatics. https://doi.org/10.1016/j.aei.2007.07.001

Flood, I., & Kartam, N. (1994). Neural networks in civil engineering. I: Principles and
understanding. Journal of computing in civil engineering, 8(2), 131-148.

George, K. P. (2000). MDOT pavement management system: prediction models and feedback

system (No. FHWA/MS-DOT-RD-00-119). Mississippi. Dept. of Transportation.

Golroo, A., & Tighe, S. L. (2012). Development of pervious concrete pavement performance
models using expert opinions. Journal of transportation engineering,  138(5), 634-648.
https://doi.org/10.1061/(ASCE)TE.1943-5436.0000356.

Graves, A. 2010. Supervised Sequence Labelling with Recurrent Neural Networks. Retrieved
from https://www.cs.toronto.edu/~graves/preprint.pdf

Haas, R., Hudson, W., & Zaniewski, J. (1994). Modern Pavement Management. In Krieger
Publishing Company (Vol. 102, p. 583).

Haas, R., & Hudson, W. R. (2015). Pavement asset management. John Wiley & Sons.
https://doi.org/10.1002/9781119038849

Hassan, R., Lin, O., & Thananjeyan, A. (2017). A comparison between three approaches for
modeling deterioration of five pavement surfaces. International Journal of Pavement
Engineering, 18(1), 26-35., https://doi.org/10.1080/10298436.2015.1030744

Horne, J. D., & Manzenreiter, W. (2004). Accounting for Mega-Events. International Review for
the Sociology of Sport. https://doi.org/10.1177/1012690204043462

Hu, Y. G., Cheng, H. D., Glazier, C., Wang, J., Chen, X. W., & Shi, X. J. (2007). Novel
Approach to Pavement Cracking Detection Based on Neural Network. Transportation
Research Record: Journal of the Transportation Research Board, 1764(1), 119–127.
https://doi.org/10.3141/1764-13

Huang, Y., & Moore, R. (1997). Roughness Level Probability Prediction Using Artificial Neural
Networks. Transportation Research Record: Journal of the Transportation Research Board,
1592, 89–97. https://doi.org/10.3141/1592-11

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
https://doi.org/10.1038/nature14539
 31

Lin, J., Yau, J.-T., & Hsiao, L.-H. (2003). Correlation Analysis Between International
Roughness Index (IRI) and Pavement Distress by Neural Network. 82nd Transportation
Research Board Annual Meeting, (January 2003).

Martens, J., & Sutskever, I. (2011). Learning recurrent neural networks with hessian-free
optimization. In Proceedings of the 28th international conference on machine learning
(ICML-11) (pp. 1033-1040). Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?
doi=10.1.1.222.6736&rep=rep1&type=pdf

de Melo e Silva, F., Van Dam, T. J., Bulleit, W. M., & Ylitalo, R. (2000). Proposed pavement
performance models for local government agencies in Michigan. Transportation research
record,  1699(1), 81-86.

Nallamothu, S., & Wang, K. C. (1996). Experimenting with recognition accelerator for pavement
distress identification. Transportation research record, 1536(1), 130-135.

Mills, L. N. O., Attoh-Okine, N. O., & McNeil, S. (2012). Developing pavement performance
models for Delaware. Transportation research record,  2304(1), 97-103.
https://doi.org/10.3141/2304-11

Laptev, N., Yosinski, J., Li, L. E., & Smyl, S. (2017, August). Time-series extreme event
forecasting with neural networks at uber. In International Conference on Machine
Learning  (Vol. 34, pp. 1-5).
http://roseyu.com/time-series-workshop/submissions/TSW2017_paper_3.pdf

Li, N., Xie, W. C., & Haas, R. (1996). Reliability-based processing of Markov chains for
modeling pavement network deterioration. Transportation research record, 1524(1), 203-
213. https://journals.sagepub.com/doi/abs/10.1177/0361198196152400124?
journalCode=trra

Lytton, R. L. (1987, November). Concepts of pavement performance prediction and modeling.

In  Proc., 2nd North American Conference on Managing Pavements (Vol. 2).

Owusu‐Ababio, S. (1998). Effect of neural network topology on flexible pavement cracking

prediction. Computer‐Aided Civil and Infrastructure Engineering, 13(5), 349-355.
https://doi.org/10.1111/0885-9507.00113

Ragnoli, A., De Blasiis, M. R., & Di Benedetto, A. (2018). Pavement distress detection methods:
A review. Infrastructures, 3(4), 58.
https://doi.org/10.3390/infrastructures3040058

Saha, P., Ksaibati, K., & Atadero, R. (2017). Developing pavement distress deterioration models
for pavement management system using Markovian probabilistic process. Advances in Civil
Engineering, 2017. https://doi.org/10.1155/2017/8292056
 32

Shabanpour, R. (2017). Pavement Performance Modeling : Literature Review and Research

Agenda, Illinois Asphalt Pavement Association
https://il-asphalt.org/files/7315/1803/1691/Ramin_Shabanpour_2017_UICg.pdf

Shahnazari, H., Tutunchian, M. A., Mashayekhi, M., & Amini, A. A. (2012). Application of soft
computing for prediction of pavement condition index. Journal of Transportation
Engineering, 138(12), 1495–1506.
https://doi.org/10.1061/(ASCE)TE.1943-5436.0000454

Sutskever, I. 2013. (n.d.). TRAINING RECURRENT NEURAL NETWORKS. Dissertation

submitted to Graduate Department of Computer Science University of Toronto Retrieved
from https://www.cs.utoronto.ca/~ilya/pubs/ilya_sutskever_phd_thesis.pdf

Taha, M. A., & Hanna, A. S. (1995). Evolutionary neural network model for the selection of
pavement maintenance strategy. Transportation Research Record, (1497).
http://onlinepubs.trb.org/Onlinepubs/trr/1995/1497/1497-009.pdf

Iowa Department of Transportation Transportation Asset Management Plan. (2018), (April),

https://iowadot.gov/systems_planning/fpmam/IowaDOT-TAMP-2018.pdf

Vasquez, C. A. (2011). Pavement management systems on a local level. Graduate Theses and
Dissertations at Utah State University https://digitalcommons.usu.edu/cgi/viewcontent.cgi?
article=1088&context=gradreports

Wolters, A. S., & Zimmerman, K. A. (2010). Research of Current practices in pavement

performance modeling (No. FHWA-PA-2010-007-080307). Retrieved from
https://trid.trb.org/view/919191

Wong, W. K., Xia, M., & Chu, W. C. (2010). Adaptive neural network model for time-series
forecasting. European Journal of Operational Research, 207(2), 807-816.
https://www.sciencedirect.com/science/article/pii/S0377221710003784