2020 IEEE 8th International Conference on Computer Science and Network Technology (ICCSNT)

Highway Toll Forecasting Model

Zhixiong Zhang Yun Wu
Inner Mongolia Baotou Teachers’ College Library Inner Mongolia Baotou Teachers’ College
Books and Materials Major Computer Software and Theory Major
Inner Mongolia, China Inner Mongolia, China
[email protected] [email protected]

Abstract—Since the complexity of artificial design features II. OPTIMIZE DEEP BELIEF NETWORK
in model training, the toll data features cannot be used
reasonably and efficiently. Toll prediction of one single station Highway toll is affected by many factors, and the process
based on toll historical data would ignore the interaction of manual design features is complicated and easy to make
between stations in the highway network. Therefore, this paper mistakes. However, DBN can learn valuable features by
constructs a highway toll forecast model, named DBN-MSVR stacking RBM for unsupervised layer by layer training and
combining deep belief network and multi-task learning for supervised parameter fine-tuning. Therefore, DBN is selected
multi-station toll forecasting. This model uses the optimized in this paper to describe the complex mapping of input and
deep belief network to perform feature learning on toll data, and features through several nonlinear feature changes to
combines multi-task learning and support vector regression on automatically learn the representative and robust high-
the top layer of the deep belief network to predict tolls. dimensional features of toll.
Experiments show that the DBN-MSVR toll prediction model
has higher prediction accuracy than traditional methods. A. Optimize Restricted Boltzmann Machines（CRBM）
Keywords—toll forecast; continuous restricted boltzmann Generally, the visual layer input of traditional RBM unit
machines; deep belief network; multi-task learning; support in DBN must be binary, but the toll data of expressway is
vector regression continuous, so it is difficult to use DBN for feature learning.
In order to solve this problem, the traditional RBM is
I. INTRODUCTION transformed into a Gaussian Bernoulli RBM cell by adding a
Expressway toll prediction is the key research direction of Gaussian noise additive term into the visual layer (Continuous
expressway operation management [1], the main methods are RBM), which overcomes the limitation that the input value of
non-parametric regression prediction model [2-4], neural the visual layer can only be binary numbers, and has the ability
network prediction model [5-6], time series model [7] and to process continuous data. The function of energy
combined prediction model [8-9]. Zhao Qi et al.[10] used the relationship between visible layer and hidden layer is changed
linear combination forecasting to establish a combined to
prediction model of toll collection station toll by combining (vi  ai )2 v
multiple neural network prediction models linearly; Liu Ning E(v, h)=   b j h j   i2 h j wij (1)
et al. [11] proposed a hybrid highway toll prediction model i 2 i2
j i j i
based on GEP algorithm, and established a modified model where, vj and hj are binary states representing the input i of
for the impact of holiday restriction and exemption policy on the visual layer and the feature j of hidden unit; ai and bj are
small cars; Pan Yong et al .[12] analyzed the characteristics of the offsets of visible layer and the hidden layer; wij is the
toll data and built short-term and long-term toll prediction weight matrix between them; and σ is the standard variance
models for a single highway station based on the weighted vector of Gaussian function. The energy relationship between
coefficient combination method. them can be obtained according to Formula 1, and their
probability distribution can be calculated as follows:
Most related researches manually select features for a n
large number of sample data based on prior knowledge in a
specific field. The manual method is complicated and error-
p (vi , h)  N (a i   i  h j wij ,  i2 ) (2)
j 1
prone, and cannot fully reflect the nature of the data. Moreover,
n
only the toll of a single toll station of the freeway is modeled,
and the relevant influence between the stations is ignored. In p(hi , v)  sigm( vi wij  b j ) (3)
view of the above two shortcomings, this paper proposes a i 1
multi-task highway toll prediction model (DBN-MSVR) where N(x) represents a Gaussian distribution. CRBM has the
based on deep learning, which mainly includes: feature same training and parameter updating process as the
learning and prediction model, which are Gaussian distributed traditional RBM. In this paper, the RBM rapid learning
DBN feature learning and Multitask support vector regression method proposed by Hinton [13] is adopted for training, and
prediction model. The bottom layer is a DBN architecture the trained RBM is connected to a regression layer for
composed of continuous restricted Boltzmann machine prediction.
(CRBM) with Gaussian distribution and RBM. Through B. CDBN Feature Learning Model
unsupervised feature learning of CRBM-DBN, the data
features representing the essential law of toll are abstracted. In the highway network, if the toll of the station i at the
The multi task support vector regression layer is added to the time t is represented, then the toll sequence of the highway can
top layer for supervised training, and the top-level weight be expressed as follows:
clustering method of deep network structure is used to group
the tasks, and the related tasks in the group jointly train the X it  {xit1 , xit2 ,…，xitn } (4)
model to improve the accuracy of prediction. where i=1,2,…，n；t=1,2,…，T. Then the tolls forecast is
based on the past historical toll series to predict the future toll
of a station in {T+Δt}, in which Δt can be adjusted.

978-1-7281-8123-3/20/$31.00 ©2020 IEEE 81 Dalian, China·Nov 20-22, 2020

Authorized licensed use limited to: University of Canberra. Downloaded on May 21,2021 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
 The whole DBN feature learning process is an A. Correlation Analysis of Expressway Stations
unsupervised learning process. As shown in Figure 1, the In highway network, regional economic development,
whole structure consists of one CRBM and two RBMs. V transportation supply and demand, road traffic flow and other
indicates that the visual layer is used to receive input signals, factors affect the level of toll revenue at different levels. This
H indicates that the hidden layer is used for feature learning, paper verifies the correlation degree of toll between stations
X and X* correspond to input and output data respectively. by Pearson correlation coefficient. Suppose the highway
Firstly, the continuous toll data X is taken as the input of the network is a network graph G, then G= (N, E) where N
visual layer Vi of the first CRBM unit, and then the output of represents the number of nodes in the road network, and E
the CRBM after training is taken as the input of the next RBM represents the collection of all stations in the whole network.
unit. After all units are trained in turn, a feature representative Assuming that there are p stations in the highway network,
model H representing the historical toll time series is obtained.
After feature learning and training, the previous toll time then there is M m  { xir , yir }ir1 , for each station si has a
series X can be transformed into another feature space X* corresponding continuous time series, which is recorded as
through H, namely H(x)=X*. Finally, a set of characteristic the matrix qi, qi  { z ( si , t1 ), z ( si , t 2 ), …, z ( si , t d )}T ,
space X* paired with the target task is given and represented
then qi represents the toll of station si in time t1 to td, where
as {(X 1 , Z1 ), (X  2 , Z 2 ), …,(X  n , Z n )} for the prediction
z ( si , t j ) represents the toll of station si in (t j  t0 , t j ) . The
model.
toll data of the whole road network constitute a two-
dimensional matrix, which is recorded as
CRBM RBM
V1 H1 V3 H3 Q  [ q1 , q2 , …, qi ], Q  R d  p . Pearson correlation
coefficient formula is used to calculate the correlation
Input X
coefficient R(i,j) of any two stations in the network. |R(i,j)| is
OutputX* used to describe the degree of correlation between the two
stations. The value range of |R(i,j)| is [0,1].
The toll data of 128 stations in a provincial highway
network in June 2019 are selected to calculate the correlation
coefficient. According to Pearson correlation coefficient
calculation formula, the R value of toll correlation coefficient
of each station is known, as shown in Table 1 (Due to the
large dimension of the R matrix, only part of the value is
listed).
V2 H2
RBM It can be seen from the table that the maximum value of
the correlation coefficient corresponding to each station is
Fig. 1. CDBN feature learning process
above 0.9, so the correlation between the corresponding
stations is relatively large, and the correlation between the
The whole process of CDBN feature learning uses less
two stations with small correlation coefficient value is
prior knowledge in the traffic field. By inputting the original
relatively small. The joint training of related sites can
toll data into the visual layer and automatically learning the
improve the overall performance of prediction, and multi-
deep network structure with multiple hidden layers, the high-
task learning can make full use of this feature and divide the
dimensional features better representing the characteristics of
related tasks into a group of common training models, so that
toll data are obtained. Since CDBN has learned representative
the weight of the shared hidden layer can be efficiently used
features, it uses the learned features in the top-level regression
and the prediction accuracy can be improved.
layer as the input vector of the prediction to make the
prediction result closer to the real value.
III. FREEWAY TOLL FORECASTING MODEL

TABLE I. R VALUE TABLE OF TOLL CORRELATION COEFFICIENT OF EACH STATION

Toll station Correlation coefficient R
number
1 1 0.9918 0.9787 0.8584 … 0.8269 0.8253 0.8351
2 0.9918 1 0.9924 0.8884 … 0.8358 0.8344 0.7984
3 0.9787 0.9924 1 0.8833 … 0.8299 0.8285 0.7906
4 0.8584 0.8884 0.8833 1 … 0.7585 0.7566 0.5259
… … … … … … … … …
126 0.8269 0.8358 0.8299 0.7585 … 1 0.9981 0.6536
127 0.8253 0.8344 0.8285 0.7566 … 0.9981 1 0.6539
128 0.8357 0.7984 0.7906 0.5259 … 0.6536 0.6539 1

82

Authorized licensed use limited to: University of Canberra. Downloaded on May 21,2021 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
B. Multitask Support Vector Regression Prediction Task1
According to the above analysis, there is a strong output(4)
correlation between the tolls of stations in the highway
network. This feature should be fully considered to improve
Task2
forecasting performance. However, most of the methods are INPUTS output(3)
only based on the historical data of highway tolls to predict a （X*）
single site which have certain limitations, ignoring the

…
interaction between sites, and cannot fully reflect the toll Task3
status of the site. MTL utilizes the inductive transfer output(3)
mechanism to select multiple training targets for the training
samples, and improves the generalization ability of the
learning system according to specific relevant information Task4
among targets. Through the mechanism of MTL sharing output(4)
W
hidden layer, the common information between tasks can be
extracted effectively and the predictive ability of tasks can be Fig. 2. Multitask support vector regression prediction
improved.
Therefore, this paper uses multi-task learning (MTL) MTL combined with SVR is used to predict toll. The tasks
combined with support vector regression (SVR) to predict toll. with similar weight distribution are divided into a group
The specific network structure is shown in Figure 2. If the according to the top weight of the network structure. The
representative and valuable features obtained from DBN tasks in the group trained the model together and Shared the
training are used as the input of multi task support vector weight of the hidden layer. Both the difference between tasks
regression layer, then the weight of support vector regression and the connections between tasks are considered. Compared
layer can be regarded as the connection between each task with single task learning, only one model needs to be trained,
and these representative features. Since MTL is implemented and the model can train several models simultaneously.
by weight sharing, weight grouping is more suitable in deep IV. EXPERIMENTS
architecture. Using k-means to cluster the top-level weights,
the tasks with similar weight distribution are divided into a A. Experimental Data Description
group, and these tasks also have the high-dimensional In this paper, a highway toll prediction model (DBN-
characteristics of feature learning. If the tasks in the group are MSVR) which combines deep belief network and multi-task
trained together, more information can be shared between the learning is constructed. In order to verify the applicability and
related tasks, which can make full use of the characteristics accuracy of the prediction model, this paper selects the
of station tolls. highway in a province as the research object, selects the data
from 2012 to 2019 in the region, and summarizes the data into
Suppose there are m tasks in total, M m  { xir , yir }ilr1 ,
a day's toll. The seven-year data from 2012 to 2018 is used as
and M is the collection of these m tasks. The learning weight the training set, and the 2019 data is used as the test set.
of each task is divided into two parts: w+wr, r
B. Evaluation Index
w  wr , r  (1, 2,…, m) , w represents the common
The performance index parameters selected in this paper
characteristics of all tasks, and wr represents the unique are root mean square error (RMSE), absolute percentage error
characteristics of the task. Then the optimization problem of (MAPE) and mean absolute error (MAE), which are defined
multi-task support vector regression prediction model is as as follows:
follows:
1 b 
 m 
m lr
1 MAE= Yi Yi ，
min ( w, w)   ( wr , wr )  C  ir ，(5) n i1
(9)
w ,b 2 2 r 1 r 1 i1
1 n 
st. . yir ((w,(xir ))  b  (wr ,r (xir ))  dr ) 1ir ，(6) RMSE= (Yi Yi )2 ， (10)
n i1
ir  0, i 1,…, lr , r 1,…, m. ， (7) 
Both w and wr are trained to calculate the result at the 1 n Yi Yi
same time. All tasks have the same w, and each task has its MAPE   100%， (11)
own wr. β is used to adjust the relative weight between the n i1 Yi
two. ξir is the relaxation variable, and C is the penalty where: 𝑌 is the actual toll value; 𝑌 is the predicted value.
coefficient. The main purpose of the prediction model is to
find m decision functions, and the formula is: C. Experimental Results and Analysis
In this paper, TensorFlow is used to realize the toll
fr (x)  (w,(xir )) b  (wr ,r (xir ))  dr , r 1,…,m (8) prediction model of DBN-MSVR highway. The number of
layers K of CDBN is set to 3 and the time interval T is 4 [14].
Each decision function consists of two parts: the The tolls of the first four time periods of 128 stations were
common weight w, whose bias term is b and the wr part for selected as input, that is, a total of 512 inputs, and the
each group, whose bias term is dr. predicted values of all stations is taken as its output. The
number of iteration of CDBN feature learning process is 1000.
In the top-level multit-ask support vector regression
prediction model, the kernel function of SVR is set as the

83

Authorized licensed use limited to: University of Canberra. Downloaded on May 21,2021 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
radial basis function, the penalty factor is 0.01, and the
training times are 1000.
This paper is divided into two parts to verify the proposed
DBN-MSVR model. One part is to compare the DBN-SVR
model with Gaussian distribution (CDBN-SVR) and the
DBN-SVR method, and use the same data set to predict a
single station; the other part is to compare DBN-MSVR
model with CDBN-SVR model, and use the DBN-MSVR
model to predict the toll of the whole road network station,
and the results are compared with those predicted by the
CDBN-SVR model for the corresponding single task.
(1) Forecast of toll fees at a single site
In order to verify the effectiveness of the deep network
structure proposed in this paper, CDBN-SVR and DBN-SVR
are used to compare the two methods for toll prediction for （a）
the same station. The performance indicators of the two
algorithms are shown in Table 2.

TABLE II. PERFORMANCE INDEX COMPARISON

Evaluation Methods
index DBN-SVR CDBN-SVR
MAE 15.78 15.36
RMSE 17.82 16.79
MAPE 6.385 5.483

（b）

Fig. 4. Comparison of two methods with real values (a) DBN-SVR

compared with real values (b) CDBN-SVR compared with real values

It can be seen from Figure 4 that in the prediction results

of the actual toll, the error of the CDBN-SVR model is
smaller than that of DBN-SVR model, and the prediction
results have a smaller deviation from the actual toll data,
Fig. 3. Comparison of the accuracy of the two algorithms which is closer to the actual toll curve.
As shown in Figure 3, it can be seen that when the number
of iterations is small, the accuracy gap between the two
algorithms was not very large. However, with the increase of
the number of iterations, the prediction accuracy of the
CDBN-SVR method gradually increases with the increase of
the number of iterations, and finally maintains at about 85%.
While the prediction accuracy of ordinary DBN-SVR model
also increases gradually with the increase of iteration times,
but the accuracy of DBN-SVR model remains at about 76.7%.
According to the above experimental results, the
prediction accuracy of CDBN-SVR model is improved
compared with the ordinary DBN-SVR model. Comparing
the DBN-SVR model and CDBN model with the actual toll,
Figure 5 shows the comparison between the predicted and
actual values of the toll No.76 in June 2017 for the two
methods.
Fig. 5. DBN-MSVR toll forecast value and actual value comparison

(2) Prediction of tolls for the entire highway road site

84

Authorized licensed use limited to: University of Canberra. Downloaded on May 21,2021 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.
 The DBN-MSVR prediction model proposed in this paper stations in the highway network, and adopt the method of top-
is used to predict the whole road network. As shown in Figure level weight clustering to group related tasks into a group of
5, the predicted value of DBN-MSVR model No. 76 in June joint training shared hidden layers to improve prediction
2019 has a small deviation from the actual value, and it performance. The experimental results show that the
reflects the basic law of toll change. Among them, the peak proposed DBN-MSVR toll prediction model has a good
stage is around the rest day of each week, and the first peak agreement between the predicted value and the actual value,
stage is where the predicted value deviates greatly from the and is an effective toll prediction method. However, this
actual value. It is mainly due to the decrease of the flow on article does not further analyze and study the impact of the
the rest day of the week after the Dragon Boat Festival, and national holiday small car exemption policy and other factors.
the fluctuation is large.
REFERENCES
In order to further verify the effectiveness of the [1] Zhao Jiandong, Wang Hao, Liu Wenhui, et al. Highway travel time
prediction model proposed in this paper, as shown in Figure prediction between stations based on toll ticket data [J]. Journal of
6, the accuracy of DBN-MSVR model for a group of related Tongji University: Natural Science, 2013, 41 (12): 1849-1854.
tasks is compared with that of CDBN-SVR. Compared with [2] Andrysiak T, Łukasz Saganowski, Choraś M, et al. Network Traffic
the corresponding single-task CDBN-SVR model, the Prediction and Anomaly Detection Based on ARFIMA Model[J].
prediction accuracy of the dbN-MSVR model with a group of Advances in Intelligent Systems & Computing, 2014, 299:545-554.
related tasks grouped by the top-level weight has been [3] Celenk M, Conley T, Graham J, et al. Anomaly prediction in network
improved by about 2.8% on average, and the station toll traffic using adaptive Wiener filtering and ARMA modeling[C]// IEEE
International Conference on Systems, Man and Cybernetics. IEEE,
prediction results with MTL have been significantly 2009:3548-3553.
improved compared with the single-task learning prediction. [4] Hong W C. Traffic Flow Forecasting by Seasonal SVR with Chaotic
Only when the related tasks are integrated into a group, the Simulated Annealing Algorithm [J]. Neurocomputing, 2011, 74(12–
overall performance can be better improved. Unrelated tasks 13):2096-2107.
may have a negative impact on model training. [5] Chan K Y, Dillon T S, Singh J, et al. Neural Network Based Models
for Short-Term Traffic Flow Forecasting Using a Hybrid Exponential
Smoothing and Levenberg–Marquardt Algorithm [J]. IEEE
Transactions on Intelligent Transportation Systems, 2012, 13(2):644-
654.
[6] Ding Hongfei, Li Yanhong, Liu Bo. Expressway’s travel time
prediction based on combined BP neural network and support vector
machine approach[J] Application Research of Computers, 2016,
33(10):2929-2932.
[7] Yu Lin, Shu Qin, Bai Jiqiong. Traffic flow prediction method based on
EMD clustering and ARMA [J]. Highway, 2015 (5): 124-129.
[8] Wang J, Deng W, Guo Y. New Bayesian combination method for
short-term traffic flow forecasting[J]. Transportation Research Part C
Emerging Technologies, 2014, 43:79-94.
[9] Qian Wei, Yang Kuangli, Yang Huihui,et al. Short-term traffic flow
prediction based on combined models [J]. Computer Simulation, 2015,
32 (2): 175-178.
[10] Zhao Qi. Highway toll prediction research [D]. Xi ‘ an: Chang ‘ an
University, 2010.
[11] Liu Ning, Huang Zhangcan, Tan Qing. Research on freeway toll
Fig. 6. Comparison of the accuracy of DBN-MSVR and CDBN-SVR for a prediction method based on GEP[J/OL] Application Research of
group of tasks Computers. 2019, 36(7). [2018-04-12].
[12] Pan Yong. Research and Application of Higway Toll Prediction[D] Xi
V. CONCLUSIONS ‘ an: Chang ‘ an University, 2010.
This paper proposes a deep network model based on deep [13] Hinton G E, Osindero S, Teh Y W. A fast learning algorithm for deep
belief network combined with multi-task support vector belief nets [J]. Neural Computation, 2006, 18 (7): 1527.
regression for highway toll forecast. Among them, CRBM [14] Huang W, Song G, Hong H, et al. Deep Architecture for Traffic Flow
Prediction: Deep Belief Networks With Multitask Learning[J]. IEEE
with Gaussian distribution is used for the problem that RBM Transactions on Intelligent Transportation Systems, 2014, 15(5):2191-
can only accept binary input and cause data loss. Introduce 2201.
MTL to take advantage of the correlation between the tolls of

85

Authorized licensed use limited to: University of Canberra. Downloaded on May 21,2021 at 14:44:18 UTC from IEEE Xplore. Restrictions apply.